From External to Swap Regret 2.0: An Efficient Reduction for Large Action Spaces

We provide a novel reduction from swap-regret minimization to external-regret minimization, which improves upon the classical reductions of Blum-Mansour and Stoltz-Lugosi in that it does not require finiteness of the space of actions. We show that, whenever there exists a no-external-regret algorithm for some hypothesis class, there must also exist a no-swap-regret algorithm for that same class. For the problem of learning with expert advice, our result implies that it is possible to guarantee that the swap regret is bounded by ϵ after (logN)^Õ(1/ϵ) rounds and with O(N) per iteration complexity, where N is the number of experts, while the classical reductions of Blum-Mansour and Stoltz-Lugosi require at least ω(N/ϵ²) rounds and at least ω(N³) total computational cost. Our result comes with an associated lower bound, which - in contrast to that of Blum-Mansour - holds for oblivious and ℓ₁-constrained adversaries and learners that can employ distributions over experts, showing that the number of rounds must be ω(N/ϵ²) or exponential in 1/ϵ. Our reduction implies that, if no-regret learning is possible in some game, then this game must have approximate correlated equilibria, of arbitrarily good approximation. This strengthens the folklore implication of no-regret learning that approximate coarse correlated equilibria exist. Importantly, it provides a sufficient condition for the existence of approximate correlated equilibrium which vastly extends the requirement that the action set is finite or the requirement that the action set is compact and the utility functions are continuous, allowing for games with finite Littlestone or finite sequential fat shattering dimension, thus answering a question left open in "Fast rates for nonparametric online learning: from realizability to learning in games"and "Online learning and solving infinite games with an ERM oracle". Moreover, it answers several outstanding questions about equilibrium computation and/or learning in games. In particular, for constant values of ϵ: (a) we show that ϵ-approximate correlated equilibria in extensive-form games can be computed efficiently, advancing a long-standing open problem for extensive-form games; see e.g. "Extensive-form correlated equilibrium: Definition and computational complexity"and "Polynomial-Time Linear-Swap Regret Minimization in Imperfect-Information Sequential Games"; (b) we show that the query and communication complexities of computing ϵ-approximate correlated equilibria in N-action normal-form games are N · poly log(N) and poly logN respectively, advancing an open problem of "Informational Bounds on Equilibria"; (c) we show that ϵ-approximate correlated equilibria of sparsity poly logN can be computed efficiently, advancing an open problem of "Simple Approximate Equilibria in Large Games"; (d) finally, we show that in the adversarial bandit setting, sublinear swap regret can be achieved in only Õ(N) rounds, advancing an open problem of "From External to Internal Regret"and "Tight Lower Bound and Efficient Reduction for Swap Regret".