TY - GEN
T1 - Self-Concordant Analysis of Frank-Wolfe Algorithms
AU - Dvurechensky, Pavel
AU - Ostroukhov, Petr
AU - Safin, Kamil
AU - Shtern, Shimrit
AU - Staudigl, Mathias
N1 - Funding Information: The authors sincerely thank Shoham Sabach for his contribution in the early stages of this project, including his part in developing the basic ideas used in this paper. We would also like to thank Quoc Tran-Dinh for sharing MATLAB codes. This research is supported by the COST Action CA16228 "European Network for Game Theory". Publisher Copyright: © Author(s) 2020. All rights reserved.
PY - 2020
Y1 - 2020
N2 - Projection-free optimization via different variants of the Frank-Wolfe (FW), a.k.a. Conditional Gradient method has become one of the cornerstones in optimization for machine learning since in many cases the linear minimization oracle is much cheaper to implement than projections and some sparsity needs to be preserved. In a number of applications, e.g. Poisson inverse problems or quantum state tomography, the loss is given by a self-concordant (SC) function having unbounded curvature, implying absence of theoretical guarantees for the existing FW methods. We use the theory of SC functions to provide a new adaptive step size for FW methods and prove global convergence rate O(1=k) after k iterations. If the problem admits a stronger local linear minimization oracle, we construct a novel FW method with linear convergence rate for SC functions.
AB - Projection-free optimization via different variants of the Frank-Wolfe (FW), a.k.a. Conditional Gradient method has become one of the cornerstones in optimization for machine learning since in many cases the linear minimization oracle is much cheaper to implement than projections and some sparsity needs to be preserved. In a number of applications, e.g. Poisson inverse problems or quantum state tomography, the loss is given by a self-concordant (SC) function having unbounded curvature, implying absence of theoretical guarantees for the existing FW methods. We use the theory of SC functions to provide a new adaptive step size for FW methods and prove global convergence rate O(1=k) after k iterations. If the problem admits a stronger local linear minimization oracle, we construct a novel FW method with linear convergence rate for SC functions.
UR - https://www.scopus.com/pages/publications/85105180324
M3 - Conference contribution
T3 - 37th International Conference on Machine Learning, ICML 2020
SP - 2794
EP - 2804
BT - 37th International Conference on Machine Learning, ICML 2020
A2 - Daume, Hal
A2 - Singh, Aarti
T2 - 37th International Conference on Machine Learning, ICML 2020
Y2 - 13 July 2020 through 18 July 2020
ER -