The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication

Blake Woodworth; Brian Bullins; Ohad Shamir; Nathan Srebro

The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication

Blake Woodworth, Brian Bullins, Ohad Shamir, Nathan Srebro

Department of Computer Science and Applied Mathematics

Weizmann Institute of Science

Research output: Contribution to journal › Conference article › peer-review

Abstract

We resolve the min-max complexity of distributed stochastic convex optimization (up to a log factor) in the intermittent communication setting, where M machines work in parallel over the course of R rounds of communication to optimize the objective, and during each round of communication, each machine may sequentially compute K stochastic gradient estimates. We present a novel lower bound with a matching upper bound that establishes an optimal algorithm.

Original language	English
Pages (from-to)	4386-4437
Number of pages	52
Journal	Proceedings of Machine Learning Research
Volume	134
State	Published - 2021
Event	34th Conference on Learning Theory, COLT 2021 - Boulder, United States Duration: 15 Aug 2021 → 19 Aug 2021

Keywords

Distributed Stochastic Convex Optimization
Oracle Complexity of Optimization

All Science Journal Classification (ASJC) codes

Artificial Intelligence
Software
Control and Systems Engineering
Statistics and Probability

Access to Document

https://proceedings.mlr.press/v134/woodworth21a.html

Cite this

@article{4c690061f43e41d1a84f6fb461004497,

title = "The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication",

abstract = "We resolve the min-max complexity of distributed stochastic convex optimization (up to a log factor) in the intermittent communication setting, where M machines work in parallel over the course of R rounds of communication to optimize the objective, and during each round of communication, each machine may sequentially compute K stochastic gradient estimates. We present a novel lower bound with a matching upper bound that establishes an optimal algorithm.",

keywords = "Distributed Stochastic Convex Optimization, Oracle Complexity of Optimization",

author = "Blake Woodworth and Brian Bullins and Ohad Shamir and Nathan Srebro",

note = "Publisher Copyright: {\textcopyright} 2021 B. Woodworth, B. Bullins, O. Shamir \& N. Srebro.; 34th Conference on Learning Theory, COLT 2021 ; Conference date: 15-08-2021 Through 19-08-2021",

year = "2021",

language = "الإنجليزيّة",

volume = "134",

pages = "4386--4437",

}

TY - JOUR

T1 - The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication

AU - Woodworth, Blake

AU - Bullins, Brian

AU - Shamir, Ohad

AU - Srebro, Nathan

PY - 2021

Y1 - 2021

N2 - We resolve the min-max complexity of distributed stochastic convex optimization (up to a log factor) in the intermittent communication setting, where M machines work in parallel over the course of R rounds of communication to optimize the objective, and during each round of communication, each machine may sequentially compute K stochastic gradient estimates. We present a novel lower bound with a matching upper bound that establishes an optimal algorithm.

AB - We resolve the min-max complexity of distributed stochastic convex optimization (up to a log factor) in the intermittent communication setting, where M machines work in parallel over the course of R rounds of communication to optimize the objective, and during each round of communication, each machine may sequentially compute K stochastic gradient estimates. We present a novel lower bound with a matching upper bound that establishes an optimal algorithm.

KW - Distributed Stochastic Convex Optimization

KW - Oracle Complexity of Optimization

UR - http://www.scopus.com/inward/record.url?scp=85151113986&partnerID=8YFLogxK

M3 - مقالة من مؤنمر

VL - 134

SP - 4386

EP - 4437

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

T2 - 34th Conference on Learning Theory, COLT 2021

Y2 - 15 August 2021 through 19 August 2021

ER -

The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication

Abstract

Keywords

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this