Fast stochastic algorithms for SVD and PCA: Convergence properties and convexity

Ohad Shamir

Fast stochastic algorithms for SVD and PCA: Convergence properties and convexity

Ohad Shamir

Department of Computer Science and Applied Mathematics

Weizmann Institute of Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We study the convergence properties of the VR-PCA algorithm introduced by (Shamir, #y2015) for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observations of independent interest, such as how pre-initializing with just a single exact power iteration can significantly improve the analysis, and what are the convexity and nonconvexity properties of the underlying optimization problem.

Original language	English GB
Title of host publication	33rd International Conference on Machine Learning, ICML 2016
Editors	Maria Florina Balcan, Kilian Q. Weinberger
Pages	392-419
Number of pages	28
ISBN (Electronic)	9781510829008
State	Published - 2016
Event	33rd International Conference on Machine Learning, ICML 2016 - New York City, United States Duration: 19 Jun 2016 → 24 Jun 2016

Publication series

Name	33rd International Conference on Machine Learning, ICML 2016
Volume	1

Conference

Conference	33rd International Conference on Machine Learning, ICML 2016
Country/Territory	United States
City	New York City
Period	19/06/16 → 24/06/16

ASJC Scopus subject areas

Artificial Intelligence
Software
Computer Networks and Communications

Access to Document

https://arxiv.org/abs/1507.08788License: Other

Cite this

@inproceedings{822658b7188f4716b2a0390ea86e3cfe,

title = "Fast stochastic algorithms for SVD and PCA: Convergence properties and convexity",

abstract = "We study the convergence properties of the VR-PCA algorithm introduced by (Shamir, \#y2015) for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observations of independent interest, such as how pre-initializing with just a single exact power iteration can significantly improve the analysis, and what are the convexity and nonconvexity properties of the underlying optimization problem.",

author = "Ohad Shamir",

year = "2016",

language = "الإنجليزيّة",

series = "33rd International Conference on Machine Learning, ICML 2016",

pages = "392--419",

editor = "Balcan, \{Maria Florina\} and Weinberger, \{Kilian Q.\}",

booktitle = "33rd International Conference on Machine Learning, ICML 2016",

note = "33rd International Conference on Machine Learning, ICML 2016 ; Conference date: 19-06-2016 Through 24-06-2016",

}

Shamir, O 2016, Fast stochastic algorithms for SVD and PCA: Convergence properties and convexity. in MF Balcan & KQ Weinberger (eds), 33rd International Conference on Machine Learning, ICML 2016. 33rd International Conference on Machine Learning, ICML 2016, vol. 1, pp. 392-419, 33rd International Conference on Machine Learning, ICML 2016, New York City, United States, 19/06/16. <https://arxiv.org/abs/1507.08788>

Fast stochastic algorithms for SVD and PCA: Convergence properties and convexity. / Shamir, Ohad.
33rd International Conference on Machine Learning, ICML 2016. ed. / Maria Florina Balcan; Kilian Q. Weinberger. 2016. p. 392-419 (33rd International Conference on Machine Learning, ICML 2016; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Fast stochastic algorithms for SVD and PCA

T2 - 33rd International Conference on Machine Learning, ICML 2016

AU - Shamir, Ohad

PY - 2016

Y1 - 2016

N2 - We study the convergence properties of the VR-PCA algorithm introduced by (Shamir, #y2015) for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observations of independent interest, such as how pre-initializing with just a single exact power iteration can significantly improve the analysis, and what are the convexity and nonconvexity properties of the underlying optimization problem.

AB - We study the convergence properties of the VR-PCA algorithm introduced by (Shamir, #y2015) for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observations of independent interest, such as how pre-initializing with just a single exact power iteration can significantly improve the analysis, and what are the convexity and nonconvexity properties of the underlying optimization problem.

UR - https://www.scopus.com/pages/publications/84997751581

M3 - منشور من مؤتمر

T3 - 33rd International Conference on Machine Learning, ICML 2016

SP - 392

EP - 419

BT - 33rd International Conference on Machine Learning, ICML 2016

A2 - Balcan, Maria Florina

A2 - Weinberger, Kilian Q.

Y2 - 19 June 2016 through 24 June 2016

ER -

Fast stochastic algorithms for SVD and PCA: Convergence properties and convexity

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this