Gradual training method for denoising auto encoders

Alexander Kalmanovich; Gal Chechik

Gradual training method for denoising auto encoders

Alexander Kalmanovich, Gal Chechik

The Leslie and Susan Gonda Multidisciplinary Brain Research Center

Bar-Ilan University

Research output: Contribution to conference › Paper › peer-review

Abstract

Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network. We investigate a training scheme of a deep DAE, where DAE layers are gradually added and keep adapting as additional layers are added. We show that in the regime of mid-sized datasets, this gradual training provides a small but consistent improvement over stacked training in both reconstruction quality and classification error over stacked training on MNIST and CIFAR datasets.

Original language	English
State	Published - 2015
Event	3rd International Conference on Learning Representations, ICLR 2015 - San Diego, United States Duration: 7 May 2015 → 9 May 2015

Conference

Conference	3rd International Conference on Learning Representations, ICLR 2015
Country/Territory	United States
City	San Diego
Period	7/05/15 → 9/05/15

All Science Journal Classification (ASJC) codes

Education
Linguistics and Language
Language and Linguistics
Computer Science Applications

Cite this

@conference{7b9bc6464404435a8af06047c601c4fd,

title = "Gradual training method for denoising auto encoders",

abstract = "Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network. We investigate a training scheme of a deep DAE, where DAE layers are gradually added and keep adapting as additional layers are added. We show that in the regime of mid-sized datasets, this gradual training provides a small but consistent improvement over stacked training in both reconstruction quality and classification error over stacked training on MNIST and CIFAR datasets.",

author = "Alexander Kalmanovich and Gal Chechik",

note = "Publisher Copyright: {\textcopyright} 2015 International Conference on Learning Representations, ICLR. All rights reserved.; 3rd International Conference on Learning Representations, ICLR 2015 ; Conference date: 07-05-2015 Through 09-05-2015",

year = "2015",

language = "الإنجليزيّة",

}

TY - CONF

T1 - Gradual training method for denoising auto encoders

AU - Kalmanovich, Alexander

AU - Chechik, Gal

PY - 2015

Y1 - 2015

N2 - Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network. We investigate a training scheme of a deep DAE, where DAE layers are gradually added and keep adapting as additional layers are added. We show that in the regime of mid-sized datasets, this gradual training provides a small but consistent improvement over stacked training in both reconstruction quality and classification error over stacked training on MNIST and CIFAR datasets.

AB - Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network. We investigate a training scheme of a deep DAE, where DAE layers are gradually added and keep adapting as additional layers are added. We show that in the regime of mid-sized datasets, this gradual training provides a small but consistent improvement over stacked training in both reconstruction quality and classification error over stacked training on MNIST and CIFAR datasets.

UR - http://www.scopus.com/inward/record.url?scp=85083954044&partnerID=8YFLogxK

M3 - محاضرة

T2 - 3rd International Conference on Learning Representations, ICLR 2015

Y2 - 7 May 2015 through 9 May 2015

ER -

Gradual training method for denoising auto encoders

Abstract

Conference

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this