RNN fisher vectors for action recognition and image annotation

Guy Lev; Gil Sadeh; Benjamin Klein; Lior Wolf

doi:https://doi.org/10.1007/978-3-319-46466-4_50

RNN fisher vectors for action recognition and image annotation

Guy Lev, Gil Sadeh, Benjamin Klein, Lior Wolf

Tel Aviv University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Recurrent Neural Networks (RNNs) have had considerable success in classifying and predicting sequences. We demonstrate that RNNs can be effectively used in order to encode sequences and provide effective representations. The methodology we use is based on Fisher Vectors, where the RNNs are the generative probabilistic models and the partial derivatives are computed using backpropagation. State of the art results are obtained in two central but distant tasks, which both rely on sequences: video action recognition and image annotation. We also show a surprising transfer learning result from the task of image annotation to the task of video action recognition.

Original language	English
Title of host publication	Computer Vision - 14th European Conference, ECCV 2016, Proceedings
Editors	Bastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Pages	833-850
Number of pages	18
DOIs	https://doi.org/10.1007/978-3-319-46466-4_50
State	Published - 2016
Event	14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands Duration: 8 Oct 2016 → 16 Oct 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9910 LNCS

Conference

Conference	14th European Conference on Computer Vision, ECCV 2016
Country/Territory	Netherlands
City	Amsterdam
Period	8/10/16 → 16/10/16

Keywords

Action recognition
Fisher vectors
Image annotation
Recurrent neural networks

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
General Computer Science

Access to Document

https://doi.org/10.1007/978-3-319-46466-4_50

Cite this

Lev, G., Sadeh, G., Klein, B., & Wolf, L. (2016). RNN fisher vectors for action recognition and image annotation. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Computer Vision - 14th European Conference, ECCV 2016, Proceedings (pp. 833-850). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9910 LNCS). https://doi.org/10.1007/978-3-319-46466-4_50

Lev, Guy ; Sadeh, Gil ; Klein, Benjamin et al. / RNN fisher vectors for action recognition and image annotation. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. editor / Bastian Leibe ; Jiri Matas ; Nicu Sebe ; Max Welling. 2016. pp. 833-850 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{7c41280122a14282a015d216cbc67ef9,

title = "RNN fisher vectors for action recognition and image annotation",

abstract = "Recurrent Neural Networks (RNNs) have had considerable success in classifying and predicting sequences. We demonstrate that RNNs can be effectively used in order to encode sequences and provide effective representations. The methodology we use is based on Fisher Vectors, where the RNNs are the generative probabilistic models and the partial derivatives are computed using backpropagation. State of the art results are obtained in two central but distant tasks, which both rely on sequences: video action recognition and image annotation. We also show a surprising transfer learning result from the task of image annotation to the task of video action recognition.",

keywords = "Action recognition, Fisher vectors, Image annotation, Recurrent neural networks",

author = "Guy Lev and Gil Sadeh and Benjamin Klein and Lior Wolf",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2016.; 14th European Conference on Computer Vision, ECCV 2016 ; Conference date: 08-10-2016 Through 16-10-2016",

year = "2016",

doi = "https://doi.org/10.1007/978-3-319-46466-4_50",

language = "الإنجليزيّة",

isbn = "9783319464657",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "833--850",

editor = "Bastian Leibe and Jiri Matas and Nicu Sebe and Max Welling",

booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",

}

Lev, G, Sadeh, G, Klein, B & Wolf, L 2016, RNN fisher vectors for action recognition and image annotation. in B Leibe, J Matas, N Sebe & M Welling (eds), Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9910 LNCS, pp. 833-850, 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, Netherlands, 8/10/16. https://doi.org/10.1007/978-3-319-46466-4_50

RNN fisher vectors for action recognition and image annotation. / Lev, Guy; Sadeh, Gil; Klein, Benjamin et al.
Computer Vision - 14th European Conference, ECCV 2016, Proceedings. ed. / Bastian Leibe; Jiri Matas; Nicu Sebe; Max Welling. 2016. p. 833-850 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9910 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - RNN fisher vectors for action recognition and image annotation

AU - Lev, Guy

AU - Sadeh, Gil

AU - Klein, Benjamin

AU - Wolf, Lior

N1 - Publisher Copyright: © Springer International Publishing AG 2016.

PY - 2016

Y1 - 2016

N2 - Recurrent Neural Networks (RNNs) have had considerable success in classifying and predicting sequences. We demonstrate that RNNs can be effectively used in order to encode sequences and provide effective representations. The methodology we use is based on Fisher Vectors, where the RNNs are the generative probabilistic models and the partial derivatives are computed using backpropagation. State of the art results are obtained in two central but distant tasks, which both rely on sequences: video action recognition and image annotation. We also show a surprising transfer learning result from the task of image annotation to the task of video action recognition.

AB - Recurrent Neural Networks (RNNs) have had considerable success in classifying and predicting sequences. We demonstrate that RNNs can be effectively used in order to encode sequences and provide effective representations. The methodology we use is based on Fisher Vectors, where the RNNs are the generative probabilistic models and the partial derivatives are computed using backpropagation. State of the art results are obtained in two central but distant tasks, which both rely on sequences: video action recognition and image annotation. We also show a surprising transfer learning result from the task of image annotation to the task of video action recognition.

KW - Action recognition

KW - Fisher vectors

KW - Image annotation

KW - Recurrent neural networks

UR - http://www.scopus.com/inward/record.url?scp=84990047800&partnerID=8YFLogxK

U2 - https://doi.org/10.1007/978-3-319-46466-4_50

DO - https://doi.org/10.1007/978-3-319-46466-4_50

M3 - منشور من مؤتمر

SN - 9783319464657

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 833

EP - 850

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

A2 - Leibe, Bastian

A2 - Matas, Jiri

A2 - Sebe, Nicu

A2 - Welling, Max

T2 - 14th European Conference on Computer Vision, ECCV 2016

Y2 - 8 October 2016 through 16 October 2016

ER -

Lev G, Sadeh G, Klein B, Wolf L. RNN fisher vectors for action recognition and image annotation. In Leibe B, Matas J, Sebe N, Welling M, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings. 2016. p. 833-850. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: https://doi.org/10.1007/978-3-319-46466-4_50

RNN fisher vectors for action recognition and image annotation

Abstract

Publication series

Conference

Keywords

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this