Bayesian relational memory for semantic visual navigation

Yi Wu; Yuxin Wu; Aviv Tamar; Stuart Russell; Georgia Gkioxari; Yuandong Tian

doi:10.1109/ICCV.2019.00286

Bayesian relational memory for semantic visual navigation

Yi Wu, Yuxin Wu, Aviv Tamar, Stuart Russell, Georgia Gkioxari, Yuandong Tian

Technion - Israel Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure.

Original language	English
Title of host publication	Proceedings - 2019 International Conference on Computer Vision, ICCV 2019
Pages	2769-2779
Number of pages	11
ISBN (Electronic)	9781728148038
DOIs	https://doi.org/10.1109/ICCV.2019.00286
State	Published - Oct 2019
Event	17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 - Seoul, Korea, Republic of Duration: 27 Oct 2019 → 2 Nov 2019

Publication series

Name	Proceedings of the IEEE International Conference on Computer Vision
Volume	2019-October

Conference

Conference	17th IEEE/CVF International Conference on Computer Vision, ICCV 2019
Country/Territory	Korea, Republic of
City	Seoul
Period	27/10/19 → 2/11/19

All Science Journal Classification (ASJC) codes

Software
Computer Vision and Pattern Recognition

Access to Document

10.1109/ICCV.2019.00286

Cite this

Wu, Y., Wu, Y., Tamar, A., Russell, S., Gkioxari, G., & Tian, Y. (2019). Bayesian relational memory for semantic visual navigation. In Proceedings - 2019 International Conference on Computer Vision, ICCV 2019 (pp. 2769-2779). Article 9009539 (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2019-October). https://doi.org/10.1109/ICCV.2019.00286

@inproceedings{720cc853eade472b9c7c4c8fef343e9e,

title = "Bayesian relational memory for semantic visual navigation",

abstract = "We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure.",

author = "Yi Wu and Yuxin Wu and Aviv Tamar and Stuart Russell and Georgia Gkioxari and Yuandong Tian",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 ; Conference date: 27-10-2019 Through 02-11-2019",

year = "2019",

month = oct,

doi = "10.1109/ICCV.2019.00286",

language = "الإنجليزيّة",

series = "Proceedings of the IEEE International Conference on Computer Vision",

pages = "2769--2779",

booktitle = "Proceedings - 2019 International Conference on Computer Vision, ICCV 2019",

}

Wu, Y, Wu, Y, Tamar, A, Russell, S, Gkioxari, G & Tian, Y 2019, Bayesian relational memory for semantic visual navigation. in Proceedings - 2019 International Conference on Computer Vision, ICCV 2019., 9009539, Proceedings of the IEEE International Conference on Computer Vision, vol. 2019-October, pp. 2769-2779, 17th IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea, Republic of, 27/10/19. https://doi.org/10.1109/ICCV.2019.00286

Bayesian relational memory for semantic visual navigation. / Wu, Yi; Wu, Yuxin; Tamar, Aviv et al.
Proceedings - 2019 International Conference on Computer Vision, ICCV 2019. 2019. p. 2769-2779 9009539 (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2019-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Bayesian relational memory for semantic visual navigation

AU - Wu, Yi

AU - Wu, Yuxin

AU - Tamar, Aviv

AU - Russell, Stuart

AU - Gkioxari, Georgia

AU - Tian, Yuandong

PY - 2019/10

Y1 - 2019/10

N2 - We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure.

AB - We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure.

UR - http://www.scopus.com/inward/record.url?scp=85081934243&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2019.00286

DO - 10.1109/ICCV.2019.00286

M3 - منشور من مؤتمر

T3 - Proceedings of the IEEE International Conference on Computer Vision

SP - 2769

EP - 2779

BT - Proceedings - 2019 International Conference on Computer Vision, ICCV 2019

T2 - 17th IEEE/CVF International Conference on Computer Vision, ICCV 2019

Y2 - 27 October 2019 through 2 November 2019

ER -

Bayesian relational memory for semantic visual navigation

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this