Reinforcement active learning hierarchical loops

Goren Gordon, Ehud Ahissar

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


A curious agent, be it a robot, animal or human, acts so as to learn as much as possible about itself and its environment. Such an agent can also learn without external supervision, but rather actively probe its surrounding and autonomously induce the relations between its action's effects on the environment and the resulting sensory input. We present a model of hierarchical motor-sensory loops for such an autonomous active learning agent, meaning a model that selects the appropriate action in order to optimize the agent's learning. Furthermore, learning one motor-sensory mapping enables the learning of other mappings, thus increasing the extent and diversity of knowledge and skills, usually in hierarchical manner. Each such loop attempts to optimally learn a specific correlation between the agent's available internal information, e.g. sensory signals and motor efference copies, by finding the action that optimizes that learning. We demonstrate this architecture on the well-studied vibrissae system, and show how sensory-motor loops are actively learnt from the bottom-up, starting with the forward and inverse models of whisker motion and then extending them to object localization. The model predicts transition from free-air whisking that optimally learns the self-generated motor-sensory mapping to touch-induced palpation that optimizes object localization, both observed in naturally behaving rats.

Original languageEnglish
Title of host publication2011 International Joint Conference on Neural Networks, IJCNN 2011 - Final Program
Number of pages8
ISBN (Electronic)9781424496372
StatePublished - Jul 2011
Event2011 International Joint Conference on Neural Network, IJCNN 2011 - San Jose, CA, United States
Duration: 31 Jul 20115 Aug 2011

Publication series

NameProceedings of the International Joint Conference on Neural Networks


Conference2011 International Joint Conference on Neural Network, IJCNN 2011
Country/TerritoryUnited States
CitySan Jose, CA

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence


Dive into the research topics of 'Reinforcement active learning hierarchical loops'. Together they form a unique fingerprint.

Cite this