Hierarchical curiosity loops and active sensing

Goren Gordon, Ehud Ahissar

Research output: Contribution to journalArticlepeer-review


A curious agent acts so as to optimize its learning about itself and its environment, without external supervision. We present a model of hierarchical curiosity loops for such an autonomous active learning agent, whereby each loop selects the optimal action that maximizes the agent's learning of sensory-motor correlations. The model is based on rewarding the learner's prediction errors in an actor-critic reinforcement learning (RL) paradigm. Hierarchy is achieved by utilizing previously learned motor-sensory mapping, which enables the learning of other mappings, thus increasing the extent and diversity of knowledge and skills. We demonstrate the relevance of this architecture to active sensing using the well-studied vibrissae (whiskers) system, where rodents acquire sensory information by virtue of repeated whisker movements. We show that hierarchical curiosity loops starting from optimally learning the internal models of whisker motion and then extending to object localization result in free-air whisking and object palpation, respectively.

Original languageEnglish
Pages (from-to)119-129
Number of pages11
JournalNeural Networks
StatePublished - Aug 2012


  • Active sensing
  • Internal models
  • Intrinsic reward
  • Object localization
  • Reinforcement learning
  • Touch
  • Vibrissa
  • Whisker

All Science Journal Classification (ASJC) codes

  • Cognitive Neuroscience
  • Artificial Intelligence


Dive into the research topics of 'Hierarchical curiosity loops and active sensing'. Together they form a unique fingerprint.

Cite this