MODEL-BASED META AUTOMATIC CURRICULUM LEARNING

Zifan Xu, Yulin Zhang, Shahaf S. Shperberg, Reuth Mirsky, Yuqian Jiang, Bo Liu, Peter Stone

Research output: Contribution to journalConference articlepeer-review

Abstract

Curriculum learning (CL) has been widely explored to facilitate the learning of hard-exploration tasks in reinforcement learning (RL) by training a sequence of easier tasks, often called a curriculum. While most curricula are built either manually or automatically based on heuristics, e.g. choosing a training task which is barely beyond the current abilities of the learner, the fact that similar tasks might benefit from similar curricula motivates us to explore meta-learning as a technique for curriculum generation or teaching for a distribution of similar tasks. This paper formulates the meta CL problem that requires a meta-teacher to generate the curriculum which will assist the student to train toward any given target task from a task distribution based on the similarity of these tasks to one another. We propose a model-based meta automatic curriculum learning algorithm (MM-ACL) that learns to predict the performance improvement on one task when the student is trained on another, given the current status of the student. This predictor can then be used to generate the curricula for different target tasks. Our empirical results demonstrate that MM-ACL outperforms the state-of-the-art CL algorithms in a grid-world domain and a more complex visual-based navigation domain in terms of sample efficiency.

Original languageEnglish
Pages (from-to)846-860
Number of pages15
JournalProceedings of Machine Learning Research
Volume232
StatePublished - 1 Jan 2023
Event2nd Conference on Lifelong Learning Agents, CoLLA 2023 - Montreal, Canada
Duration: 22 Aug 202325 Aug 2023

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability

Fingerprint

Dive into the research topics of 'MODEL-BASED META AUTOMATIC CURRICULUM LEARNING'. Together they form a unique fingerprint.

Cite this