Curriculum Learning with Hindsight Experience Replay for Sequential Object Manipulation Tasks

Binyamin Manela, Armin Biess

Research output: Working paper/PreprintPreprint

30 Downloads (Pure)


Learning complex tasks from scratch is challenging and often impossible for humans as well as for artificial agents. A curriculum can be used instead, which decomposes a complex task (target task) into a sequence of source tasks (the curriculum). Each source task is a simplified version of the next source task with increasing complexity. Learning then occurs gradually by training on each source task while using knowledge from the curriculum's prior source tasks. In this study, we present a new algorithm that combines curriculum learning with Hindsight Experience Replay (HER), to learn sequential object manipulation tasks for multiple goals and sparse feedback. The algorithm exploits the recurrent structure inherent in many object manipulation tasks and implements the entire learning process in the original simulation without adjusting it to each source task. We have tested our algorithm on three challenging throwing tasks and show vast improvements compared to vanilla-HER.
Original languageEnglish GB
StatePublished - 2020

Publication series

NameArxiv preprint


  • Computer Science - Machine Learning
  • Computer Science - Artificial Intelligence
  • Computer Science - Neural and Evolutionary Computing
  • Computer Science - Robotics
  • Statistics - Machine Learning


Dive into the research topics of 'Curriculum Learning with Hindsight Experience Replay for Sequential Object Manipulation Tasks'. Together they form a unique fingerprint.

Cite this