Overcoming Obstacles with a Reconfigurable Robot Using Reinforcement Learning

Liran Yehezkel, Sigal Berman, David Zarrouk

Research output: Contribution to journalArticlepeer-review

8 Scopus citations


The reconfigurable robot RSTAR (Rising Sprawl Tuned Autonomous Robot) is a newly developed crawling robot that can reconfigure its shape and shift the location of its center of mass. These features endow the RSTAR with inherent robustness and enhanced ability to overcome obstacles and crawl on a variety of terrains for a vast range of applications. However, defining the trajectories to fully exploit the robot's capabilities is challenging, especially when complex maneuvers are required. Here, we show how reinforcement learning can be used to determine the optimal strategies to overcome three typical obstacles: squeezing through two adjacent obstacles, ducking underneath an obstacle and climbing over an obstacle. We detail the implementation of the Q learning algorithm in a simulation environment with a physical engine (UNITY™) to learn a feasible path in a minimum number of steps. The results show that the algorithm successfully charted a feasible trajectory in all cases. Comparing the trajectory found by the algorithm to trajectories devised by 12 human experts with discrete or continuous control showed that the algorithm trajectories were shorter than the expert trajectories. Experiments showing how the physical RSTAR robot can overcome different obstacles using the trajectories found in the simulation by the Q Learning algorithm are described and presented in the attached video.

Original languageEnglish
Article number9272763
Pages (from-to)217541-217553
Number of pages13
JournalIEEE Access
StatePublished - 1 Jan 2020


  • Q learning
  • Reinforcement learning
  • crawling robot
  • reconfigurable robot
  • sprawl tuning

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering


Dive into the research topics of 'Overcoming Obstacles with a Reconfigurable Robot Using Reinforcement Learning'. Together they form a unique fingerprint.

Cite this