Overcoming Obstacles with a Reconfigurable Robot Using Reinforcement Learning

The reconfigurable robot RSTAR (Rising Sprawl Tuned Autonomous Robot) is a newly developed crawling robot that can reconfigure its shape and shift the location of its center of mass. These features endow the RSTAR with inherent robustness and enhanced ability to overcome obstacles and crawl on a variety of terrains for a vast range of applications. However, defining the trajectories to fully exploit the robot's capabilities is challenging, especially when complex maneuvers are required. Here, we show how reinforcement learning can be used to determine the optimal strategies to overcome three typical obstacles: squeezing through two adjacent obstacles, ducking underneath an obstacle and climbing over an obstacle. We detail the implementation of the Q learning algorithm in a simulation environment with a physical engine (UNITY™) to learn a feasible path in a minimum number of steps. The results show that the algorithm successfully charted a feasible trajectory in all cases. Comparing the trajectory found by the algorithm to trajectories devised by 12 human experts with discrete or continuous control showed that the algorithm trajectories were shorter than the expert trajectories. Experiments showing how the physical RSTAR robot can overcome different obstacles using the trajectories found in the simulation by the Q Learning algorithm are described and presented in the attached video.