Wheel Loader Scooping Controller Using Deep Reinforcement Learning

Osher Azulay, Amir Shapiro

Research output: Contribution to journalArticlepeer-review

26 Scopus citations


This article presents a deep reinforcement learning-based controller for an unmanned ground vehicle with a custom-built scooping mechanism. The robot's aim is to autonomously perform earth scooping cycles with three degrees of freedom: lift, tilt and the robot's velocity. While the majority of previous studies on automated scooping processes are based on data recorded by expert operators, we present a method to autonomously control a wheel loader to perform the scooping cycle using deep reinforcement learning methods without any user-provided demonstrations. The controller's learning approach is based on the actor-critic, Deep Deterministic Policy Gradient algorithm which we use to map online sensor data as input to continuously update the actuator commands. The training of the scooping policy network is done solely in a simplified simulation environment using a virtual physics engine, which converges to an average of a 65% fill factor from the full bucket capacity and a 5 [ sec] average cycle time. We illustrate the performance of the trained policy in simulations and in real-world experiments with 3 different inclination angles of the earth. An additional scooping experiment compared the performance of our controller to remote manual human control. Overall, the deep reinforcement learning-based controller exhibited good performance in terms of both achieved visually bucket fill with varying scooped earth weights of 4.1-7.2[kg], and a 5.1-7.1[sec] cycle time. The experimental results confirm the ability of our planner to fill bucket as required, indicating that our controller can be used for excavation purposes.

Original languageEnglish
Article number9344588
Pages (from-to)24145-24154
Number of pages10
JournalIEEE Access
StatePublished - 1 Jan 2021


  • Robotics in construction
  • agricultural automation
  • machine learning

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering


Dive into the research topics of 'Wheel Loader Scooping Controller Using Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this