Overcoming Obstacles With a Reconfigurable Robot Using Deep Reinforcement Learning Based on a Mechanical Work-Energy Reward Function

Or Simhon, Zohar Karni, Sigal Berman, David Zarrouk

Research output: Contribution to journalArticlepeer-review

Abstract

This paper presents a Deep Reinforcement Learning (DRL) method based on a mechanical (work) Energy reward function applied to a reconfigurable RSTAR robot to overcome obstacles. The RSTAR is a crawling robot that can reconfigure its shape and shift the location of its center of mass via a sprawl and a four-bar extension mechanism. The DRL was applied in a simulated environment with a physical engine (UNITY TM). The robot was trained on a step obstacle and a two-stage narrow passage obstacle composed of a horizontal and a vertical channel. To evaluate the benefits of the proposed Energy reward function, it was compared to time-based and movement-based reward functions. The results showed that the Energy-based reward produced superior results in terms of obstacle height, energy requirements, and time to overcome the obstacle. The Energy-based reward method also converged faster to the solution compared to the other reward methods. The DRL's results for all the methods (energy, time and movement- based rewards) were superior to the best results produced by the human experts (see attached video).

Original languageAmerican English
Article number10121761
Pages (from-to)47681-47689
Number of pages9
JournalIEEE Access
Volume11
Issue number99
DOIs
StatePublished - 9 May 2023

Keywords

  • Obstacle negotiation
  • reconfigurable robot
  • reinforcement learning (RL)
  • reward shaping

All Science Journal Classification (ASJC) codes

  • General Engineering
  • General Computer Science
  • General Materials Science

Fingerprint

Dive into the research topics of 'Overcoming Obstacles With a Reconfigurable Robot Using Deep Reinforcement Learning Based on a Mechanical Work-Energy Reward Function'. Together they form a unique fingerprint.

Cite this