A System for Applying Deep Reinforcement Learning to Soft Robotic Control

Abstract

Soft robots offer a host of benefits over traditional rigid robots, including inherent compliance that lets them passively adapt to variable environments and operate safely around humans and fragile objects. However, compliance means there is a high degree of freedom (DOF), which in turn makes it hard to use model-based methods in planning tasks requiring high precision or complex actuation sequences. Reinforcement learning (RL) can potentially find effective control policies, but training RL using physical soft robots is often infeasible, and training using simulations has had a high barrier to adoption due to the challenge of configuring the required systems and the high computational cost required to simulate high-DOF systems.

To accelerate research in control and RL for soft robotic systems, we introduce SoMoGym Soft Motion Gym (SoMoGym) and Soft Motion RL (SoMo-RL), software toolkits that facilitate the training and evaluation of controllers for continuum robots. SoMoGym provides a series of benchmark tasks involving soft robots interacting with various objects and environments. SoMo-RL implements associated RL functionality, enabling the use of RL to generate new controllers and evaluate their performance. Custom environments, hardware platforms, and RL configurations can be added easily.

We demonstrate the utility of SoMoGym and SoMo-RL by providing baseline RL policies for each of the benchmark tasks and conducting a series of experiments that develop a set of interesting manipulation behaviors. Overall, this system enables the use of RL for continuum robots, a class of robots not covered by existing benchmarks, giving these robots the capability to autonomously solve tasks that were previously unattainable.