Optimal control and reinforcement learning in simple physical systems

Abstract

Motor control is a ubiquitous theme in the natural world and its richness demands we approach it from several directions. Alongside the challenge of understanding perception and coordinated muscular activation we need to account for the fact that all motion takes place in a body embedded in a dynamic environment. Inspired by extreme feats of athletic and animal performance we bring together ideas from physics and optimization to approach problems in motor control. First we study the Estonian sport Kiiking and show how to augment basic physical models to account for physiological limitations. We use methods from both optimal control theory and reinforcement learning to understand optimal strategies in the sport and compare our findings to observations. Next we describe an approach for the control of gliding flight by dynamically changing the shape of the gliding body. We compare strategies that are time optimal to those which require the least control input. Finally we use elasticity theory to understand the mechanics of snakes as they traverse large vertical gaps. After describing our data analysis procedure we detail the optimal control problem we formulated to understand the posture of the animals as they complete their ascent.