Learning, Optimization, and Control for Real-World Physical Systems

Abstract

Recent breakthroughs in machine learning and artificial intelligence have led to key advancements in many areas such as natural language understanding, game playing, and simulated locomotion. Two important algorithmic frameworks underpinning this success are reinforcement learning (RL) and data-driven, gradient-based optimization. Leveraging these RL and optimization frameworks for real-world physical systems, however, remains challenging: real systems yield limited data, rarely offer accurate analytical models, suffer from sim-to-real discrepancies, and often operate as large, interconnected networks. In this thesis, I seek to remedy these challenges in the following ways. First, I present a theoretically principled representation-based RL framework for stochastic nonlinear control which utilizes dynamics knowledge and structure to improve sample efficiency in RL. I will then discuss how this approach can be extended to the important challenge of scalable control in networked dynamical systems. Next, I will introduce efficient Bayesian and zeroth-order toolkits designed for sample-constrained black-box optimization, a pervasive challenge in many real-world engineering systems. Together, these contributions chart a path toward reliable, data-efficient learning-based optimization and control of complex real-world physical systems.