Real-time Decision Making in Control and Optimization with Performance and Safety Guarantees

Abstract

With the rapid evolution of sensing, communication, computation, and actuation technology, recent years have witnessed a growing interest in real-time decision making, which leverages the data received in real time to improve the decision-making performance. Real-time decision making enjoys great potentials in two major domains: (i) in applications with time-varying environments, harnessing real-time data can allow quick adaptation to the new environments, (ii) when the underlying models are unknown, one can learn the models and improve the decisions by real-time data from the interaction with the systems. However, when applying real-time decision making to these domains, there are several major challenges, three of which are discussed below. Firstly, the real-time data available are usually limited and imperfect, which calls for more understanding on the effective usage and fundamental values of the real-time data. Secondly, the implementability on large-scale systems calls for computation-efficient and communication-efficient algorithm design. Thirdly, safety is crucial for the applications in the real world.

This thesis aims to tackle these challenges by studying real-time decision making in control and optimization with time-varying environments and/or unknown models. Specifically, the thesis consists of three parts.

In Part I, we consider online convex optimization and online optimal control with time-varying cost functions. The future cost functions are unknown but some predictions on the future are available. We design gradient-based online algorithms to leverage predictions. Further, we consider different models of prediction errors for generality, and study our algorithms' regret guarantees in these models. We also provide fundamental lower bounds, which provide more insights into the fundamental values of the prediction information.

In Part II, we consider optimal control with constraints on the states and actions. We consider two problems, one with time-varying cost functions but known systems, and the other with unknown systems in a time-invariant setting. We design online algorithms for both settings. We provide safety guarantees of our online algorithms, i.e., constraint satisfaction and feasibility. We also provide sublinear regret guarantees for both algorithms, which indicate that our algorithms achieve desirable performance while ensuring safety requirements.

In Part III, we consider a decentralized linear quadratic control problem with unknown systems. We design a distributed policy gradient algorithm that only requires limited communication capacities. We provide a sample complexity bound for reaching a stationary point of the distributed policy optimization. We also provide stability guarantees for the controllers generated along the way.