Reinforcement Learning Design: Modifying Stochastic Environments to Improve the Performance of Reinforcement Learning Agents

Abstract

In this thesis, I present the Reinforcement Learning Design (RLD) problem: the question of how to design training environments for reinforcement learning agents. Specifically, if given an evaluation environment, a set of allowable modifications, and a budget constraint on the number of modifications to apply, my goal is to suggest a training environment such that an agent that trains in the training environment will perform better in the evaluation environment than will an agent that trains in the evaluation environment. RLD has applications to areas where a designer is willing to make temporary modifications to an evaluation environment in order to help an autonomous agent learn a good policy. For example, the user of an automated insulin pump for diabetes management might be willing to modify their diet during the first week of use in order to help the pump learn a dosing regimen that will work well once the diet returns to normal. In this work, I propose two methods for solving the RLD problem and show their applicability through empirical evaluation.