An Introduction to Reinforcement Learning

Abstract

This thesis presents a new course textbook on reinforcement learning (RL) with a focus on algorithms and their properties. The textbook is suitable for a one-semester introductory undergraduate course on RL for students with prior experience in basic probability, linear algebra, and multivariable calculus. We systematically cover the fundamentals of Markov decision processes, optimal control, multi-armed bandits, fitted dynamic programming algorithms, policy gradient methods, imitation learning, and tree search-based planning methods. Our contribution to the RL literature is an approachable and concise presentation of core RL algorithms that balances practical considerations with theoretical rigour. Each chapter includes extensive bibliographic notes that survey recent advances. We hope this textbook will equip the reader with a workable mental model for navigating modern RL research.