Exact Asymptotics of Linear Quadratic Adaptive Control and Population-level Comorbidity Analysis

Abstract

We present two self-contained topics in this thesis: exact asymptotics of linear quadratic adaptive control (LQAC) and population-level comorbidity analysis. LQAC is perhaps the simplest non-bandit reinforcement learning problem. Existing work on LQAC is focused almost exclusively on characterizing rates of regret and their ability to learn the underlying system dynamics, with little attention paid to the constants multiplying those rates that can be critically important in practice. By carefully combining recent finite-sample performance bounds for the LQAC problem with a particular (less-recent) martingale central limit theorem, we are able to derive asymptotically-exact expressions for the regret, estimation error, and prediction error of a rate-optimal stepwise-updating LQAC algorithm. In simulations on both stable and unstable systems, we find that our asymptotic theory also describes the algorithm’s finite-sample behavior remarkably well. In the same LQAC setting, we closes a log(T) rate gap between regret upper bound and lower bound by establishing a novel regret upper-bound of Op(√T). Population-level comorbidity analysis can be conducted with Electronic health records (EHRs), which have more data, lower cost, and less ethical concern comparing with clinical trials. With nationwide insurance claims data of 85.97 million enrollees across 8 years, our study quantifies the bi-directional causal effect: getting one of cancers and autoimmune diseases could increase the risk of getting the other. We identified a significantly increased risk of developing autoimmune diseases among patients receiving immunotherapy agents in all seven cancer types commonly treated with immunotherapy. On the reversal direction, we suggested that the underlying immune system dysregulation of rheumatoid arthritis (a common autoimmune disease), rather than its treatments, implicates the development of subsequent cancers. We applied the matching methods to balance the treatment and control group patients by sex, race, age, and inferred health and economic status. Our method is extensible to investigating the connections among drugs, diseases, and comorbidities at scale.