Frequentist Properties of Adaptive Bayesian Uncertainty Quantification

Abstract

Adaptive experiments provide increased flexibility and accuracy when constructing efficient estimators. Examples of such adaptive sampling frameworks include reinforcement learning, multi-armed bandit algorithms, Bayesian optimization procedures, and other dynamic systems. However, the inherent sequential dependency within these adaptive systems makes statistical inference more challenging. In general, adaptive sampling may break many of the useful properties of estimation procedures that hold in i.i.d. sampling contexts, such as consistency and asymptotic normality of the MLE. In this project, we present a version of the Bernstein von Mises theorem which analyzes the frequentist behavior of the Bayesian posterior distribution. Our result is that under mild assumptions, the posterior distribution is asymptotically normal with mean equal to the MLE and variance equal to the inverse Fisher Information in the adaptive setting. Thus, many positive results shown in adaptive frequentist inference imply that adaptive Bayesian uncertainty quantification is asymptotically valid and statistically efficient from a frequentist perspective. Furthermore, we show that if the number of samples from each treatment goes to infinity, the posterior distribution is asymptotically not dependent on the prior; then, from a Bayesian perspective, a credible interval will be asymptotically valid even if the prior is misspecified, provided the prior's support contains the support of the true parameter distribution. Our result also gives a counterexample that even in an adaptive setting where the prior is supported on an entire finite-dimensional parameter space and the number of samples from each treatment goes to infinity in a way where Bayesian inference asymptotically does not depend on the prior, the resulting Bayesian uncertainty quantification need not be asymptotically frequentist-valid.