Making Decisions Under High Stakes: Trustworthy and Expressive Bayesian Deep Learning

Abstract

Machine learning models applied to high-stakes domains must navigate the tradeoff between (i) having enough representation power to learn effectively from high-dimensional, high-volume datasets, while (ii) avoiding cost-prohibitive errors due to model overconfidence and poor generalization. This need has led to growing interest in Bayesian neural networks (BNN), models that perform Bayesian inference on deep neural networks. BNNs are able to quantify predictive uncertainty over an inherently rich hypothesis space, which is crucial for high-stakes decision-making. In this thesis, we present two contributions that tackle the shortcomings of BNNs. First, BNN priors are defined in uninterpretable parameter space, which makes it difficult for end users to express functional prior knowledge independent of training data. We formulate two novel priors that incorporate functional constraints (i.e. what values the output should hold for any given input) that can easily be specified by end users. The resulting model is amenable to black-box inference. We demonstrate its efficacy on two high-stakes domains: (i) enforcing physiologically feasible interventions on a clinical action prediction task, and (ii) enforcing racial fairness constraints on a recidivism prediction task where the training data is biased. Next, variational approximations that are typically used for BNN posterior inference do not come with provable error guarantees, making it difficult to trust their predictive estimates. By exploiting the functional form of BNNs, we bound the predictive mean error of such approximations via maximum mean discrepancy on a reproducing kernel Hilbert space. Our bound is easily estimable and directly useful to end users as it is specified in predictive space.