Generalizability Methods for Estimating Causal Population Effects

Abstract

Studies are often performed in samples that do not resemble the target populations relevant for policy, treatment, or other decisions. Much of the causal inference literature has focused on addressing internal validity bias; however, both internal and external validity are necessary for unbiased estimates in a target population. The generalizability methods presented in this thesis allow for inference on the population of interest rather than the one in the study.

Chapter 1 presents a framework for addressing external validity bias, including a synthesis of approaches for generalizability and transportability, the assumptions they require, as well as tests for the heterogeneity of treatment effects and differences between study and target populations. The chapter concludes with practical guidance for researchers and practitioners.

Chapter 2 presents an innovative class of estimators, conditional cross-design synthesis (CCDS), for combining randomized and observational data to eliminate their respective external and internal validity biases. CCDS uses the region of covariate overlap between data types to remove potential unmeasured confounding bias in the observational data in order to extend inference beyond the support of the randomized data to the full target population. We derive outcome regression, propensity weighting, and double robust approaches under the CCDS framework. We illustrate the methods to estimate the causal effect of health insurance plans on cost among New York City Medicaid enrollees.

Chapter 3 introduces novel approaches for generalizing from an evaluation study of a voluntary intervention to estimate population average treatment effects for future treated individuals, which can accommodate nonparametric outcome regression approaches such as Bayesian Additive Regression Trees and Bayesian Causal Forests. The generalizability approach incorporates uncertainty regarding target population treated group membership into the posterior credible intervals to better-reflect the uncertainty of scaling up a voluntary intervention. In a simulation based on real data, we estimate impacts of a national scale-up of a voluntary health policy model and highlight the benefit of using flexible regression approaches for generalizability.