New Directions for Causal Inference With Complex Data in Health Care, Social Science, and Beyond

Abstract

With the rise of the online marketplace and the digitization of data from administrative databases and electronic health records, we now have access to more comprehensive and more complex data than ever before. And while the digital revolution has created a new frontier for empirical research that promises exciting insights about human health and behavior, it has also given rise to a number of theoretical and computational complexities that call for statistical innovation. This dissertation focuses on research directions that apply classical statistical frameworks to large, messy modern datasets with a focus on settings where inferences are complicated due to the presence of unstructured text or electronic health data. Both types of data have the potential to help advance scientific understanding - for example, about the impacts of a non-randomized intervention on the health or sentiments of a population - but these data are commonly overlooked because of the theoretical or computational obstacles they impose. In particular, text is inherently high-dimensional and difficult to model, and electronic health data are often inconsistent or incomplete due to large measurement errors and non ignorable missing values. In the chapters that follow, we first provide some background on the potential outcomes framework, describe how the literature has evolved over time, and characterize a new frontier for methodological research in causal inference with complex, modern data. We then present new statistical methods to address challenges that arise when making causal inferences with text data and in longitudinal observational studies. Throughout, we illustrate the proposed methods with case studies examining real data in health care and social science.