Regressions for Estimating Main and Principal Causal Effects in Multi-Site Randomized Trials and Small Sample Designs

Abstract

Randomized controlled experiments have long been considered one of the best settings for evaluating causal impacts for a population of interest. Myriad experimental designs call for different statistical analysis methods. For example, randomized trials are often conducted with separate randomizations across multiple sites such as schools, voting districts, or hospitals. These sites can differ in important ways, including the site's implementation, local conditions, and the composition of individuals. An important question in practice is whether --- and under what assumptions --- researchers can leverage this cross-site variation to learn more about the intervention. Chapters 1 and 2 address these questions in the principal stratification framework, which describes causal effects for subgroups defined by post-treatment quantities. We show that researchers can estimate certain principal causal effects via site-level regressions within the multi-site design if they are willing to impose the strong assumption that the site-specific effects are uncorrelated with the site-specific distribution of stratum membership. Chapter 3 discusses regression methods to estimate causal effects in small sample experiments. Much of the statistical regression adjustment literature focuses on ways to improve the precision of treatment impact estimators that are asymptotically unbiased. However, an asymptotically unbiased estimator can be biased in a finite population. This is a well-known fact that typically is not addressed in practice. We address this issue by formally characterizing the finite population bias of ordinary least squares adjustment estimators in completely randomized experiments. Furthermore, we propose a bias-corrected estimator that, under a range of data generating mechanisms, has not only smaller finite population bias, but also smaller mean squared error than state-of-the-practice regression estimators. In particular, our bias correction is most useful when there is a heterogeneous treatment effect and when the squared covariates are highly correlated with the residual potential outcomes of each treatment group. In all chapters, we investigate the operating characteristics of our methodologies through simulations, and apply our methods to real datasets to demonstrate their meaningfulness in practical applications.