Essays on measurement and causal inference in education

Abstract

This dissertation consists of three essays at the intersection of psychometrics and causal inference in education. The first essay investigates the properties and quality of measures of learning outcomes in education randomized controlled trials (RCTs) conducted in low- and middle-income countries (LMICs). I review test properties across 158 studies and conduct item-level psychometric analysis of a subset of these studies to show that current tests vary widely in scope, content coverage, administration conditions, and score aggregation and analysis. These results suggest that comparisons of treatment effects must take into account degrees of measurement error and the content breadth of the tests to contextualize why effects may differ on substantively different outcome variables. The second essay presents a formal meta-analysis that systematizes the evidence base on education interventions conducted in LMICs since 2009. In order to use the meta-analytical technique, I assume that the latent outcomes I compare have the same amount of measurement error. I find that overall education interventions have a positive and significant effect of 0.10 standard deviation units. The third essay focuses on estimating heterogeneous treatment effects but moves beyond traditional subgroup analysis. I estimate individual conditional average treatment effects (ICATEs) and explore them against students’ characteristics to illustrate the proportion of students who benefit or lose from the intervention and by how much. This chapter highlights the importance of the ICATEs for taking a more informed and action-oriented approach to scalability and generalizability of the results.