Statistical Explorations of Testing and Testing Culture

Abstract

This dissertation is composed of three studies. The first and second examine the U.S. private tutoring industry which heretofore has gone largely undocumented by academic research. I show that not only has the industry grown precipitously within the last few decades, but that a disproportionate amount of that growth has taken place among areas with higher income and greater educational attainment. This intensifying concentration of educational resources within traditionally privileged communities points out private tutoring prevalence as a measure, and a mechanism, for deepening social inequality. I also find evidence for a cultural dimension to private tutoring demand. Furthermore, the private tutoring industry in the United States and its relationship to school markets is unique. I estimate charter school openings have a positive causal effect on private tutoring prevalence, despite research from other national contexts suggesting a negative association between school choice and private tutoring.

The third study of this dissertation offers an innovation on traditional teacher value-added models. Value-added models use student performance metrics to estimate teacher ability. The Signal Weighted Value-Added Model explicitly models the scenario that students differ in their capacity as signals of teacher ability. Specifically, some students may be more sensitive to teaching ability, and some students may be more reliable indicators of teaching ability. Theoretically, the model resembles an Item Response Theory model, wherein teachers are examinees and students are exam items. Simulation data suggests the signal weighted model can substantially improve the recovery of true teacher ability if underlying assumptions are met, but still has comparable performance otherwise. The model also demonstrates better fit on an empirical data set provided by the NCTE.

Through a descriptive study, a causal study, and a simulation study, this dissertation presents a broad inquiry into the topic of testing and testing culture. Its findings encourage greater scrutiny into the methods and data education policy research employs, and often takes for granted.