Short-run Learning Dynamics under a Test-based Accountability System: Evidence from Pakistan

Abstract

Low student learning is a common finding in much of the developing world. This paper uses a relatively unique dataset of five semiannual rounds of standardized test data to characterize and explain the short-term changes in student learning. The data are collected as part of the quality assurance system for a public-private partnership program that offers public subsidies conditional on minimum learning levels to low-cost private schools in Pakistan. Apart from a large positive distributional shift in learning between the first two test rounds, the learning distributions over test rounds show little progress. Schools are ejected from the program if they fail to achieve a minimum pass rate in the test in two consecutive attempts, making the test high stakes. Sharp regression discontinuity estimates show that the threat of program exit on schools that barely failed the test for the first time induces large learning gains. The large change in learning between the first two test rounds is likely attributable to this accountability pressure given that a large share of new program entrants failed in the first test round. Schools also qualify for substantial annual teacher bonuses if they achieve a minimum score in a composite measure of student test participation and mean test score. Sharp regression discontinuity estimates do not show that the prospect of future teacher bonus rewards induces learning gains for schools that barely did not qualify for the bonus.