Three Essays on Improving Measurement in the Study of Early Childhood Policy

Abstract

As early childhood has grown as a subject of academic study and policy intervention internationally, there is increasing demand for measures of early childhood development and the quality of early childhood care and education settings that function well across diverse settings. This dissertation presents three essays on improving measurement of these important constructs and defines improvement as better understanding the psychometric properties of assessments, designing strategies that enhance reliability and validity, and addressing logistical and practical constraints alongside psychometric ones. The first essay introduces a human-centered framework for evaluating a longer assessment of early childhood development (ECD) and creating a shorter version that maintains good measurement properties. This framework considers statistical, conceptual, and practical facets of items to maximize reliability, reduce complexity, and the conceptual breadth of the construct being studied. The framework is then applied to the International Development and Early Learning Assessment (IDELA), an assessment of ECD for three- to six-year-old children used in over 90 countries worldwide, to propose a balanced short form. The second essay compares direct and caregiver-reported measures of ECD to evaluate the convention that direct assessment is a superior form of measurement. Using two datasets from the creation of the caregiver-reported ECDI2030 tool used to assess ECD with millions of two- to four-year-old children worldwide, this study evaluated the performance of item-pairs and compares the functioning of a directly assessed item against a caregiver-reported counterpart. The results suggest that direct assessment may deserve its status as a preferred form of ECD assessment, but that many applications of ECD measurement may be appropriate with caregiver reported tools. The third essay evaluates the psychometric properties of the International Development and Early Learning Assessment-Classroom Environment (IDELA-CE), an assessment of process and structural quality of ECCE settings. It examined the factor structure of the IDELA-CE and how well it aligns with the hypothesized structure of the tool across countries. It also examines sources of measurement error stemming from the rater observing the class and the particular lesson observed. The analysis found moderate support for the factor structure hypothesized by the tool's creators and that while most variance in observed scores is attributable to stable differences between classrooms, that day-to-day fluctuations should not be ignored. The overarching theme of this dissertation is that measurement in early childhood is a study of tradeoffs. The three chapters highlight tensions in ensuring that the validity & relevance, reliability & precision, and feasibility & cost of measurement is tailored to the use of scores.