Discreteness Causes Bias in Percentage-Based Comparisons: A Case Study From Educational Testing

DSpace/Manakin Repository

Discreteness Causes Bias in Percentage-Based Comparisons: A Case Study From Educational Testing

Citable link to this page

 

 
Title: Discreteness Causes Bias in Percentage-Based Comparisons: A Case Study From Educational Testing
Author: Yee, Darrick Shen-Wei; Ho, Andrew Dean

Note: Order does not necessarily reflect citation order of authors.

Citation: Yee, Darrick S. and Andrew D. Ho. 2015. Discreteness causes bias in percentage-based comparisons: A case study from educational testing. American Statistician, 69: 174-181.
Full Text & Related Files:
Abstract: Discretizing continuous distributions can lead to bias in parameter estimates. We present a case study from educational testing that illustrates dramatic consequences of discreteness when discretizing partitions differ across distributions. The percentage of test-takers who score above a certain cutoff score (percent above cutoff, or “PAC”) often describes overall performance on a test. Year-over-year changes in PAC, or ΔPAC, have gained prominence under recent U.S. education policies, with public schools facing sanctions if they fail to meet PAC targets. In this paper, we describe how test score distributions act as continuous distributions that are discretized inconsistently over time. We show that this can propagate considerable bias to PAC trends, where positive ΔPACs appear negative, and vice versa, for a substantial number of actual tests. A simple model shows that this bias applies to any comparison of PAC statistics in which values for one distribution are discretized differently from values for the other.
Published Version: doi:10.1080/00031305.2015.1031828
Terms of Use: This article is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#OAP
Citable link to this page: http://nrs.harvard.edu/urn-3:HUL.InstRepos:27471534
Downloads of this work:

Show full Dublin Core record

This item appears in the following Collection(s)

 
 

Search DASH


Advanced Search
 
 

Submitters