Auditing for Score Inflation Using Self-Monitoring Assessments: Findings from Three Pilot Studies

DSpace/Manakin Repository

Auditing for Score Inflation Using Self-Monitoring Assessments: Findings from Three Pilot Studies

Citable link to this page

 

 
Title: Auditing for Score Inflation Using Self-Monitoring Assessments: Findings from Three Pilot Studies
Author: Koretz, Daniel; Jennings, Jennifer L.; Yu, Carol; Braslow, David Arthur; Langi, Meredith; Hui, Leng Ng

Note: Order does not necessarily reflect citation order of authors.

Citation: Koretz, Daniel, Jennifer L. Jennings, Hui Leng Ng, Carol Yu, David Braslow, and Meredith Langi. 2016. Auditing for score inflation using self-monitoring assessments: findings from three pilot studies. Educational Assessment
Full Text & Related Files:
Abstract: Research has shown that test-based accountability programs often produce score inflation. Most studies have evaluated inflation by comparing trends on a high-stakes test and a lower-stakes audit test. However, Koretz and Benguin (2010) noted the weaknesses of using external audit tests and suggested instead using self-monitoring assessments (SMAs), which incorporate into high-stakes tests audit items that are not susceptible to test preparation aimed at more predictable items. This paper reports the results of the first three trials of the SMA approach, evaluating whether SMAs can detect inflation in a context in which it has been demonstrated to exist. The studies were conducted with the New York State mathematics tests in grades 4, 7, and 8 in 2011 and 2012. Despite a severe conservative bias created by numerous aspects of the study designs, we found that the audit component functioned as expected in many of the trials. The difference in performance between nonaudit and audit items was associated with factors that earlier research showed to be related to test preparation and score inflation, such as "bubble-student" status (scoring just below the Proficient cut in the previous year) and school poverty. However, a number of trials yielded null findings. These findings underscore the need for additional research investigating the optimal characteristics of audit items.
Terms of Use: This article is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#OAP
Citable link to this page: http://nrs.harvard.edu/urn-3:HUL.InstRepos:28269315
Downloads of this work:

Show full Dublin Core record

This item appears in the following Collection(s)

 
 

Search DASH


Advanced Search
 
 

Submitters