Why It’s Important To Be Wrong: Systematic Bias and the Limits of Accuracy in 2024 Pre-Election Presidential Polls

Abstract

This research aims to investigate systematic errors in 2024 pre-election presidential polling and to assess the standard methods used to report and analyze polls. The thesis employs four methodologies to examine polling errors, including cross-sectional analysis to identify potential bias in key subsets of the data; multivariate OLS regression to address predictors of bias across time and geographies; Kalman filtering and smoothing to determine the trajectory of poll-implied latent opinion and structural and random bias over the polling period; and the data defect framework replicated and extended from Meng (2018) to evaluate the presence of selection bias and its inconsistency with conventional certainty models. Across all four methods, there is evidence of persistent systematic bias overstating Kamala Harris’ support relative to Donald Trump that is directly associated with structural poll characteristics and electoral context, and it is not captured by standard uncertainty conventions. The findings support top-line analysis conducted on the 2024 pre-election polling, but given the unique scope and methodological approach, they also offer a new lens into this arena without direct comparison. In the broader context of presidential polling literature, the results align with the recurring understatement of Donald Trump across the past three election cycles, while highlighting unique patterns in each election’s “what went wrong.” This analysis invites new perspectives outside of conventional simple random sampling assumptions, which govern much of poll analysis, and argues for the exploration of alternative metrics of evaluation and greater transparency in methodological reporting.