Essays on Political Methodology

Abstract

This dissertation consists of four independent essays on political methodology. These studies center around the following research programs: (1) measuring ideological scores beyond a single-dimensional scale, (2) addressing bias from missing values when estimating causal effects using panel data, and (3) assessing decision-making systems with algorithmic recommendations. In the first study, I develop a method to estimate ideal points specific to a single issue area using roll call votes and user-supplied issue labels. Ideal point estimation is widely used to measure the ideology and policy preferences of political actors. Yet, an outstanding challenge is to estimate ideal points specific to a single issue area. A common practice is to subset the voting data and fit a model for a specific issue, a method that not only discards valuable information but also hampers the comparison across multiple issue areas. To address this, I propose IssueIRT, a hierarchical Item Response Theory (IRT) model that estimates issue-specific axes within a latent policy space to generate single-dimensional issue-specific ideal points. Contrary to the common practice of subsetting approach, this method enables comparison of ideal points across different issue areas and across time span. Using this method, I examine varying degrees of polarization in US Congress across 33 issue areas from 1979 to 2023. In the second study, I develop methods for estimating causal effects in difference in differences (DID) designs with nonignorable missing outcomes. Missing outcomes in panel data are prevalent and particularly problematic in DID settings, as either selection into treatment or the treatment effect itself may influence outcome missingness. A common approach, known as complete case analysis, drops any units with missing values over time, potentially leading to biased estimates. In this study, I propose alternative identification strategies based on the parallel trends within each principal strata (e.g., always respondents, if treated respondents). Building on this, I introduce two methods: (1) point identification of the average treatment effect for the treated (ATT) using an instrumental variable approach, and (2) partial identification of the ATT for always respondents by leveraging panel data on missingness. Unlike complete case analysis, the partial identification approach does not require independence between treatment selection and principal strata, nor does it assume homogeneous effects across these strata. The third study, coauthored with Naijia Liu and Soichiro Yamauchi, proposes a sensitivity analysis for assessing the robustness of the synthetic control methods (SCM) when units are dropped from an analysis due to missing data. SCM is a widely used causal inference method for policy interventions, yet handling missing values, such as those in country-year economic indicators, remains challenging. We leverage vertical regression as an estimation strategy, where the control units serve as the independent variables and SCM weights correspond to the regression coefficients. Using this framework, we apply omitted variable bias to derive the exact bias formula in SCM estimates. We then propose a sensitivity analysis that utilizes partially observed, often neglected, data as benchmarks. This simple tool allows researchers to evaluate the robustness of their SCM estimates with respect to bias from different configurations of control units. The fourth study, coauthored with Eli Ben-Michael, D. James Greiner, Melody Huang, Kosuke Imai, and Zhichao Jiang, proposes a causal inference framework to compare human decisions with those assisted by Artificial Intelligence (AI). Today, data-driven recommendations based on AI play a central role in human decision-making. The critical question is whether AI helps humans make better decisions compared to a human-alone or AI-alone system. We introduce a new methodological framework to empirically answer this question with minimal assumptions, where we measure a decision maker's ability to make correct decisions using standard classification metrics based on the baseline potential outcome. Under this framework, we show how to compare the performance of three alternative decision-making systems---human-alone, human-with-AI, and AI-alone. This also enables policy learning for better decision-making systems: when AI recommendations should be provided to a human-decision maker, and when one should follow such recommendations. We apply the proposed methodology to our own randomized controlled trial that evaluates a pretrial risk assessment instrument in the US criminal justice system.