Building upon Bradley-Terry and Plackett-Luce: some methods for modeling paired comparison and rank order data

Abstract

A common model used for analyzing paired comparison data is the Bradley-Terry model. An implicit assumption embedded within the Bradley-Terry framework is stochastic transitivity. This implies that if A is preferred to B, and B is preferred to C, then A must be preferred to C. Chapter 1 proposes two Bayesian hierarchical paired comparison models that extend the Bradley-Terry model by introducing a stability parameter for each player. This additional dimension on player characteristics permits the violation of stochastic transitivity among players. This can improve paired comparison inferences fit and out-of-sample prediction accuracy when there is evidence of violation of stochastic transitivity in the data. Chapter 2 extends the model developed in Chapter 1 by developing a rating system that characterizes athletes through strength and stability parameters. We find that the new rating system is capable of identifying athletes with high level of instability.

Chapter 3 evaluates a common practice of analyzing rank order data (e.g., from multi-competitor races) by converting them into paired comparison data and using Bradley-Terry models to make inferences and predictions. We establish this practice within existing statistical frameworks, assuming the rank order data can be analyzed via the conventionally used Plackett-Luce model, and prescribe the necessary adjustments to the covariance matrix of the Bradley-Terry estimators. We also describe the loss of efficiency in doing so. The second part of Chapter 3 draws on the formulation of rank order data as paired comparisons to propose a new model diagnostic tool for the Plackett-Luce model. In a simulation study, this new model diagnostic has good power in detecting departures from Plackett-Luce models.