Modeling and Diagnostics for Paired Comparison Data and Rank Order Data

Abstract

Paired comparison models are used for analyzing data that involves pairwise comparisons among a set of objects. When the outcomes of the pairwise comparisons have no ties, the paired comparison models can be generalized as a class of binary response models. Receiver operating characteristic (ROC) curves and their corresponding areas under the curves (AUC), or the concordance-statistic (c-statistic), are commonly used as performance metrics to evaluate the discriminating ability of binary response models. Despite their individual wide range of usage and their close connection to binary response models, ROC analysis to our knowledge has never been extended to paired comparison models since the problem of using different objects as the reference in paired comparison models prevents traditional ROC approach from generating unambiguous and interpretable curves. In Chapter 1, we focus on addressing this problem by proposing two novel methods to construct ROC curves for paired comparison data which provide interpretable statistics and maintain desired asymptotic properties. The methods are then applied and analyzed on head-to-head professional sports competition data. While the original ROC analysis only applies to problems involving binary outcomes, extensions have been made on binary ROC analyses to accommodate multi-class outcomes. In Chapter 2, we focus on rank order data, which consist of rankings among a set of items, and extend the approaches proposed in Chapter 1 to perform multi-class ROC analyses as diagnostics tools for ranking models. For each extended method, a corresponding generalized c-statistic is defined as a measure of discrimination ability. The extended multi-class ROC analyses can be applied in various cases involving complete rankings, partial rankings, rankings with ties, and data that assume rankings arise from models with mixture components. We also discuss the use of the generalized c-statistics for model assessment and model selection via simulation studies and apply the proposed diagnostic tool to sushi ranking data. In Chapter 3, we propose a generalized Bradley-Terry model to estimate individual ratings within head-to-head team sports. We define the overall team rating as a "smooth maximum'' function (smooth-max) of the rating parameters of the individual players on the team and a "smooth'' parameter. Depending on the sign of the "smooth'' parameter, the team rating can depend either more on the stronger players or more on the weaker players on the team. In addition, a cyclic Minorization-Maximization(MM)-gradient algorithm is proposed for maximum likelihood estimation. We utilize the general MM-algorithm framework developed for a wide class of generalized Bradley-Terry models and make a few modifications for our proposed model to account for the difficulty of constructing a proper minorizing function and the lack of a closed-form maximizer of the function. The proposed algorithm is proven to converge to a stationary point of the log-likelihood function under mild conditions. We apply the proposed model and algorithm on League of Legends (LOL) esports data to estimate individual skills of 2022 Season League Championship Series (LCS) players.