Exploring Avenues Toward Improving Prediction Markets as Tools to Forecast Paper Replication Probability

Abstract

Replication studies are important security mechanisms against the proliferation of false discoveries and misconceptions in the scientific and public communities. However, their high costs and low scalability mean that they are infrequently conducted. This has led to a search for methods that are both accurate and scalable. Some promising research has been done into the efficacy of prediction markets for this purpose as well as machine learning approaches targeting the features of the studies in question. My thesis explores whether combining information from different sources, and also applying the objective discrimination of machines to the subjective beliefs of humans can work toward improving the accuracy of these replication tools while remaining low cost. While the application of machine learning models in this paper to features drawn from survey, study, and transactional market data have only seen marginal improvements, this approach opens the door to other integrative ways to improve the efficacy of these pivotal tools.