Peer Prediction with Private Beliefs

Abstract

Reputation mechanisms at online opinion forums, such as Amazon Reviews, elicit ratings from their users about the experiences with products of unknown quality and critically rely on these ratings being truthful. The peer prediction method by Miller, Resnick and Zeckhauser is arguably the most prominent truthful feedback mechanism in the literature. An obstacle with regard to its application are the strong common knowledge assumptions. Especially the commonly held prior belief about a product’s quality, although prevailing in economic theory, is too strict for this setting. Two issues stand out in particular: first, that different buyers hold different beliefs and, second, that the buyers’ beliefs are often unknown to the mechanism. In this paper, we develop an incentive-compatible peer prediction mechanism for these reputation settings where the buyers have private beliefs about the product’s inherent quality and the likelihood of a positive experience given a particular quality. We show how to exploit the temporal structure and truthfully elicit two reports: one before and one after the buyer’s experience with the product. The key idea is to infer the experience from the direction of the belief change and to use this direction as the event that another buyer is asked to predict.