Learning to Promote Cooperation in the Collective Risk Dilemma

Abstract

The first part of this thesis provides a survey of the game theory relevant to the analysis of multi-player player games, including the existence of mixed-strategy Nash equilibria, and the connections between Nash equilibria and evolutionary game theory through replicator dynamics. This exposition also reviews the Q-learning algorithm introduced in Watkins 1989, including a proof of its convergence. The second part of this thesis is a study of social dilemmas, which are games where the payoff to an individual player for cooperative behavior is lower than for defecting behavior, but players are worse off if all defect. Sustaining cooperative action in social dilemmas is challenging due to this tension between unaligned individual short-term incentives and group long-term incentives. One solution is to introduce a social planner whose interventions can resolve the dilemma. In this work we see the application of a social planner to the collective risk dilemma (CRD). The CRD captures the setting in which individuals can contribute (cooperate) towards some collective target. If the target is not reached, i.e. too many agents did not contribute (defect), all agents suffer with some probability (the risk). Prior analysis of the CRD showed that there exist cooperative strategies which are stable steady states in settings with high risk Santos and Pacheco (2011). This work strengthens this result and shows that these cooperative strategies are also evolutionary stable strategies. Furthermore, this work specifically addresses the setting of low perceived risk in the population, which a social planner learns to mitigate through economic intervention in a new intertemporal framing of the CRD. Through Q-learning, a budgeted social planner can learn to push players from non-cooperative to cooperative equilibria and improve the level of cooperation.