Computational Mechanisms Underlying the Influence of Agency on Learning

Abstract

We live in an uncertain environment where making flexible predictions about the occurrence of positive and negative events is necessary for maximizing rewards, minimizing punishments, and guiding future behavior. Predictions are most accurate, and feedback most useful, when our own actions are responsible for the outcomes we receive. Both humans and animals exhibit a bias toward presuming control, or agency, over outcomes (Mineka, 1985), and extensive work suggests that beliefs about agency have substantial effects on how individuals learn from different types of outcomes. By utilizing computational models, neuroimaging, and flexible behavioral tasks, this dissertation investigated the behavioral and neurobiological pathways through which humans make inferences about hidden information and determined how these inferences influence learning processes. We first found that inference about hidden agents modulated biased learning of positive and negative outcomes within the same individuals. A novel Bayesian model could account for this learning asymmetry, demonstrating a mechanistic framework for understanding how causal attributions contribute to learning (Paper 1). Next, we showed that RPE signals in the dorsal and ventral striatum were scaled by both subjective and model-derived beliefs about agency, but in opposite directions, providing preliminary evidence that striatal learning is gated by causal inference (Paper 2). Finally, we show that controllability arbitrates the use of a Pavlovian over an instrumental learning system (Paper 3). Together, these results suggest that beliefs about agency are at least one factor that influences how we learn from feedback and how we decide the types of learning processes to utilize.