Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems
Access StatusFull text of the requested work is not available in DASH at this time ("dark deposit"). For more information on dark deposits, see our FAQ.
MetadataShow full item record
CitationKool, Wouter, Samuel Gershman, and Fiery Cushman. 2017. Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems. Psychological Science.
AbstractHuman behavior is sometimes determined by habit and other times by goal-directed planning. Modern reinforcement-learning theories formalize this distinction as a competition between a computationally cheap but inaccurate model-free system that gives rise to habits and a computationally expensive but accurate model-based system that implements planning. It is unclear, however, how people choose to allocate control between these systems. Here, we propose that arbitration occurs by comparing each system’s task-specific costs and benefits. To investigate this proposal, we conducted two experiments showing that people increase model-based control when it achieves greater accuracy than model-free control, and especially when the rewards of accurate performance are amplified. In contrast, they are insensitive to reward amplification when model-based and model-free control yield equivalent accuracy. This suggests that humans adaptively balance habitual and planned action through on-line cost-benefit analysis.
Citable link to this pagehttp://nrs.harvard.edu/urn-3:HUL.InstRepos:41467544
- FAS Scholarly Articles