Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems
View/ Open
43786107.pdf (1.138Mb)
Access Status
Full text of the requested work is not available in DASH at this time ("dark deposit"). For more information on dark deposits, see our FAQ.Published Version
https://doi.org/10.1177/0956797617708288Metadata
Show full item recordCitation
Kool, Wouter, Samuel Gershman, and Fiery Cushman. 2017. Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems. Psychological Science.Abstract
Human behavior is sometimes determined by habit and other times by goal-directed planning. Modern reinforcement-learning theories formalize this distinction as a competition between a computationally cheap but inaccurate model-free system that gives rise to habits and a computationally expensive but accurate model-based system that implements planning. It is unclear, however, how people choose to allocate control between these systems. Here, we propose that arbitration occurs by comparing each system’s task-specific costs and benefits. To investigate this proposal, we conducted two experiments showing that people increase model-based control when it achieves greater accuracy than model-free control, and especially when the rewards of accurate performance are amplified. In contrast, they are insensitive to reward amplification when model-based and model-free control yield equivalent accuracy. This suggests that humans adaptively balance habitual and planned action through on-line cost-benefit analysis.Citable link to this page
http://nrs.harvard.edu/urn-3:HUL.InstRepos:41467544
Collections
- FAS Scholarly Articles [17559]
Contact administrator regarding this item (to report mistakes or request changes)