Credit assignment to state-independent task representations and its relationship with model-based decision making

Nitzan Shahar, Rani Moran, Tobias U. Hauser, Rogier A. Kievit, Daniel McNamee, Michael Moutoussis, Consortium Nspn, Raymond J. Dolan

Research output: Contribution to journalArticlepeer-review

37 Scopus citations

Abstract

Model-free learning enables an agent to make better decisions based on prior experience while representing only minimal knowledge about an environment’s structure. It is generally assumed that model-free state representations are based on outcome-relevant features of the environment. Here, we challenge this assumption by providing evidence that a putative model-free system assigns credit to task representations that are irrelevant to an outcome. We examined data from 769 individuals performing a well-described 2-step reward decision task where stimulus identity but not spatial-motor aspects of the task predicted reward. We show that participants assigned value to spatial-motor representations despite it being outcome irrelevant. Strikingly, spatial-motor value associations affected behavior across all outcome-relevant features and stages of the task, consistent with credit assignment to low-level state-independent task representations. Individual difference analyses suggested that the impact of spatial-motor value formation was attenuated for individuals who showed greater deployment of goal-directed (model-based) strategies. Our findings highlight a need for a reconsideration of how model-free representations are formed and regulated according to the structure of the environment.

Original languageEnglish
Pages (from-to)15871-15876
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Volume116
Issue number32
DOIs
StatePublished - 6 Aug 2019
Externally publishedYes

Keywords

  • Decision making
  • Motor learning
  • Reinforcement learning

ASJC Scopus subject areas

  • General

Fingerprint

Dive into the research topics of 'Credit assignment to state-independent task representations and its relationship with model-based decision making'. Together they form a unique fingerprint.

Cite this