Basal ganglia neurons dynamically facilitate exploration during associative learning

Sameer A. Sheth, Tarek Abuelem, John T. Gale, Emad N. Eskandar

Research output: Contribution to journalArticle

33 Citations (Scopus)

Abstract

The basal ganglia (BG) appear to play a prominent role in associative learning, the process of pairing external stimuli with rewarding responses. Accumulating evidence suggests that the contributions of various BG components may be described within a reinforcement learning model, in which a broad repertoire of possible responses to environmental stimuli are evaluated before the most profitable one is chosen. The striatum receives diverse cortical inputs, providing a rich source of contextual information about environmental cues. It also receives projections from midbrain dopaminergic neurons, whose phasic activity reflects a reward prediction error signal. These coincident information streams are well suited for evaluating responses and biasing future actions toward the most profitable response. Still lacking in this model is a mechanistic description of how initial response variability is generated. To investigate this question, we recorded the activity of single neurons in the globus pallidus internus (GPi), the primary BG output nucleus, in nonhuman primates (Macaca mulatta) performing a motor associative learning task. A subset (29%) of GPi neurons showed learning-related effects, decreasing firing during the early stages of learning, then returning to higher baseline rates as associations were mastered. On a trial-by-trial basis, lower firing rates predicted exploratory behavior, whereas higher rates predicted an exploitive response. These results suggest that, during associative learning, BG output is initially permissive, allowing exploration of a variety of responses. Once a profitable response is identified, increased GPi activity suppresses alternative responses, sharpening the response profile and encouraging exploitation of the profitable learned behavior.

Original languageEnglish (US)
Pages (from-to)4878-4885
Number of pages8
JournalJournal of Neuroscience
Volume31
Issue number13
DOIs
StatePublished - Mar 30 2011
Externally publishedYes

Fingerprint

Basal Ganglia
Learning
Neurons
Globus Pallidus
Exploratory Behavior
Dopaminergic Neurons
Mesencephalon
Macaca mulatta
Reward
Primates
Cues

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Basal ganglia neurons dynamically facilitate exploration during associative learning. / Sheth, Sameer A.; Abuelem, Tarek; Gale, John T.; Eskandar, Emad N.

In: Journal of Neuroscience, Vol. 31, No. 13, 30.03.2011, p. 4878-4885.

Research output: Contribution to journalArticle

Sheth, Sameer A. ; Abuelem, Tarek ; Gale, John T. ; Eskandar, Emad N. / Basal ganglia neurons dynamically facilitate exploration during associative learning. In: Journal of Neuroscience. 2011 ; Vol. 31, No. 13. pp. 4878-4885.
@article{a6b5b816cb284d9fa1c96908a6e9197f,
title = "Basal ganglia neurons dynamically facilitate exploration during associative learning",
abstract = "The basal ganglia (BG) appear to play a prominent role in associative learning, the process of pairing external stimuli with rewarding responses. Accumulating evidence suggests that the contributions of various BG components may be described within a reinforcement learning model, in which a broad repertoire of possible responses to environmental stimuli are evaluated before the most profitable one is chosen. The striatum receives diverse cortical inputs, providing a rich source of contextual information about environmental cues. It also receives projections from midbrain dopaminergic neurons, whose phasic activity reflects a reward prediction error signal. These coincident information streams are well suited for evaluating responses and biasing future actions toward the most profitable response. Still lacking in this model is a mechanistic description of how initial response variability is generated. To investigate this question, we recorded the activity of single neurons in the globus pallidus internus (GPi), the primary BG output nucleus, in nonhuman primates (Macaca mulatta) performing a motor associative learning task. A subset (29{\%}) of GPi neurons showed learning-related effects, decreasing firing during the early stages of learning, then returning to higher baseline rates as associations were mastered. On a trial-by-trial basis, lower firing rates predicted exploratory behavior, whereas higher rates predicted an exploitive response. These results suggest that, during associative learning, BG output is initially permissive, allowing exploration of a variety of responses. Once a profitable response is identified, increased GPi activity suppresses alternative responses, sharpening the response profile and encouraging exploitation of the profitable learned behavior.",
author = "Sheth, {Sameer A.} and Tarek Abuelem and Gale, {John T.} and Eskandar, {Emad N.}",
year = "2011",
month = "3",
day = "30",
doi = "10.1523/JNEUROSCI.3658-10.2011",
language = "English (US)",
volume = "31",
pages = "4878--4885",
journal = "Journal of Neuroscience",
issn = "0270-6474",
publisher = "Society for Neuroscience",
number = "13",

}

TY - JOUR

T1 - Basal ganglia neurons dynamically facilitate exploration during associative learning

AU - Sheth, Sameer A.

AU - Abuelem, Tarek

AU - Gale, John T.

AU - Eskandar, Emad N.

PY - 2011/3/30

Y1 - 2011/3/30

N2 - The basal ganglia (BG) appear to play a prominent role in associative learning, the process of pairing external stimuli with rewarding responses. Accumulating evidence suggests that the contributions of various BG components may be described within a reinforcement learning model, in which a broad repertoire of possible responses to environmental stimuli are evaluated before the most profitable one is chosen. The striatum receives diverse cortical inputs, providing a rich source of contextual information about environmental cues. It also receives projections from midbrain dopaminergic neurons, whose phasic activity reflects a reward prediction error signal. These coincident information streams are well suited for evaluating responses and biasing future actions toward the most profitable response. Still lacking in this model is a mechanistic description of how initial response variability is generated. To investigate this question, we recorded the activity of single neurons in the globus pallidus internus (GPi), the primary BG output nucleus, in nonhuman primates (Macaca mulatta) performing a motor associative learning task. A subset (29%) of GPi neurons showed learning-related effects, decreasing firing during the early stages of learning, then returning to higher baseline rates as associations were mastered. On a trial-by-trial basis, lower firing rates predicted exploratory behavior, whereas higher rates predicted an exploitive response. These results suggest that, during associative learning, BG output is initially permissive, allowing exploration of a variety of responses. Once a profitable response is identified, increased GPi activity suppresses alternative responses, sharpening the response profile and encouraging exploitation of the profitable learned behavior.

AB - The basal ganglia (BG) appear to play a prominent role in associative learning, the process of pairing external stimuli with rewarding responses. Accumulating evidence suggests that the contributions of various BG components may be described within a reinforcement learning model, in which a broad repertoire of possible responses to environmental stimuli are evaluated before the most profitable one is chosen. The striatum receives diverse cortical inputs, providing a rich source of contextual information about environmental cues. It also receives projections from midbrain dopaminergic neurons, whose phasic activity reflects a reward prediction error signal. These coincident information streams are well suited for evaluating responses and biasing future actions toward the most profitable response. Still lacking in this model is a mechanistic description of how initial response variability is generated. To investigate this question, we recorded the activity of single neurons in the globus pallidus internus (GPi), the primary BG output nucleus, in nonhuman primates (Macaca mulatta) performing a motor associative learning task. A subset (29%) of GPi neurons showed learning-related effects, decreasing firing during the early stages of learning, then returning to higher baseline rates as associations were mastered. On a trial-by-trial basis, lower firing rates predicted exploratory behavior, whereas higher rates predicted an exploitive response. These results suggest that, during associative learning, BG output is initially permissive, allowing exploration of a variety of responses. Once a profitable response is identified, increased GPi activity suppresses alternative responses, sharpening the response profile and encouraging exploitation of the profitable learned behavior.

UR - http://www.scopus.com/inward/record.url?scp=79955746277&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79955746277&partnerID=8YFLogxK

U2 - 10.1523/JNEUROSCI.3658-10.2011

DO - 10.1523/JNEUROSCI.3658-10.2011

M3 - Article

VL - 31

SP - 4878

EP - 4885

JO - Journal of Neuroscience

JF - Journal of Neuroscience

SN - 0270-6474

IS - 13

ER -