TY - JOUR
T1 - Valuation of uncertain and delayed rewards in primate prefrontal cortex
AU - Kim, Soyoun
AU - Hwang, Jaewon
AU - Seo, Hyojung
AU - Lee, Daeyeol
N1 - Funding Information:
We thank Dominic Barraclough for his contribution to some of the studies described in this manuscript. This study was supported by the grants from the National Institute of Health (MH 073246 and DA 024855).
PY - 2009/4
Y1 - 2009/4
N2 - Humans and animals often must choose between rewards that differ in their qualities, magnitudes, immediacy, and likelihood, and must estimate these multiple reward parameters from their experience. However, the neural basis for such complex decision making is not well understood. To understand the role of the primate prefrontal cortex in determining the subjective value of delayed or uncertain reward, we examined the activity of individual prefrontal neurons during an inter-temporal choice task and a computer-simulated competitive game. Consistent with the findings from previous studies in humans and other animals, the monkey's behaviors during inter-temporal choice were well accounted for by a hyperbolic discount function. In addition, the activity of many neurons in the lateral prefrontal cortex reflected the signals related to the magnitude and delay of the reward expected from a particular action, and often encoded the difference in temporally discounted values that predicted the animal's choice. During a computerized matching pennies game, the animals approximated the optimal strategy, known as Nash equilibrium, using a reinforcement learning algorithm. We also found that many neurons in the lateral prefrontal cortex conveyed the signals related to the animal's previous choices and their outcomes, suggesting that this cortical area might play an important role in forming associations between actions and their outcomes. These results show that the primate lateral prefrontal cortex plays a central role in estimating the values of alternative actions based on multiple sources of information.
AB - Humans and animals often must choose between rewards that differ in their qualities, magnitudes, immediacy, and likelihood, and must estimate these multiple reward parameters from their experience. However, the neural basis for such complex decision making is not well understood. To understand the role of the primate prefrontal cortex in determining the subjective value of delayed or uncertain reward, we examined the activity of individual prefrontal neurons during an inter-temporal choice task and a computer-simulated competitive game. Consistent with the findings from previous studies in humans and other animals, the monkey's behaviors during inter-temporal choice were well accounted for by a hyperbolic discount function. In addition, the activity of many neurons in the lateral prefrontal cortex reflected the signals related to the magnitude and delay of the reward expected from a particular action, and often encoded the difference in temporally discounted values that predicted the animal's choice. During a computerized matching pennies game, the animals approximated the optimal strategy, known as Nash equilibrium, using a reinforcement learning algorithm. We also found that many neurons in the lateral prefrontal cortex conveyed the signals related to the animal's previous choices and their outcomes, suggesting that this cortical area might play an important role in forming associations between actions and their outcomes. These results show that the primate lateral prefrontal cortex plays a central role in estimating the values of alternative actions based on multiple sources of information.
KW - Game theory
KW - Inter-temporal choice
KW - Reinforcement learning
KW - Temporal discounting
KW - Utility theory
UR - http://www.scopus.com/inward/record.url?scp=66749182947&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=66749182947&partnerID=8YFLogxK
U2 - 10.1016/j.neunet.2009.03.010
DO - 10.1016/j.neunet.2009.03.010
M3 - Article
C2 - 19375276
AN - SCOPUS:66749182947
SN - 0893-6080
VL - 22
SP - 294
EP - 304
JO - Neural Networks
JF - Neural Networks
IS - 3
ER -