This setting provides a natural model for situations such as commuters in traffic or bargaining in bazaars. Importantly, in minimizing the role of reputation and higher-order belief considerations, the population setting using a random matching protocol is perhaps the most widely studied experimental setting and has served as a basic building block for a number of models in evolutionary biology and game theory 7 , 25 , 26 , Lesion reconstruction and task schematic.
All BG lesions were shown overlaid on the left hemisphere for comparison purposes 4L; 2R. BG group mean lesion volume was Maximal lesion overlap was in the putamen and encompassed the head and body of the caudate as well as the globus pallidus in some patients. OFC group mean lesion volume was In the particular payoff structure we used, the prize is worth 10 units, and the Strong Weak player is endowed with five 4 units at the beginning of each round. Subjects then inputted the decision self-paced by pressing a button mapped to the desired investment amount from the initial endowment.
In the event of a tie, both lost the prize. In either case, the subject kept the portion of the endowment not invested. In the non-strategic condition, participants were told to choose an investment that must exceed a randomly generated hurdle to win the prize, but were not told how the computer generated the random hurdle. The hurdle followed the same empirical frequency of decisions as in the strategic condition.
The experiment consisted of rounds of Patent Race game, alternating between the strategic and non-strategic conditions over 80 rounds, counterbalanced. Specifically, in the Patent Race, players of two types, Strong and Weak, are randomly matched at the beginning of each round and compete for a prize by choosing an investment in integer amounts from their respective endowments.
The player who invests more wins the prize, and the other loses. In the event of a tie, both lose the prize. Regardless of the outcome, players lose the amount that they invested. In the particular payoff structure we use, the prize is worth 10 units, and the Strong Weak player is endowed with 5 4 units Fig. Substantial evidence has shown that learning in economic games including the Patent Race can be parsimoniously explained using two learning rules across a wide-range of strategic contexts and experimental conditions: i reinforcement-based learning RL through trial and error, and ii belief-based learning through anticipating and responding to actions of others In particular, RL models posit that learning is driven by a prediction error defined as the difference between expected and received rewards and have been highly successful in connecting behavior to the underlying neurobiology 28 , In contrast, belief-based learning posits that players make use of knowledge of the structure of the game to update value estimates of available actions and comes in two computationally equivalent interpretations.
One interpretation assumes the existence of latent beliefs and requires players to form and update first-order beliefs regarding the likelihood of future actions of opponents. Under the alternative interpretation, beliefs and mental models are not assumed and action values are updated directly by reinforcing all actions proportional to their foregone or fictive rewards The equivalence of these two mathematical interpretations thus makes it clear that belief-based learning does not necessarily imply the learning of mental, verbalizable beliefs commonly referred to in the cognitive and social sciences, because specific beliefs about likely strategies of opponents are sufficient but not necessary for this type of learning.
Importantly, previous neuroimaging results have been able to disaggregate distinct computational signatures of reinforcement-based learning RL and belief learning processes based on trial-by-trial variation in neural responses along frontostriatal circuits. Specifically, whereas the medial prefrontal cortex mPFC selectively responds to belief learning prediction error signals, activity in the putamen, a substructure of the BG, is correlated with prediction errors associated with both RL and belief-based learning 7.
Building on these findings, therefore, we sought to investigate the extent to which putative computational processes such as RL and belief-based learning would reflect functions of BG necessary for social and strategic learning.
Following informed consent, subjects were tested in the Patent Race as well as a matching non-strategic version where we replaced the human pool players with a computer algorithm. Specifically, in the non-strategic task, participants were told to choose an investment that must exceed a randomly generated hurdle to succeed, but were not told how the computer generated the random hurdle choices Methods. Importantly, whereas in strategic learning both belief and reinforcement components were engaged, our previous work has shown that learning in the non-strategic environment was driven primarily by reinforcement learning 7.
Thus the inclusion of both strategic and non-strategic conditions makes opposing predictions about how BG lesion will affect learning in the economic game. If BG are involved in both trial-and-error learning as well as social functioning such as strategic learning, damage to BG will affect performances in both strategic and non-strategic environments. Alternatively, if, in the strategic environment where multiple learning processes are engaged, learning inputs originated from prefrontal areas provide compensatory functions for trial-and-error deficits resulted from the BG damage, we should expect that damage to the BG selectively impairs learning capacity in the non-strategic, reward-reinforcing environment, as opposed to the more complex, interpersonal, strategic environment.
Consistent with neuroimaging evidence suggesting dissociable contributions of the BG and prefrontal cortex PFC to multiple learning rules, we found that patients with BG damage performed similarly as participants in the HC cohort in the strategic condition, where learning was driven by a mixture of reinforcement and belief learning.
In contrast, BG patients were markedly impaired when information about the strategic context was removed, such that participants must rely primarily on learning based on reinforcement. Moreover, these differences were qualitatively distinct from those observed in patients with OFC damage, suggesting that findings related to BG cannot be attributed to the specific method used in the study or a general deficit associated with reward circuit damage.
To characterize overall task performance and differences across cohorts in the Patent Race game Fig. In particular, this analysis captures the idea from previous theoretical and empirical studies showing that whereas behavior of pure reinforcement learners should be sensitive only to received payoffs, belief-based learners will be in addition sensitive to foregone payoffs. Overall task performances. For example, if the subject chose 5 and the opponent 0, the maximal forgone payoff was 14, as the optimal action would have been to invest 1, as opposed to the received payoff Had the opponent chosen 3, however, the received payoff would remain to be 10 but the maximal payoff would be 11, as choosing 4 would have been the best strategy.
Y -axes represent the percentage of trials in which subjects chose to stay with the same decision on the next trial i. Error bars represent S. To illustrate this, suppose that the Weak player observes the Strong players frequently investing five units.
She may subsequently respond by playing zero to keep her initial endowment. This behavior may, in turn, entice the Weak player to move away from investing zero to win the prize. In contrast, pure RL players will respond to these changes in behavior of the opponents in a much slower manner, because they behave by comparing received payoffs from past investments without consideration for the strategic behavior of others Supplementary Figure 1.
We therefore conducted model-free logistic regression of the probability that HC participants would choose the same strategy on the received payoff and foregone payoff on a round-by-round basis. Consistent with previous theoretical and empirical findings, we found that the extent to which HC participants would stay switch with the same strategy was associated with having received high low payoff or low high regret. Previous work has shown that, whereas participants respond to both received and foregone payoffs in a strategic environment, in the non-strategic environment learning is driven primarily by reinforcing actions associated with the received payoff 18 , Using these measures, we next investigated how lesion to BG or OFC affected performance across strategic and non-strategic conditions.
All results were robust to analyses controlling for demographic variables and neuropsychological assessments, as well as non-parametric permutation tests sampling null distribution for each cohort and each condition Supplementary Table 2 and Supplementary Figure 2.
The above results therefore argue in favor of a model where effects of the BG damage on social functioning were buffered by other learning processes, but not when the social context was removed. To more formally test this dissociation, and to connect behavioral differences to underlying cognitive mechanisms, we applied a computational approach using the Experience-Weighted Attraction EWA model, which nests reinforcement and belief-based learning algorithms as special cases and has been highly successful in connecting these computational components with neural responses along frontostriatal circuits Supplementary Figure 1 ; see Methods 7 , 31 , We tested the hypothesis that the extent to which BG are asymmetrically involved in learning in strategic and non-strategic environments would be reflected by the differential ability of EWA to explain choice behavior.
That is, BG patients should benefit more when shifting from the non-strategic to strategic environment, in terms of the EWA model fit either in-sample e. Specifically, comparing pseudo- R 2 values 31 , we found no significant difference in how well EWA explained choice behavior under the strategic condition in BG vs.
Computational modeling. The bar plots depict values of pseudo- R 2 derived from the best-fitting EWA, defined as the difference between the log-likelihood of the EWA model and a random choice model, scaled by log-likelihood of the random model. Higher pseudo- R 2 values indicate better model fit relative to chance level. The means and error bars were constructed using a bootstrap procedure with 10, iterations pooling over cohorts for each condition. Error bars indicated bootstrap S. Moreover, there was a significant cohort BG vs.
HC by condition strategic vs. To address potential concerns regarding overfitting and spurious cohort differences arising from natural variations in learning across individuals, we performed additional analyses using out-of-sample tests and permutation tests shuffling cohort labels.
Both yielded similar results Supplementary Figure 6 — 7. HC: 0. However, unlike in BG patients, OFC effects were sensitive to alternative specifications such as the self-tuning estimation, where some of the EWA parameters were replaced by functions of experience of OFC patients 37 Supplementary Figure 8. Model estimates and additional robustness checks are reported in the Supplement Supplementary Figure 9 , Supplementary Tables 3 — 4.
To more formally test the hypothesis that BG damage spares the capacity to engage in belief-based learning, we used the EWA model to disentangle the relative contributions of different decision rules across cohorts in strategic and non-strategic conditions. Specifically, we examined the extent to which EWA improved the explanatory power above and beyond the basic RL algorithm.
That is, if strategic learning capacity in BG patients was compensated using high-order learning processes, EWA should significantly improve the fit relative to the baseline RL Supplementary Figure 1. By focusing on model comparison as opposed to specific parameters calibrated from the behavior e. Importantly, this test also serves as a more stringent test, because choices that were equally explainable by RL and other learning rules nested within EWA were attributed solely to the RL algorithm.
Using the Bayesian Information Criterion BIC to penalize for the number of free parameters, we found that in control subjects, consistent with previous studies, EWA significantly improved the fit only in the strategic but not the non-strategic condition Strategic: 7.
A wealth of neuroimaging data has implicated the involvement of the BG, and in particular the striatum, in a striking variety of goal-directed decisions, including those involving acquiring rewards for oneself as well as in the social domain where actions and outcomes depend on rewards of others 4 , 7 , 24 , In the former, these correlational findings have been corroborated with findings from causal studies using focal lesion patients and those with neurodegenerative disorders known to affect BG 11 , 19 , 22 , 23 , 24 , 39 , 40 , In contrast, surprisingly little evidence exists, either in support of or argue against, the causal involvement of BG in social decision making 11 , 13 , Here by connecting the lesion method with neuroeconomic tools, we show that capacity for strategic learning in the presence of competitive, intelligent opponents can be preserved in patients with focal BG damage, despite having deficits in learning in a non-social, probabilistic environment.
Model comparisons further show that damage to BG spares strategic learning capacity possibly through compensatory processes such as belief-based learning when the social context is available for anticipating future actions of others. Owing to variation in lesion location and extent across patients, it is possible that our findings were driven by damage to specific BG nuclei or adjacent regions outside of BG. In particular, the putamen and caudate nucleus have been previously implicated in learning about actions and their reward consequences in action-contingent learning 42 , 43 , 44 , 45 and in social exchanges involving trust and reputation that requires learning about social agents based on their previous actions 4.
Similarly, damage extending to the insular cortex, which was observed in three of six patients in the BG lesion cohort, was not associated with performance Supplementary Table 5. In contrast, behavior in patients with damage to the OFC, another critical node within the reward circuit, shows a qualitatively distinct pattern, suggesting that findings related to BG cannot be attributed to the specific method used in the study or considered as a general property associated with the reward circuit.
Together with prior neuroimaging findings, our data provide insights into the computational underpinnings of social decision making and the apparently contradictory findings from past neuroimaging and lesion studies. Specifically, both set of findings are consistent with a model of BG functioning in receiving higher-order learning signals broadcasted from other regions involved in social cognition to the striatal input areas.
In line with this model, BG activations identified in prior fMRI studies of social decision-making were typically accompanied by concurrent activations in other brain regions involved in social cognitive processing, including the rostral anterior cingulate cortex ACC 46 , mPFC 7 , 47 , and temporoparietal junction TPJ 48 , Within the Patent Race itself, BOLD responses in the putamen were found to be associated with prediction errors arising from both belief-based and reinforcement learning, whereas activity in the medial PFC is correlated with belief-based learning prediction errors 7.
Our results are consistent with past studies in BG disorders, suggesting the presence of compensatory processes when the task can be solved through multiple learning strategies.
For example, although BG damage is associated with impaired learning in changing, probabilistic environments 19 , 23 , 39 , there is some evidence that learning capacity is intact when patients are able to engage in declarative learning strategies which do not depend on the integrity of BG Results of these studies thus raise intriguing questions regarding whether the asymmetrical functions of BG are specific to strategic vs.
For example, in observational learning where individuals can learn from either the actions or choice outcomes of others in a non-strategic manner, it remains unclear whether the putative contribution of BG in the processing of outcome-based learning signals is necessary for learning from observations, or can be compensated by action-based learning that depends on the dorsolateral PFC 6. An alternative possible explanation is that the preserved strategic learning capacity reflects the compensatory role of the intact contralateral BG.
Indeed, owing to the rarity and often devastating motor deficits of bilateral BG damage 11 , our BG cohort consisted only of those with unilateral lesion. As a result, it is possible that the intact hemisphere alone is sufficient for learning in social settings, but not in non-social environment. More broadly, it suggests the possibility where social learning capacity following BG damage may crucially depend on intact functional coupling between preserved portions of the BG and cortical regions involved in social cognitive processes.
This is consistent with existing causal evidence from both lesion and TMS studies, demonstrating the causal involvement of cortical regions, including the ACC 50 , 51 , 52 , mPFC 10 , 51 , 53 , and right TPJ 54 , 55 , in social decision-making in humans and non-human primates. Future studies comparing the functional connectivity of these regions in patient vs. Interestingly, although pseudo- R 2 values were fairly consistent over time in the strategic condition, they were more variable in the non-strategic condition.
This is particularly true for the BG patients during rounds 15—35, where pseudo- R 2 dropped sharply following a rise at the start of the experiment. This may reflect the engagement, albeit less successfully than in the strategic condition, of compensatory mechanisms in BG patients in the non-strategic condition, for example, through relying on working memory WM systems. Indeed, past studies of reward learning suggest that prediction errors produced by RL systems include significant contribution from WM systems, especially during early learning 56 , Future studies are needed to more firmly establish this effect and the underlying neural mechanisms.
Our findings also contribute to the understanding of social deficits associated with OFC lesions. Deficits observed in our OFC patients were particularly marked in the strategic condition, hinting at a more pronounced impairment during decision-making in social contexts.
This is consistent with the wealth of neuropsychological findings documenting profound changes in social behavior following the OFC damage, including impaired capability in perspective taking and inferring mental states of others The OFC effects observed in our experiment, however, were more heterogeneous and sensitive to the specific analytic choices.
One candidate explanation is that this reflects the greater variation in the damage extent in our OFC cohort, which in some cases extended into the lateral and dorsal regions. Previous literature suggests lateral and medial OFC differentially contribute to processes entailing, respectively, learning and updating versus those involved in value comparisons especially in decisions among three or more options 58 , 59 , Moreover, owing to the presence of white matter damage and in some cases adjacent regions including the lateral OFC, we cannot completely rule out the contribution from non-mOFC based processes Future experiments with larger sample sizes in combination with lesion mapping techniques will be needed to test these possibilities.
A more general concern with our model-based approach is the possibility of model misspecification due to participants engaging in decision rules beyond the EWA model space This is particularly the case with lesion cohort behavior, as model-based approaches such as RL or EWA inherently make strong assumptions regarding how past experiences are integrated over the course of learning.
Our study addressed this in two ways. First, we focused on cross-cohort comparisons using goodness-of-fit measures, rather than specific parameters calibrated from behavior e. In particular, comparisons based on individual parameters provide meaningful insights into cognitive components if cohorts behave in accordance to model assumptions. For example, using comparison between HC strategic and non-strategic conditions, the belief learning parameter provided a good indication that HC relied less on belief learning in the non-strategic condition.
On the other hand, a poor model fit raises the possibility that one or more model assumptions are violated. This misspecification issue is equally true for both frequentist and Bayesian approaches.
More importantly, such model misspecification can result in either upward or downward biases in parameter estimates. This makes it difficult, even in the presence of significant differences in parameter estimates, to draw firm conclusions regarding differences in cognitive mechanisms between cohorts or conditions.
Second, consistent with other neuroeconomic studies using the lesion method e. A more thorough investigation using data-driven approaches will be needed to further assess and compare choice predictability of lesion patients vs. Issues of whether, and under what circumstances, cognitive processes supported by BG reflect the computational properties necessary for social behavior have important implications for understanding the interaction between parallel cognitive processes, as well as the neural mechanisms necessary for arbitrating between such processes.
The present study thus demonstrates the utility of combining the lesion method with formal models of behavior in addressing these questions. At the same time, an important limitation of our study concerns the limited sample size of patient cohorts, particularly given the inherent rarity of focal BG lesion. Future studies can address this issue by using lesion analytical methods, such as model-based lesion symptom mapping, to identify the distributed patterns of brain areas within BG that subserve social functions.
In addition, future studies also need to address whether our findings generalize to other types of social decisions, including those involving prosocial motivations 47 , 65 , 66 , reciprocity 67 , and social dominance See Supplementary Table 1 for demographic information and neuropsychological background of lesion patients. Software reconstructions were performed using MRIcron A neurologist R. Following task instructions and a comprehension quiz, participants were administered two blocks of strategic and non-strategic condition trials, each containing 80 rounds.
All choices were conducted using hypothetical payoffs and no feedback, with order of the strategic and non-strategic blocks counterbalanced across participants within each cohort. When the blood flow to an area of the brain is restricted or stopped, the brain does not receive enough oxygen. Oxygen deprivation injures brain cells in that area, and they die as a result.
A collection of cell bodies called the basal ganglia lies deep in the center of the brain. The basal ganglia serve as the message center for a range of bodily functions. There are several types of basal ganglia stroke, all with different causes. The three main types are as follows:. This common stroke occurs when a blood clot blocks a vessel carrying blood to the brain, making it impossible for blood to reach its target.
An estimated 87 percent of all strokes are ischemic. This less common type of stroke accounts for about 40 percent of all stroke deaths, according to the National Stroke Association. This type of stroke occurs when blood leaks from a burst, torn, or unstable blood vessel into the tissue in the brain.
The buildup of blood can create swelling, pressure, and, ultimately, brain damage. Many basal ganglia strokes are hemorrhagic strokes, which often result from uncontrolled high blood pressure. More accurately, a TIA means that stroke-like symptoms occur for several minutes but always less than 24 hours. They resolve with no lasting damage. These events can be a warning sign that a more severe type of stroke might occur in the future.
Strokes have a common set of signs and symptoms. Everyone should be aware of these because recognizing them when they occur and taking action could help to save lives. A basal ganglia stroke also has some unique signs and symptoms that might make it harder to identify than other types of stroke. These include:. Anyone who notices these symptoms in themselves or others should seek immediate medical attention. Treatment for a basal ganglia stroke depends on the type of stroke and how quickly a person receives medical attention.
If stroke victims reach the hospital in good time, they might receive a drug that breaks up the blood clots causing the stroke symptoms. Call your provider if you have any abnormal or involuntary movements, falls without known reason, or if you or others notice that you are shaky or slow.
Jankovic J. Parkinson disease and other movement disorders. Bradley's Neurology in Clinical Practice. Philadelphia, PA: Elsevier; chap Other movement disorders. Goldman-Cecil Medicine. Disorders of the basal nuclei. Umphred's Neurological Rehabilitation. St Louis, MO: Elsevier; chap Updated by: Amit M.
0コメント