Dopamine neurons are thought to facilitate learning by comparing actual and expected reward. Despite two decades of investigation, little is known about how this comparison is made. To determine how dopamine neurons calculate prediction error, we combined optogenetic manipulations with extracellular recordings in the ventral tegmental area while mice engaged in classical conditioning. Here we demonstrate, by manipulating the temporal expectation of reward, that dopamine neurons perform subtraction, a computation that is ideal for reinforcement learning but rarely observed in the brain. Furthermore, selectively exciting and inhibiting neighbouring GABA (γ-aminobutyric acid) neurons in the ventral tegmental area reveals that these neurons are a source of subtraction: they inhibit dopamine neurons when reward is expected, causally contributing to prediction-error calculations. Finally, bilaterally stimulating ventral tegmental area GABA neurons dramatically reduces anticipatory licking to conditioned odours, consistent with an important role for these neurons in reinforcement learning. Together, our results uncover the arithmetic and local circuitry underlying dopamine prediction errors.
Haroush and Williams trained pairs of monkeys to play in a prisoner’s dilemma game, a model of social interactions. Recording from the dorsal anterior cingulate cortex (dACC), they find neurons whose activity reflects the anticipation of the opponent’s yet unknown choice, which may be important in guiding animals’ performance in the game.
Serotonin's function in the brain is unclear. One challenge in testing the numerous hypotheses about serotonin's function has been observing the activity of identified serotonergic neurons in animals engaged in behavioral tasks. We recorded the activity of dorsal raphe neurons while mice experienced a task in which rewards and punishments varied across blocks of trials. We 'tagged' serotonergic neurons with the light-sensitive protein channelrhodopsin-2 and identified them based on their responses to light. We found three main features of serotonergic neuron activity: (1) a large fraction of serotonergic neurons modulated their tonic firing rates over the course of minutes during reward versus punishment blocks; (2) most were phasically excited by punishments; and (3) a subset was phasically excited by reward-predicting cues. By contrast, dopaminergic neurons did not show firing rate changes across blocks of trials. These results suggest that serotonergic neurons signal information about reward and punishment on multiple timescales.
Serotonin and dopamine are major neuromodulators. Here, we used a modified rabies virus to identify monosynaptic inputs to serotonin neurons in the dorsal and median raphe (DR and MR). We found that inputs to DR and MR serotonin neurons are spatially shifted in the forebrain, and MR serotonin neurons receive inputs from more medial structures. Then, we compared these data with inputs to dopamine neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc). We found that DR serotonin neurons receive inputs from a remarkably similar set of areas as VTA dopamine neurons apart from the striatum, which preferentially targets dopamine neurons. Our results suggest three major input streams: a medial stream regulates MR serotonin neurons, an intermediate stream regulates DR serotonin and VTA dopamine neurons, and a lateral stream regulates SNc dopamine neurons. These results provide fundamental organizational principles of afferent control for serotonin and dopamine.
How is sensory information represented in the brain?Along-standing debate in neural coding is whether and how timing of spikes conveys information to downstream neurons. Although we know that neurons in the olfactory bulb (OB) exhibit rich temporal dynamics, the functional relevance of temporal coding remains hotly debated. Recent recording experiments in awake behaving animals have elucidated highly organized temporal structures of activity in the OB. In addition, the analysis of neural circuits in the piriform cortex (PC) demonstrated the importance of not only OB afferent inputs but also intrinsicPCneural circuits in shaping odor responses. Furthermore, new experiments involving stimulation of the OB with specific temporal patterns allowed for testing the relevance of temporal codes. Together, these studies suggest that the relative timing of neuronal activity in the OB conveys odor information and that neural circuits in the PC possess various mechanisms to decode temporal patterns of OB input.
Krashes MJ, Shah BP, Olson DP, Strochlic DE, Garfield AS, Vong L, Pei H, Watabe-Uchida M, Uchida N, Liberles SD, et al.
Hunger is a hard-wired motivational state essential for survival. Agouti-related peptide (AgRP)-expressing neurons in the arcuate nucleus (ARC) at the base of the hypothalamus are crucial to the control of hunger. They are activated by caloric deficiency and, when naturally or artificially stimulated, they potently induce intense hunger and subsequent food intake. Consistent with their obligatory role in regulating appetite, genetic ablation or chemogenetic inhibition of AgRP neurons decreases feeding. Excitatory input to AgRP neurons is important in caloric-deficiency-induced activation, and is notable for its remarkable degree of caloric-state-dependent synaptic plasticity. Despite the important role of excitatory input, its source(s) has been unknown. Here, through the use of Cre-recombinase-enabled, cell-specific neuron mapping techniques in mice, we have discovered strong excitatory drive that, unexpectedly, emanates from the hypothalamic paraventricular nucleus, specifically from subsets of neurons expressing thyrotropin-releasing hormone (TRH) and pituitary adenylate cyclase-activating polypeptide (PACAP, also known as ADCYAP1). Chemogenetic stimulation of these afferent neurons in sated mice markedly activates AgRP neurons and induces intense feeding. Conversely, acute inhibition in mice with caloric-deficiency-induced hunger decreases feeding. Discovery of these afferent neurons capable of triggering hunger advances understanding of how this intense motivational state is regulated.
Wu Z, Autry AE, Bergan JF, Watabe-Uchida M, Dulac CG.
Mice display robust, stereotyped behaviours towards pups: virgin males typically attack pups, whereas virgin females and sexually experienced males and females display parental care. Here we show that virgin males genetically impaired in vomeronasal sensing do not attack pups and are parental. Furthermore, we uncover a subset of galanin-expressing neurons in the medial preoptic area (MPOA) that are specifically activated during male and female parenting, and a different subpopulation that is activated during mating. Genetic ablation of MPOA galanin neurons results in marked impairment of parental responses in males and females and affects male mating. Optogenetic activation of these neurons in virgin males suppresses inter-male and pup-directed aggression and induces pup grooming. Thus, MPOA galanin neurons emerge as an essential regulatory node of male and female parenting behaviour and other social responses. These results provide an entry point to a circuit-level dissection of parental behaviour and its modulation by social experience.
Normalizing neural responses by the sum of population activity allows the nervous system to adjust its sensitivity according to task demands, facilitating intensity-invariant information processing. In this issue of Neuron, two studies, Kato et al. (2013) and Miyamichi et al. (2013), suggest that parvalbumin-positive interneurons in the olfactory bulb play a role in this process.
Dopamine neurons are thought to promote learning by signaling prediction errors, that is, the difference between actual and expected outcomes. Whether these signals are sufficient for associative learning, however, remains untested. A recent study used optogenetics in a classic behavioral paradigm to confirm the role of dopamine prediction errors in learning.
Haddad R, Lanjuin A, Madisen L, Zeng H, Murthy VN, Uchida N.
Odor stimulation evokes complex spatiotemporal activity in the olfactory bulb, suggesting that both the identity of activated neurons and the timing of their activity convey information about odors. However, whether and how downstream neurons decipher these temporal patterns remains unknown. We addressed this question by measuring the spiking activity of downstream neurons while optogenetically stimulating two foci in the olfactory bulb with varying relative timing in mice. We found that the overall spike rates of piriform cortex neurons (PCNs) were sensitive to the relative timing of activation. Posterior PCNs showed higher sensitivity to relative input times than neurons in the anterior piriform cortex. In contrast, olfactory bulb neurons rarely showed such sensitivity. Thus, the brain can transform a relative time code in the periphery into a firing rate–based representation in central brain areas, providing evidence for the relevance of a relative time–based code in the olfactory bulb.
Decision making requires an actor to not only steer behavior toward specific goals but also determine the optimal vigor of performance. Current research and models have largely focused on the former problem of how actions are directed while overlooking the latter problem of how they are energized. Here we designed a self-paced decision-making paradigm, which showed that rats' performance vigor globally fluctuates with the net value of their options, suggesting that they maintain long-term estimates of the value of their current state. Lesions of the dorsomedial striatum (DMS) and, to a lesser degree, in the ventral striatum impaired such state-dependent modulation of vigor, rendering vigor to depend more exclusively on the outcomes of immediately preceding trials. The lesions, however, spared choice biases. Neuronal recordings showed that the DMS is enriched in net value–coding neurons. In sum, the DMS encodes one's net expected return, which drives the general motivation to perform.