Which approach to motivation explains behavior in terms of external stimuli and their rewarding properties?

Motivated behaviors are fundamental components of vertebrate life. These behaviors control an animal's interactions with those goal objects in the environment that are important for the survival of the individual and the species; for example, food, water, or conspecifics. This article will emphasize that motivated behaviors are generated by the complex interactions of neural systems distributed throughout the brain. Pre-eminent among these are the cerebral hemispheres, the hypothalamus, and hindbrain. By means of complex sets of neural connections, these regions process a wide variety of sonsory inputs, control arousal state, and organize the different motor components that make up motivated behaviors.

Conclusions

Motivated behavior arises through expectation of reward or avoidance of punishment. Many factors contribute to the computation of the current value of a particular state. These include whether a reward will be delivered, how soon and how desirable a reward that might be delivered will be, and how aversive the effort to gain the reward is likely to be. Over the past 30 years some theoretical constructs have been developed that provide structures for organizing and evaluating results from behavioral, pharmacological, and physiological experiments. There seem to be connections between these biological experiments and the factors that these theories demonstrate are needed to influence motivation. The next few years will provide the data to refine the theories and help choose among them.

Abstract

Motivated behavior is considered to be a product of integration of a behavior's subjective benefits and costs. As such, it is unclear what motivates “habitual behavior” which occurs, by definition, after the outcome's value has diminished. One possible answer is that habitual behavior continues to be selected due to its “intrinsic” worth. Such an explanation, however, highlights the need to specify the motivational system for which the behavior has intrinsic worth. Another key question is how does an activity attain such intrinsically rewarding properties. In an attempt to answer both questions, we suggest that habitual behavior is motivated by the influence it brings over the environment—by the control motivation system, including “control feedback.” Thus, when referring to intrinsic worth, we refer to a representation of an activity that has been reinforced due to it being effective in controlling the environment, managing to make something happen. As an answer to when does an activity attain such rewarding properties, we propose that this occurs when the estimated instrumental outcome expectancy of an activity is positive, but the precision of this expectancy is low. This lack of precision overcomes the chronic dominance of outcome feedback over control feedback in determining action selection by increasing the relative weight of the control feedback. Such a state of affairs will lead to repeated selection of control relevant behavior and entails insensitivity to outcome devaluation, thereby producing a habit.

Publisher Summary

Specific motivated behaviors are linked to states of CNS arousal, which are, in turn, fueled by a primitive, fundamental, generalized CNS arousal force. With respect to drive reduction and need reduction theories of reward, this chapter focuses on what causes increased drive, following increased bodily needs, to begin with. Ascending neuroanatomical systems supporting generalized CNS arousal are well known, and include noradrenergic (NA), dopaminergic, histaminergic, serotonergic, and cholinergic systems, augmented by the actions of orexin/hypocretin. For both females and males, hormonal “drives” on sexual behavior are not limited to consummatory responses themselves. Animals show increased motivation, as proven by their courtship behaviors, conditioned place preferences, willingness to learn arbitrarily chosen operants, and to suffer pain in order to gain access to the mating partner. Sex drive is discussed not only in the context of the performance of mating behaviors, but also as a product of more fundamental, primitive CNS arousal processes. These have been conceived as a “generalized arousal” force that depends on dopaminergic systems as well as several other transmitters and peptides. How generalized arousal transmitters affect ventromedial hypothalamic neurons that serve sexual arousal is studied. Furthermore, the chapter reviews the generation of sexual arousal at three different levels of the neuraxis.

Introduction

Feeding is a motivated behavior necessary for survival. It is comprised of food seeking and consumption and these behaviors are coordinated with accompanying cognitive and bodily (metabolic, autonomic, and visceral) responses. The control of feeding behavior therefore necessitates an integrated network across behavioral, cognitive, and physiological neural systems (Swanson, 2000). That complex neural network mediates the expression of feeding behavior via innate and learned mechanisms, in response to internal and external signals—physiological signals related to energy and nutrient needs and environmental signals that function through hedonic and cognitive processes to drive appetite and eating.

Innate, hard-wired mechanisms can overtake the control of feeding behavior when a fast (reflex-type) response is required, for instance, when an organism's survival is threatened. Under most other circumstances a more complex process underlies the control of feeding behavior. That process necessitates continuous assessments of available energy and nutrients against anticipated usage and gains, which require learning and memory and other cognitive processes (e.g., planning, decision-making) to guide feeding behavior. Learning and memory provide adaptive control of feeding behavior that is important in health and disease and recent research has begun deciphering how these cognitive processes are integrated within the feeding neural network.

A main node of the feeding network, the lateral hypothalamus (lateral hypothalamic area (LHA); Swanson, 2004) was recently proposed to serve as a cognition-motivation interface in the control of feeding behavior (Petrovich, 2018) and the neuropeptides orexins/hypocretins are among key LHA substrates mediating this function. This chapter will review the evidence for a causal function of the orexin system in the control of feeding behavior by learned cues—a model of cognitive motivation to eat.

Understanding how reward-motivated behavior is affected by delay of reinforcement is an important goal in behavioral neuroscience. Recent progress in this area has been helped by theoretical and methodological advances in the experimental analysis of behavior; in particular, new models of intertemporal choice can overcome the perennial difficulty of discriminating between the effects of neurobiological interventions on sensitivity to delay of reinforcement and reinforcer magnitude. Central 5-hydroxytryptamine depletion has been associated with increases in the rate of delay discounting in some studies but not in others. Psychostimulants increase delay discounting, although this effect can be counteracted by these drugs’ ability to enhance the rewarding value of intradelay stimuli. Evidence is emerging that the nucleus accumbens and the orbital prefrontal cortex contribute to the control of behavior by delayed reinforcement.

Motivational Considerations

SDT suggests that all motivated behaviors can be classified along a continuum from motivations or behavioral regulations that are more autonomous and integrated to the self, to those that are more controlled or experienced as external to the self. There are five points along the continuum of motivated behaviors, the first four of which are extrinsically motivated. External regulation is the least autonomous of these, and indicates behaviors that are enacted purely to gain externally controlled rewards or avoid punishments. Introjected regulation represents an external regulation that’s been partially internalized, meaning that the external contingencies are no longer present, but taking their place is a framework of contingent self-worth and shame- or guilt-avoidance that is present within the individual. A major form of introjected regulation is represented by ego involvement (Ryan, 1982), in which a person’s behavior is driven by self-esteem-related concerns. Identified regulation is yet more autonomous, characterizing actions that are engaged because they are consciously valued and endorsed, but not necessarily integrated with the larger ecosystem of values and beliefs within an individual. The final type of extrinsic regulation is integrated regulation, which describes behaviors that are brought into cohesion, or reciprocally assimilated, with the individual’s other values and beliefs. Integrated regulation is still extrinsically motivated, in that the intent is to attain outcomes independent of the satisfactions inherent in acting, but it is nonetheless highly volitional and autonomous, being wholeheartedly endorsed by the individual. The fifth type of motivation, intrinsic motivation, entails behaving because of the activity’s inherent enjoyment and satisfactions. It is this criterion that separates intrinsically motivated behaviors from those that are extrinsically motivated.

Because SDT is focused on the content of life leading to wellness, it thus follows that engaging in behaviors, identities, vocations, and avocations autonomously, rather than heteronomously, is expected to lead to greater basic psychological need satisfaction, and thus consequent well-being and happiness. Indeed, autonomous motivation has been linked to many positive outcomes. For example, motivation at work has been identified as an important predictor of outcomes. Ilardi, Leone, Kasser, and Ryan (1993) examined the role of motivation in the workplace on employees’ satisfaction with work and general well-being. To the extent that employees were autonomously motivated, they evidenced greater need satisfaction, greater satisfaction with work, and higher well-being, findings that have since been widely replicated (Gagné, Deci, & Ryan, 2013; Van den Broeck, Lens, De Witte, & Coillie, 2013).

In line with a eudaimonic perspective, when people direct their autonomous activities toward meaningful and valued endeavors, they especially derive basic need satisfactions. For example, in a set of studies, Weinstein and Ryan (2010) examined the role of helpers’ autonomous motivation in fostering both their own and recipients’ well-being and positive experience. An initial diary method assessed individuals’ engagement in prosocial behavior. On average, helping others was related to small increases in subjective well-being, vitality, and self-esteem. However, when proscocial behavior was autonomously motivated, these outcomes were significantly enhanced. In addition, the three basic psychological needs were shown to mediate between autonomous helping and these enhanced well-being outcomes. Another study provided an experimental analogue, showing that participants given a choice to donate to others experienced more positive affect and vitality than those instructed to do so. Once again, the three basic psychological needs mediated the relation between autonomous helping and changes in well-being. Additional experiments in this series showed further that only when helping was enacted autonomously did recipients of help show well-being benefits. That is, when recipients felt they were being willingly helped, both their wellness and that of the helper were enhanced. It seems clear that in the realm of prosocial behavior, motivation and need satisfaction are critical predictors in the happiness and well-being experienced during these activities.

4.1 Internalization of Extrinsic Motivation

Within SDT, extrinsically motivated behaviors can become more autonomous through a process in which people internalize the values and regulations associated with the behaviors. Internalization is the process of taking in a value or regulation and making it one’s own. When the process functions optimally, individuals will have transformed an externally regulated extrinsic motivation into an internally regulated extrinsic motivation by integrating the regulation and its value into their sense of self.

Internalization has long been an important concept in many psychological theories. The idea is that the challenge of becoming self-regulating of activities that are important for people but are not themselves personally compelling can be met through the process of internalization. To be self-regulating, people must make internal what was initially external. The thing that makes SDT’s conceptualization of internalization different from most others is that it includes different types or degrees of internalization. In other words, SDT proposes that regulations can be internalized more or less fully, such that people will, to differing degrees, accept the regulations as personally important for themselves and, thus, be more or less autonomous in enacting them. As such, there will be differences in the extent to which people do a behavior because they want to rather than because they believe that they have to, and this will reflect the degree to which the regulation has been internalized.

Thus, SDT views internalization in terms of a continuum that describes how fully the person has transformed an external prompt into an internal regulation that will allow volitional or “choiceful” behavior. This continuum, therefore, illustrates how instrumental (i.e., extrinsic) motivation for a behavior can range from passive compliance to active personal commitment.

One reason why this relative autonomy continuum is so important is that it addresses an important problem about internal regulation of behavior. Specifically, there are behaviors that people force themselves to do because they think they should do them and know they will feel guilty if they do not. The regulation of these behaviors is certainly internal to these people, but it does not exhibit the qualities of volition or autonomy that, for example, are so evident in intrinsic motivation. SDT uses the concept of introjection to refer to the type of internalization that leads to this internally pressured regulation. In contrast, there are behaviors that people do not find interesting but that have become meaningful and important for their own self-selected goals and personal life plans. As such, people do these behaviors quite autonomously, even though the behaviors themselves are not inherently satisfying. In SDT, this is said to occur as people identify with the importance of the activities for themselves and then integrate that identification with other aspects of themselves. When this has occurred, people will feel fully autonomous as they enact an extrinsically motivated behavior because it would then emanate from their integrated sense of self.

The former example, of introjection, is about controlling oneself, and the process bears considerable similarity to being controlled by other people. The latter example, in contrast, is more about making a choice to do the activity because, all things considered, doing so feels desirable and right. Thus, in the latter case, people experience a truly internal PLOC, even though the behavior is still instrumental, that is, done for reasons other than the enjoyment of the activity itself. This important contrast, between what might be thought of as self-control and self-regulation, is highlighted in the SDT model of extrinsic motivation.

More specifically, the theory states that there are various forms of extrinsic motivation: one where the regulation is external to the person, one where the regulation has been merely introjected and so must be buttressed by internal reward or punishment (e.g., guilt) contingencies, one where the person has identified with the importance of the behavior for himself or herself, and one where the regulation has been fully integrated with other aspects of the person’s self. Although the latter three of these all represent internal motivation, they vary in the degree to which the behaviors they motivate are autonomous. Integrated regulation represents a relatively full sense of volition and personal commitment, with introjection representing a relative lack of these qualities (Fig. 1).

Intrinsic motivation and fully integrated extrinsic motivation are the two bases for autonomous or self-determined behaviors. More than three decades of research have now shown that the quality of people’s experience and performance vary as a function of the degree to which a behavior is autonomous or self-determined. Thus, when people have identified with and integrated the regulation of an extrinsically motivated behavior, the behavior shares many of the qualities of behaviors that are intrinsically motivated. Not only do people feel a sense of choice and experience an internal PLOC, but more importantly, autonomous extrinsic motivation, like intrinsic motivation, is positively related to psychological well-being as well as to learning outcomes and effective performance, especially on activities that require a deeper or fuller engagement with the activity.

Internalization of Extrinsic Motivation

Within SDT, extrinsically motivated behaviors can become more autonomous through a process in which people internalize the values and regulations associated with the behaviors. Internalization is the process of taking in a value or regulation and making it one's own. When the process functions optimally, individuals will have transformed an externally regulated extrinsic motivation into an internally regulated extrinsic motivation by integrating the regulation and its value into their sense of self.

Internalization has long been an important concept in many psychological theories. The idea is that the challenge of becoming self-regulating of activities that are important for people but are not themselves personally compelling can be met through the process of internalization. To be self-regulating, people must make internal what was initially external. The thing that makes self-determination theory's conceptualization of internalization different from most others' is that it includes different types or degrees of internalization. In other words, SDT proposes that regulations can be internalized more versus less fully, such that people will, to differing degrees, accept the regulations as personally important for themselves and thus be more versus less autonomous in enacting them. As such, there will be differences in the extent to which people do a behavior because they want to rather than because they feel that they have to, and this will reflect the degree to which the regulation has been internalized.

SDT thus views internalization in terms of a continuum that describes how fully the person has transformed an external prompt into an internal regulation that will allow volitional or “choiceful” behavior. This continuum, therefore, illustrates how extrinsic (ie, instrumental) motivation for a behavior can range from passive compliance to active personal commitment.

One reason this relative-autonomy continuum is so important is that it addresses an important problem about internal regulation of behavior. Specifically, there are behaviors that people force themselves to do because they think they should do them and they know they will feel guilty if they do not. The regulation of these behaviors is certainly internal to the people, but it does not exhibit the qualities of volition or autonomy that, for example, are so evident in intrinsic motivation. SDT uses the concept of introjection to refer to the type of internalization that leads to this internally pressured regulation. In contrast, there are behaviors that people do not find interesting but that have become meaningful and important for their own self-selected goals, desires, and personal life plans. As such, people do these behaviors quite autonomously even though the behaviors themselves are not inherently satisfying. In SDT, this is said to occur as people identify with the importance of the activity for themselves and then integrate that identification with other aspects of themselves. When this has occurred, people will feel fully autonomous as they enact an extrinsically motivated behavior, for it would then emanate from their integrated sense of self.

The former example, of introjection, is about controlling oneself, and the process bears considerable similarity to being controlled by other people. The latter example, in contrast, is more about making a choice to do the activity because, all things considered, it feels desirable and right. Thus, in the latter case, people experience a truly internal PLOC even though the behavior is still instrumental—that is, done for some reasons other than the enjoyment of the activity itself. This important contrast—between what might be thought of as self-control versus self-regulation—is highlighted in the SDT model of extrinsic motivation.

More specifically, the theory states that there are different forms of extrinsic motivation: one in which the regulation is external to the person; one in which the regulation has been merely introjected and thus must be buttressed by internal reward or punishment contingencies such as guilt and contingent self-esteem; one in which the person has identified with the importance of the behavior for himself or herself; and, finally, one in which the regulation has been fully integrated with other aspects of the person's self. Although the last three all represent internal motivation, they vary in the degree to which the behaviors they motivate are autonomous. Integrated regulation represents a relatively full sense of autonomy, volition, and personal commitment, with introjection representing a relative lack of these qualities (Fig. 1).

Intrinsic motivation and fully integrated extrinsic motivation are the two bases for highly autonomous or self-determined behaviors. More than three decades of research has now shown that the quality of people's experience and performance vary as a function of the degree to which a behavior is autonomous or self-determined. Thus, when people have identified with and integrated the regulation of an extrinsically motivated behavior, the behavior shares many of the qualities of behaviors that are intrinsically motivated. Not only do people feel a sense of choice and experience an internal PLOC, but more importantly, autonomous extrinsic motivation, like intrinsic motivation, is positively related to psychological well-being, as well as to learning outcomes and effective performance especially on activities that require a deeper or fuller engagement with the activity.

2.1 Dopaminergic Deficits in Nonhuman Animals Modulate Reward and Effort Sensitivity

A key feature of motivated behavior is that it requires one to decide whether to embark on a course of action for a given reward given the associated costs (Chong et al., 2016). Thus, motivation requires an animal to be sensitive to the rewards on offer for its actions (“reward sensitivity”), as well as the costs associated with it, such as the effort required to obtain it (“effort sensitivity”). This cost–benefit computation is thought to be underpinned by a distributed network of brain areas, including the ventral striatum, ventral pallidum, medial prefrontal and anterior cingulate cortices (ACC), and basolateral amygdala (Fig. 1; Farrar et al., 2008; Floresco and Ghods-Sharifi, 2007; Hauber and Sommer, 2009; Walton et al., 2003). The core of this network is composed of reciprocal connections between the basal ganglia and prefrontal cortex, particularly the dopaminergic mesocorticolimbic and nigrostriatal pathways (Levy, 2012).

The mesocorticolimbic dopamine system is considered to be central to the brain's reward and motivational circuitry (Robbins and Everitt, 2006; Salamone et al., 2006). It projects from the ventral tegmental area of the midbrain to a widespread area of cortical and subcortical regions, including the ventral striatum/nucleus accumbens (NAcc), medial prefrontal areas, the amygdala, and the hippocampus (Fig. 1). The NAcc is a subcortical structure comprising an inner core and outer shell, with the shell being one of the major projection areas of mesolimbic dopamine neurons, which also receives important connections from the hippocampus and amygdala (Sokoloff et al., 2006). Dopamine released in the NAcc is thought to play a central role in effort-based decisions, and some have proposed that the NAcc plays a critical role as a “limbic–motor interface” to translate motivation into action (Mogenson et al., 1980). The prefrontal cortex, in particular ventromedial prefrontal areas and the ACC, is functionally interconnected with basal ganglia and limbic structures through different circuits (Mega and Cummings, 1994) and plays a critical role in reward processing, initiation, planning, and monitoring of goal-directed behavior (Fuster, 2008).

There is extensive evidence that a dopaminergic deficit, or selective lesions to the mesocorticolimbic system, results in less-motivated behavior, which resembles the behavior of patients with apathy. For example, typical studies in rodents require the animal to decide how much effort it is willing to invest for various rewards. Such paradigms may include operant conditioning tasks or dual alternative, effort discounting tasks (Chong et al., 2016). These tasks are able to quantify motivation in terms of the animal's sensitivity to available rewards—that is, how much reward is required to incentivize it to act. They are also able to quantify the animal's sensitivity to effort costs—that is, how much effort it is willing to exert to obtain those rewards. The animal's reward and effort sensitivities can then be compared before and after lesions to the mesocorticolimbic system. Typically, dopamine transmission is disrupted through systemic administration of low doses of dopamine antagonists, or selective dopamine depletion or antagonism (eg, with 6-hydroxy-dopamine, SCC 23390, ecopipam, haloperidol, flupenthixol; Salamone and Correa, 2012).

A vast volume of literature has been built on this approach, and the results from many of these tasks are strikingly similar. The overall pattern is that disrupting dopamine transmission reduces an animal's sensitivity to reward and increases its sensitivity to effort. Thus, it will require greater rewards to incentivize it to act, and it will be willing to invest less effort for given rewards. This is a consistent finding across a range of paradigms and can occur in the context of systemic dopaminergic depletion, disruption of dopaminergic input to the basal ganglia and/or frontal lobes, or from selective antagonism of basal ganglia and cortical dopamine receptors (Cousins and Salamone, 1994; Cummings, 1993; Denk et al., 2005; Farrar et al., 2010; Floresco et al., 2008; Hauber and Sommer, 2009; Mai et al., 2012; Nowend et al., 2001; Nunes et al., 2010, 2013; Pardo et al., 2012; Randall et al., 2012; Salamone and Correa, 2012; Salamone et al., 1991, 1994, 2003, 2007; Schweimer and Hauber, 2006; Sink et al., 2008; Walton et al., 2005). Together, this literature highlights the importance of the reward- and effort-related functions of dopaminergic systems.