Which of the following pairs occur in higher-order conditioning?

(v) Higher-order conditioning

Once a CS has been associated with a UCS and is capable of eliciting a reliable CR, that CS can then be used to reinforce other potential CSs. For instance, second-order conditioning can be demonstrated using the following procedure: a CS1 (e.g., a light) is paired with a UCS (e.g., food); then CS2 (e.g., a tone) is paired with CS1 (the light). This will usually result in a CR relevant to the original UCS (food) being evoked by CS2, even though CS2 has never been directly paired with food (e.g., Rescorla, 1980; Rizley & Rescorla, 1972). This phenomenon also has potential importance for conditioning models of psychopathology because it implies that emotional reactions can be acquired through higher-order conditioning in which the potentially phobic CS has never been paired directly with a traumatic UCS.

Olfactory Learning Limax

Olfactory learning in the terrestrial mollusk Limax maximus exhibits first-order classical conditioning as well as a variety of higher-order conditioning phenomena, such as US pre-exposure effects, second-order conditioning, and blocking (for review, see41). To examine neuronal and network processes that may underlie this repertoire of learning phenomena, Goel and Gelperin42 (see also43,44] constructed a neural network model (Figure 7.2A). Synaptic plasticity in the network is governed by a learning rule that embodied the concept of activity-dependent heterosynaptic facilitation (see Figure 7.3A and61,62). Specifically, activity in the facilitatory neuron (FN) strengthens all co-active synapses, and synaptic plasticity occurs only during FN activity. For example, co-activating the US with the CS1 (i.e., first-order classical conditioning) strengthens the CS1-to-motor neuron (MN) and CS1-to-FN connections. Strengthening these connections allows CS1 to acquire properties similar to US, such as eliciting the CR. Moreover, preconditioning CS1 subsequently allows this stimulus to reinforce CS2 activity during second-order conditioning. This simple network simulates (1) first-order conditioning, in which CS1 elicits the CR after training; (2) second-order conditioning, in which a previously conditioned CS1 reinforces a naive CS2 in the absence of a US; and (3) blocking, in which a preconditioned CS1 subsequently blocks conditioning of CS2 during pairing of a compound CS (CS1/CS2) with the US. This model of Limax learning illustrates that a single learning rule (e.g., heterosynaptic facilitation) can simulate multiple forms of learning when incorporated into an appropriate neural network (see also Figure 7.3).

Learning of Tentacle Position

Learned modifications of tentacle positioning have been used in H. aspersa to demonstrate blocking of conditioned tentacle lowering137 and conditioned inhibition, verified using both retardation and summation tests.138 Latent inhibition, second-order conditioning, and sensory preconditioning have also been demonstrated.139 The tentacle positioning system has also been used to demonstrate the rewarding properties of direct brain stimulation in Helix.140 Lowering of the tentacles is also used as an index of appetitive conditioning to food141 and can be dissociated from food finding after conditioning.142 Some elements of the motor system for tentacle positioning have been elucidated,143,144 including modulatory giant neurons in Achatina fulica145 and motoneurons in Ariolimax columbianus146,147 and Helix.148Tritonia peptide is widely distributed in central neurons of H. aspersa, including motoneurons sending processes to the tentacle retractor muscles,149 probably representing another example of a peptide co-transmitter co-localized in neurons liberating a classical neurotransmitter such as glutamate,150,151 acetylcholine,152,153 or serotonin.154

Second-Order Conditioning

Although the VET model has proven to be superior to the SBD model in providing flexibility in CR topography and timing, it has limitations of its own. The main limitation is that it cannot generate second-order conditioning. The VET model is incapable of generating second-order conditioning because, like the Rescorla-Wagner model, its learning rule assumes that learning occurs only on time steps or trials where a discrepancy exists between the magnitude of the US predicted by CS elements and the magnitude of the US as represented by the scalar value of the US, λ. In second-order conditioning, the US does not occur, so there is no mechanism that permits the formation and modification of connections between the second-order CS and the CR.

In contrast, Ẏ learning models such as SB are capable of generating second- order conditioning, so long as some portion of the second-order CS precedes the first-order CS (see Barto & Sutton, 1982, Figure 5, p. 230). If the would- be second-order CS occupies precisely the same time steps as the first-order CS, the SB model predicts blocking of conditioning to the would-be second- order CS. If the would-be second-order CS follows the first-order CS, its connection weights to the response become negative in value. The would-be second-order CS becomes a conditioned inhibitor. The TD model possesses these same attributes.

1.27.2.1.2 Reconsolidation in Limax flavus

The first study that found the reconsolidation phenomenon in an invertebrate was done in Limax flavus. In this study the authors focused on the temporal evolution of a memory (Sekiguchi et al., 1997). Although the authors did not term their findings reconsolidation, they revealed retrieval-dependent amnesia when trained snails were exposed to the CS and were cooled immediately afterward.

The combination of memory retrieval and cooling was applied at different time points after training, and the resulting memory was tested 1 day later. It turned out that by memory retrieval, a cooling-sensitive process can be induced until 3 days after training (Figure 3). After 3 days the memory becomes insensitive to retrieval-dependent amnesia. Nevertheless, when an additional CS–US pairing was presented before combining memory retrieval and cooling, a retrieval-dependent amnesia occurred even though the initial training had been applied more than 3 days before (Figure 4).

Interestingly, this induced susceptibility for retrieval-dependent amnesia followed the same temporal gradient as occurred after initial training, and it was supposed that the additional CS–US pairing results in a new memory with the same temporal gradient as the initial memory. To test this, a second-order conditioning trial, where a CS (CS 2) is paired with the formerly reinforced CS (CS 1), was presented instead of the additional CS–US pairing. The presentation of the CS1 in combination with cooling after the second-order conditioning resulted in a retrieval-dependent amnesia for the initial memory. But the presentation of the CS 2 in combination with cooling leads to retrieval-dependent amnesia for the initial memory and the second-order conditioned memory. Sekiguchi et al. (1997) concluded that the CS 1 in the second-order conditioning trial activates the initial memory. Accordingly, an additional CS–US pairing should also activate the initial memory rather than resulting in the formation of a new memory. According to this conclusion, Sekiguchi et al. (1997) stated that a memory becomes inactive after it matures and can no longer be inhibited by cooling. When the memory is retrieved, it becomes active again but might still be insensitive to cooling, depending on its age. Only additional CS–US pairings or second-order conditioning trials activate the initial memory that was insensitive to cooling, pushing it from a cooling-insensitive state back to a cooling-sensitive state. This model is based on Lewis’s theory of activated and inactivated memories (1979) (see section titled “Memory Consolidation after Training and Retrieval”) but extends beyond it. In contrast to Lewis (1979), Sekiguchi et al. (1997) posed an active and an inactive memory state, but in addition, showed a cooling-sensitive and a cooling-insensitive state. Accordingly, although memories are retrieved and are thus activated, they are not necessarily cooling sensitive. Only additional CS–US pairing or second-order conditioning leads to a memory that gets cooling sensitive by retrieval 24 h later. Interestingly, the findings by Sekiguchi et al. (1997) cannot be easily explained by the reconsolidation theory (Nader, 2003). The reconsolidation theory proposes a direct reactivation of a consolidated memory by a retrieval trial. Here, instead, the additional CS–US pairing enables a memory to be reactivated 24 h later.

4.4 Organization of fear memory associations

Pavlovian fear conditioning involves establishment of associations between neural representations of learning events (Konorski, 1967). The organization of associations developed during a learning experience puzzled early learning theorists (Hull, 1943; Konorski, 1967; Rizley & Rescorla, 1972). Typically, the strength of proposed associations is assessed using various post-training manipulations (Gewirtz & Davis, 2000). In this approach, especially second-order conditioning procedures, compared to first-order learning protocols, were found to be useful, offering a tool for analyzing the structure of associations and the number of possible associations (Gewirtz & Davis, 2000). In second-order conditioning, the CS2 may become associated with the CS1, forming a CS2–CS1 association. Alternatively, the CS2 may become associated with the US, establishing a CS2–US association, or with the fear response, producing a CS2–response memory. Manipulating the value of the CS memory reveals the structure of existing associations. For example, extinction of responding to CS1 resulting in attenuation of responding to CS2 suggests that during second-order learning, a CS2–CS1 association is developed (Gewirtz & Davis, 2000). In contrast, if extinction of responding to CS1 has no effect on memory for CS2, the existence of CS2–US or CS2–response associations would be more plausible (Rizley & Rescorla, 1972). Some authors have proposed to use this approach in analyzing the structure of Pavlovian fear conditioning in the amygdala (Gewirtz & Davis, 2000). We used this methodology, combining extinction procedures with reconsolidation protocols (Debiec et al., 2006, 2010; Diaz-Mataix et al., 2011; Doyere et al., 2007). We found that when two distinct auditory CSs are paired with the same US, exposure to one of these CSs followed by the intra-LA microinfusions of a reconsolidation blocker results in a selective disruption of responding to this CS (Debiec et al., 2010; Doyere et al., 2007). However, an exposure to the shared US followed by the disruption of reconsolidation affects responding to both CSs (Debiec et al., 2010). These findings suggest that in our protocol, each distinct CS has a distinct representation in the LA, although each of these representations is associated with a shared element (representation of the US). However, if these same distinct CSs are used in a second-order conditioning protocol, the post-CS1 disruption of reconsolidation in the LA or extinction of CS1 both affect freezing responding to the CS2 (Debiec et al., 2006). This demonstrates that the same representation of the auditory CS, depending on the learning conditions, forms distinct associations. In another series of experiments, we used two distinct auditory CSs, each paired with a distinct US (either electric foot- or eyelid shock) (Debiec et al., 2010; Diaz-Mataix et al., 2011). Using reconsolidation protocols, we found that exposure to one of the USs followed by the pharmacological disruption of reconsolidation in the LA selectively affects responding to the CS that was paired with this US, leaving responding to the other US intact (Debiec et al., 2010; Diaz-Mataix et al., 2011). These findings were paralleled by the extinction experiments as described in the previous section (Diaz-Mataix et al., 2011). US-selective character of reconsolidation processes suggests that the amygdala distinguishes between these USs and encodes their specific sensory values.

Our findings demonstrate that reconsolidation protocols in combination with extinction procedures provide a powerful tool to gain insights into the architecture of fear memories in the LA.

Olfactory Learning in Crickets

Crickets exhibited excellent olfactory learning. In appetitive olfactory conditioning, for example, one conditioning trial was sufficient to establish a memory lasting for several hours (midterm memory; Figure 41.2A).16 Two15 or four (Figure 41.2A16) conditioning trials (with a 5-min intertrial interval (ITI)) induced memory that lasted for at least 1 day, which reflects a protein synthesis-dependent long-term memory (LTM).17 After aversive olfactory conditioning, one trial was sufficient to establish 30-min retention, and six trials with a 5-min ITI were needed to establish 1-day retention.15 Our subsequent study demonstrated that the time course of memory retention after aversive conditioning and that after appetitive conditioning are fundamentally different in crickets.18

Next, we examined olfactory learning of crickets with respect to (1) durability of memory retention19; (2) capacity of memory storage20; and (3) higher order learning, namely context-dependent discrimination learning21 and second-order conditioning.22

First, we showed that crickets retain memory for life.19 Previously, convincing reports of lifetime memory retention in insects were limited to adults of honeybees.23 Third- or fourth-instar nymphal crickets were trained to associate one odor (CS+) with water and another odor (CS−) with sodium chloride solution. Six and 10 weeks after training, adult crickets exhibited higher preferences for CS+ compared to CS−. The learned preference was altered when they were given reversal training 6 weeks after training.

Next, we investigated whether crickets simultaneously memorize seven odor pairs.20 Fourteen odors were grouped into seven A/B pairs, and crickets in one group were subjected to differential conditioning to associate A odors with water (appetitive US) and B odors with sodium chloride solution (aversive US) for all seven pairs. Crickets in another group were trained with the opposite stimulus arrangement. Crickets in both groups exhibited significantly greater preference for the odors associated with appetitive US compared to the odors associated with aversive US for all seven odor pairs, demonstrating that crickets can retain memory of seven pairs of odors at the same time.

We then studied whether crickets select one of a pair of odors and avoid the other in one context and the opposite pairing in another context.21 One group of crickets received differential conditioning to associate one of a pair of odors (CS1) with water and another odor (CS2) with sodium chloride solution under illumination and to the reversed pair in the dark (CS1 with aversive US and CS2 with appetitive US). Another group of crickets received training of the opposite stimulus arrangement. One day after completion of the 3-day training, the former group preferred CS1 over CS2 under illumination but preferred CS2 over CS1 in the dark, and the latter group exhibited the opposite odor preference. Results of control experiments showed that background light conditions had no significant effects on memory formation or retrieval unless the background light was explicitly associated with US during training. We concluded that crickets use visual context stimuli to disambiguate the meaning of CSs and to predict USs.

Second-order conditioning is a procedure for testing whether a stimulus (CS1) can acquire the reinforcing properties of a US by conditioning with the US and whether the stimulus can support a new conditioning thereafter. In our experiment, an olfactory stimulus (CS1) was paired with water or sodium chloride solution (US), and then a visual stimulus (CS2) was paired with the CS1. Crickets exhibited significantly changed preference for the CS2 that had never been paired with the US, indicating that second-order conditioning was successful.22

Whether and to what extent insects perform social or observational learning are interesting topics in neurobiology and behavioral biology (see Chapters 29 and 40Chapter 29Chapter 40). Coolen et al.24 showed that wood crickets, Nemobius sylvestris, exhibit predator-avoidance behavior after having observed other crickets being attacked by a spider. This finding, as well as findings in other species of insects, including honeybee foragers transferring information about the location of a profitable food source to nestmates by waggle dances25 and foraging bumblebees copying flower choice of other individuals,8 raise the possibility that social learning is more widespread in insects than previously considered.

1.30.5.1 The Complexity of Molluscan Learning

Molluscan studies are focused on implicit forms of memory such as classical/operant conditioning and sensitization. Initially, simple forms of associative and nonassociative learning behavior were investigated. However, gastropod mollusks are capable of showing more complex types of associative learning behavior with features that are similar to those found in vertebrates (See Chapters 1.18, 1.36). For instance, differential conditioning has been described in a number of mollusks (Hawkins et al., 1983; Kemenes et al., 1989a; Jones et al., 2001; Inoue et al., 2006). In addition, second-order conditioning and blocking of aversive-odor conditioning has been demonstrated in Limax (Sahley et al., 1981b), and stimulus generalization, goal tracking, and context dependence (increased learning in a novel environment) were found in Lymnaea tactile conditioning (Kemenes and Benjamin, 1989a,b, 1994). The circuits underlying these behaviors are more complicated than those originally used for the study of reflexive defensive withdrawal responses and require the understanding of CPG and other interneuronal circuits mediating multimodality sensory responses. A key finding in these studies is that conditioning-induced synaptic and nonsynaptic changes occur at several sites within the same network (Benjamin et al., 2000; Crow, 2004; Baxter and Byrne, 2006; Kemenes et al., 2006; Straub et al., 2006). These include sensory neurons, modulatory and pattern-generating interneurons, and motor neurons. A future task will require us to understand how these various changes may be integrated to generate the final behavioral output.

Why Study Learning in Invertebrates?

Deciphering the neuronal mechanisms by which we acquire, store, and recall memories represents one of the major challenges for modern neuroscience. This topic is being pursued simultaneously in both vertebrate and invertebrate preparations. Given the indisputable relevance of studying learning mechanisms in vertebrates, it seems appropriate to consider why many investigators choose to work on invertebrates. First, invertebrates display many of the same types of learning enjoyed by vertebrates, including habituation, dishabituation, sensitization, classical conditioning, second order conditioning, sensory preconditioning, latent inhibition, overshadowing, blocking, context conditioning and, operant conditioning. Second, the synaptic plasticity mechanisms that appear to encode learning turn out to be highly conserved. Thus, invertebrates have been shown to have synapses that undergo the same types of experience-dependent plasticity studied in vertebrates, including paired-pulse facilitation, homosynaptic depression, presynaptic inhibition, presynaptic facilitation, posttetanic potentiation, long-term potentiation, and long-term depression. Third, molecular mechanisms also appear to be conserved, including the involvement of particular second messenger systems in short-term memory and the requirement for gene expression for the formation of stable, long-term memories. Finally, a key reason for studying learning mechanisms in invertebrates is their seemingly made-for-science nervous systems. For example, invertebrate neurons are larger (up to 1 mm in diameter) and far fewer in number than those of vertebrates and are conveniently located on the outside of their ganglia, where they can be seen and penetrated with electrodes or observed with optical recording techniques. Many of these neurons can be recognized as unique individuals, making it possible to map out discrete circuits and to return to the same neurons and synapses in successive preparations in order to characterize their role in information storage. Furthermore, because these neurons and circuits can often be studied in situ, the specific behavioral function of these synapses, and thus the information they encode, can often be determined, a feat seldom possible in vertebrates.

5.4.1 Reconsolidation and memory updating

At least two forms of memory updating have been investigated to date: the linking of novel, distinct information to an old memory trace and the addition of information regarding the same experience (i.e., incremental learning). Although both can be defined as memory updating, these processes are likely controlled by very different underlying mechanisms.

With respect to the first form of memory updating, that of incorporation of new information into an existing trace, in our laboratory, Tronel et al. (2005) found that second-order conditioning—that is, the association of a novel and distinct conditioned stimulus (CS2) to a previously formed association (CS1–US)—is mediated by consolidation mechanisms. This conclusion was reached using a molecular requirement that doubly dissociates consolidation from reconsolidation of IA memory in rats, namely that C/EBPβ is required in the dorsal hippocampus for consolidation but not reconsolidation, whereas C/EBPβ is required in the basolateral amygdala for reconsolidation but not consolidation.

Specifically, inhibition of C/EBPβ in the amygdala after reactivation of IA memory (CS1–US) in a new context (i.e., CS2) did not prevent the formation of the CS2–(CS1–US) association through second-order conditioning because rats subsequently tested in this new context showed clear memory. However, as would be predicted, inhibition of C/EBPβ in the amygdala interfered with reconsolidation of the first trace (i.e., CS1–US) because rats no longer showed memory for the original context (Figure 5.5). In contrast, second-order conditioning was prevented by inhibition of C/EBPβ in the dorsal hippocampus, suggesting that this memory underwent a new process of consolidation. Disruption of C/EBPβ in the hippocampus did not affect the reconsolidation of the original memory (Figure 5.5). Together, these data indicate that reconsolidation does not mediate memory updating through second-order conditioning but, rather, that updating engages a new consolidation process. In fact, disrupting the reconsolidation of the original CS1–US memory does not affect the new CS2–(CS1–US) association, supporting the conclusion that the two memories exist completely independent of each other.

Our findings that memory updating requires consolidation, for the purpose of adding new information to an existing trace, suggest that this form of updating is not a primary function of reconsolidation. In fact, given that memories are continuously updated throughout the life span, how could a reconsolidation-based mechanism that is temporally restricted and mainly occurs when a memory is recent be the process that mediates all updating of old memories? New information is constantly integrated into a network of memory traces, and this seems to occur via new consolidation processes. We speculate that for very long-lasting memories, it would be less adaptive that the updating of an existing memory would transform the original memory into a new one, whereas it seems more advantageous that to provide behavioral adaptability and choices, the old memories continue to coexist with new memories that comprise both old and new linked information. An example of this situation is extinction, whereby a new extinction memory trace coexists with the original conditioning trace. Memory updating, consolidation, and reconsolidation are all important pieces of the dynamic memory trace and storage, and understanding these processes and how they evolve over time is key for developing effective treatments of psychopathologies, whether they are pharmacological, behavioral, or the combination of approaches. This knowledge will also be critical for developing more effective psychotherapeutic treatments (see Chapter 14).

Data from other memory paradigms and other model organisms support the conclusion that integrating new information occurs through consolidation and that two related traces can continue to exist in parallel after updating. For example, Debiec et al. (2006), using rat auditory fear conditioning, found that disrupting the reconsolidation of a CS1–US memory by inhibiting amygdala protein synthesis after reactivation does not affect the formation of a new, related second-order association (i.e., CS2–CS1–US). Similar results were also found with associative memories in the crab Chasmagnathus (Suárez, Smal, & Delorenzi, 2010), as well as in humans (Forcato, Rodríguez, Pedreira, & Maldonado, 2010). Hence, we conclude that the primary function of reconsolidation must be different from that of linking an existing, reactivated memory to a novel, distinct experience. In general terms, we can infer that memory updating via formation of complex networks requires memory reactivation but not reconsolidation, and that reconsolidation of a reactivated memory does not alter the entire network of updated associative memories.

Other studies have tested whether reconsolidation mediates memory updating by examining the contribution of the repetition of similar training trials compared to that of encoding new information. These studies concluded that with multiple learning trials or reactivations, memory does indeed become labile. However, the fragility is seen only when learning is in a non-asymptotic mode. Specifically, inhibition of protein synthesis after repeated training trials during a non-asymptotic phase of learning returns memory performance to a pretraining chance level, suggesting that the original trace had remained labile. This effect has been shown using spatial learning (Meiri & Rosenblum, 1998; Touzani, Puthanveettil, & Kandel, 2007) and motor learning (Luft, Buitrago, Kaelin-Lang, Dichgans, & Schulz, 2004; Luft, Buitrago, Ringer, Dichgans, & Schulz, 2004). In contrast, when performance on a task has reached ceiling levels, and memory has therefore reached an asymptotic phase, memory is resistant to disruption by post-reactivation amnesic treatments (Morris et al., 2006; Rodriguez-Ortiz, De la Cruz, Gutiérrez, & Bermudez-Rattoni, 2005; Winters, Tucci, & DaCosta-Furtado, 2009).

Interestingly, memory becomes labile once again if conditions are adjusted to induce a new phase of encoding. For example, using a delayed matching-to-place task in rats requiring that they locate a platform in a water maze, Morris et al. (2006) found that reconsolidation of a well-trained spatial memory is engaged only when there is a shift in location of the escape platform, representing a mismatch in contextual information that triggers updating of their cognitive representation of space. With a combination of taste recognition and taste aversion learning in rats, Rodriguez-Ortiz et al. (2005) found that inhibition of protein synthesis in the gustatory cortex disrupted a well-trained memory for saccharin taste only when malaise was introduced in conjunction with reactivation, another situation in which a mismatch between prior and current conditions triggers updating of the memory for familiar taste. Winters et al. (2009) noted a similar effect using spontaneous object recognition in rats: Inhibition of NMDA receptor function interfered with expression of the original object memory when reactivation introduced new contextual information but not when reactivation was identical to the initial training sessions.

As in studies discussed previously in relation to memory updating, it is unclear whether disruption of the memory occurs, because during reactivation the rat associates old information (e.g., the water maze, saccharin taste, or objects) with novel information (e.g., a new platform location, malaise, or new contextual cues), and the interference disrupts the consolidation of a new memory trace containing information about the contextual mismatch rather than the reconsolidation of the original trace. Much like the case for extinction learning in conditioning paradigms, two memory traces could potentially coexist and both contribute to the expression of behavior in the tasks described previously. Also, because both consolidation and reconsolidation are sensitive to similar forms of molecular disruption, using an interference approach that does not doubly dissociate the two processes cannot irrefutably demonstrate that reconsolidation mediates this type of updating.

Thus, an alternative interpretation from that offered by authors who suggest that reconsolidation mediates memory updating when novel information is linked to an existing trace is twofold. First, in agreement with the authors’ conclusions, during a learning curve, post-trial application of amnesic treatments disrupts memory retention only when the memory is not at an asymptotic level. However, when retention has reached an asymptotic level and no further learning or increased retention is evident, a previously consolidated memory remains stable and resistant to disruption. Second, if new events are then presented and associated with this memory, a new trace is formed that is labile because it undergoes consolidation. If part of the old information is incorporated in a new trace, its retention might be disrupted if the new trace is challenged by amnesic treatments, following the rules of the predominant active trace (Eisenberg et al., 2003).

Hence, we speculate that for memories that undergo system consolidation through the medial temporal lobe, reconsolidation is engaged only to change the strength of the original memory; in other words, reactivation of the memory strengthens or further consolidates the original memory without changing its content.