Classical conditioning

File:MonumentIPAVLOV.jpg

A statue of Ivan Pavlov and one of his dogs.

Classical conditioning (also Pavlovian conditioning or respondent conditioning) is a form of learning in which one stimulus, the conditioned stimulus or CS, comes to signal the occurrence of a second stimulus, the unconditioned stimulus or US. The US is usually a biologically significant stimulus such as food or pain that elicits a response from the start; this is called the unconditioned response or UR. The CS usually produces no particular response at first, but after conditioning it elicits the conditioned response or CR. Classical conditioning differs from operant or instrumental conditioning, in which behavior emitted by the organism is strengthened or weakened by its consequences (e.g. reward or punishment).

Conditioning is usually done by pairing the two stimuli, as in Pavlov’s classic experiments. Pavlov presented dogs with a ringing bell (CS) followed by food (US). The food (US) elicited salivation (UR), and after repeated bell-food pairings the bell also caused the dogs to salivate (CR).

It used to be thought that the basic process in classical conditioning is that the conditioned stimulus becomes associated with, and elicits, the unconditioned response. However, many observations are inconsistent with this view. For example, the conditioned response is often quite different than the unconditioned response. Learning theorists now commonly say that the conditioned stimulus comes to signal or predict the unconditioned stimulus. Robert A. Rescorla provided clear summary of this change in thinking, and its consequences, in his article “Pavlovian conditioning: It's not what you think it is.”

Origins

I. P. Pavlov provided the most famous example of classical conditioning, though E. B. Twitmyer published his findings a year earlier (a case of simultaneous discovery). During his research on the physiology of digestion in dogs, Pavlov noticed that, rather than simply salivating in the presence of food, the dogs began to salivate in the presence of the lab technician who normally fed them. Pavlov called this anticipatory salivation psychic secretion. From this observation he predicted that, if a particular stimulus in the dog's surroundings was present when the dog was given food, then this stimulus would become associated with food and cause salivation on its own. In his initial experiment, Pavlov used a bell to call the dogs to their food and, after a few repetitions, the dogs started to salivate in response to the bell. Pavlov called the bell the conditioned (or conditional) stimulus (CS) because its effect depended on its association with food. He called the food the unconditioned stimulus (US) because its effect did not depend on previous experience. Likewise, the response to the CS was the conditioned response (CR) and that to the US was the unconditioned response (UR). The timing between the presentation of the CS and US is integral to facilitating the conditioned response. Pavlov found that the shorter the interval between the bell's ring and the appearance of the food, the more quickly the dog learned the conditioned response and the stronger it was.

As noted earlier, it is often thought that the conditioned response is a replica of the unconditioned response, but even Pavlov noted that saliva produced by the CS differs in composition from that produced by the US. In fact, the CR may be any new response to the previously neutral CS that can be clearly linked to experience with the conditional relationship of CS and US. It was also thought that repeated pairings are necessary for conditioning to emerge, however many CRs can be learned with a single trial as in fear conditioning and taste aversion learning.

Diagram representing forward conditioning. The time interval increases from left to right.

Classical conditioning procedures

Forward conditioning

Learning is fastest in forward conditioning. During forward conditioning the onset of the CS precedes the onset of the US in order to signal that the US will follow. Two common forms of forward conditioning are delay and trace conditioning.

• Delay conditioning: In delay conditioning the CS is presented and is overlapped by the presentation of the US.
• Trace conditioning: During trace conditioning the CS and US do not overlap. Instead, the CS is presented, a period of time is allowed to elapse during which no stimuli are presented, and then the US is presented. The stimulus-free period is called the trace interval. It may also be called the conditioning interval.

Simultaneous conditioning

During simultaneous conditioning, the conditioned stimulus and unconditioned stimulus are presented and terminated at the same time.

Second-order conditioning

Main article: Second-order conditioning

This form of conditioning follows a two-step procedure. First a neutral stimulus (“CS1”) comes to signal a US through forward conditioning. Then a second neutral stimulus (“CS2”) is paired with the first (CS2-CS1) and comes to yield its own conditioned response. For example, a bell (CS1) might paired with food (US) until the bell elicits salivation. If a light is then paired with the bell the light also may come to elicit salivation.

Backward conditioning

Backward conditioning occurs when a conditional stimulus immediately follows an unconditional stimulus. Unlike traditional conditioning models, in which the conditional stimulus precedes the unconditional stimulus, the conditional response tends to be inhibitory. This is because the conditional stimulus serves as a signal that the unconditional stimulus has ended, rather than as a signal that the unconditioned stimulus is about to appear.

Temporal conditioning

When a US is presented at a number of regularly timed intervals a CR often develops shortly before the US is scheduled to appear. This suggests that animals have a biological clock that can serve as a CS. This method has also been used to study timing ability in animals. (see Animal cognition).

Zero contingency procedure

In this procedure, the CS is paired with the US as usual, but the US also occurs at other times, so that the US is just as likely to happen in the absence of the CS as it is following the CS. Here, in other words, the CS does not "predict" the US. In this case, conditioning fails: the CS does not come to elicit a CR. This finding - that prediction rather than CS-US pairing is the key to conditioning - greatly influenced subsequent conditioning research and theory.

Extinction

Main article: Extinction (psychology)

Extinction refers to the repeated presentation of a CS in the absence of a US. This is done after a CS has been conditioned by one of the methods above. When this is done the CR frequency eventually returns to pre-training levels. However, spontaneous recovery (and the related phenomena, see "Recovery from extinction" below) show that extinction does not completely eliminate the effects of the prior conditioning.

Phenomena observed in classical conditioning

Acquisition

During acquisition the CS and US are paired in one of the ways described above. The extent of conditioning may be tracked by test trials in which the CS is presented alone and the CR is measured. A single CS-US pairing may suffice to yield a CR on test, but usually a number of pairings are necessary, during which the strength and/or frequency of the CR gradually increases. The speed of conditioning depends on a number of factors, including the nature and strength of the CS and US, the animal's motivational state, and previous experience

Extinction

The procedure of presenting a CS alone, without the US, is called extinction. When this is done often enough the CS ordinarily stops eliciting a CR; that is, it has been extinguished.

External inhibition

External inhbition may be observed if a strong or unfamiliar stimulus is presented just before the CS. This causes a reduction in the conditioned response to the CS.

Recovery from extinction

Several procedures can cause an extinguished CS to once again trigger a CR. The outcomes of these procedures suggest that extinction does not entirely erase the effects of conditioning. Each of the following descriptions assumes that a CS has been conditioned and then extinguished through the procedures described above.

• Reacquisition:

If an extinguished CS is again paired with the US, a CR is again acquired, but this second acquisition usually happens much faster than the first one.

• Spontaneous recovery:

Main article: Spontaneous recovery (psychology)

If an extinguished CS is tested at a later time (for example an hour or a day) after conditioning it will often again elicit a CR, a phenomenon called spontaneous recovery. This renewed CR is usually much weaker than the CR observed prior to extinction

• External disinhibition:

If an extinguished CS is tested just after intense but associatively neutral stimulus has occurred, there may be a temporary recovery of the conditioned response to the CS

• Reinstatement:

If the US used in conditioning is presented to a subject in the same place where conditioning and extinction occurred, but without the extinguished CS being present, the extinguished CS often elicits a response when it is tested later.

• Renewal:

If conditioning and extinction are conducted in different contexts (for example, conditioning occurs in one experimental chamber and extinction occurs in a chamber with a different color and shape), a return to the conditioning context may cause the CS to again elicit a CR.

Stimulus Generalization

Stimulus generalization is said to occur if, after a particular CS has come to elicit a CR, another test stimulus elicits the same CR. Usually the more similar are the CS and the test stimulus the stronger is the CR to the test stimulus.

Stimulus discrimination

One observes stimulus discrimination when one stimulus ("CS1") elicits one CR and another stimulus ("CS2") elicits either another CR or no CR at all. This can be brought about by, for example, pairing CS1 with an effective US and presenting CS2 in extinction, that is, with no US.

Latent inhibition

Main article: Latent inhibition

In this procedure, a CS is presented several times before paired CS–US training commences. This slows the rate of CR acquisition relative to that observed without such pre-exposure.

Conditioned suppression

This is one of the most common ways to measure the strength of learning in classical conditioning. In a typical example of this procedure, a rat first learns to press a lever through operant conditioning. Classical conditioning follows: in a series of trials the rat is exposed to a CS, often a light or a noise. This CS is followed by the US, a mild electric shock. As an association between the CS and US develops, the rat slows or stops its lever pressing when the CS comes on. The rate of pressing during the CS measures the strength of classical conditioning; that is, the slower the rat presses, the stronger is the association of the CS and the US. (Slow pressing indicates a "fear" conditioned response, and it is an example of a conditioned emotional response, see section below.)

Conditioned inhibition

Three phases of conditioning are typically used:

Phase 1

A CS (CS+) is paired with a US until asymptotic CR levels are reached.

Phase 2

CS+/US trials are continued, but interspersed with trials on which the CS+ is paired with a second CS, (the CS-) but not with the US (i.e. CS+/CS- trials). Typically, organisms show CRs on CS+/US trials, but stop responding on CS+/CS− trials.

Phase 3

• Summation test for conditioned inhibition: The CS- from Phase 2 is presented together with a new CS+ that was conditioned as in Phase 1. Conditioned inhibition is found if the response is less to the CS+/CS- pair than it is to the CS+ alone.
• Retardation test for conditioned inhibition: The CS- from Phase 2 is paired with the US. If conditioned inhibition has occurred, the rate of acquisition to the previous CS− should be less than the rate of acquisition that would be found without the Phase 2 treatment.

Blocking

Main article: Blocking effect

This form of classical conditioning involves two phases.

Phase 1

A CS (CS1) is paired with a US.

Phase 2

A compound CS (CS1+CS2) is paired with a US.

Test

A separate test for each CS (CS1 and CS2) is performed. The blocking effect is observed in a lack of conditional response to CS2, suggesting that the first phase of training blocked the acquisition of the second CS.

Theories of classical conditioning

Data sources

Experiments on theoretical issues in conditioning have mostly been done on vertebrates, especially rats and pigeons, but conditioning has also been studied in invertebrates, and very important data on the neural basis of conditioning has come from experiments on the sea slug, Aplysia. Most relevant experiments have used a classical conditioning procedure, though instrumental (operant) conditioning experiments have also been used, and the strength of classical conditioning is often measured through its operant effects, as in conditioned suppression (see Phenomena section above) and autoshaping .

The Rescorla-Wagner model

Main article: Rescorla-Wagner model

The Rescorla-Wagner (R-W) model is a relatively simple yet powerful model of conditioning. The model predicts a number of important phenomena, but it also fails in important ways, thus leading to number modifications and alternative models. However, because much of the theoretical research on conditioning in the past 40 years has been instigated by this model or reactions to it, the R-W model deserves a brief description here.

A key idea behind the R-W model is that a conditioned stimulus (CS) signals or predicts the unconditioned stimulus (US). One might also say that before conditioning the subject is "surprised" by the US, but after conditioning the subject is no longer surprised, because the CS predicts the coming of the US. (Note that the model can be described mathematically and that words like "predict", "surprise", and "expect" are only used to help explain the model.) Here the workings of the model are illustrated with brief accounts of acquisition, extinction, and blocking. The model also predicts a number of other phenomena, see main article on the model.

R-W model: acquisition

The R-W model measures conditioning by assigning an "associative strength" to the CS. Before a CS is conditioned its associative strength is zero. Pairing the CS and the US causes a gradual increase in the associative strength of the CS from zero to a maximum that is determined by the nature of the US (e.g. its intensity). The amount of learning that happens during any single CS-US pairing depends on the difference between the current associative strength of the CS and the maximum set by the US. On the first pairing of CS and US this difference is large and the associative strength of the CS takes a big step up. As CS-US pairings accumulate, the US becomes more predictable, and the increase in associative strength on each trial becomes smaller and smaller. Finally the difference between the associative strength of the CS and the maximum strength reaches zero. That is, the CS fully predicts the US, the associative strength of the CS stops growing, and conditioning is complete.

R-W model: extinction

The associative process described by the R-W model also accounts for extinction (see Procedures above). The extinction procedure starts with a CS with a positive associative strength, which means the CS predicts that a US will occur. However, in the extinction procedure, the US fails to occur. As a result of this “surprising” outcome, the associative strength of the CS takes a step down. Extinction is complete when the strength of the CS reaches zero; no US is predicted, and no US occurs.

R-W model: blocking

Main article: Blocking effect

The most important and novel contribution of the R-W model is its assumption that the conditioning of a CS depends not just on that CS alone, and its relationship to the US, but also on all other stimuli present in the conditioning situation. In particular, the model states that the US is predicted by the sum of the associative strengths of all stimuli present in the conditioning situation. Learning is controlled by the difference between this total associative strength and the strength supported by the US. When this sum of strengths reaches a maximum set by the US, conditioning ends as just described.

The R-W explanation of the blocking phenomenon illustrates one consequence of the assumption just stated. In blocking (see Phenomena above), CS1 is paired with a US until conditioning is complete. Then on additional conditioning trials a second stimulus (CS2) appears together with CS1, and both are followed by US. Finally CS2 is tested and shown to produce no response because learning about CS2 was “blocked” by the initial learning about CS1. The R-W model explains this by saying that after the initial conditioning CS1 fully predicts the US. Because there is no difference between what is predicted and what happens, no new learning occurs on the additional trials with CS1+CS2, hence CS2 later yields no response.

Theoretical issues and alternatives to the Rescorla-Wagner model

One of the main reasons for the importance of the R-W model is that it is relatively simple and makes clear predictions. Tests these predictions have led to a number of important new findings and a considerably increased understanding of conditioning. Some of this new information has supported the theory, but much has not, and it is generally agreed that the theory is, at best, too simple. As yet no single model seems to account for all the phenomena that experiments have produced. Following are brief summaries of some related theoretical issues.

The content of learning

The R-W model reduces conditioning to the association of a CS and US, and measures this with a single number, the associative strength of the CS. A number of experimental findings indicate that more is learned than this. Among these are two phenomena described earlier in this article

• Latent inhibition: If a subject is repeatedly exposed to the CS before conditioning starts, then conditioning takes longer. The R-W model cannot explain this because preexposure leaves the strength of the CS unchanged at zero
• Recovery of responding after extinction: It appears that something remains after extinction has reduced associative strength to zero because several procedures cause responding to reappear without further conditioning.

The role of attention in learning

Latent inhibition might happen because a subject stops attending to a CS that is seen frequently before it is paired with a US. In fact, changes in attention to the CS are at the heart of two prominent theories that try to cope with experimental results that give the R-W model difficulty. In one of these, proposed by Nicholas Mackintosh, the speed of conditioning depends on the amount of attention devoted to the CS, and this amount of attention depends in turn on how well the CS predicts the US. A related model based on a different attentional principle was proposed by Pearce and Hall Although neither model explains all conditioning phenomena, the attention idea still has an important place in conditioning theory.

Context

As stated earlier, a key idea in conditioning is that the CS signals or predicts the US (see Zero contingency procedure above). But the room or chamber in which conditioning takes place also “predicts” that the US may occur, although with much less certainty than does the experimental CS itself. Such so-called “context” stimuli are always present; they have been found to play an important role in conditioning and they help to account for some otherwise puzzling experimental findings. Context plays an important role in the comparator and computational theories outlined below.

Comparator theory

To find out what has been learned, we must somehow measure behavior ("performance") in a test situation. However, as students know all too well, performance in a test situation is not always a good measure of what has been learned. As for conditioning, there is evidence, for example, that subjects in a blocking experiment do learn something about the “blocked” CS but fail to show this learning because of the way that they are usually tested.

“Comparator” theories of conditioning are “performance based;” that is, they stress what is going on at the time of test. In particular, they look at all the stimuli that are present during testing and at how the associations acquired by these stimuli may interact. To oversimplify somewhat, comparator theories assume that during conditioning the subject acquires both CS-US and context-US associations. At test, these associations are compared, and a response to the CS occurs only if the CS-US association is stronger than the context-US association. After a CS and US are repeatedly paired, as in simple acquisition, the CS-US association is strong and the context-US association is relatively weak, so the CS elicits a strong CR. In “zero contingency” (see above), the conditioned response is weak or absent because the context-US association is about as strong as the CS-US association. Blocking and other more subtle phenomena can also be explained by comparator theories, though, again, they cannot explain everything.

Computational theory

An organism's need to predict future events is central to modern theories of conditioning. Most theories use associations between stimuli to take care of this prediction; for example, in the R-W model, the associative strength of a CS tells us how strongly that CS predicts a US. A different approach to prediction is suggested by models such as that proposed by Gallistel & Gibbon (2000, 2002). Here response is not determined by associative strengths. Instead, the organism records the times of onset and offset of CSs and USs and uses these to calculate the probability that the US will follow the CS. A number of experiments have shown that humans and animals do learn to time events (see Animal cognition), and the Gallistel & Gibbon model yields very good quantitative fits to a variety of experimental data.

Applications

Neural basis of learning and memory

Pavlov proposed a physiological account of conditioning that involved connections between brain centers for the conditioned and the unconditioned stimuli. Though Pavlov's physiology has been abandoned, classical conditioning is extensively used in modern studies of the neural structures and functions that underlie learning and memory. Forms of classical conditioning that are used for this purpose include, among others, fear conditioning, eyeblink conditioning, and the foot contraction conditioning of Hermissenda crassicornis, a sea-slug.

Behavioral therapies

Main article: Behavior therapy

Some therapies associated with classical conditioning are aversion therapy, systematic desensitization and flooding. Aversion therapy attempts to associate an unpleasant CR, such as nausea, with stimuli associated with unwanted behavior, such as the smell of alcohol. Systematic desensitization attempts to elminate an unwanted CR, such as anxiety, by gradually exposing the patient to associated CS's (e.g. angry words) in a relaxing situation. Flooding attempts to eliminate an unwanted CR through massive exposure of the associated CS, as in extinction. Such therapies and treatments using classical conditioning differ from those using operant conditioning.

Classical conditioning usually takes less time with therapists and less effort from patients than humanistic therapies.

Conditioned drug response

A stimulus that is present when a drug is administered or consumed may eventually evoke a conditioned physiological response that mimics the effect of the drug. This is sometimes the case with caffeine; habitual coffee drinkers may find that the smell of coffee gives them a feeling of alertness. In other cases, the conditioned response is a compensatory reaction that tends to offset the effects of the drug. For example, if a drug causes the body to become less sensitive to pain, the compensatory conditioned reaction may be one that makes the user more sensitive to pain. This compensatory reaction may contribute to drug tolerance. If so, a drug user may increase the amount of drug consumed in order to feel its effects, and end up taking very large amounts of the drug. In this case a dangerous overdose reaction may occur if the conditioned stimulus happens to be absent, so that the conditioned compensatory effect fails to occur. For example, if the drug has always been administered in the same room, the stimuli provided by that room may produce a conditioned compensatory effect; then an overdose reaction may happen if the drug is administered in a different location where the conditioned stimuli are absent.

Conditioned hunger

Signals that consistently precede food intake can become conditioned stimuli for a set of bodily responses that prepares the body for food and digestion. These reflexive responses include the secretion of digestive juices into the stomach and the secretion of certain hormones into the blood stream, and they induce a state of hunger. An example of conditioned hunger is the "appetizer effect." Any signal that consistently precedes a meal, such as a clock indicating that it is time for dinner, can cause people to feel hungrier than before the signal. The lateral hypothalamus (LH) is involved in the initiation of eating. The nigrostriatal pathway, which includes the substantia nigra, the lateral hypothalamus, and the basal ganglia have been shown to be involved in hunger motivation.

Conditioned emotional response

Further information: conditioned emotional response and fear conditioning

The influence of classical conditioning can be seen in emotional responses such as phobia, disgust, nausia, anger, and sexual arousal. A familiar example is conditioned nausea, in which the CS is the sight and/or smell of a particular food that in the past has resulted in an unconditioned stomach upset. Similarly, when the CS is the sight of a dog and the US is the pain of being bitten, the result may be a conditioned fear of dogs.

As an adaptive mechanism, emotional conditioning helps shield an individual from harm or prepare it for important biological events such as sexual activity. Thus, a stimulus that has occurred before sexual interaction comes to cause sexual arousal, which prepares the individual for sexual contact. For example, sexual arousal has been conditioned in human subjects by pairing a stimulus like a picture of a jar of pennies with views of an erotic film clip. Similar experiments involving blue gourami fish and domesticated quail have shown that such conditioning can increase the number of offspring. These results suggest that conditioning techniques might help to increase fertility rates in infertile individuals and endangered species.

In popular culture

One of the earliest literary references to classical conditioning can be found in the comic novel The Life and Opinions of Tristram Shandy, Gentleman (1759) by Laurence Sterne. The narrator Tristram Shandy explains how his mother was conditioned by his father's habit of winding up a clock before having sex with his wife:

My father [...] was, I believe, one of the most regular men in every thing he did [...] [H]e had made it a rule for many years of his life,—on the first Sunday-night of every month throughout the whole year,—as certain as ever the Sunday-night came,—to wind up a large house-clock, which we had standing on the back-stairs head, with his own hands:—And being somewhere between fifty and sixty years of age at the time I have been speaking of,—he had likewise gradually brought some other little family concernments to the same period, in order, as he would often say to my uncle Toby, to get them all out of the way at one time, and be no more plagued and pestered with them the rest of the month. [...] [F]rom an unhappy association of ideas, which have no connection in nature, it so fell out at length, that my poor mother could never hear the said clock wound up,—but the thoughts of some other things unavoidably popped into her head—& vice versa:—Which strange combination of ideas, the sagacious Locke, who certainly understood the nature of these things better than most men, affirms to have produced more wry actions than all other sources of prejudice whatsoever.

Another example is in the dystopian novel, A Clockwork Orange in which the novel's anti-hero and protagonist, Alex, is undergoes a procedure called the Ludovico technique, where he is fed a solution to cause severe nausea and then forced to watch violent acts. This renders him unable to perform any violent acts without inducing similar nausea. Unintentionally, he also forms an aversion to classical music.

In the science-fiction book Ender's Shadow, "Pavlovian mental bans" are also used to prevent crime. In the book, a controversial scientist, Anton, is kept from researching genetic experimentation by associating his work with anxiety. A device is then surgically placed in his head that would increase detected anxiety, sending him into a panic attack. The result is that Anton must remain good humored at all times, can only speak of his work through self-deceptive metaphors, and even after his Pavlovian mental ban is lifted can no longer study science.

The metal band Rorschach have a song titled "Pavlov's dogs" (the title being an obvious reference to Ivan Pavlov's experiment) whose lyrics also treat about classical conditioning.

Another example is from the TV series The Office. In the episode Phyllis' Wedding Jim conditions Dwight to want a breath mint whenever there is a computer chime.

See also

• Behaviorism
• Little Albert experiment
• Eyeblink conditioning
• Fear conditioning
• Latent inhibition
• Learned helplessness
• Nocebo
• Measures of conditioned emotional response
• Operant conditioning
• Placebo (origins of technical term)
• Quantitative analysis of behavior
• Rescorla-Wagner model of conditioning
• Reward system
• Preparedness (learning)
• Second-order conditioning
• Stimulus control
• Taste aversion
• Edwin B. Twitmyer
• Proboscis extension reflex

Further reading

• Dayan, P.; Kakade, S. & Montague, P.R. (2000). Learning and selective attention. Nature Neuroscience, 3, 1218–23. Full text
• Jami, S.A.; Wright, W.G. & Glanzman, D.L. (2007). Differential Classical Conditioning of the Gill-Withdrawal Reflex in Aplysia Recruits Both NMDA Receptor-Dependent Enhancement and NMDA Receptor-Dependent Depression of the Reflex. The Journal of Neuroscience, 27, 3064–8.
• Kirsch, I.; Lynn, S.J.; Vigorito, M. & Miller, R.R. (2004). The role of cognition in classical and operant conditioning. Journal of Clinical Psychology, 60, 369–92.
• Pavlov, I.P. (1927). Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex (translated by G.V. Anrep). London: Oxford University Press. Google preview of Dover 2003 reprint
• Rescorla, R.A. & Wagner, A.R. (1972). A theory of Pavlovian conditioning. Variations in effectiveness of reinforcement and non-reinforcement. In A. Black & W.F. Prokasky, Jr. (eds.), Classical Conditioning II New York: Appleton-Century-Crofts.
• wiki book on Animal behavior

External links

• Scholarpedia Classical conditioning
• Scholarpedia Computational models of classical conditioning
• Scholarpedia Hermissenda