Theoretical behaviorism

• Comment: Please revise your draft to remove the editorializing. Rhetorical questions (e.g. "Which suggests that response rate is an independent indicator of…what? The learning process?") aren't appropriate for an encyclopedia. Take some time to read through Philosophy of mind, Attachment theory, and the other featured/good articles in WikiProject Psychology for examples of strong articles written in an encyclopedic tone. Please submit the draft again after your revisions. Thanks! — Newslinger talk 15:43, 9 August 2018 (UTC)

• Comment: The material copied from elsewhere is correctly in quotation marks or block quotes, but is not adequately attributed. WP:MINREF requires inline citations for all direct quotations. Worldbruce (talk) 04:02, 27 July 2018 (UTC)

---

Experimental psychologist B. F. Skinner proposed radical behaviorism as an alternative to the then-prevailing more mentalistic philosophies of mind and behavior. His new philosophy accompanied a novel program of experimental research based on the study of individual organisms rather than groups. Two books, The behavior of organisms (1938) and Schedules of reinforcement (1957, with C. B. Ferster), describe the methods of operant conditioning and present much experimental data.

The new experimental methods were very successful, producing hundreds of papers reporting many new phenomena. But radical behaviorism was criticized as a philosophy for neglecting the processes that interpose between stimulus and response . The movement was in fact actively hostile to theory; but experimental success overshadowed theoretical weakness.

Operant Conditioning

Two inventions, the Skinner box and the cumulative recorder, were central to Skinner’s new experimental method. The Skinner box facilitated long-term (many weeks rather than a few days) automated experiments on learned behavior in individual organisms. The cumulative recorder showed real-time data as opposed to the group averages commonly used by other researchers.

Skinner was able to demonstrate rapid learning in individual animals. The method was to present small rewards (now called reinforcements) right after the animal makes a desired response. The process could begin with approximations to the target behavior. Skinner called the technique shaping by successive approximations. The process as a whole he termed 'operant conditioning', a re-naming of what was already known as ‘instrumental learning’.

Skinner recognized that behavior must occur spontaneously before it can be reinforced. He called behavior that is both spontaneous and reinforcible emitted behavior, in contrast to the elicited, reflex-like behavior of classical (Pavlovian) conditioning (respondent behavior). The operant-respondent distinction turned out to be flawed, but the idea that behavior must be emitted before it can be rewarded is critical.

The operant method showed that a given response, be it lever-pressing by a rat or key-pecking by a pigeon, need not be reinforced on every occasion. Responding can be maintained by various partial-reinforcement schedules. For example, if only the first response after 30 s is reinforced (a fixed-interval-30 s schedule), pigeons, rats and humans will soon learn to wait for 15 s or so after each reinforcement before resuming responding, yielding the familiar scalloped cumulative record. If reinforcement is intermittent and unpredictable, responding may be maintained for many hours in the absence of further reinforcement.

Experiment soon revealed hitherto unsuspected regularities: the stable cumulative records associated with different schedules. Most important, these stable patterns could be recovered after exposure to another schedule. The typical scallop pattern on an FI schedule, for example, would reappear after a few days on second exposure after an intervening experience with another procedure. Behavior in condition A would be the same after different prior experiences B, C, D, etc.

Learning is, almost by definition, irreversible. The effect of treatment X will therefore be different if preceded by treatment A than if it is preceded by B. Two learning treatments cannot be meaningfully compared successively in the same subject. Most learning psychologists therefore assumed that learning must be studied by comparing groups of subjects. In fact, the group method, and its associated statistical analyses, has turned out to have unsuspected pitfalls - problems which are avoided by the single-subject approach.

On the other hand, the fact that behavior under a given reinforcement schedule is recoverable, the same no matter what the preceding treatment, suggested to Skinner and his followers that learning – operant conditioning – can be studied in single subjects. Neither averaging across individuals nor comparisons between groups is required. Since the individual, not the group, is the target of all psychological investigation, and since there were known to be serious problems inferring the properties of individuals from group averages , Skinner’s method seemed to provide a powerful technique for understanding the effects of reward and punishment on the behavior of individual organisms.

Rate of response is visible as the slope of a cumulative record. As a subject learns a typical operant task, the slope of the record, the rate, increases: "The rate at which a response is emitted in such a situation comes close to our preconception of the learning process. As the organism learns, the rate rises." [1] Skinner continued:

"It is no accident that rate of responding is successful as a datum...If we are to predict behavior (and possibly to control it), we must deal with probability of response...Rate of responding is not a ‘measure’ of probability but it is the only appropriate datum in a formulation in these terms."

So, response rate was used by most operant conditioners as a measure of response strength/probability. Researchers noticed that if reinforcement is available only at random times (a random-interval – RI – schedule, one kind of variable-interval, VI), a procedure which ensures that reinforcement probability is essentially constant, subjects adapt by responding at a constant rate which is positively related to reinforcement rate.

In this way, average response rate became the standard dependent variable for operant psychology.

Discriminative stimuli

Pigeons, rats, and people can easily be trained to respond differentially in the presence of different stimuli, depending on consequences. If a hungry pigeon, confronted with two adjacent pecking keys, is paid off with bits of grain only for pecking the red, and not the green, key, he will soon learn to peck only the red; and similarly if the payoffs are reversed. Skinner called discriminations like this examples of stimulus control.

Skinner went on to propose the three-term contingency as a behavioral unit incorporating stimulus, response and reinforcement. The idea is that reinforcing a response in the presence of a given stimulus establishes control by the stimulus of the pattern of behavior established by the prevailing reinforcement schedule. Skinner called this unit the operant, his word for what might previously have been called a habit and resembles the theoretical behaviorist notion of state.

By inventing new concepts, and re-naming several old ones, Skinner created a distinctive terminology that helped to define a new and self-contained movement: the experimental analysis of behavior, aka behavior analysis aka operant conditioning.

With the sole exception of the three-term contingency, these ideas were summarized by Skinner in an influential 1950 paper , Are theories of learning necessary?. He defined theory as "any explanation of an observed fact which appeals to events taking place somewhere else, at some other level of observation, described in different terms, and measured, if at all, in different dimensions." This definition would rule out many well-accepted theories in other areas of science. The temperature of a liquid, for example, is directly related to movement. It is not clear that the “dimensions” of temperature are the same as the kinetic energy of molecules. The spectra of hot elements – the red flame of lithium, for example – can be derived from the element’s atomic properties. Again, it is not the case that the atomic properties that underlie emission spectra have same dimensions as wavelength. It cannot be right to rule out theories like this .

Skinner argued that learning theories are for the most part impediments to scientific advance: "Much useless experimentation results from theories, and much energy and skill are absorbed by them", although he also conceded that, "It would be foolhardy to deny the achievements of theories of this sort in the history of science. This sort refers to a previous paragraph in which he attempts to distinguish between postulates", "theorems", and "theories".'He admits, in a widely cited phrase, there is a "need for a formal representation of the data reduced to a minimal number of terms" but at the end of his article says that, "We do not seem to be ready for theory in this sense." By the 1990s, the field was ready.

Problems with atheoretical behaviorism

The shaky epistemological basis for Skinner’s anti-theory argument was overshadowed by the very compelling experimental examples he described in the rest of the 1950 article. His novel method produced strikingly orderly real-time patterns of behavior in individual organisms. He proceeded to use these data to identify what he called "controlling variables", those aspects of the training procedure responsible for the observed patterns: "the independent variables of which probability of response is a function." When we know the controlling variables, he argued, theory is unnecessary.

Defending his idea that response probability is the correct dependent variable for learning psychology, he showed that the alternative favored by reflex-type theorists, latency, did not behave in the appropriate way. Motivated and unmotivated animals show the same modal response latency in a suitable task. Motivated animals do not respond sooner, as they should if latency is an adequate measure of response strength. As hunger motivation is reduced, latencies are more variable however (what Skinner referred to as a "scattering of latencies"). The increase in variability as motivation is reduced is consistent with the selectionist view of learning espoused by theoretical behaviorism

In another experiment, arguing against the inhibition theory of extinction, Skinner showed that well-trained pigeons forget little even after a lapse of four years between successive exposures to a task. He also showed that the pattern of a cumulative record in extinction is related to the pattern built up during training. He attributed the difference between extinction of a periodic vs. an aperiodic schedule to novelty and dissipation of emotional responses. He described the method that would later be used by Guttman and Kalish to measure stimulus generalization.

Skinner’s examples were striking. His conclusion was persuasive. Many readers came to accept his claim that theories of learning – not just the poor theories then current but perhaps all learning theories – are in fact impediments to progress in scientific psychology.

But atheoretical behaviorism was criticized by theorists on several points, which can be illustrated by re-visiting three of Skinner’s examples: response rate as a dependent variable, extinction and memory, and the operant-respondent distinction. These problems led to the theory and philosophy of theoretical behaviorism.

Response rate

Skinner wrote in the 1950 paper:

"Rate of responding appears to be the only datum that varies significantly and in the expected direction under conditions which are relevant to the “learning process.” We may, therefore, be tempted to accept it as our long-sought for measure of strength of bond, excitatory potential, etc. Once in possession of an effective datum, however, we may feel little need for any theoretical construct of this sort".

Which suggests that response rate might be an independent indicator of something like 'strength of learning'. Skinner wasn’t sure, but succumbed to temptation and settled on response rate as the most useful response measure.

A nagging problem for the idea that response rate is always valid as a measure of response strength is that rate of response can itself be controlled by the appropriate contingencies of reinforcement. For example, animals will learn, albeit with some difficulty, to space their pecks or lever presses 10-s apart (spaced-responding schedule) if that is a condition for reinforcement, – even though the ‘natural’ rate for an equally rewarding schedule that lacks the spaced-responding requirement is much higher, perhaps 60 pecks per minute. Since rate of response, over the typical period of 30-min or so, on a spaced-responding schedule is low, then probability of response hence response 'strength'must also be low, according to one reading of Skinner – lower than on, say, a variable-interval schedule dispensing reinforcements at the same rate. This is clearly wrong. Response rate, per se, is not an adequate measure of response strength.

Skinner never wrote explicitly about this problem. But an obvious response is that the spaced-responding schedule involves the explicit discrimination of time. Response probability, hence, response strength is high at some times and low at others. It is high close to 10 s after each response and low in between, for example. Much the same is true of a fixed-interval schedule. This is of course true. But it makes response rate less attractive as a universal measure of response strength. Indeed, perhaps time should take over as the appropriate dependent variable? Perhaps the question should be not “how does schedule X affect response rate?” but, “How does schedule X affect the temporal location of behavior?”

Using time as a dependent measure also avoids a problem that is rarely addressed: over what time period (minutes? hours?) should rates be computed? In operant-conditioning experiments, rates are usually computed over intervals of 30-min or more. The choice of denominator is justified not by any theoretical rationale, but by the orderly functional relations that result. In Skinner’s [2] own words: “Science is...is an attempt to discover order, to show that certain events stand in lawful relations to other events.” Order was its own justification for Skinner.

Extinction and Memory

Skinner never mentioned the word memory in the 1950 article, and rarely afterwards. But he did discuss spontaneous recovery, a paradoxical property of experimental extinction: After sufficient training, an organism responds. If reinforcement is withdrawn, responding ceases (extinction), usually within a single experimental session. But the next day, returned to the apparatus, the animal begins to respond again. Since we know (argued Skinner) that little or no forgetting should occur from one day to the next, this recovery of the extinguished response, an apparent forgetting of the extinction on the previous day, needs explaining.

Until Skinner’s paper, the standard explanation for spontaneous recovery was that during the extinction session, inhibition builds up, but by the next day it has dissipated so responding recovers, at least for a while. But Skinner already showed that mere passage of time has little effect on level of responding -- although we will have reason to question that in a moment. So perhaps some other variables are operating? In the 1950 paper, Skinner proposed two: emotion and novelty:

"When we fail to reinforce a response that has previously been reinforced, we not only initiate a process of extinction, we set up an emotional response....The pigeon coos in an identifiable pattern, moves rapidly about the cage, defecates, or flaps its wings rapidly in a squatting position that suggests treading (mating) behavior. This competes with the response of striking a key and is perhaps enough to account for the decline in rate in early extinction…Whatever its nature, the effect of this variable is eliminated through adaptation." [emphases added]

Skinner said no more than this about “emotion,” but his description is interesting for two reasons. First, it involves observation, actually watching the (pigeon, rat) subjects. This practice soon fell out of fashion in behavior analysis. Yet direct observation of behavior was later to prove critical in shedding new light on Skinner’s theoretical approach. Second, he might have said something more about competition, which is apparently also involved. As it is, emotion is unsatisfactory as an explanation because the new process he invokes to explain its dissipation, adaptation, cannot be independently measured.

But novelty is the variable Skinner thought most important: “Maximal responding during extinction is obtained only when the conditions under which the response was reinforced are precisely reproduced.” First Skinner describes stimulus generalization, the decline in responding in the presence of stimuli different from the training stimulus. Then he goes on:

"Something very much like this must go on during extinction. Let us suppose that all responses to a key have been reinforced and that each has been followed by a short period of eating. When we extinguish the behavior, we create a situation in which responses are not reinforced, in which no eating takes place, and in which there are probably new emotional responses. The situation could easily be as novel as a red triangle after a yellow [his earlier example of stimulus generalization]. If so, it could explain the decline in rate during extinction."

Novelty, as subsequently precisely measured in the stimulus generalization experiments of Guttman and Kalish and many others, is the real explanation for spontaneous recovery, said Skinner. But again, this is an incomplete account, because we cannot measure the stimulus in this case. In regular stimulus generalization, to a color or a shape, for example, both physical stimulus properties and the effects of changes on responding can be measured objectively. Not so in the case of extinction, the case that Skinner is attempting to explain. How exactly should ‘novelty’ be measured/manipulated? Something more was needed: a theory of memory, perhaps?

A relevant theory was in fact available. At the end of the previous century, Adolf Jost proposed two memory laws, the second of which is: Given two associations (equivalently, habits, memories, operants) of the same strength, but of different ages, the older one will fall off less rapidly with time. Jost’s law implies that the strength of a habit does not decay exponentially, by the same fixed fraction each day, because if it did, the relative strength of two memories would not change with lapse of time.

On the other hand, suppose that the decline in strength of a habit, V_i, after time t_i, is not exponential but hyperbolic, like this:

V_i = K/(A+t_i),.... (1)

where K/A is the salience – strength – of habit i at time zero and A is a parameter representing the memory decay rate. If we look at the rate of change of V_i with time we get

(dV_i)/dt = -K/(V_i+t_i )².... (2)

Now, suppose that at a particular time after learning, with values t₁ and t₂, t₁ > t₂, representing the ages of two memories, the memory strengths, V_i, of events 1 and 2 are equal. Then the rate of decline for each strength will be given just by Eq. 2, with V the same for both memories. Since t₂ < t₁ clearly the rate of decline of memory strength will be greater for t₂, the more recent memory. Equation 1 is hyperbolic, but many other monotonic, decreasing functions will do as well to model Jost’s Law.

Jost’s Law explains spontaneous recovery. Since the first extinction session is necessarily more recent than the many days of conditioning that preceded it, the associated behavior should lose more strength from one day to the next than the earlier conditioning. At the end of the first day of extinction, responding ceases, which means that the strengths of the two memories, for responding and for not responding, must be roughly equal. Thereafter both memories decline in strength, the new faster than the older. On the next day, therefore, the older tendency – to respond – must gain (according to Jost) over the more recent one (not responding), hence: spontaneous recovery. With continued extinction session, the "respond" memory continues to decline, but the "not-respond"tendency is strengthened day by day.

A model like this could make predictions about the effects of different delays before returning the animal to the apparatus, and on different amounts of training on subsequent extinction . If the second extinction session follows closely on the first, recovery should be less, for example. In other words, the theory draws attention to historical variables as possibly involved in recovery after extinction, a useful advance over the “novelty” idea, which looks only at a contemporary cause, and one that is difficult to measure objectively.

A Jost’s Law account implies that memories compete in some way. The competition idea also speaks to the apparent contradiction between the very slow decay of well-learned operant behavior demonstrated in Skinner’s 4-year experiment and the apparent rapid forgetting of an extinction experience illustrated by spontaneous recovery. The key is competition between memories. In the absence of any competition, a long-learned habit may persist for a long time, as Skinner’s pigeons showed. But when the competition is weak – just one extinction session – memory for many earlier conditioning sessions reasserts itself, and responding recovers until more extinction experience has accumulated.

Hyperbolic discounting is a phenomenon much studied by behavioral economists with both human and animal subjects. In a choice situation, subjects usually prefer a reward of size 2 after a delay of 10-s, say, over a reward of size 5 after a delay of 20-s, even though the rate of return is better for the larger, later reward. This contradicts the standard exponential discounting assumption, which assumes that rate of return is key.

It is tempting to relate hyperbolic memory decay to hyperbolic discounting in choice experiments and there may be some theoretical link. But also involved is the fact that organisms typically time their responses to be proportional to the expected time of a reward. There is also some evidence that the larger the anticipated reward, the sooner animals will respond. Offered a choice, therefore, between two stimuli, one signaling a small reward after 5-s vs one more than twice the size after 10-s, preference will be a balance between the tendency to wait a time proportional to the expected delay (which favors the smaller, sooner reward) and an opposite tendency to respond sooner if the expected reward is larger. The experimental evidence seems to suggest that the latter effect is smaller than the former. Animals are likely to respond sooner to the shorter delay, even if the associated reward size is smaller – and even if the overall rate of reward associated with the smaller choice is less than for the larger.

These two accounts are not in conflict: discounting is a functional, economic account; the timing account represents the potential underlying mechanism.

The Operant-Respondent Distinction

Ivan Petrovich Pavlov (1849-1936) was not a psychologist. His pioneering work on conditioned responses like salivation (typically by a dog following several pairings between a buzzer or a metronome, say, and the delivery of food) was physiology not psychology. The focus of Pavlov, and many who followed him, was on the reflex-like behavior maintained by classical or Pavlovian or, in Skinner’s terms respondent, conditioning. Pavlov found that the conditioned response was most rapidly obtained if the food followed closely on the stimulus (temporal contiguity). Subsequent work by Robert Rescorla and others showed that the key was prediction. The signaling stimulus need not be close in time to the reinforcer so long as it is closer than any other signal: relative proximity. Skinner justified his term respondent by pointing out that conditioned responses like salivation are products of the autonomic (involuntary), not the somatic (skeletal) nervous system. Operant behavior, he thought, depended on the somatic system.

But after Pavlov, researchers began to pay attention to responses other than salivation that might occur during a conditioning experiment. An early experiment by Karl Zener showed that salivation is not the only activity that classically conditioned dogs exhibit. A story recounted by the great ethologist Konrad Lorenz provides a clue:

"My late friend Howard Liddell told me about an unpublished experiment he did while working as a guest in Pavlov’s laboratory. It consisted simply in freeing from its harness a dog that had been conditioned to salivate at the acceleration in the beat of a metronome. The dog at once ran to the machine, wagged its tail at it, tried to jump up to it, barked, and so on; in other words, it showed as clearly as possible the whole system of behavior patterns serving, in a number of Canidae, to beg food from a conspecific. It is, in fact, this whole system that is being conditioned in the classical experiment."

This rich repertoire of operant behavior even if the conditioned stimulus is too long to produce much salivation but is sufficiently predictive to allow the dog to anticipate food.

Another sign that something was wrong with the rigid dichotomy between operant and respondent was provided by a pair of experiments: a very influential short paper "'Superstition' in the pigeon" by Skinner and a much longer experimental and theoretical paper by Staddon and Simmelhag many years later. Here is what Skinner did in 1948: Hungry pigeons were placed in a box and given brief access to food at fixed periods – 15-s for some animals, longer periods for others. This is temporal conditioning (a fixed-time – FT – schedule in operant terminology), which is a Pavlovian procedure since the animal’s behavior has no effect on food delivery.

Despite the absence of an operant contingency, all the animals developed vigorous stereotyped, apparently operant, activities in between feeder operations. Skinner attributed this behavior to accidental contiguity between some spontaneous behavior by the pigeon and the delivery of food. He called it adventitious reinforcement. Since these conjunctions were accidental, not causal, Skinner termed the activities “superstitious” and likened them to human superstitions

More than 20 years after the superstition paper, Staddon and Simmelhag repeated Skinner’s experiment, and observed the pigeons’ behavior second-by-second in each interfood interval from the very beginning of training. Their aim was atheoretical. They were simply curious: let’s see what happens, in detail, and let’s see if the interfood interval has to be constant (as in Skinner’s experiment) or can it be variable?

It turns out that variable intervals work fine; a variable-time schedule also induces “superstitious” activity. But Staddon & Simmelhag also found three things that differ from Skinner’s account:

1. The activities that develop are of two kinds: interim activities that occur in the first two-thirds or so of the fixed interfood interval, and a single terminal response, that occurs during the last third of the interval. Both interim and terminal responses trend to occur at a higher rate the shorter the interval; the reverse.
2. The terminal response is either pecking or a stereotyped pacing activity related to it; the terminal response does not differ from animal to animal in the capricious way implied by Skinner’s account.
3. Terminal pecking often appeared suddenly after a little training. It did not develop following an accidental conjunction with food as the adventitious reinforcement hypothesis implies. Interim activities are rarely contiguous with food, so also cannot be explained by adventitious reinforcement.

In short, Skinner’s account is wrong. The superstitious behavior he observed was not the result of happenstance, accidental contiguity between an emitted behavior and response-independent food. This experiment, and an earlier one showing that untrained pigeons will learn to peck in a Pavlovian procedure where an intermittent 7-s light (conditioned stimulus: CS) that ends with free food (unconditioned stimulus: US. Brown and Jenkins termed this effect autoshaping), showed that Skinner’s dichotomy between operant (somatic) and respondent (autonomic) behavior does not hold, since pecking – the prototypical operant response – can behave just like salivation, the prototypical respondent.

These results demanded a revision of the standard framework for the study of operant conditioning. If pecking is both an operant and a respondent, but salivation (for example) can be classically but not operantly conditioned, if supposedly ‘instinctive’ activities can supersede already learned operant behavior, the sharp distinction between classical and operant conditioning becomes untenable.

Selection and variation

Beginning in the early 1950s, people began to point out the similarities between the learning process and evolution through variation and selection. Recently, models explicitly analogous to gene mutation and selection by reinforcement have successfully duplicated many operant conditioning phenomena. Skinner’s idea of emitted behavior fits quite naturally into a Darwinian scheme. Behavior varies; a variant that is contiguous with reward is strengthened and thus increases in frequency.

Unlike Darwin, Skinner had little to say about the causes and types of variation. He left the impression that variation is unstructured, ‘random’. On the other hand, observations like Zener's and Liddell’s show that the repertoire from which reinforcement selects is very far from random. It is different for food than for sex or social reward, for example. Lorenz, an ethologist, identified the dog’s behavior as a particular instinctive pattern. A cognitive psychologist might say that the dog is showing an expectation of food. A theoretical behaviorist account is that the conditioning process causes the conditioned stimulus to evoke a particular behavioral repertoire. The emitted behavior to which that repertoire gives rise is not at all random.

The Repertoire

The composition of the repertoire will depend on the animal’s training – learning the signal properties of the metronome – motivational state and species. Anticipation of food will lead to a different repertoire than anticipation of electric shock. Food → vigorous activity, tail-wagging, etc. Electric shock → ‘freezing,’ crouching – suppression of all activity. Indeed, conditioned suppression is the name for the shock-anticipation procedure used by Estes and Skinner and others to establish the necessary and sufficient conditions for respondent conditioning.

The idea of a repertoire implies that some behaviors are potential, lying in wait, but ready to occur if the active behavior goes unrewarded. The stronger the animal’s motivation and the better the predictive properties of the stimulus (how close, how big, the reward?) the more restricted the repertoire is likely to be. In the limit, if the stimulus (as in the autoshaping experiments) or the interfood interval (as in the superstition experiment) is very short, the pigeon’s repertoire may be limited to a single response: pecking. But if the situation is not too “hot” the repertoire will be larger although perhaps less vigorous.

In addition to the active behavior, at any moment, a repertoire comprises latent or covert activities that can occur. This idea of a latent response was in fact suggested by Skinner himself in the same year that he published the “superstition” experiment:

"Our basic datum…is the probability that a response will be emitted…We recognize …that … every response may be conceived of as having at any moment an assignable probability of emission... A latent response with a certain probability of emission is not directly observed. It is a scientific construct. But it can be given a respectable status, and it enormously increases our analytical power…. It is assumed that the strength of a response must reach a certain value before the response will be emitted. This value is called the threshold." [emphases added]

Skinner was writing about language and never extended the idea to the operant behavior of non-human animals. But his proposal is different from theoretical behaviorism in only one respect: for the ThB hypothesis, the threshold is simply competition from other latent/silent responses.

The idea that any predictive relation between a stimulus and a reward creates an expectation, equivalently, a repertoire of potential actions, implies that a repertoire will be created, conditioning will occur, even if the dog doesn’t salivate. Imagine a conditioning situation in which the CS is just a bit too long to yield conditioning, as measured by, say, salivation, or an auto-shaped key-peck. So long as the CS is still predictive (e.g., signals a shorter time to the US than other signals) the animal may still form an expectation, and develop a repertoire. Members of the repertoire will be available as candidates for operant conditioning, which is to say selection by temporal contiguity. But the repertoire itself, active response excepted, will be covert and may not reveal itself at once. If the animal, like Pavlov’s dog, is restrained, for example, its behavioral potential is necessarily limited. But freed from restraint, the dog shows at once the wide range of activities induced by a stimulus that signals imminent food.

Emitted responses can be induced in other ways. An unexpected reward will at once elicit a range of food-related activities, for example. Similarity of a new situation to one associated with food or a mate will similarly elicit a historically relevant repertoire.

Extinction, cessation of an established reinforcement schedule, shows the effects of relaxing selection. When reinforcement is withdrawn, the selection process ceases, and the trained response declines. But observation, and Skinner’s "scattering of latencies", show that other activities, suppressed by the training schedule, now occur again. This is the normal increase in variability when selection is relaxed, either natural selection or selection by reinforcement schedule . Extinction usually leads to more variable behavior.

SUMMARY

Treating classical (respondent) conditioning and operant conditioning as different processes has taught us much about the necessary and sufficient conditions for conditioning to occur. But it has also, says ThB, led learning psychology into a blind alley. Learning researchers were misled by Pavlov’s genius and the neurophysiological differences between typical classically conditioned responses and typical responses conditioned operantly. Salivation and lever pressing are obviously very different.

Theoretical behaviorism treats classical and operant conditioning as parts of the same psychological process. Classical conditioning detects correlations between environmental features and something of value, positive or negative, to the organism. This correlation induces a repertoire from which operant conditioning can select. If the correlation is very strong and he unconditioned stimulus is imminent, then the induced repertoire may be limited – to pecking (in a hungry pigeon) or to salivation (in a restrained dog). Selection, in the sense of a response contingency, may be unnecessary. The result may look like a reflex, but isn’t, although restricted behavioral options and extreme motivation may make it appear so.

If the selection is weaker, some ‘expectation’ may still be formed and the repertoire may comprise many responses. Operant reinforcement must select from this pool. If there is no reinforcement, the behaviors that comprise the repertoire will occur one after another, back and forth, each time weaker and weaker. Eventually vigorous activity may cease altogether, leaving a passive, behavioral residue.

The old Yerkes-Dodson law (1908) shows that learning is fastest at intermediate levels of motivation, which suggests that the size of the repertoire is then at its maximum. As the organism learns, behavior adapts, reinforcement rate increases, and the repertoire shrinks to a class of responses defined by their consequences and controlled by a class of stimuli that are a reliable signal of the contingencies. This is Skinner’s three-term operant. Another name for it is state – not internal state or physiological state or even mental state, but state as repertoire controlled, in the well-trained organism, by identifiable stimuli under certain motivational conditions. (For the computational details on state as equivalent history, see The New Behaviorism but these details are not necessary to see the need to add state to stimulus and response to arrive at an accurate picture of the behaving organism.)

Theoretical behaviorism repeals Skinner’s proscription of theory. The “ism” is unfortunate because ThB is not a rigid ideology that rules things out. It is theoretical but eclectic. It does require that ideas be testable by a third party. But in that sense it is just. . . science. Concepts like memory and expectation are perfectly acceptable, just so long as they can be given some explanatory and predictive meaning.

The selection/variation view of learning implies that there is no sharp distinction between classical and operant conditioning. Operant reinforcement selects from a repertoire, just as Skinner argued. But that repertoire comes from somewhere. It has causes. One of them is stimulus-stimulus correlations detected by the processes labeled as classical conditioning. Classical and operant conditioning are a team, but one process (a repertoire set by classical conditioning) always limits the other (selection from the repertoire by response-reinforcer contiguity). Autoshaping, superstitious behavior, memory, and expectation are problems for Skinner’s radical behaviorism. Theoretical behaviorism offers a solution.