learning slides

Learning

1

Overview2

How do we develop passions, aversions, and responses to so many objects and activities?

3

Some of these objects and activities evolved after our birth—like a passion or aversion to Facebook

4

Such passions, aversions, & responses cannot be innate

5

Classical conditioning explains many of these tendencies

6

Classical conditioning was discovered by Ivan Pavlov

7

Pavlov’s conclusion

Pavlov’s discovery

Pavlov’s objectives

Ivan Pavlov, a Russian researcher, was interested in

the sources of salivation 8




He wanted to determine which foods and conditions promoted salivation in dogs

9




To examine this issue, he developed a means to

measure salivation in dogs10




In essence, the salivary gland, was connected to a clear tube

that dripped into a beaker11




Each food would evoke different levels of salivation

12




But gradually he noticed something unusual

13




When he or his assistant entered the room, the

dog would salivate14




Dogs could not have evolved to recognize Russian scientists

15




He decided to study this anticipation

16




A metronome was activated before food was presented

17




Later, a metronome without food evoked salivation

18




If a signal tends to precede an event that provokes a reflex…

19




this signal alone will evoke the reflex…

20

Classical conditioning extends reflexes to new situations

21

What are reflexes? 22

Reflexes are unlearned, involuntary responses to stimuli

23

Stepping reflex

Swimming reflex

Grasping reflex

When placed in water, hold breath; move arms & legs

24

Stepping reflex

Swimming reflex

Grasping reflex

Disappears after 4-6 months

25

Stepping reflex

Swimming reflex

Grasping reflex

When held upright, with their feet touching the ground

26

Stepping reflex

Swimming reflex

Grasping reflex

They will step, as if walking

27

Stepping reflex

Swimming reflex

Grasping reflex

Unless practiced often, disappears after 8 weeks

28

Stepping reflex

Swimming reflex

Grasping reflex

If touch palm, babies curl their finger around your finger

29

Stepping reflex

Swimming reflex

Grasping reflex

After 3 to 4 months, replaced by a voluntary grasp

30

Classical conditioning also extends to involuntary acts that are learned, called elicited behaviours

31

Classical conditioning can also explain fears

32

Conclusion from this finding

Little Albert afterwards

Little Albert initially

Little Albert was a happy child, with no obvious problems

33




But he was subjected to an experiment…

34




He received electric shocks whenever a rat approached

35




He became fearful and avoidant of rats as a consequence

36




This fear even generalized to other animals such as rabbits

37




Perhaps classical conditioning can explain phobias…

38

Suppose a sound always precedes an electric shock

39

Over time, people associate the sound with the shock

40

The sound itself is sufficient to evoke fear or phobias

41




But his experience does not typify phobias in general

42




Unlike Little Albert, one pairing is enough to generate phobias

43

The study was actually flawed & not only unethical

44

The results, for example, were quite mixed.

45

Nomenclature 46

The concept is simple… The nomenclature is confusing…

47

You need to learn five terms

48

• neutral stimulus • unconditioned stimulus and unconditioned response • conditioned stimulus and conditioned response

49

The metronome is initially a neutral stimulus—not associated with a particular response

Neutral stimulus

Unconditioned stimulus

Unconditioned response

50

The food and salivation are called unconditioned.They are innate rather than conditioned over time.

Neutral stimulus



51

Over time, the metronome is conditioned to evoke salivation—and is called the conditioned stimulus

Conditioned stimulus

Conditioned response 52

Yet, the metronome produces less salivation than food



We distinguish the conditioned & unconditioned response



Abbreviations of these terms: NS, US, UR, CS, and CR

55

Observations 57

FAQ: Will the metronome evoke salivation after preceding the food say 5 times?

58

That is, how many pairings are necessary

59

CR

inte

nsi

ty li

ke s

aliv

atio

n

Number of paired stimuli

60

CR

inte

nsi

ty li

ke s

aliv

atio

n


No precise number

61

CR

inte

nsi

ty li

ke s

aliv

atio

n


A few pairings might evoke a modest CR

62

Overview Nomenclature Observations

Stimulus-response Substitution Preparation

CR

inte

nsi

ty li

ke s

aliv

atio

n


Many pairings evoke stronger CRs

63

CR

inte

nsi

ty li

ke s

aliv

atio

n


…until a plateau is reached

64

FAQ: Can the metronome precede the food by 1 hour?

65

or appear at the same time or after the food?

66

What is the optimal timing of the NS and US?

67

To answer this question, consider four arrangements…

68

Delayed: The NS precedes but overlaps with the US

Delay

69

This arrangement facilitates conditioning

70

After a few pairings, the metronome will evoke salivation

71

Trace: The NS precedes but does not overlap with the US

Only a trace of the sound remains

72

Prevents conditioning unless organism is concentrating

Only a trace of the sound remains

73

and the hippocampus, vital to memory, is intact

74

Simultaneous impedes conditioning

75

Backward--the NS follows the US--impedes conditioning

76

But simultaneous and backward can be effective if…

77

the neutral stimulus has been significant during evolution

78

Consider this example

79

Suppose the sound always followed the shock

80

This backward arrangement will not enable conditioning

81

But now suppose the NS is significant, like a snake

82

The snake does become conditioned to provoke fear

83

Why?

84

Perhaps we are more sensitive to objects that have been significant, or threatening, during evolution

85

Complications86

Suppose you want to feel inspired by the textbook

87

Before each event that could be inspiring, like a movie

88

you should read the book. The book will become a CS

CS US UR

89

That is, over time, the book will elicit inspiration

CS CR

90

But actually many complications need to be addressed

CS CR

91

One complication is called overshadowing

92

Example: If the book coincides with an intense stimulus

CS URUS

93

such as listening to loud music

CS URUS

Introduction Formula Classical

Over-expectation Contradictions Refinements94

the intense stimulus is conditioned

CS CR



The loud music, even by itself, will evoke inspiration

CS CR

96

But the other stimulus—the book—is not conditioned

97

The book will not elicit inspiration

98

The intense stimulus overshadowed the other stimulus

Refinements99

A second complication is called blocking

100

If you wear the same clothes before each inspiring event

101

these clothes will become conditioned to the stimulus

CS US UR

102

That is, the clothes even by itself will evoke inspiration

CS CR

103

But if you now also read the text before inspiring events

104

the book will not be conditioned or evoke inspiration

105

Why not?

106

That is, how did the clothes block this conditioning?

107

Because the clothes already predict the US

CS US

108

The book does not help us predict the US

CS US

109

So the book in essence is disregarded

CS US

110

A third complication is called latent inhibition



Example: If the book coincides with an unfamiliar object

CS URUS

112

the unfamiliar stimulus is conditioned

CS CR

113

That is, the unfamiliar sound by itself evokes inspiration

CS CR

114

But the other stimulus—the book—is not conditioned

115

The book will not elicit inspiration

116

The unfamiliar stimulus inhibited the other stimulus

117

But how does the NS evolve into a CS?

118

What cognitive processes underlie this change?

119

Some researchers ascribed classical conditioning to a stimulus response model

120

That is, individuals learn to associate the neutral stimulus with the unconditioned response

121

The neutral stimulus will thus prime the unconditioned response

122

The sound is associated with salivation & other activities

123

So the sound alone primes salivation & other activities

124

But this model cannot explain some interesting findings

125

One example is sign tracking

126

Sign tracking is the tendency of organisms to approach a stimulus that tends to precede a pleasant event…

127

Suppose a light always precedes food and thus pecking

Neutral stimulus



128

Unsurprisingly the light will evoke pecking

129

But the pigeon also pecks at the light

130

Could not be ascribed to a stimulus-response model

131

The pigeon had never pecked at the light initially. So the light was never associated with this behaviour

132

Instead, the pigeon seemed to substitute the light & food

133

The pigeons regarded the light and food as equivalent, called a stimulus-stimulus model

134

But organisms do not always apply the same behaviour to the CS and US—contrary to the substitution model

135

Heroin tends to decrease blood pressure

Decrease blood

pressure

136

But cues that precede heroin increase blood pressure

Increase blood

pressure

137

So heroin and its precursors could not be substitutes—because they evoke different responses

Increase blood

pressure

138

Perhaps the precursors—and all CSs—evoke a tendency to prepare before the US is presented

139

That is, suppose these cues always precede heroin use—and this heroin use tends to decrease blood pressure

Decrease blood

pressure

140

The cue thus precedes a decrease in blood pressure

Decrease blood

pressure

141

Overview Nomenclature Observations

Stimulus-response Substitution Preparation

To compensate, the cue increases blood pressure

Decrease blood

pressure

142

The cue thus evokes a preparatory response

Decrease blood

pressure

143

Perhaps stimulus-response, stimulus substitution, and response preparation all apply in various settings

144

Operant conditioning 145

Nomenclature of operant conditioning 146

To describe this theory, specific labels are often used

147

Individuals are exposed to some setting or stimulus

148

called the antecedent or discriminating stimulus

Antecedent

149

They choose some behaviour in response, like press a bar

Antecedent Behaviour

150

and receive some outcome as a consequence, like milk

Antecedent Behaviour Consequence

151

If the consequence is pleasant, they will repeat this act


152

Pleasant consequences are called reinforcers or rewards


153

If the consequence is unpleasant, they will inhibit this act


154

Unpleasant consequences are called punishers


155

Positive versus negative reinforcers and punishers156

Besides reinforcers and punishers…

157

consequences can be divided into positive or negative

158

Oddly, not the same as pleasant and unpleasant !

159

Positive—added to. Negative—withdrawn from

160

Consider these examples

161

Positive reinforcement: Adding pleasant thing

162

like receiving ice cream after studying

163

Negative reinforcement: withdrawing unpleasant thing

164

like removing handcuffs or some other restriction

165

Positive punishment: adding unpleasant thing

166

like receiving a smack

167

Negative punishment: removing pleasant thing

168

like removing the right to leave the house

Introduction Nomenclature Delays

Reinforce vs punish Predictions Positive vs negative169

Why do we distinguish positive & negative outcomes?

170

Because positive consequences affect some people,

171

and negative consequences affect other people more

172

How do you reinforce acts the are not initiated otherwise

173

Apply shaping

174

To illustrate, suppose you want a dog to shake hands

175

Initially reinforce sitting

176

Then reinforce only acts that are closer to shaking

177

like sitting with one paw in the air

178

Then reinforce only acts that are even closer to shaking

179

like the paw touching a hand

180

Eventually reinforce only the desired act of shaking

181

Shaping can be used to develop social skills too

182

Example: Socially withdrawn girl who spoke too quietly

183

Initially, if she exceeded 20 dB, she was rewarded

“Excellent”21

184

Then, if she exceeded 25 dB, she was rewarded

“Excellent”21

“Excellent”28

185

Eventually, if she exceeded 50 dB, she was rewarded

“Excellent”21

“Excellent”28 “Excellent”

“Excellent”32 “Excellent”“Excellent”

“Excellent”52

186

Did you know when people speak louder and act extraverted their mood, & thus social skills, improve

187

Shaping has also been shown to overcome stuttering

Aim Prompting & fading New behaviours

Habits Openness Transfer 188

Person is reinforced for increasing rate of speech

Aim Prompting & fading New behaviours

Habits Openness Transfer 189

Classical versus operant conditioning

190

Classical conditioning extends our innate tendencies to a broader range of settings

191

For example, we innately smile in response to warmth

192

If a parent always coincides with this warmth…

193

the parent alone will evoke a smile

194

The smile is elicited in a broader range of settings

195

But respondent conditioning does not extend our repertoire of behaviours

196

Operant conditioning does extend our repertoire of behaviours

197

That is, in response to various events, we attempt variations of existing behaviours

198

Behaviours that are rewarded, not punished, are repeated and, over time, modified

199

Without operant conditioning, we would still attempt a diversity of behaviours but…

200

these behaviours would not become increasingly useful

201

Reinforcement schedules 202

How can you inspire children to tell the truth?

203

Perhaps each time the child concedes or discloses a fault,

I was rude

today

204

you could offer reassurance—a positive reinforcement

That’s OK; you’re

usually very good

I was rude

today

205

But this approach generates two problems

I was rude

today

206

First, you might be unable to reinforce each disclosure

207

For example, you might be distracted or far away

I was rude

today

208

Second, if you fail to reinforce only a few times,

I was

rude…

I was

rude…

I was

rude…

209

the behaviour will tend to vanish or extinguish quickly

I was

rude…

I was

rude…

I was

rude…

210

So how can you prevent these problems?

211

Reinforce after only a portion of suitable behaviors

212

Example: You could reinforce after every third disclosure

I was

rude… I was

rude… I was

rude…

I was

rude… I was

rude… I was

rude…

213

or after a delay of at least 1 hourI was

rude…

I was

rude…

I was

rude…

2.00 pm 4.00 pm 6.00 pm

I was

rude…

214

• Continuous reinforcement: Reinforce after each time• Intermittent reinforcement: Do not reinforce each time

215

Types of intermittent schedules 216

Intermittent schedules are classified as rate or interval

217

If you reinforce after every x acts, like 3 disclosures

I was

rude… I was

rude… I was

rude…

I was

rude… I was

rude… I was

rude…

218

the arrangement is called a rate schedule

I was

rude… I was

rude… I was

rude…

I was

rude… I was

rude… I was

rude…

219

Why is this arrangement called a rate schedule

220

bc the child is reinforced at a rate of 1 per 3 disclosures

221

If you reinforce an act only after a delay, such as 1 hour

I was

rude…I was

rude…

I was

rude…

2.00 pm 4.00 pm 6.00 pm

I was

rude…

222

the arrangement is called an interval scheduleI was

rude…

I was

rude…

I was

rude…

2.00 pm 4.00 pm 6.00 pm

I was

rude…

223

bc the child is reinforced only after an interval of 1 hour

224

Note the child is not always reinforced after an hourI was

rude…

I was

rude…

I was

rude…

2.00 pm 4.00 pm 6.00 pm

I was

rude…

225

Instead the first disclosure after this hour is reinforcedI was

rude…

I was

rude…

I was

rude…

2.00 pm 4.00 pm 6.00 pm

I was

rude…

226

Rate and interval schedules can also be divided…

227

into fixed and variable

228

Fixed rate: Child is reinforced after exactly 3 disclosures

I was

rude… I was

rude… I was

rude…

I was

rude…I was

rude…I was

rude…

229

Variable: Child is reinforced after 3 disclosures on average

I was

rude… I was

rude…

I was

rude… I was

rude…I was

rude…I was

rude…

230

Why do we need to know about these schedules?

231

Each schedule is common in real life

232

and generates different effects

233

Examples of intermittent schedules 234

Here are some typical examples

235

Many states resemble death

…. … …

… … …Each time salespeople sell 5 home loans, they receive a $100 bonus

Fixed rate

Fixed interval

Variable rate

Variable interval236


…. … …

… … …So, the act is the sale, and

the reinforcement is the bonus

Fixed rate

Fixed interval

Variable rate



…. … …

… … …called fixed rate 5 or FR 5

Fixed rate

Fixed interval

Variable rate



…. … …

… … …Slot machines pay a bonus after

about 10 lever presses on average

Fixed rate

Fixed interval

Variable rate



…. … …

… … …The behaviour is pressing a lever; the reinforcement is the money

Fixed rate

Fixed interval

Variable rate



…. … …

… … …called variable rate 10 or VR10

Fixed rate

Fixed interval

Variable rate



…. … …

… … …To teach children to be quiet, you

serve food only when they are quiet

Fixed rate

Fixed interval

Variable rate

Variable interval

7.00 am 1.00 pm 6.00 pm

242


…. … …

… … …but only after an interval of at least 5

hours from their previous meal

Fixed rate

Fixed interval

Variable rate

Variable interval

7.00 am 1.00 pm 6.00 pm

243


…. … …

… … …Called fixed interval 5 or FI5

Fixed rate

Fixed interval

Variable rate

Variable interval

7.00 am 1.00 pm 6.00 pm

244


…. … …

… … …Or, if the minimum delay varied

but was 5 hours on average

Fixed rate

Fixed interval

Variable rate

Variable interval

7.00 am 1.00 pm 6.00 pm

245


…. … …

… … …the schedule is VI5 instead

Fixed rate

Fixed interval

Variable rate

Variable interval

7.00 am 1.00 pm 6.00 pm

246

Benefits of the various intermittent schedules 247

So which schedule is preferable?

248

And what is the effect or benefit of each schedule?

249

These effects vary on four attributes

250


…. … …

… … …

Rate of response

Pause after reinforcement

Distribution of responses

Extinction Variable interval generates the most responses

FR VR

FI VI

251


…. … …

… … …

Rate of response



Extinction Why?

FR VR

FI VI

252


…. … …

… … …

Rate of response



Extinction If fixed interval, bc of the delay, frequent responses are not always reinforced

FR VR

FI VI

253


…. … …

… … …

Rate of response



Extinction So rate of responses diminishes

FR VR

FI VI

254


…. … …

… … …

Rate of response



Extinction If variable, reinforcement may be very soon—enhances energy

FR VR

FI VI

255


…. … …

… … …

Rate of response



Extinction For fixed, we may know the reward will not arrive until 6 actions later256


…. … …

… … …

Rate of response



Extinction For variable, the reward could be sooner, igniting anticipation 257


…. … …

… … …

Rate of response



Extinction Could explain why some people gamble so often

FR VR

FI VI

258


…. … …

… … …

Rate of response



Extinction because any moment they could enjoy a reward

FR VR

FI VI

259


…. … …

… … …

Rate of response



Extinction For rate schedules, the responses are evenly distributed over time

Time

260


…. … …

… … …

Rate of response



Extinction For interval schedules, the responses escalate just before the delay ends

7.00 am 1.00 pm 6.00 pm

261


…. … …

… … …

Rate of response



Extinction rather than earlier

7.00 am 1.00 pm 6.00 pm

262


…. …

… … …

Rate of response



Extinction

FR VR

FI VI

If fixed, often refrain from response immediately after the reward263


…. …

… … …

Rate of response



Extinction

FR VR

FI VI

If variable, still respond immediately after the reward264


…. …

… … …

Rate of response



Extinction

FR VR

FI VI

because, in these settings, an immediate reward is still possible 265


…. …

… … …

Rate of response



Extinction

FR VR

FI VI

Finally, after you withdraw reinforcement altogether 266


…. …

… … …

Rate of response



Extinction

FR VR

FI VI

responses will not vanish or extinguish as immediately if variable267


…. …

… … …

Rate of response



Extinction

FR VR

FI VI

Because of the variability, not as sure the reinforcement has vanished268

Observational learning

269

At one time, researchers assumed that people will learn only from the experiences in which they participated

270

Classical conditioning: People learn only when they experience the unconditioned response, like salivating

271

Operant conditioning : People learn only if they receive a reward or punishment themselves

272

Overview Bobo 1961 study Bobo 1963 study

Role of attention Ubiquity

Albert Bandura challenged

this assumption

Role of motivation 273



He showed that people—and even many animals—can learn merely by watching other individuals




Observing and participating in some activity is more effective than either of these modes alone


1961 Bobo study

276

A famous study with Bobo, a doll, illustrated his theory

277

Phase 1

Phase 2

Measures

Results

During Phase 1, some children observed violence

278

Phase 1

Phase 2

Measures

Results

That is, 3 to 6 year old children played in one

corner of a room 279

Phase 1

Phase 2

Measures

Results

while an adult played in another corner

280

Phase 1

Phase 2

Measures

Results

The adult played with some toys including a

mallet and a doll: Bobo281

Phase 1

Phase 2

Measures

Results

Half the children watched an aggressive adult—saying “Pow”, using a mallet…

282

Phase 1

Phase 2

Measures

Results

The other children watched an adult who did not play with Bobo

283

Phase 1

Phase 2

Measures

Results

Did observing aggression affect their behaviour?

284

Phase 1

Phase 2

Measures

Results

Phase 2 was designed to answer this question

285

Phase 1

Phase 2

Measures

Results

After playing with fun toys, the children were told to shift

rooms—to evoke frustration

286

Phase 1

Phase 2

Measures

Results

In the other room were some modest toys and Bobo

287

Phase 1

Phase 2

Measures

Results

A variety of behaviours were coded to assess the hypotheses

288

Phase 1

Phase 2

Measures

Results

• Physical aggression, like kicking

• Verbal aggression, like yelling

• Mirroring adults, using the mallet

289

Phase 1

Phase 2

Measures

Results

If exposed to aggression, children were more likely to

demonstrate aggression

290

Phase 1

Phase 2

Measures

Results

especially in boys, particularly when

the adult was male

291

1963 Bobo study

292

The 1963 study was similar except…

293

First, the children watched only a film of the adult

294

Second, the adult was either rewarded with food or punished with an angry warning or tirade

295

Third, both the memory of this film and behaviour of the children were assessed

296

Again, children mirrored the aggressive adult—but only if this adult was rewarded

297

Yet the reward did not improve memory of aggression

298

So, the rewards or punishment that other people receive affect behaviour but not memory

299

…the determinants of learning & performance differ

300

learning slides

Education