Forty-five years of PBL: Does it deliver on what was promised?

Henk G. Schmidt

Department of Psychology Erasmus University

JIM ANDERSON – JOHN EVANS – T. FRASER MUSTARD – BILL SPAULDING – BILL WALSH

Growth of number of PBL-schools worldwide 1969-2005

0

20

40

60

80

100

120

140

160

1970 1975 1980 1985 1990 1995 2000 2005

Year

Number of PBL schools

Characteristics of PBL   Problem is main stimulus for learning: “The problem

comes first.”   Students work in small groups guided by tutor (tutor

does not teach; “facilitates”)   Use systematic approach: 7-jump   Limited number of lectures; ample time for self-

directed learning   Horizontal and vertical integration of subject matter

(in the health sciences)   Skills training integrated in PBL-curriculum   Serial rather than parallel curriculum   Information needed for learning provided online   Communication with students online

Gaining weight suddenly: a normal process when you are over forty? A man in his forties comes to your office complaining that he has grown fat in a short time. His abdomen is swollen and he finds it difficult to fasten his belt. His eyelids and the skin around his eyes appear to be swollen too. The percentage of plasma protein in his blood is lower than normal. How would you manage his problem?

Gaining weight suddenly: a normal process when you are over forty? A man in his forties comes to your office complaining that he has grown fat in a short time. His abdomen is swollen and he finds it difficult to fasten his belt. His eyelids and the skin around his eyes appear to be swollen too. The percentage of plasma protein in his blood is lower than normal. What is the matter with this man?

Blood transfusion for a child of Jehovah's witnesses.

A child is admitted to the Emergency ward of a hospital. She has been knocked down by a motorcar and has lost a substantial amount of blood. The only way to save the child's life in the judgement of the attending physician will be transfusion. However, there is a problem, namely that the parents are not likely to give their consent because their religion forbids transmission to their child's body of another person's blood. “She is in God’s hand.” Should transfusion take place in spite of this, the parents will reject the child. What do you think the physician should do?

Procedural

Descriptive

Strategy problem

Declarative

Explanatory

Explanation problem Fact-finding problem

Normative

Knowledge of the world

Public knowledge (e.g., to be found in books)

Personal knowledge

Moral dilemma resolution problem

An example: Love at first sight?

The picture shows the Capilano Bridge in the neighborhood of Vancouver. On this bridge, Dutton and Aron conducted in 1974 a curious experiment. A beautiful research assistant (‘Gloria’) was placed halfway on the bridge. She asked passing males to write, on the spot, a short story about a number of ambiguous pictures shown. In addition, she offered the subjects the opportunity to phone her a few days later to be informed about their score for the story. Exactly the same procedure was carried out at another site, a short bridge that crosses the Capilano River three meters above water level. The stories were scored with regard to the amount of erotic content. In addition, the percentage of participants that called the research assistant was recorded. About 12% of the low-bridge-males called Gloria, whereas almost half of the Capilano bridge participants called. In addition, the stories produced on the Capilano Bridge contained significantly more erotic material.

cell level

tissue level

organ (system) level

person level

family level

population level

“Problems” of pathological functioning

“Problems” of normal functioning

Levels of human functioning

Varieties of written problems

How did PBL change the face of conventional HPE?

  Many schools have integrated basic-science with clinical-science teaching: both horizontally and vertically

  Greater emphasis on community health needs   Earlier introduction into the health care system   Behavioral sciences included to larger extent   Many schools use patient problems as focus of learning   Small-group tutorials have been introduced even in

conventional programs   Less lectures and less parallel programming   More emphasis on professional skills, communication

and ethics

Do you expect PBL-graduates to be different? If so: in what ways?

What was promised?

  According to the early literature, PBL would contribute to   renewal of the medical curriculum   a more student-centered learning environment   students acquiring better clinical reasoning

skills   students acquiring better interpersonal and

other relevant professional skills   (acquisition and retention of relevant medical

knowledge)

Did PBL act as a enzyme for curriculum renewal?

  Generally: Yes   Many schools have integrated basic-science with

clinical-science teaching: both horizontally and vertically

  Greater emphasis on community health needs   Earlier introduction into the health care system   Behavioral sciences included to larger extent   Many schools use patient problems as focus of learning   Small-group tutorials have been introduced even in

conventional programs   Less lectures and less parallel programming   More emphasis on professional skills, communication

and ethics

Does PBL provide a more student-centered learning environment?   According to most studies: Yes   Students report less stress, fewer feelings of

being powerless, and less fatalism (Kuhnigk & Schauenburg, 1999)

  Students feel more supported by learning environment, and experience more social support (Kiessling, et al., 2004)

  In Dutch nationwide surveys among students, problem-based curricula always finish first in their category

Student ratings of Dutch medical curricula 2006

Source: Keuzegids 2006-2007

Student ratings of Dutch psychology curricula 2004

Source: Keuzegids 2004-2005

Does PBL help students acquire clinical reasoning skills?   According to few studies available:

Yes   Hmelo (1997, 1998a, 1998b): PBL has

strong effects on clinical reasoning and representation skills

  Schmidt et al. (1995)   Schuwirth et al (1999): PBL improves on

diagnostic competence with 8%

Schuwirth’s (1999) study of diagnostic competence

40

45

50

55

60

65

70

1 2 3 4 5 6

Year

Dia

gnos

tic s

core

Maastricht PBL

Groningen

Interlude: Potentially important professional competencies of university graduates   Getting much work done

  productivity, efficiency, effectiveness   Getting work done with the help of others

  team work, leadership   Getting work done in new innovative ways

  creativity, problem-solving   Getting work done using science

  finding, reviewing, producing, disseminating scientific knowledge

Does PBL help students acquire such professional competencies?   All studies point in the same direction:

Yes   e.g., Woodward & McAuley (1983) study

using supervisor ratings   Van Dalen et al. (2002)   Schmidt et al. (2006) using self-ratings

How good are you, compared to colleagues from other universities in: Communication skills?

1 2 3 4 5 Much poorer Much better Equally good

Schmidt’s et al. (2006) comparison of professional skill levels

  Comparing performance of about 1400 graduates from two medical schools in the Netherlands

  Using self-ratings on five-point scales

Schmidt’s et al. (2006) comparison of professional skill levels

  Comparing performance of about 1400 graduates from two medical schools in the Netherlands

  Using self-ratings on five-point scales

Maastricht PBL-curriculum

Rotterdam conventional curriculum

Interpersonal competencies e.g., communication skills, leadership skills, and collaboration skills 3.94 3.24

Schmidt’s et al. (2006) comparison of professional skill levels

  Comparing performance of about 1400 graduates from two medical schools in the Netherlands

  Using self-ratings on five-point scales

Maastricht PBL-curriculum

Rotterdam conventional curriculum

Interpersonal competencies e.g., communication skills, leadership skills, and collaboration skills 3.94 3.24

Medical knowledge level 3.16 3.21

General academic competencies e.g., conducting research, report writing skills, and statistics knowledge level 3.21 3.12

Schmidt’s et al. (2006) comparison of professional skill levels

  Comparing performance of about 1400 graduates from two medical schools in the Netherlands

  Using self-ratings on five-point scales

Maastricht PBL-curriculum

Rotterdam conventional curriculum

Problem-solving and self-regulation competencies e.g., problem-solving, use of resources, self-directed learning

3.70 3.28

Physical examination skills 3.60 3.19

Task-supporting competencies, e.g., productivity, efficiency 3.58 3.42

Does PBL support acquisition and retention of medical knowledge?

  According to most curriculum-level studies: No

Van der Vleuten et al. (2004): Retention of medical knowledge

Assessments of the value of PBL   Colliver (2001): “There is no

difference in knowledge acquired with students from conventional schools. So why bother?”

  Shanley (2007): “Let’s leave the empty glass of PBL behind.”

  Kirschner (2007): “Minimally guided instruction does not work.”

Does PBL support acquisition and retention of medical knowledge?

  According to most curriculum-level studies: No

  According to micro-level experimental studies: Yes, and with large effects (e.g., De Grave et al., 2001)

A stinging incident Mr Laeven practises a hobby that most of us find rather scary to say the least. At the back of his yard, he has a number of beehives. He always visits his bees first thing in the morning. One morning, when he arrives at the back of the yard, he stumbles and hits a hive. No need of telling what happened. He hardly manages to reach the house and alarms his wife, who calls their family physician. Mr Laeven's sting wounds are seriously aching and after some five minutes he feels faint and gets dizzy. He is in a cold sweat and is getting sick. The family physician arrives after 15 minutes. She sees a conscious looking man with a regular breath. The family physician counts a pulse rate of 88 beats a minute and takes the blood pressure: 120/80 mmHg. After five minutes, she takes the pulse rate, 104 beats/minute, the blood pressure then being 115/80 mmHg. After another five minutes Mr Laeven has a pulse of 120 beats/minute and a blood pressure of 85/55 mmHg. The acra are hot. The family physician injects adrenaline intravenously.

Number of propositions recalled (De Grave, et al, 2001)

Does PBL support acquisition and retention of medical knowledge?

  According to most curriculum-level studies: No

  According to micro-level experimental studies: Yes, and with large effects (e.g., De Grave et al., 2001)

  My interpretation: Serious methodological errors were made in the Colliver and Kirschner curriculum-level reviews

Cohen’s d

Curriculum-comparisons approaches to PBL: Maastricht findings (Schmidt, 2009)

26.9

17.1

6.6

0

5

10

15

20

25

30

Conventionalcurriculum

PBL in basic sciences Full PBL -curriculum

Study Group

Per

cen

t

Attrition rates in subsequent Harvard dental curricula

Graduation rates Dutch medical students 1989-1998

70

75

80

85

90

95

Study duration Dutch medical students 1989-1998

6.4

6.6

6.8

7

7.2

7.4

7.6

7.8

Hypothetical effects of two treatments

7

7 8

6 7 8

6 7 8

5 6 7 8

4 5 6 7 8 9

4 5 6 7 8 9

3 4 5 6 7 8 9

1 2 3 4 5 6 7 8 9 10

7

6 7 8

6 7 8

6 7 8

6 7 8

6 7 8 9

5 6 7 8 9

4 5 6 7 8 9

4 5 6 7 8 9

3 4 5 6 7 8 9 10

Treatment A Treatment B

MeanA = 6.81 MeanB = 6.31

Hypothetical effects of two treatments (poor performers removed)

7

7 8

6 7 8

6 7 8

5 6 7 8

4 5 6 7 8 9

4 5 6 7 8 9

3 4 5 6 7 8 9

1 2 3 4 5 6 7 8 9 10

7

6 7 8

6 7 8

6 7 8

6 7 8

6 7 8 9

5 6 7 8 9

4 5 6 7 8 9

4 5 6 7 8 9

3 4 5 6 7 8 9 10

Treatment A Treatment B

MeanA = 7.38 MeanB = 7.41

Dropout = 19% Dropout = 30%

Summary statistics of the Schmidt (2012) study and the effects of correcting for differences in attrition and study duration