Formative Assessment:

Effectiveness of

ConcepTests and

Conceptual Probes in Math

and Science

Dr. George Collison &

Dr. Judith Collison

Who are we?

Why are we here?

We will make concrete a set of

effective links between high

quality assessment and

effective pedagogy in physics

and mathematics education.

What is the

Concord

Consortium?

CC is a non-profit educational think tank located in Concord Massachusetts. The small group of scientists and educators at CC is dedicated to “realizing the educational potential of technology”

The work and products of CC

are widely respected.

�

“Writing grants dries up our pens.”Bob Tinker, founder and President of CC

Some of CC‟s Products

We invented the MBL�

The Virtual High School�

Global Lab�

More of CC‟s Products

Facilitating Online Learning

A guide for moderators

Seeing Math™ web-video case studies

CAPA - assessments

in electronics.

More of CC‟s Products

INTEC

Molecular Workbench�

Hands On Physics

Outline of this talk

Halloun & Hestenes

FCIMazur @ Harvard

Peer Instruction 1995

ConcepTests™

Web-based Conceptual Probes

INTEC

Facilitating Online

Learning 2003

ILT @ Harvard

DVD 2007

Molecular Workbench 2008

Data driven change:

The mirror of the FCI

On Dr. Mazur‟s first use of conceptual

questions in his physics class, a student

asked:

“Do you want me to answer

these questions the way you

are teaching me or the way I

really think?”�

Data from the FCI

Physics for Non-Majors

Mazur, Peer Instruction 1995

��Pedagogical change

INSERT list of changes in pedagogy

induced by attention to student

understanding.

Big move: out of delivery system into

confirmation.

��Other Physics Education

Centers

Other centers in the physics ed.

research.

University of Arizona, U. of Maryland. R.

Hake, U. Seattle, U. Oregon.

University of Massachusetts Amherst.

“the 8am to 12noon physicist”

�

“At M.I.T., Large Lectures Are Going the Way of

the Blackboard” 01/12/09

“Using the „Beauties of Physics‟ to Conquer

Science Illiteracy.” 07/17/07

Dozens of powerful universities are doing away

with the physics lecture in favor of peer instruction.

�

The measure of Change

Normalized gain

Normalized gain on the FCI test to be the average increase in students' scores on the FCI divided by the average increase that would have resulted if all students had perfect scores on the post-instruction test.

(R. Hake 1992 Am. J. of Phys, 66, 1, pp. 64-74)

Normal.Gain AVG FCIpost FCIpre AVG(FCIperfect FCIpre )

)

�

Normalized gain Traditional vs.

Peer Instruction teaching

N=6,000 college students

Traditional Instruction: gain = 0.23 ± 0.04

Peer Instruction: gain = 0.48 ± 0.14

Gains as high as 0.60 to 0.80 have consistently been measured with PI.

Similar results hold for secondary instruction.

(R. Hake, 1992 Am. J. of Phys, 66, 1, pp. 64-74)

Normal.Gain AVG FCIpost FCIpre AVG(FCIperfect FCIpre )

)

Structure of a PI driven class

Structure of a PI driven class

Structure of a PI driven class

ConcepTest

Brief lecture

Explanation

Repeat from

Start

>70% Correct

Poll #1

30-70%

Correct

Poll #2

<30% Correct

Peer

Discussion

Revisit

Concept

Sample ConcepTest

A piece of metal is

heated 500 degrees

Centigrade.

Please vote as

individuals:

1: The hole diameter will expand

2: The hole diameter will contract.

3: The hole diameter will stay

the same.2 minutes

Sample ConcepTest Group Work

A piece of metal is

heated 500 degrees

Cent.

Please vote as

a group:

1: The hole‟s diameter will expand

2: The hole‟s diameter will

contract.

3: The hole‟s diameter will stay

the same.4 minutes

Video from the ILT DVD

INSERT video from ILT

Discussion of PI 5 minutes

Resources on Peer Instruction

• 2,000 + published articles

• Interactive Learning Technologies DVD +

online databank of questions

• Calculus Concept Inventory, Epstein of Duke

University, and ConcepTests by Hughes-Hallet

from Arizona

Conceptual Probes

• Independently developed at CC parallel to

Dr. Mazur’s work

• Online “conceptual temperature taking”�

for CC’s online PD courses

Conceptual Probes

Characteristics

• Anonymous, anywhere, anytime

• Possibly dynamic with video or simulations

• Very wide administration + database of

responses.

• Nations or regions can administer

conceptual probes.

Sample Conceptual

Probe

Take a piece of paper.

Roll it into a cylindrical shape with the

long edge as the cylinder’s end and

imagine it capped at both ends.

Take the same paper into a cylindrical

shape, this time with the short side as

the cylinder’s end.

Sample Conceptual

Probe

Consider the two cylinders formed.

Please vote as individuals:

1: If the volumes are the same

2: If the volumes are different

Sample Conceptual

Probe

Consider the two cylinders formed.

Please vote as a group:

1: If the volumes are the same

2: If the volumes are different

Discuss for 4 minutes

Conceptual Probe

Discussion

What did you learn from the exercise?

Conceptual probes can be administered anytime, anyplace.

A database holds all responses, including reasons for answers.

Conceptual Probe

Design

Molecular Workbench permits construction and administration of conceptual probes, including animations or models.

Where is the innovation?

Confluence of

– High quality testing

– Effective pedagogy

Formative Assessment

Peer instruction

&

Conceptual Probes

Inform teaching and individual accomplishment