Alina Goldman

Committee Members:

Benjamin Bederson

June Ahn

Tammy Clegg

Predicting and Motivating

Achievement in Self-Paced LearningA Formative Design, Study and

Evaluation

HumanComputerInteractionLaboratory

Online Tradition

al

Paths to Computer Science

(CMSC 198 C, D, E)

Septembe

r

December………………………

In Class Activities

30 minutes Mini Lecture

• Short in class lectures and student presentations

90 minutesActivity/HW

• In class programming activity

• Working on homework

30 minutesAssessment

• Students who finish a module take an in class assessment or midterm

Students have trouble succeeding in

self-paced courses

Gilovich, Griffin and Kahneman (2002). Heuristics and Biases: The Psychology of

Intuitive Judgment

Fishbach and Dhar (2005). A. Journal of Consumer Research

Torres, Gross, and Dadashova (2010). A. Journal of College Student Retention

Ariey and Wertenbroch (2002). Psychological Science.

Ferrari (1992). A. Journal of Psychopathology and Behavioral Assessment

Dunning, Heath, and Suls (2010). Psychological Science in the Public Interest

Bauman (2002). Distance Education Report

What factors predict

student success in self-

paced environments?

How can we encourage

students to set

realistic goals?

How can we motivate

students to work

persistently and meet

course goals?

17 men/14 women (31 total)

6 colleges across campus

4 ethnicities

Mean GPA: 3.5

Participants

`

Design Goals

Achievable

goals

Goal

follow-through Meaningful

incentives

Meaningful

feedback

Peer motivation

Success Capacity

Calibration

Anonymous and

personalized

Progress Feedback

Course Plan

System Overview

Leaderboard, Badges,

Points and Prizes

Pre Course

Survey

Course Plan

Post-Credit survey

Post-Credit survey

Post-Credit survey

Calibration

Observation

Computer Science

Background

Self-Efficacy

Time and Goal

Commitments

Calibration

Calibration

Composite Score

Credit Suggestion For

Calibration

Composite Score

Credit Suggestion For

Num

ber

of S

tude

nts

Course Plan

Anonymous Progress Feedback

Personal Feedback and Forecasting

Badges, Points & Prizes

Progress Leaderboard

Evaluation

Credits Pursued Vs. CompletedN

um

ber

of

Cre

dits

= 4 students

0

5

10

15

20

25

4 5 6 7 8 9 10 11 12 13 14

Nu

mb

er

of

Stu

de

nts

Weeks

Not Started

C1.M1

C1.M2

C1.M3

C1.M4

C2.M1

C2.M2

C2.M3

Assignments Viewed ( Credit 2)

Last Module Completed Week 4-14

Was the calibration suggestion

a good predictor of credits

students completed?

No,

however…

Which background variables

predicted the total number of

credits students completed?

Programming Experience

Procrastination

Self-Efficacy

other classes got in

the way and this

[course] got pushed

to the back burner

Design Considerations

Some calibration predictors

are significant

Compare students on

individual predictors

Did students follow the

suggested credit goal?

Not explicitly,

but they considered it

When choosing their

credit goal

I am not at all familiar

with coding and I don't

know how difficult its

going to be. Neither

am I good at

meeting goals.

I have lots of other commitments

this semester, such as GRE,

grad school apps, service [gap],

and research. I picked 2

credits instead of 1

because the course is

really interesting

Design Considerations

Students anchored and

adjusted credit suggestion

Give lower suggestions

expecting that students

adjust upward

What did students think of

the course plan

and deadlines?

Most students had trouble

meeting deadlines

but…

found them

useful & necessary

I liked setting my own

schedule. I had actually planned

when I should finish each credit

before I got to the first class,

but it helped me to be able to

schedule each module.

deadlines were set really

high and after I fell behind

I never saw a reason to

catch up to them.

although we didn't use a course

plan in the Spring, we did have a

much harder deadline (May 30th)

to finish all our work. This helped

me manage my time and

finish the work accordingly

Design Considerations

Deadlines were not effective

Deadlines should be broken

into smaller increments and

spaced more evenly

Students need multiple “hard”

deadlines that shouldn’t be

extended

Students paid attention to

aggregate feedback

but…

were demotivated by

stagnation

Were students motivated by

group feedback?

watching half of the

class fall behind at the

same time was

definitely a non-

motivating factor

Everything is fine for me,

[but] I have been noticing

that a lot of other student

s are having trouble

understanding a lot of the

core concepts needed to

complete the assignments

Comparing themselves to the

class can be both motivating

and demotivating

Rather than showing

everyone’s progress,

compare students in

progress cohorts

Design Considerations

Students did not actively

respond to individual

feedback

but enjoyed the process of

tracking their progress

Were students motivated by

individual feedback?

I find it motivating to

track my progress

Through each credit

with the little check

marks and such

Students don’t pay attention to

personal feedback

Students should receive

dynamic feedback whenever

they log in to work

Design Considerations

Students enjoying marking

progress path

Create intrinsic motivation

from allowing students to

manually check off

assignments

Design Considerations

Some students were

motivated to see

themselves at the top

other students wanted

more meaningful

recognition

Were students motivated by

the leaderboard?

once you're in the

top three, you feel

like you want

to keep going

I had a chemistry teacher who

would ask those students who

had done an especially nice job

with a problem on a test

to describe what they had

done …in front of the whole

class. It was an honor to be

called up, and a [real] motivator.

initially motivated

by badges

Prizes were motivating

only when students really wanted them

Were students motivated by

Badges, points and prizes?

At first, getting badges…

was exciting, but it

did not continue to be

motivating because

it became normal.

prizes became

meaningless for

students who

fell behind

Were students motivated by

Badges, points and prizes?

once someone is

behind, they might

as well not even try,

as the motivators

are unattainable

Some didn’t find the

prizes meaningful

Some didn’t appreciate

their visibility

Were students motivated by

Badges, points and prizes?

There were no motivation

practices that could make

up for the difficulty of the

class. Chocolate doesn't

Really matter when

the object is to learn

something

I don't really care

about the stickers, and

I don't like other

people seeing how

many I got

Students need meaningful

recognition and incentives

Reward students by letting

them take on new roles (e.g.

TA badge for a specific skill)

Design Considerations

Students need to be able

to recover from mistakes

Only positive reinforcement

Reward students for weekly

progress instead of overall

progress

Design Considerations

Visualize achievement

predictors meaningfully

Help students motivate

each other

Future Work

Thanks to…

Ben Bederson

Tammy Clegg

June Ahn

My Family

HCIL

Questions?

alina-

goldman.comalina.goldman@gmail.c

om