kalie’s test a workshop for parents on standardized testing

Kalie’s testA workshop for parents on standardized testing

Master’s Project ReviewStanford UniversityMay 17, 2002Michael Griffin

Michael Griffin / Master’s Project Review / 17 May 2002 / [email protected]

Presentation overview

• Thumbnail description• Design history• Approach• Next steps


Kalie’s test: description

• Workshop for middle school parents on standardized testing– Story of a character named Kalie– Structured interrogation of interactive

models– Participation in scenario– Role-playing– Reflection– Discussion


Kalie’s test: context of use

• Live group setting• Computer-equipped facilities• Disseminated over the web


Design history

• Primes (NSF-funded)• Early work• Seattle


Primes

• NSF-funded• Housed at IRL, WestEd, Stanford• Learning goals

– You can do math, important in middle school, today’s classrooms, help your child

• Products– TV special– Parent Guide – 9 workshops (*)


Primes workshops

• 2-hour experiences• Workshop-in-a-box• Get parents in the door • Build math confidence, understanding

of school setting > more involvement• Middle school math + relevant topics


Early work

• Testing is relevant topic• Math: percentile• Learn percentile using hands-on

activities (model reform curriculum)• Paper materials• Idea: use simulation


Early work: simulation


Seattle

• Met with parents, teachers at a public elementary school in Seattle

• 1st meeting: show-and-tell, listen– “Why percentile?”– “What about big issues?”

• 2nd meeting: informational workshop– Topics: choosing schools? legal issues? what

do scores mean? how to improve scores?


Rationale

• Learning problem• Learning goals• Conceptual framework


Learning problem

• Parents need more knowledge to advocate effectively

• No standards for content• Discourse is partisan, controversial• Piecemeal content used to support

opinions• Reasoning requires schematic

understanding


Learning goals

• Parents need to be able to:– Engage in discourse– Make informed decisions– Develop and defend arguments– Flexibly accommodate new information– Develop informed opinions


Conceptual framework

• Situative, socio-historic perspective– Social practices of inquiry and learning– Support for identity– Discourse and representation– Realistic problems and settings– Cognitive apprenticeship

• Echoed in Friere’s Pedagogy of the Oppressed– Attention to cultural situation– Encourage voice, criticality, praxis


Approach

• Workshop overview• Tour


Workshop overview:Activities

Intro Software ScenarioRoleplays Reflection

20 min 20 min 20 min 25 min 15 min 20 min

Group


Tour: Introduction

• Personal introductions• QuickTime video viewed as whole group

– Explain learning goals, workshop plan– Overview of testing system– Perspectives of various stakeholders– Why test?– Hot topics

Intro - Software - Scenario - Role plays - Reflection - Group


Tour: Software description

• Model stands in for an expert’s understanding

• ‘Glass box’ design, weighting/rationale• Actually useful for understanding NYT!• Doesn’t include

– Situative perspective (e.g., identity); testing is developed and discussed from cognitive/behavioral perspective

– Detailed model of knowledge; ‘alignment’



Tour: Software activities

• Intuition:model is complex to novices (computer users, content learners)

• Short, structured activities– Learn skills, concepts and interface

simultaneously– Staging, bridging

(Roy Pea’s principles for learning from visualizations, WorldWatcher)



Tour: Software activities example

• Small groups explore model• Worksheet scaffolds inquiry

– What happens when Kalie’s knowledge in math increases?

• To her score• In the knowledge map• Does she get a passing grade?

– Can you find a way to improve her score without increasing her knowledge?



Tour: ScenarioIntro - Software - Scenario - Role plays - Reflection - Group

• QuickTime video viewed by small groups– Story about Kalie taking a test– Results are low, though she does well in

class

• Worksheet– Structures inquiry– Team builds rationale for confused parents


Tour: Role plays

• QuickTime video viewed by small groups– Stories about Kalie, class taking a test– Each group watches different case– Results are confusing to variety of

stakeholders

• Role play– Individuals take notes, make case– Debate each other– Share experiences across groups



Tour: Reflection

• Worksheet to guide reflection– What new ideas have I come across?– What factors might be at play with my

child?– How do I feel about tests?– What might I try to do to make my child’s

experience better?– Can I help my community?



Tour: Group discussion

• Discussion, sharing of ideas that come from reflection

• QuickTime video viewed by whole group– Next steps– Resources for more information

• Take-home booklet with resources



Next steps

• User studies• Learner study• More content/model vetting• Repurpose for other users?• Expand from glass box > creative tool


Additional details


Learner study


Relationships


Usability

• Nielsen’s usability measures– Learnable?– Efficient?– Memorable?– Errors?– Satisfaction?

•Nielsen’s heuristics–Simple, natural dialogue–Speak the user’s language–Minimize cog. Load–Consistency–Feedback–Exits–Shortcuts–Good error msgs.–Prevent errors–Help


Interface design considerations

• Rao’s heuristics– Graphics first– Use data’s grain– Focus, context– Animate transitions (esp.

for discontinuous changes, on/off)

– Stable, consistent spaces– Direct manipulation– Spotlight computational

results

• Direct manipulation– Bridge the gulfs– Directness– Semantic, articulatory

distance

kalie’s test a workshop for parents on standardized testing

Documents

middle school parents

worldwatcherintro software

software descriptionmodel

software activitiesintuition

boxget parents

hot topicsintro software

public elementary school

relevant topicmath