assessment: not just a stop along the way · assessment: not just a stop along the way barbara d....

Assessment: Not Just a Stop Along the Way

Barbara D. Wright

[email protected]

Our Roadmap

• The “disrupted” context for assessment of student learning

• Unbundling, rebundling, and assessment

• Closing the loop – and why it matters

• How assessment pulls it all together

• Assessment, equity, and GEMS

3/20/2015 CSU Fullerton 2

“Disruptions”

• Open courseware (e.g., Carnegie Mellon, MIT, Kahn Academy)

• Massive, open, online courses (MOOCs)

• Course Redesign (Carol Twigg)

• Competency-based education (CBE; USDOE)

– Southern New Hampshire University

– Western Governors University

– Capella, Brandman, others

June 2014 IGEA, Burlington, VT 3

“Disruptions,” cont.

• Challenges, “testing out” (CLEP, CSU system, University of Akron)

• DIY trends (A. Kamenetz)

• Alternatives to traditional degrees, e.g.,

– Certificates

– Badges

– Peer reviews

– Social networks


The Mother of All Disruptions:

• The shift from teaching to learning – From inputs to outcomes

– From teacher- to student-centered

– From covering the material to deep learning

– From time held constant and varied learning (bell curve) to time varied, learning constant

– From grading to assessing, credits to competency

– Competency or mastery learning as goal

– From filter to pump, i.e., sorting to supporting



“Colleges maintain fixed and fossilized structures and tell their students to be flexible and open to new things.” (John Tagg, The Learning Paradigm College, p. 352)

A fundamental challenge:

What’s happening to HE?

• Unbundling

• Deinstitutionalization

• Disintegration

• Incoherence

• Fragmentation

• “A tsunami of chaos”


Maybe that’s why we need maps and markers.

Unbundling is promoted by . . .

• A changing student population

• Students’ need for flexibility, alternatives

• Student mobility

• Cost, debt

• For-profit institutions

• Technology

• Trend toward externalization of services

• Society-wide: increasing personalization; growing skepticism toward traditional institutions


Unbundling Traditional Institutional Roles:

• Knowledge

• Instruction

• Validation of learning

• Degrees

• Social networking

• Cultural opportunities

• Personal development


Unbundling Traditional Faculty Roles:

• Faculty have traditionally

– Owned the knowledge, expertise

– Designed the curriculum

– Designed the course

– Delivered the course

– Conducted the testing/assessment of what’s learned


Unbundling the Student Experience:

• Commuting

• Part-time attendance

• Attendance at multiple institutions

• Swirling

• Transfer

• Face to face, hybrid, and online coursework

• Varying levels of preparation, strengths


There’s a countervailing trend: rebundling

• “Intentionality”

• Integrative learning

• High-impact practices

• From factoids to deep, inquiry-based learning

• LEAP

• The Degree Qualifications Profile (DQP)

• General Education Maps and Markers (GEMS)


The Mother of All Rebundling Tools: Assessment


Assessment is the key – done right

Assessment and GEMS Principles

• Proficiency

• Agency and Self-Direction

• Integrative Learning and Problem-Based Inquiry

• Equity

• Transparency and Assessment


June 2014 15

Assessment is the key to . . .

• Improving and validating students’ knowledge, skills, proficiencies

• Engaging all students in self-assessment, agency, self-direction

• Choosing, using the right HIPS

• Achieving equity

• Communicating with multiple audiences (transparency)

IGEA, Burlington, VT

Assessment is the key if . . .

• The approaches are direct, embedded

• They encourage deep, integrative learning

• They elicit, assess higher-order intellectual skills

• Demonstrate levels of achievement

• Are educative for students and faculty

• Promote self-assessment, life-long learning

• (Not checklists or standardized testing)


Assessment is the key if

• We actually close the loop at all levels

– Student

– Course

– Program

– Institution

– Institutional structures and practices (e.g., planning, budgeting, oversight)


Assessment vs. Evaluation:

• Inquiry

• Improvement

• Curiosity

• Candor, honesty

• Transparency

• Collegiality

• Rewards for quality, equity

• Judgment

• Consequences

• Anxiety

• Circling the wagons

• Concealment

• Avoiding blame

• Rewards for looking good, “excellence”


Key definitions . . .

• Criterion-referenced: testing or assessment in which student performance is judged in relation to pre-established standards and not in relation to the performance of other students.

• Norm-referenced: testing or assessment in which student performance is judged in relation to the performance of a larger group of students, not measured against a pre-established standard.



Thinking about standards of student performance . . .

• Absolute standards: knowledge/skill level of champions, award winners, top experts

• Contextual standards: appropriate expectations for, e.g., a 10-year old, a college student, an experienced professional

• Developmental standards: amount of growth, progress over time, e.g., 2 years of college, 5 years

• Benchmarked standards: performance goals set by a group or consortium

• Institutional, regional, national standards – emerging?

1. Outcomes, questions

2. Gathering evidence 3. Interpretation

4. Use

The Assessment Loop


Why is it so

hard to

close the

loop? Black

Black Hole

of

Assessment


THE

#3 Interpretation, or making meaning out of evidence. . .

• Is it included as an explicit step?

• Is the “community of interpretation” inclusive?

• Is it characterized by courage & candor?

• Is this a non-judgmental process of inquiry and understanding, not blaming?


Questions for Interpretation. . . . • What did we find?

• Did we disaggregate for subpopulations?

• What do the findings mean?

• Who can shed light on them?

• What do we want to do about them?

• Is the plan feasible (e.g., sustainable, aligned with campus culture)?

• Are we communicating with the campus?

• Where is institutional support – or not?

• What will it cost?


4. Using findings . . .


• Does the notion of “use” go beyond – Collecting data? – Writing reports? – Making recommendations? – Planning for changes? – Refining the assessment process?

• And “close the loop,” i.e.,

– Implement recommended changes? – Retrace the loop to determine the effect of

changes? On all student populations?

Using findings, cont.: Do we have

• Credible recommendations? • Ongoing communication, tailored to different

audiences? • Broad buy-in (e.g., faculty, administrators, other

educators, students, employers)? • Budget? • Structures for systematic reporting and feedback? • Steady, knowledgeable leadership? • A schedule for revisiting, adapting, moving on? • Data management and analysis capability? • Public praise and rewards for problems solved?


Making it happen . . .

• Students as partners in closing the loop – and a vast, untapped resource?


If it’s really all about learning for all students, then . . .

• Assessment is the key to dealing with disruptions

• Closing the assessment loop is the key to equity


assessment: not just a stop along the way · assessment: not just a stop along the way barbara d....

Documents