Lesson TwelveLesson Twelve

Alternative Alternative AssessmentAssessment

Yun-Pi Yuan 2

ContentsContents Tests vs. AssessmentTests vs. Assessment DefinitionDefinition & Characteristics of Alternative Assessm & Characteristics of Alternative Assessm

entent Traditional Traditional vsvs “Alternative” Assessment “Alternative” Assessment Performance-based AssessmentPerformance-based Assessment Why Alternative AssessmentWhy Alternative Assessment PortfoliosPortfolios

definition, characteristicsdefinition, characteristics advantages, guidelinesadvantages, guidelines

ObservationObservation JournalJournal SummarySummary

Yun-Pi Yuan 3

Questions to DiscussQuestions to Discuss What’s the difference between tests What’s the difference between tests

and assessment?and assessment? What does “alternative assessment” What does “alternative assessment”

mean?mean? What’s the difference between What’s the difference between

traditional and alternative traditional and alternative assessment?assessment?

Is performance-based assessment Is performance-based assessment the same as alternative assessment?the same as alternative assessment?

Yun-Pi Yuan 4

Tests vs. Assessment (1)Tests vs. Assessment (1) All tests are assessments, but not all All tests are assessments, but not all

assessments are tests.assessments are tests.

(Brown (Brown 5)5)

Tests

AssessmentTeaching

Yun-Pi Yuan 5

Tests vs. Assessment (2)Tests vs. Assessment (2) Tests:Tests:

Formal proceduresFormal procedures Strict time limitationsStrict time limitations Sample the performance of an Sample the performance of an

individual in a specific domainindividual in a specific domain Assessment:Assessment:

Includes all occasionsIncludes all occasions Both formal and informalBoth formal and informal Alternative assessment: early 1990sAlternative assessment: early 1990s

FairnessFairness Balance of power relationships in the Balance of power relationships in the

classroomclassroom

Yun-Pi Yuan 6

Alternative AssessmentAlternative Assessment

Definition:Definition: Any method of finding out what a Any method of finding out what a

student knows or can do that is student knows or can do that is intended to show growth and inform intended to show growth and inform instruction and is alternative to instruction and is alternative to traditional forms of testing, namely, traditional forms of testing, namely, multiple-choice tests.multiple-choice tests.

Multi-assessment methods, rather Multi-assessment methods, rather than sticking to traditional paper-than sticking to traditional paper-and-pencil tests.and-pencil tests.

Yun-Pi Yuan 7

Characteristics of Alternative Characteristics of Alternative AssessmentAssessment

Require Ss to perform, create, produce, or do Require Ss to perform, create, produce, or do somethingsomething

Use real-world contexts or simulationsUse real-world contexts or simulations Assess Ss on what they do in class every dayAssess Ss on what they do in class every day Focus on processes as well as productsFocus on processes as well as products Higher-level thinking & problem-solving skillsHigher-level thinking & problem-solving skills Provide info. about Ss’ strengths & weaknessesProvide info. about Ss’ strengths & weaknesses Use human judgment in scoringUse human judgment in scoring More . . . . More . . . . (cited in Brown 252; Brown & Hudson 654-55)(cited in Brown 252; Brown & Hudson 654-55)

Yun-Pi Yuan 8

Traditional versus “Alternative” Traditional versus “Alternative” Assessment (1)Assessment (1)

One-shot testsOne-shot tests Continuous, longitudinal Continuous, longitudinal assessmentassessment

Indirect testsIndirect tests Direct assessmentDirect assessment

Inauthentic testsInauthentic tests Authentic assessmentAuthentic assessment

Individual projectsIndividual projects Group projectsGroup projects

No feedback provided to No feedback provided to learnerslearners

Feedback provided to learnersFeedback provided to learners

Speeded examsSpeeded exams Untimed examsUntimed examsDecontextualized test tasksDecontextualized test tasks Contextualized test tasksContextualized test tasks

Norm-referenced score Norm-referenced score interpretationinterpretation

Criterion-referenced score Criterion-referenced score interpretationinterpretation

Standardized testsStandardized tests Classroom-based testsClassroom-based tests

Bailey, Kathleen M. (1998) Learning about Language Assessment . (p. 207)

Yun-Pi Yuan 9

Traditional versus “Alternative” Traditional versus “Alternative” Assessment (2) Assessment (2) (Brown 13)(Brown 13)

One-shot, standardized One-shot, standardized examsexams

Continuous long-term Continuous long-term assessmentassessment

Timed, multiple-choice Timed, multiple-choice formatformat

Untimed, free-response formUntimed, free-response formatat

Decontextualized test itemsDecontextualized test items Contextualized Contextualized communicativecommunicative

Scores suffice for feedbackScores suffice for feedback Individualized Individualized feedback/washbackfeedback/washback

Norm-referenced scoresNorm-referenced scores Criterion-referenced scoresCriterion-referenced scores

Focus on the “right” Focus on the “right” answeranswer

Open-ended, creative Open-ended, creative answersanswers

SummativeSummative FormativeFormative

Oriented to productOriented to product Oriented to processOriented to process

Non-interactive Non-interactive performanceperformance

Interactive performanceInteractive performance

Fosters extrinsic motivationFosters extrinsic motivation Fosters intrinsic motivationFosters intrinsic motivation

Yun-Pi Yuan 10

AdvantagesAdvantages

Traditional multiple-choice tests:Traditional multiple-choice tests: Highly practical Highly practical Highly reliableHighly reliable

Alternative assessment:Alternative assessment: Beneficial washbackBeneficial washback AuthenticityAuthenticity greater face validity greater face validity

Some suggestions:Some suggestions: See Brown 254.See Brown 254. (See figure 10.1, Brown 253) (See figure 10.1, Brown 253)

Yun-Pi Yuan 11

Performance-based AssessmentPerformance-based Assessment Productive, observable skills of content-Productive, observable skills of content-

valid tasksvalid tasks A subset of authentic assessment, but not A subset of authentic assessment, but not

all authentic assessment is performance-all authentic assessment is performance-basedbased

Alternative assessment could be Alternative assessment could be performance-based.performance-based.

Characteristics of performance assessment:Characteristics of performance assessment: Constructed responseConstructed response Higher-order thinking involved, with open-ended, Higher-order thinking involved, with open-ended,

meaningful, engaging, and authentic tasksmeaningful, engaging, and authentic tasks Integration of language skillsIntegration of language skills Both process and product are assessedBoth process and product are assessed A student’s mastery is emphasizedA student’s mastery is emphasized

Yun-Pi Yuan 12

Three Basic Assessment TypesThree Basic Assessment Types Following Brown & Hudson’s Following Brown & Hudson’s (1998) (1998)

classification:classification: Alternatives in assessment:Alternatives in assessment:

Selected response: Selected response: T/F, matching, multiple-choiceT/F, matching, multiple-choice

Constructed response:Constructed response: Fill-in, short answer, performance assessmentsFill-in, short answer, performance assessments

Personal response:Personal response: Conferences, portfolios, self/peer assessmentsConferences, portfolios, self/peer assessments

Yun-Pi Yuan 13

Another ClassificationAnother Classification Evaluation with testsEvaluation with tests

Purposes/usesPurposes/uses KindsKinds FormatsFormats Characteristics of good testsCharacteristics of good tests

Evaluation without tests Evaluation without tests (alternative (alternative assessment)assessment) PortfoliosPortfolios JournalsJournals Conferences, interviewsConferences, interviews ObservationsObservations Performance assessmentPerformance assessment Self & peer assessmentSelf & peer assessment

Yun-Pi Yuan 14

Why Alternative Assessment?Why Alternative Assessment? Uncertain about test scoresUncertain about test scores

What’s the real difference between scores of 59 What’s the real difference between scores of 59 and 61 (esp. 60 = passing score)and 61 (esp. 60 = passing score)

Small or chance difference in test scoresSmall or chance difference in test scores No such thing as a perfect testNo such thing as a perfect test Multi-method assessment in order to account Multi-method assessment in order to account

for multi-culture, multi-intelligences of for multi-culture, multi-intelligences of learnerslearners

It can assess learning processes in an on-It can assess learning processes in an on-going mannergoing manner

Many Ts become dissatisfied with the Many Ts become dissatisfied with the mismatch between how they teach & how mismatch between how they teach & how assessment is doneassessment is done

Yun-Pi Yuan 15

Definition of PortfoliosDefinition of Portfolios ““A portfolio is a purposeful collection of A portfolio is a purposeful collection of

student work that exhibits the student’s student work that exhibits the student’s efforts, progress, and achievements in one or efforts, progress, and achievements in one or more areas. The collection must include more areas. The collection must include student participation in selecting contents, the student participation in selecting contents, the criteria for judging merit, and evidence of criteria for judging merit, and evidence of student self-reflection” (Paulson, Paulson & student self-reflection” (Paulson, Paulson & Meyer, 1991). Meyer, 1991).

““A A purposeful collection of students’ work that purposeful collection of students’ work that tell the story of their achievements, skills, efforts, tell the story of their achievements, skills, efforts, abilities, and contributions to a particular class” abilities, and contributions to a particular class” (Brown & Hudson, p. 664)(Brown & Hudson, p. 664)

Yun-Pi Yuan 16

Teacher and students can negotiate to decide Teacher and students can negotiate to decide what is included.what is included.

The learner has some control over what is being The learner has some control over what is being assessed. (learner-centered)assessed. (learner-centered)

Not necessary to include everything, depending Not necessary to include everything, depending on the purposeon the purpose

To document development over time (e.g., To document development over time (e.g., drafts)drafts)

Typical portfolio: four sectionsTypical portfolio: four sections Introduction: portfolio contents, reflective Introduction: portfolio contents, reflective

essayessay Academic worksAcademic works Personal section, e.g. journals, photosPersonal section, e.g. journals, photos Assessment section: evaluation from peers, Assessment section: evaluation from peers,

teachersteachers

Characteristics of PortfoliosCharacteristics of Portfolios

Yun-Pi Yuan 17

GuidelinesGuidelines State objectives clearly.State objectives clearly. Give guidelines on what materials to include.Give guidelines on what materials to include. Communicate assessment criteria to students.Communicate assessment criteria to students. Designate time within the curriculum for Designate time within the curriculum for

portfolio development.portfolio development. Establish periodic schedules for review and Establish periodic schedules for review and

conferencing.conferencing. Designate an accessible place to keep Designate an accessible place to keep

portfolio.portfolio. Provide positive washback-giving final Provide positive washback-giving final

assessments.assessments.

Yun-Pi Yuan 18

AdvantagesAdvantages Foster intrinsic motivation, responsibility & Foster intrinsic motivation, responsibility &

ownershipownership Promote S-T interaction with T as facilitatorPromote S-T interaction with T as facilitator Individualize learning & celebrate Individualize learning & celebrate

uniqueness of each studentuniqueness of each student Provide tangible evidence of a S’s workProvide tangible evidence of a S’s work Facilitate critical thinking, self-assessment, Facilitate critical thinking, self-assessment,

& revision processes& revision processes Opportunities for collaborative work w/ Opportunities for collaborative work w/

peerspeers Assessment of multiple dimensions of Assessment of multiple dimensions of

language learninglanguage learning

Yun-Pi Yuan 19

Good Assessment Method?Good Assessment Method?

Is portfolio a good assessment tool?Is portfolio a good assessment tool? Check against the characteristics of Check against the characteristics of

good assessment:good assessment: ValidityValidity ReliabilityReliability PracticalityPracticality AuthenticityAuthenticity WashbackWashback

(Brown 259; Bailey 218)(Brown 259; Bailey 218)

Yun-Pi Yuan 20

Observation (1)Observation (1)

Purpose: Purpose: for teachers to make inferences about for teachers to make inferences about

instructional or learning processes or instructional or learning processes or strategiesstrategies

to explain failure to learnto explain failure to learn

Systematic, planned procedure Systematic, planned procedure for real-time recording of S for real-time recording of S verbal and nonverbal behaviorverbal and nonverbal behavior

Yun-Pi Yuan 21

Observation (2)Observation (2) Planning classroom observationPlanning classroom observation

the objectives of the observationthe objectives of the observation aspects of teaching/learning included in obs.aspects of teaching/learning included in obs. keep elements of observation at one time limitedkeep elements of observation at one time limited number of Ss being observed at one timenumber of Ss being observed at one time how many observations, one occasion or repeatehow many observations, one occasion or repeate

dly?dly? how to record your observationshow to record your observations

Anecdotal records, checklists, rating scales Anecdotal records, checklists, rating scales (Brown 268)(Brown 268) how you will use the resultshow you will use the results

Yun-Pi Yuan 22

Journal (1)Journal (1)

Definition:Definition: An account of one’s thoughts, feelings, An account of one’s thoughts, feelings,

reactions, assessments, ideas, or progress reactions, assessments, ideas, or progress toward goalstoward goals

With little attention to structure, form or With little attention to structure, form or correctnesscorrectness

Features:Features: Self-reflectionSelf-reflection Writing practice; writing as a thinking processWriting practice; writing as a thinking process IndividualizationIndividualization Communication with the teacherCommunication with the teacher Most formativeMost formative

Yun-Pi Yuan 23

Journal (2)Journal (2) Guidelines:Guidelines:

Introduce Ss to the concept of journal writing.Introduce Ss to the concept of journal writing. State the objectives of the journal. State the objectives of the journal. (Brown 262)(Brown 262)

Give guidelines on what kinds of topics to Give guidelines on what kinds of topics to include.include.

Provide optimal feedback in your responses.Provide optimal feedback in your responses. Designate appropriate time frames & Designate appropriate time frames &

schedules for review.schedules for review. Provide formative, washback-giving final Provide formative, washback-giving final

comments.comments. A good assessment measure?A good assessment measure?

Yun-Pi Yuan 24

Summary of Alternative Summary of Alternative AssessmentAssessment

principleprinciple portfoliportfolioo

journaljournal confereconferencence

intervieintervieww

observobservationation

self/self/peerpeer

prcticalitprcticalityy

lowlow lowlow lowlow modmod modmod modmod

reliabilitreliabilityy

modmod modmod lowlow modmod modmod lowlow

Face Face validityvalidity

highhigh modmod highhigh highhigh highhigh modmod

Content Content validityvalidity

highhigh highhigh highhigh highhigh highhigh highhigh

washbacwashbackk

highhigh highhigh highhigh modmod modmod highhigh

authentiauthenticitycity

highhigh highhigh highhigh modmod highhigh highhigh

Yun-Pi Yuan 25

Performance TestsPerformance Tests

Second Language Assessment: Second Language Assessment: performance of a particular job or set performance of a particular job or set of situated functions.of situated functions.

Strengths: using stimulus materialsStrengths: using stimulus materials AuthenticAuthentic DirectDirect Highly contextualizedHighly contextualized

(Bailey 208 - 215)(Bailey 208 - 215)