educational assessment of students 524 - january 2020/textbook...rating scales •whenever possible,...

Educational Assessment of Students7e

Susan M. Brookhart & Anthony J. Nitko

Copyright © 2015, 2012, 2009 by Pearson Education, Inc. All Rights Reserved

Chapter 13

Performance and Portfolio Assessments

Educational Assessment of Students, 7eBrookhart & Nitko

Copyright © 2015, 2012, 2009 by Pearson Education, Inc.

All Rights Reserved

Key Concept #1

• A performance assessment

a) requires students to create a product, demonstrate a process, or both.

b) uses clearly defined criteria to evaluate the qualities of student work.



All Rights Reserved

Two Components of Performance Assessment

1. Performance task

direct demonstration of students’ achievement of learning objective

2. Rules for scoring

rubrics, rating scales, or checklists



All Rights Reserved

Key Concept #2

• Types of performance tasks range from structured, on-demand tasks to longer-term projects and portfolios.



All Rights Reserved

Types of Performance Tasks

• Structured, on-demand tasks

Paper-and-pencil (e.g., constructed response, essay)

Tasks requiring equipment/resources(e.g., science experiments)

Demonstrations

• Naturally occurring tasks (e.g., observation, assessment)



All Rights Reserved

Types of Performance Tasks

• Long-term projects (individual, group)

Reports

Experiments

Oral presentations and dramatizations

• Simulations (live, computerized)

Adaptive scenarios

• student’s response to one situation will determine what the next presentation will be



All Rights Reserved

Key Concept #3

• Advantages of performance assessment stem from its ability to assess complex learning outcomes.

They clarify the meaning of complex learning objectives.

They allow assessment of the “ability to do.”

They are consistent with modern learning theory.

They require integration of knowledge, skills, and abilities.



All Rights Reserved

Key Concept #3

• Advantages of performance assessment stem from its ability to assess complex learning outcomes.

They may be closely linked with teaching activities.

They broaden the approach to student assessment and offer alternative ways for students to express learning.

They let teachers assess the processes students use as well as the products they produce.



All Rights Reserved

Key Concept #3

• Disadvantages of performance assessment stem from the difficulties arising from that complexity.

They are difficult to create.

Completing them takes a lot of time.

Scoring them takes a lot of time.

Scores may have low scorer reliability.

Students’ performance on one task provides little information about performance on other tasks.



All Rights Reserved

Key Concept #3

• Disadvantages of performance assessment stem from the difficulties arising from that complexity.

They do not assess all learning objectives.

They may be discouraging to less able students.

They may underestimate the learning of some cultural groups.

They may be corruptible (i.e., easy to coach).



All Rights Reserved

Key Concept #4

• To create a performance assessment, first be clear about the performance you want to assess.

Identify objectives to be assessed

Identify criteria for assessment

Define quality levels (along continuum)

Decide whether to assess a process, a product, or both



All Rights Reserved

Key Concept #5

• The second step is to design the task to elicit the desired performance.

Tasks should be meaningful.

Tasks should be valid.

Tasks should be completed in a short period of time.

Tasks should be structured as much as necessary.



All Rights Reserved

Additional Considerations When Creating Performance Tasks

• Participation: Group or individual

• Assessment: Product or process

• Response mode

• Directions that make the task clear to students

• Number of tasks in one assessment (scope, available time, available resources)



All Rights Reserved

Key Concept #6

• The third step is to design a scoring scheme that reflects the performance criteria.

The scoring scheme may be one of several types of rubrics, a checklist, or a rating scale.



All Rights Reserved

Rubrics

• Coherent sets of criteria for students’ work

Criteria are based on the learning objectives.

Include descriptions of levels of performance quality on the criteria.

• Decide how many.

• Make sure they are descriptions, not judgments (e.g., “complete explanation,” not “excellent explanation”).



All Rights Reserved

Creating Rubrics: Top-Down Approach

1. Adapt or create a list of achievement dimensions that describe the qualities you will be assessing.

2. Develop descriptions of student performance at each performance level of each dimension.

3. Share the rubric with students.

4. Use the rubric to assess the performances of several students.

5. Revise the rubric as necessary.



All Rights Reserved

Creating Rubrics: Bottom-Up Approach

1. Obtain copies of students’ responses to a performance item, illustrating various levels of quality.

2. Read and sort the responses: high-quality, medium-quality, and low-quality.

3. Write very specific reasons why each response is in a particular group.

4. Identify emerging dimensions.

5. Write a specific student-centered description of what the responses at each level are typically like.



All Rights Reserved

Types of Rubrics

• General

Description of work gives characteristics that apply to family of tasks

• Task specific

Description of work refers to content of individual tasks

• Analytic

each criterion evaluated separately

• Holistic

All criteria evaluated simultaneously



All Rights Reserved

Checklists

• Procedure checklist

assess whether students follow appropriate steps

• Product checklist

assess what students make

• Behavior checklist

assess whether students show set of discrete behaviors

• Self-evaluation checklist



All Rights Reserved

Rating Scales

• Assess the degree to which students have attained the achievement dimensions in the performance task.

Numeric rating scales

• Judgments translated into numbers

• Similar in function to rubric

Graphic rating scales

• Unbroken line represents achievement dimension; student performance is marked on the line



All Rights Reserved

Rating Scales

• Whenever possible, use rubrics instead of rating scales for academic work.

exception: ratings of behavior

• Construct rating scales with substance in mind first, then consider the research results.



All Rights Reserved

Scoring Errors: Rating Scales & Rubrics

• Leniency error

Almost all ratings toward the high end of the scale

• Severity error

Almost all ratings toward the low end of the scale.

• Central tendency error

Almost all ratings at the middle part of the scale.



All Rights Reserved


• Halo effect

Teachers’ general impressions of students affect ratings on specific dimensions

• Personal bias

Ratings based on inappropriate or irrelevant stereotypes

• Logical error

Ratings on unrelated dimensions are similar, because teacher believes they are related



All Rights Reserved

Reliability Coefficients

• Estimating reliability over time

Test-retest

Alternate forms on different occasions

• Estimating reliability on a single occasion

Alternate forms

Coefficient alpha

Split-halves coefficient

• Estimating scorer reliability

Correlation of two scorers’ results

Percentage of agreement

Kappa coefficient



All Rights Reserved

To Improve Reliability

• Organize achievement dimensions into logical groups that match the content and process framework of the curriculum.

• Use behavioral descriptors to define each level of performance.

• Provide examples of students’ work to help define each level of an achievement dimension.

• Have several teachers work together to develop

• a scoring rubric or rating scale.

• Have several teachers review and critique the

• draft of a scoring rubric or rating scale.



All Rights Reserved

To Improve Reliability

• Provide training and supervised practice for everyone who will use the rubric or rating scale.

• Have more than one rater rate each student’s

• performance.

• Monitor raters by periodically sampling their ratings, and retrain if ratings are inaccurate or inconsistent.



All Rights Reserved

Key Concept #7

• Projects can be used as performance assessments if they are designed according to the three steps (clearly describe the performance to be assessed, then match both the project task and scoring scheme to it).



All Rights Reserved

Key Concept #8

• For purposes of assessment, a portfolio is a limited collection of a student’s work used either to present the student’s best work(s) or to demonstrate the student’s educational growth over a given period of time.

e-portfolios are becoming more common



All Rights Reserved

Growth Portfolios

• Examples of a student’s work, along with comments, that demonstrate how well the student’s learning has progressed over a given period.

Used for formative purposes

Student should play a significant role in deciding what should be included in his/her portfolio.

• Students should also provide reflections and self-evaluations.



All Rights Reserved

When Planning Growth Portfolios

• Be very clear about the learning objectives or standards.

• Have a firm understanding of a learning progress theory and use it to guide you to identify what you should look for.

• Collaborate and work cooperatively with other teachers.

• Use some type of rubric to define assessment criteria and to help you be consistent.



All Rights Reserved

Steps for Crafting a Portfolio System

1. Identify purpose and focus

2. Identify general achievement dimensions to be assessed

3. Identify appropriate organization (outline/framework)

4. Determine how it will be used

5. Determine how it will be evaluated

6. Evaluate scoring rubrics

educational assessment of students 524 - january 2020/textbook...rating scales •whenever possible,...

Documents