Assessment and Evaluation

1

Martin Valcke Martin.Valcke@UGent.be

http://users.ugent.be/~mvalcke/CV_2012/

Structure •  Consider evidence-base about

assessment and evaluation •  Definition assessment and evaluation

– Activity 1: develop definitions

•  Trends in adult A & E •  Self assessment and peer assessment •  Rubrics

– Activity 2: develop a rubric 2

Advance organizer

•  Where is the evidence about assessment and evaluation?

Hattie, 2009

5

Assessment & Evaluation

Activity 1: define •  What is assessment? •  What is evaluation?

Definition assessment & evaluation

•  Three elements: – Measuring: get data from student about his/her

answer or behavior (test, exam, task) – Evaluating: what is the value of the “answer/

behavior”? Separately –  Scoring: what score will be attributed to a

quality level in the answer or behavior?

Assessment en Evaluation in adult education

•  Characteristics of adult learners! – Voluntary learning –  Informal – Self-directed – personal goals

•  Motivational dimensions: autonomy, belonging and competence

8

Trends in A & E of adults

•  Major changes in the place, role, function, focus, approach, tools, … of A & E in adult education

•  Trends in WHO is responsible for the evaluation and in the TOOLS being used

Trends in A & E

•  Trends share characteristics: –  Focus on “behavior” –  Focus on “authentic” behavior –  Foicus on “complex” behavior –  Explicit “criteria” –  Explicit “standards” –  Need for concrete feedback –  Focus on “consequential validity”

Gielen, Dochy & Dierick (2003) Examples

Rubrics

11 http://www.siue.edu/~lmillio/IT598/Resources/04_assessment/Rubrics%20and%20Adult%20Learners.pdf

Self assessment

12 Boud, 2005

Trends in A & E

Trends according to Fant et al. (1985, 2000): •  Assessment centres •  Self and Peer assessment •  Portfolio assessment •  Logbooks •  Rubrics

Individual learner Group learner

External institution

Teachers Expert teacher

Assessment system Institutional level

Who is responsible for A & E?

Self- and peer assessment

•  Learn about your own learning process. •  Schmitz (1994): “assessment-as-learning”. •  ~ self corrective feedback •  See experiential learning cycle of Kolb. •  Boekaerts (1991) self evaluation as a

competency. 15

•  Development of metacognitive knowledge and skills (see Brown, Bull & Pendlebury, 1998, p.181).

•  Freeman & Lewis (1998, p.56-59): developing pro-active learners

16

17 The Learning Cycle Model

Learning to evaluate

•  Develop checklists •  Give criteria •  Ask to look for quality indicators. •  Analysis of examples good and less

good practices: develop a quality “nose”

18

Learning to evaluate •  Freeman & Lewis (1998, p.127) :

• Learner develops list of criteria. • Pairs of learners compare listed criteria. • Pairs develop a criterion checklist. •  Individual application of checklist. • Use of checklist to evalute work of other learner. •  Individual reworks his/her work. • Final result checkeed by teacher and result compared to learner

evaluation. • Pairs recheck their work on the base of teacher feedback.

19

Learning to evaluate

•  Peer evaluation is not the same as Peer grading

•  Part of score could build on accuracy of self/peer evaluation and self-correction

•  Example: peer evaliuation and feedback in writing

20

21

Collaborative writing space

22

23

Feedback

24

25

Feedback on feedback

26

27

Analysis example

•  Authentic task •  Collaborative setting •  Clear criteria •  Peer feedback •  Re-writing •  Feedback on feedback

28

Fits adult learner

Trends in A & E

Trends (Fant et al., 1985, 2000): •  Assessment centres •  Self and Peer assessment •  Portfolio assessment •  Logbooks •  Rubrics

Rubrics

30

Quality indicators

Criteria

Quality levels

Rubrics •  Rubric: assessment tool for a qualitative

assessment of complex authentic activity. – A rubric builds on criteria that are enriched

with a scale that help to detremine mastery levels

– For each mastery level, standards/benchmarks can be defined.

– A rubric helps both the trainer and the trainee in view of what is expected at process/product level.

32

33

34

Critical thinking rubric

http://academic.pgcc.edu/~wpeirce/MCCCTR/Designingrubricsassessingthinking.html

Activity 2

•  Develop a rubric for assessing the quality of a conference contribution: – Define criteria – Define at least 4 quality levels with quality

indicators

36

Quality indicators

Criteria

Research about rubrics •  Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and

educational consequences. Educational Research Review, 2, 130–144.

– Reliable, especially if rubrics are analytic, topic-specific, and complemented with exemplars and/or rater training;

– Rubrics promote learning. Main reason: rubrics make expectations and criteria explicit, which also facilitates feedback and self-assessment.

Conclusions

•  Adult learning: assessment & evaluation special position

•  Trends in assessment & evaluation align with adult learning – Self assessment and peer assessment – Rubrics

Assessment and Evaluation

39

Martin Valcke Martin.Valcke@UGent.be

http://users.ugent.be/~mvalcke/CV_2012/

Informative websites •  Overview tools, examples, theory, background, research: http://

school.discoveryeducation.com/schrockguide/assess.html •  Critical thinking rubrics: http://academic.pgcc.edu/~wpeirce/MCCCTR/

Designingrubricsassessingthinking.html •  Rubric generators: http://www.teach-nology.com/web_tools/rubrics/ •  Intro on interesting rubric sites: http://web.njit.edu/~ronkowit/teaching/rubrics/

index.htm •  Rubric APA research paper: http://web.njit.edu/~ronkowit/teaching/rubrics/

samples/rubric_apa_research.pdf •  General intro and overview:

http://web.njit.edu/~ronkowit/teaching/rubrics/index.htm