Mathematics and Statistics Leaders Symposium

September 2011Waipuna Conference Centre

Overall Teacher Judgments and Moderation

Christine Hardie

Purpose

• To explore the multiple sources of evidence teachers can use to make overall teacher judgments in mathematics

• To exemplify a moderation process

• To examine the key principles of moderation within and between schools

Effective Assessment – NZC – p.40

• Benefits students• Involves students• Supports teaching and learning goals• Is planned and communicated• Is suited to the purpose• Is valid and fair

Overall Teacher Judgment (OTJ)

What is an overall teacher judgment?

Overall Teacher Judgment

Standards based assessment – shared benchmarks of expected performance, supported by exemplars.Summative judgment – a point- in- time description of student achievement and as such should be based on evidence that is relevant at the time the judgment is being made.Holistic judgment, defensible judgment, on-balanced judgmentPart of the knowledge building inquiry cycle

Overall teacher judgment

An overall teacher judgment involves drawing on and applying the evidence gathered up to a particular point in time in order to make an overall judgment about a student’s progress and achievement. Using a range of approaches allows the student to participate throughout the assessment process, building their assessment capability……No single source of information can accurately summarise a student’s achievement or progress. A range of approaches is necessary in order to compile a comprehensive picture of the areas of progress, areas requiring attention, and what a student’s progress looks like.

(MOE – fact sheet 2010.)

Evidence

What counts as evidence?

Evidence

Any facts, circumstances or perceptions that that can be used as an input for an analysis or decision.

• What are some school examples?

Evidence

All schools have data about student achievement. To make the most of these data to improve learning, we need to be aware of the evidence that describes our students’ wider learning environment.

• Demographics• Student achievement• Perceptions• School processes• Other practice

Evidence for overall teacher judgments in mathematics

What evidence are you collecting to inform your overall teacher judgment in mathematics?

Are these sources of evidence adequate?

What might count as evidence:

• GloSS tool – assesses the strategy number framework which aligns with the National Standards - (Use of tool needs be moderated)

• PAT• E-asTTle• School designed tests and questions• ARB’s• NEMP tasks

What might count as evidence:

• Modelling book evidence –work alongside the teacher

• Students books/worksheets as well as models, graphs and diagrams

• Learning conversations• Student profiles – nzmaths• Numeracy planning sheets – used as a working

document• Rich tasks – diagnosis• Post it notes – anecdotal notes• Self and peer assessment

Students place in the process

How are you including the student in the process of evidence collection and analysis?

Students place in the process

Ability to articulate their learning and their next steps.

Teachers need to consider how their practice

supports students to discuss their learning, and to explain and justify their mathematical thinking.

School-wide systems to collect evidence

As a leader of mathematics what school-wide processes and systems have you put in place to support the collection and analysis of evidence?

Where to next?

What other systems could you put in place?

Summary: Consistency of overall teacher judgment is developed by…

• Discussing what sources of evidence you currently collect to inform teaching and learning decisions

• Critiquing whether these sources of evidence are adequate

• Developing a process/template for the school (or areas within the school – e.g. junior and senior school) so that teachers are using similar types of evidence to inform OTJs.

The big picture - Moderation

What is moderation?

The big picture -Moderation

Moderation is the process of teachers sharing their expectations and understandings of standards with each other in order to improve the consistency of their decisions about student learning and achievement.

The process where teachers compare judgments to either confirm or adjust them.

Moderation processes require

• Interpersonal/social skills

• Theoretical and content knowledge and pedagogical content knowledge

• Familiarity with the agreed frame of reference or standard

• Staff culture-disposition towards, and systems for on-going professional learning

Teachers need to be safe to:• express and clarify thinking• ask questions• explore solutions• adapt thinking after listening to informed

ideas of others• tolerate and appreciate differences in

perspectives• view differences as opportunities to deepen

knowledge base

Scenarios to discuss moderation issues with National Standards

National Standards Moderation Discussions

(a) A principal was concerned after the mid-year ‘progress’ teacher judgements. He felt some teachers mark ‘hard’ and some teachers mark ‘easy’. How can we ‘moderate’ so all teachers are marking the same.

Assessment Key Messages (pages 10-12)

• Meeting a standard depends on the nature of a students response to given problems, not just their ability to solve the problems. (p.10, paragraph 6)

• When assessing a student’s achievement and progress, the teacher needs to make an overall teacher judgement (OTJ) about the student in relation to the whole standard. (p.12 paragraph 1)

• The expectations for Number are the most critical requirement for meeting a standard” (p.12. paragraph 5)

• Number knowledge is for facilitating problem solving, just demonstrating number knowledge e.g. basic facts, is insufficient to meet a standard. (p.10, paragraph 3)

• independently and most of the time. (p.12 paragraph 4).

National Standards Moderation Discussions

(b) How do we determine if a student is ‘early’ or ‘at’ in stage 5 and stage 6?

Level 2 Stage 5: Early Additive

Differentiating Between Early and Late Stage 5Which problems, if solved correctly using part-whole thinking, would indicate late stage 5 thinking, e.g. End of Y4

83 – 9 59 + 26 74 + 30 8 + 29 97 - 43

Additional Stage 5 questions

Level 2 Stage 6: Advanced Additive

How would you differentiate between Early and Late Stage 6?

National Standards Moderation Discussions

(c) Making overall teacher judgments can be time consuming and often judgments are being made on old data.

What are some ways to make the collection of evidence to support overall teacher judgments easy and manageable?

Tickled pink / green for growth highlighting or stickers on…• Modelling book• Planning units• Photocopied students whiteboards• Teachers feedback comments in

student books• Other anecdotal notebook• Self/peer assessment in maths

diaries• Photograph (hands on work, etc)• “I can Sheets”

Student C

National Standards Moderation Discussions

(d) Around report writing time, a group of teachers had a heated debate in their syndicate meeting. How were teachers supposed to reconcile the student strategy stage with their knowledge stage when for some students the gap between their knowledge and strategy was as much as 2 or 3 stages?

What do you understand by this student’s IKAN sheet?

Moderation processes

What moderation processes are you leading in your school?

What are the moderation processes currently being used in your school?

This is what a moderation process could look like…..Select the children whose evidence is conflicting or who sit

on the cusp.

•Each teacher from the team/syndicate brings evidence of one

of those students.

•Using evidence, the National Standards and other curriculum

resources, each teacher makes an independent judgment about

each student.

•Teachers discuss their judgments and use evidence, the

National Standards, and other curriculum resources to come to

an agreed judgment.

•The team selects one student for whom it has reached

agreement and take to school-wide moderation.

Each teacher from a team/syndicate

brings evidence of one of

those students

Select the children whose evidence

is conflicting or who sit on the cusp.

The team selects one student’s

results for which they have

come to agreement and take to whole-school (vertical)

moderation.

Teachers compare independent OTJs and come to an agreed judgment for each student using evidence / NS / resources.

An example of a moderation process

Each teacher uses evidence/ NS/resources to make an independent OTJ for each

student

Whole school moderation and between schools moderation

How can we moderate across the whole school?

How can we moderate between schools?

Access Mathematics Symposium resources and links online

http://teamsolutions.wikispaces.com