Choosing An Assessment For Your school

Chris Jellis and Peter Olsen

What Makes a Good Assessment?

• Provides useful information to both teachers and managers

• Fits into the current assessment structure• Manageable (in terms of time and cost)• Trusted• Reliable• Valid

Managing Assessment Data

• Provides useful information to both teachers and managers– In a format that is readily understandable

• Fits into the current assessment structure– Fits with other assessment data– School tracking system

• Manageable (in terms of time and cost)• Trusted

Reliability

• What does it mean when we say that a test is “reliable”?

• A reliable test, if applied repeatedly to the same subject, will yield the same result each time

• Test-retest—same person takes test at two different times (and gets a similar result)

• Interrater—two people score the same test with similar scores

Validity

• What does it mean when we say that a test is “valid”?

• Is your test measuring what you say you are measuring?• Face validity—common agreement• Content validity—degree to which it covers the range of

meanings• Criterion-related validity—extent to which it matches

outcomes of a similar, but different measure

The Dartboard Analogy

Reliable (but not valid)

Valid (but not reliable)

Valid and reliable

Just In Case You Ask…

• Validity– ASPECTS (September) to ASPECTS (June) = 0.68– ASPECTS (September) to PIPS (September) = 0.72– (GCSE to A-Level = approx. 0.7)

• Reliability– PIPS Test/retest reliability for reading and maths =

0.98

So Why Use CEM Assessments?

• Based on sound educational research• Technical data available• Used by CEM for wider research projects• Ability to discuss assessments with specialists

who really understand what makes a good assessment

Things To Consider

• What do I want to know?– Pupil ability, Progress

• What format do I need the results to be in?– Age scores, standardised scores

• When do I want to test?– Fixed periods, anytime

• Do I want to test every year?• How much will it cost me?

Comparing InCAS and PIPS

• PIPS• Standardised scores are more difficult to

understand, but good for comparing children and groups

• InCAS• Age scores are clearer, but with no standard

deviation, it is difficult to tell what the range is

InCAS can be diagnostic

PIPS is good for value addedEn

d of

Yer S

tand

ardi

sed

Read

ing

Scor

e

Zen

Yolanda

Xavier

Wendy

Veronique

Usworth

Terry

Stephen

Richard

Quentin

PatriciaOliver

NedMike

Lucy

Kalvinder

Jeremy

Ishmael

Gilbert

David

CharlieBethany

Adam

25

30

35

40

45

50

55

60

65

70

75

25 30 35 40 45 50 55 60 65 70 75

InCAS can show progress

PIPS has IDEAS+ software

-15

-5

5

15

-15 -5 5 15

Prior Value-added

Con

curr

ent V

alue

-add

ed

Doing better than expected, however they are no longer as far ahead as they used to be

Doing better than expected and this may have been a consistent characteristic over time

Probably on track before, they have made excellent progress and have now moved

further ahead

Doing as well as expected. However, they have moved from a position where they were

ahead of similar children

On track and is probably a consistent characteristic over time

Probably underachieving before, however they have made excellent progress and are now on

track.

Probably on track before but has fallen behind and is now underachieving

Underachieving and this may have been a consistent characteristic over time

Probably underachieving before. They have made good progress but they still have some

catching up to do

-10

-8

-6

-4

-2

0

2

4

6

8

10

Year Tw o Year Three End Year Four Year Five End Year Six

Year Group

Mat

hs C

onte

xt V

A M

ean

Scor

e

-10

-8

-6

-4

-2

0

2

4

6

8

10

Administration

• InCAS– Administer at any time of year– CD (installed on school network)– Group assessment– Feedback

• Within 24 hours

• PIPS– Fixed assessment periods– Pencil and paper or CD (installed on school

network)– Group assessment– Feedback

• Approx 6 weeks for pencil and paper• Quicker for CD

So, which do you choose?

• To establish a baseline (particularly in the early years) and show progress - PIPS

• If the group is not average – InCAS• If you need diagnostic information – InCAS• Some schools do both

Thank You For Your Time

Any Questions?