using cem data for self-evaluation and improvement
DESCRIPTION
Using CEM Data for Self-Evaluation and Improvement. Running Your School on Data 7 th June 2011. [email protected]. Running your school on data. CEM data includes: School profile bands Baseline test acquired ability data, inc. IPRs Predictive data inc. chances graphs - PowerPoint PPT PresentationTRANSCRIPT
Using CEM Data for Self-Evaluation and Improvement
Running Your School on Data
7th June 2011
Running your school on data
CEM data includes:
• School profile bands• Baseline test acquired ability data, inc. IPRs• Predictive data inc. chances graphs• Value-added data• Attitudinal data• Curriculum assessments (Insight)• Software programmes (PARIS) with databases
So much data................................!
1. What data is needed? (What data is not needed?)
2. And for Whom is the data ?
3. What does the data mean? (What does the data not mean?)
Some key questions:
1. What data do I need? A key first step!
e.g. using MidYIS GCSE predictive data for target setting
• Point and/or grade predictions? (e.g. 5.8/grade B)• Based on whole cohort?• Based on the 75th percentile?• Based on prior value-added?• The chances graphs?
Individual Chances Graph for ADAM BECKSMITH - GCSE EnglishMidYIS Score 121 MidYIS Band A
Teacher's Adjustment : 0 grades / levels / points
0 0 0 0
3
17
34 34
12
0
5
10
15
20
25
30
35
40
U G F E D C B A A*
Grade
Per
cen
t
Prediction 6.3 A/B
MidYIS Score 121 MidYIS Band A Prior VA Adjustment : 1 grades / levels
0 0 0 0 0
4
16
3941
0
5
10
15
20
25
30
35
40
45
50
U G F E D C B A A*
Grade
Per
cen
t
Prediction 7.4 A*/A
Discussion: setting the targets...........
• From your perspective, assess the merit of each type of predictive data and the associated chances graphs
• Which type of predictive data would you use to set the targets, and why?
• Would your students be involved as part of the target setting process?
• Would parents be informed about the process and outcome?
• How would you ensure that HoDs were involved to ensure overview the process?
2. CEM data: who is it for? For example:
• SMT/SLT: summary data, attitudinal• HoDs: Subject VAD, predictive data• Subject teachers:• FormTutors:• Head of Learning/Year/House:• Parents:• Pupils:
and not forgetting:• Governors:• Media:
3. What does the data mean?
e.g. value-added data:
• The difference between raw and standardised residuals
• The use of confidence limits to distinguish between average and statistically significant data, and to allow for small entry subjects
• Can ‘zero’ or even ‘negative value-added’ be acceptable?
Raw residual Bar chart
(MIDYIS and YELLIS only)
Standardised Residual Bar Chart
0.0
-0.2
0.3
-0.1 -0.2 -0.2
0.1 0.1
0.9
0.00.1
-0.3
0.3
-0.3
0.2 0.2 0.2 0.1
-0.3
0.1 0.1 0.20.0
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
Art
& D
esig
n
Bio
logy
Bus
ines
s S
tudi
es
Che
mis
try
Des
ign
& T
echn
olog
y
Dra
ma
Eng
lish
Eng
lish
Lite
ratu
re
Fre
nch
Geo
grap
hy
Ger
man
His
tory
Hom
e E
cono
mic
s
ICT
Mat
hem
atic
s
Mus
ic
Phy
sica
l Edu
catio
n
Phy
sics
Rel
igio
us S
tudi
es
Wel
sh
Sci
ence
: O
ther
Sci
ence
: G
CS
E
Sci
ence
: G
CS
E A
dditi
onal
SC
IC
T
SC
Phy
sica
l Edu
catio
n
SC
Rel
igio
us S
tudi
es
Voc
Hea
lth &
Soc
ial C
are
Av
era
ge
Sta
nd
ard
ise
d R
es
idu
al
99.7% confidence limit
95% confidence limit
ANY VALUE IN THE INNER SHADED AREA IS CONSIDERED TO BE AVERAGE VALUE ADDED
Raw residual Bar chart
(MIDYIS and YELLIS only)
Standardised Residual Bar Chart
0.0
-0.2
0.3
-0.1 -0.2 -0.2
0.1 0.1
0.9
0.00.1
-0.3
0.3
-0.3
0.2 0.2 0.2 0.1
-0.3
0.1 0.1 0.20.0
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
Art
& D
esig
n
Bio
logy
Bus
ines
s S
tudi
es
Che
mis
try
Des
ign
& T
echn
olog
y
Dra
ma
Eng
lish
Eng
lish
Lite
ratu
re
Fre
nch
Geo
grap
hy
Ger
man
His
tory
Hom
e E
cono
mic
s
ICT
Mat
hem
atic
s
Mus
ic
Phy
sica
l Edu
catio
n
Phy
sics
Rel
igio
us S
tudi
es
Wel
sh
Sci
ence
: O
ther
Sci
ence
: G
CS
E
Sci
ence
: G
CS
E A
dditi
onal
SC
IC
T
SC
Phy
sica
l Edu
catio
n
SC
Rel
igio
us S
tudi
es
Voc
Hea
lth &
Soc
ial C
are
Av
era
ge
Sta
nd
ard
ise
d R
es
idu
al
99.7% confidence limit
95% confidence limit
ANY VALUE IN THE INNER SHADED AREA IS CONSIDERED TO BE AVERAGE VALUE ADDED
Standardised Residual Bar Chart
Average Standardised Residuals by SubjectMidYIS Year 7 2004/2005 to GCSE 2009
0.0 0.2
-0.1
0.8
0.4 0.30.0 0.0
0.5 0.50.2
0.9
-0.9
0.0 0.1
0.6 0.50.6
0.3 0.30.3
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Additional S
cience
Art &
Design
Bus. &
Com
m.
System
s
Business S
tudies
Design &
Technology
Dram
a
English
English Literature
French
Geography
Germ
an
History
Hom
e Econom
ics
Latin
Mathem
atics
Music
Physical E
ducation
Religious S
tudies
Science
Spanish
Statistics
Ave
rag
e S
tan
dar
dis
ed R
esid
ual
Case study: Alis value-added dataFour sets of VAD are available!
From average GCSE baseline:• all Alis cohort • type of Institution• Syllabus
From the baseline test• all Alis cohort
SPC Chart with confidence limits: WHOLE SCHOOL
All Alis Cohort: Institution
Syllabus
-0.3
-0.2
-0.1
-0.0
0.1
0.2
0.3
0.420
04
2005
2006
2007
2008
2009
2010
Ave
rage
Sta
ndar
dise
d R
esid
ual
Exam Year
-0.3
-0.2
-0.1
-0.0
0.1
0.2
0.3
0.4Final_Result
Using PARIS software: Baseline Test
Whole School
From your perspective, which set of VAD would you use for the different user groups? (Governors, HoDs, Media, Parents, SMT/SLT...)
0.8
0.5
0.5
0.7
0.4
0.6
0.2
-0.1
0.4
1.2
-0.2
0.5
0.3
0.3
0.8
0.2
-0.7
0.7
0.6
0.7
0.5
-3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0
Spanish
Science
Religious Studies
Physics
Physical Education
Music
Mathematics
Latin
History
German
Geography
French
English Literature
English
Drama
Design & Technology
Classical Civilisation
Chemistry
Biology
Art & Design
Additional Science
Average Raw Residual
A final thought..........
Is it possible to keep adding value at each key stage?
0.5 0.6 0.6
-0.6
0.2
0.6
0.3 0.3 0.3
-0.2
0.9
0.3
-0.1
0.2
0.50.3
0.6
0.30.5 0.6
0.4
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Additional S
cience
Art &
Design
Biology
Chem
istry
Classical
Civilisation
Design &
Technology
Dram
a
English
English Literature
French
Geography
Germ
an
History
Latin
Mathem
atics
Music
Physical E
ducation
Physics
Religious S
tudies
Science
Spanish
Ave
rag
e S
tan
dar
dis
ed R
esid
ual
2009 A2 2010 A2
2009
2010
• USE ONE YEARS DATA WITH CAUTION!
• Better to use three years data as patterns over time are more significant.
• Teachers to help learners• Curriculum and staffing decisions• Target setting and monitoring pupil progress• Inspection evidence• Self-evaluation• Monitoring changes over time such as pupil
ability intake profiles and VAD• Asking the question ‘can we do better?’
i.e. the data is used to aid and to support professional judgement
CEM data is used for:
with due consideration to:
• Ethos and tradition of ‘my’ institution• Accountability • Parental expectations• Staff training in use of, and ability to cope with, data (data
overload)• Management of data: integrating the data into school
procedures, storage, retrieval, distribution and access……policy?
i.e. doing our best to help every pupil to at least achieve, if not exceed, their potential