PrimaryNational StrategyUnderstanding and using data
© Crown Copyright 2005
© Crown Copyright 2005
•The content of this presentation may be reproduced free of charge by schools and local education authorities provided that the material is acknowledged as Crown copyright, the publication title is specified, it is reproduced accurately and not used in a misleading context. Anyone else wishing to reuse part or all of the content of this publication should apply to HMSO for a core licence.
•The permission to reproduce Crown copyright protected material does not extend to any material in this publication which is identified as being the copyright of a third party.
•Applications to reproduce the material from this publication should be addressed to:
•HMSO, The Licensing Division, St Clements House, •2–16 Colegate, Norwich NR3 1BQ•Fax: 01603 723000•e-mail: [email protected]
Crown copyright statement
© Crown Copyright 2005
Aims
• developing a strategic approach to the management of inclusion;
• developing their middle management skills;• making effective use of data to improve
teaching and learning and to celebrate children’s progress.
To support those who lead or coordinate inclusion in:
2.1
© Crown Copyright 2005
School improvement questions
• How well are we doing?
• How do we compare with similar schools?
• How well should we be doing?
• What more can we aim to achieve next year?
• What must we do to make it happen?
2.2
© Crown Copyright 2005
The cycle for school improvement
1
25
34
How well are we doing?
How well should
we be doing?
Taking action and reviewing
What must we do to make it
happen?
What more can we aim to achieve?
Cycle for school
improvement
2.3
© Crown Copyright 2005
Activity 1
Discussion questions:
• Are there particular groups of children at risk of underachievement in your school?
• How do you currently track and evaluate their progress?
• What types of data might you draw on?
2.4
© Crown Copyright 2005
Possible groups
• girls and boys• children who are disabled or who have special educational
needs• minority ethnic and faith groups• children learning English as an additional language • Travellers, asylum seekers, refugees• children with social and emotional difficulties• children in public care• sick children • young carers • children from families under stress• children at risk of disaffection, poor attendance or exclusion 2.5
© Crown Copyright 2005
Possible data
• Foundation Stage Profile• end of key stage data• optional tests• teacher assessments, P scales assessment• achievement of targets – curricular, IEP, PSP• attendance and exclusions• relevant behaviour measures • value-added – progress as well as achievement• QCA Language in common step descriptors and
any other EAL or first language assessments2.6
© Crown Copyright 2005
Inclusion and school improvement – an increasingly data rich context
• variance in children’s outcomes – a concern to raise achievement for all
• informed, systematic approach to improve children’s progress – every child matters and dealing with inequalities
• inclusion coordinators and other managers using data to inform strategy and teaching at school, class, group and individual levels
• increasing availability of data via PLASC, National Curriculum and other assessment results
• the power of ICT to process the information 2.7
© Crown Copyright 2005
Excellence and enjoyment
‘Ensure every child succeeds: provide an inclusive education within a culture of high expectations.’
Target setting:‘A whole-school approach which focuses on meeting pupils’ individual needs and on using detailed information to set stretching and appropriate targets for each pupil.’
2.8
© Crown Copyright 2005
Primary National Strategy
A great deal to celebrate:
• standards have risen dramatically;
• quality of teaching and learning improved;
• world-class comparisons
2.9
© Crown Copyright 2005
‘Off the plateau’ at level 4+
provisional
© Crown Copyright 2005
% l
evel
4+
2.10
© Crown Copyright 2005
Percentage of all children below level 3 at the end of Key Stage 2
1998 1999 2000 2001 2002 2003 2004
English 7 7 6 7 7 7 7
Maths 7 6 6 5 5 6 6
2.11
© Crown Copyright 2005
The percentage of children
• below levels 1 and 2 at end of Key Stage 1
• below level 3 at end of Key Stage 2
• below level 2 at end of Key Stage 1 who attain
level 3+ at end of Key Stage 2
2.12
© Crown Copyright 2005
100% -
-
-
-
-
80%
60%
40%
20%
0%A B C D E F G H
Schools
Percentage of all children below level 3 at the end of Key Stage 2
2.13
Below level 3 Complex SEN EAL
© Crown Copyright 2005
The effects of social disadvantageThe achievement gap between low FSM and high FSM schools has narrowed …
Source: Form 7 and ASC data. Performance tables dataCoverage: Maintained mainstream schools
Dif
fere
nc
e b
etw
een
th
e K
ey
Sta
ge
2 l
eve
l 4
+ %
of
the
lo
we
st
and
h
igh
es
t F
SM
ba
nd
s
– p
erc
enta
ge
po
ints
2.14
© Crown Copyright 2005
…but the achievement gap between non-FSM and FSM children has stayed the same
Source: NPD CubeviewerCoverage: Maintained mainstream schools. 2003 Attainment matched to 2003 PLASC and 1997 Attainment (as prior attainment) matched to 2002 PLASC
Dif
fere
nc
e b
etw
een
th
e K
ey
Sta
ge
2 l
eve
l 4
+ %
of
no
n-F
SM
an
d F
SM
p
up
ils
– p
erc
en
tag
e p
oin
ts
2.15
© Crown Copyright 2005
Outcomes for children from differentethnic groups – English
Effect of ethnicity and deprivation on KS2 English results
-60 -50 -40 -30 -20 -10 0 10
ChineseIrish
White and AsianIndian
Any other mixed backgroundWhite British
White and Black AfricanAny other White backgroundAny other Asian backgroundWhite and Black Caribbean
UnclassifiedAny other Black background
BangladeshiBlack Caribbean
Black AfricanAny other ethnic group
PakistaniGypsy/Roma
Traveller of Irish Heritage
Difference in percentage of pupils achieving level 4+
Observed ethnicityeffect
Traveller of Irish heritage
Gypsy/Roma
Pakistani
Any other ethnic group
Black African
Black Caribbean
Bangladeshi
Any other black background
Unclassified
White and black Caribbean
Any other Asian background
Any other white background
White and black African
White British
Any other mixed background
Indian
White and Asian
Irish
Chinese
Effect of ethnicity and deprivation on Key Stage 2 English results
Difference in percentage of pupils achieving level 4+
-
2.16
© Crown Copyright 2005
Outcomes for children from differentethnic groups – English
Traveller of Irish heritage
Gypsy/Roma
Pakistani
Any other ethnic group
Black African
Black Caribbean
Bangladeshi
Any other black background
Unclassified
White and black Caribbean
Any other Asian background
Any other white background
White and black African
White British
Any other mixed background
Indian
White and Asian
Irish
Chinese
Effect of ethnicity and deprivation on Key Stage 2 English results
Difference in percentage of pupils achieving level 4+
Effect of ethnicity and deprivation on KS2 English results
-60 -50 -40 -30 -20 -10 0 10
ChineseIrish
White and AsianIndian
Any other mixed backgroundWhite British
White and Black AfricanAny other White backgroundAny other Asian backgroundWhite and Black Caribbean
UnclassifiedAny other Black background
BangladeshiBlack Caribbean
Black AfricanAny other ethnic group
PakistaniGypsy/Roma
Traveller of Irish Heritage
Difference in percentage of pupils achieving level 4+
-60 -50 -40 -30 -20 -10 0 10
Effect of difference inFSM rates
Observed ethnicityeffect
-
2.17
© Crown Copyright 2005
Outcomes for children from differentethnic groups – English
Traveller of Irish heritage
Gypsy/Roma
Pakistani
Any other ethnic group
Black African
Black Caribbean
Bangladeshi
Any other black background
Unclassified
White and black Caribbean
Any other Asian background
Any other white background
White and black African
White British
Any other mixed background
Indian
White and Asian
Irish
Chinese
Effect of ethnicity and deprivation on Key Stage 2 English results
Difference in percentage of pupils achieving level 4+
Effect of ethnicity and deprivation on KS2 English results
-60 -50 -40 -30 -20 -10 0 10
ChineseIrish
White and AsianIndian
Any other mixed backgroundWhite British
White and Black AfricanAny other White backgroundAny other Asian backgroundWhite and Black Caribbean
UnclassifiedAny other Black background
BangladeshiBlack Caribbean
Black AfricanAny other ethnic group
PakistaniGypsy/Roma
Traveller of Irish Heritage
Difference in percentage of pupils achieving level 4+
-60 -50 -40 -30 -20 -10 0 10
Effect of difference inFSM rates
Effect of ethnicity net ofFSM
Observed ethnicityeffect
Effect of difference in FSM rates
Effect of ethnicity net of FSM
Observed ethnicity effect
-
2.18
© Crown Copyright 2005
Outcomes for children from differentethnic groups – Mathematics
Traveller of Irish heritage
Gypsy/Roma
Pakistani
Any other ethnic group
Black African
Black Caribbean
Bangladeshi
Any other black background
Unclassified
White and black Caribbean
Any other Asian background
Any other white background
White and black African
White British
Any other mixed background
Indian
White and Asian
Irish
Chinese
Effect of ethnicity and deprivation on Key Stage 2 maths results
Difference in percentage of pupils achieving level 4+
Effect of ethnicity and deprivation on KS2 Maths results
-60 -50 -40 -30 -20 -10 0 10 20
ChineseIrish
White and AsianIndian
Any other mixed backgroundWhite British
White and Black AfricanAny other White backgroundAny other Asian backgroundWhite and Black Caribbean
UnclassifiedAny other Black background
BangladeshiBlack Caribbean
Black AfricanAny other ethnic group
PakistaniGypsy/Roma
Traveller of Irish Heritage
Difference in percentage of pupils achieving level 4+
Observed ethnicityeffect
-
2.19
© Crown Copyright 2005
Outcomes for children from differentethnic groups – Mathematics
Traveller of Irish heritage
Gypsy/Roma
Pakistani
Any other ethnic group
Black African
Black Caribbean
Bangladeshi
Any other black background
Unclassified
White and black Caribbean
Any other Asian background
Any other white background
White and black African
White British
Any other mixed background
Indian
White and Asian
Irish
Chinese
Effect of ethnicity and deprivation on Key Stage 2 maths results
Difference in percentage of pupils achieving level 4+
Effect of ethnicity and deprivation on KS2 Maths results
-60 -50 -40 -30 -20 -10 0 10 20
ChineseIrish
White and AsianIndian
Any other mixed backgroundWhite British
White and Black AfricanAny other White backgroundAny other Asian backgroundWhite and Black Caribbean
UnclassifiedAny other Black background
BangladeshiBlack Caribbean
Black AfricanAny other ethnic group
PakistaniGypsy/Roma
Traveller of Irish Heritage
Difference in percentage of pupils achieving level 4+
-60 -50 -40 -30 -20 -10 0 10 20
Effect of difference inFSM rates
Observed ethnicityeffect
-
2.20
© Crown Copyright 2005
Outcomes for children from differentethnic groups – Mathematics
Traveller of Irish heritage
Gypsy/Roma
Pakistani
Any other ethnic group
Black African
Black Caribbean
Bangladeshi
Any other black background
Unclassified
White and black Caribbean
Any other Asian background
Any other white background
White and black African
White British
Any other mixed background
Indian
White and Asian
Irish
Chinese
Effect of ethnicity and deprivation on Key Stage 2 maths results
Difference in percentage of pupils achieving level 4+
Effect of ethnicity and deprivation on KS2 Maths results
-60 -50 -40 -30 -20 -10 0 10 20
ChineseIrish
White and AsianIndian
Any other mixed backgroundWhite British
White and Black AfricanAny other White backgroundAny other Asian backgroundWhite and Black Caribbean
UnclassifiedAny other Black background
BangladeshiBlack Caribbean
Black AfricanAny other ethnic group
PakistaniGypsy/Roma
Traveller of Irish Heritage
Difference in percentage of pupils achieving level 4+
-60 -50 -40 -30 -20 -10 0 10 20
Effect of difference inFSM rates
Effect of ethnicity net ofFSM
Observed ethnicityeffect
-
Effect of difference in FSM rates
Effect of ethnicity net of FSM
Observed ethnicity effect
2.21
© Crown Copyright 2005
Children looked after by the local authority (LAC)
% achieving level 2 at the end of Key Stage 1
% achieving level 4 at the end of Key Stage 2
All children LAC All children LAC
2002 85% 50% 78% 40%
2003 85% 53% 78% 42%
2.22
© Crown Copyright 2005
Transitions Key Stage 1 to Key Stage 2
The percentage of children achieving below level 2 at end of Key Stage 1 who achieve level 3+ at end of Key Stage 2
2003 2004
English 67 64
Mathematics 70 69
2.23
© Crown Copyright 2005
Value-added
• looks at progress as well as attainment;
• assumes prior attainment is correlated with later attainment;
• can show ‘average’ performance for individuals with same prior attainment;
• statutory key stage and optional QCA test and teacher assessment data can be recorded using Pupil Achievement Tracker;
• data can be filtered to show progress for different groups of children;
• can incorporate contextual factors. 2.24
© Crown Copyright 2005
What is a value-added linegraph?
0 3 6 9 12 15 18 21
40
35
30
25
20
15
10
5
0
Key
Sta
ge
2 2
003
aver
age
po
ints
sco
re
Key Stage 1 average point score
UQ
LQ
NationalMedian Line
SchoolMedian Line
2.25
© Crown Copyright 2005
Pupil attributes
• class / year group
• Wave 2 and 3 interventions
• involvement of agencies
• additional interventions via IEPs, PEPs, PSPs
• exclusions
Data about provision, targeted interventions or sanctions for example.
• date of birth• gender• type of SEN• pupil in care• ethnicity• EAL step descriptors,
(early, advanced)
Contextual data about individuals, for example.
2.26
© Crown Copyright 2005
National value-added line graphs
•
2004 Key Stage 2 English value-added line
20
30
40
50
60
70
80
90
5 7 8 9 10 11 12 13 14 15 16 17 18 19 20+
Average 2000 Key Stage 1 points score
200
4 K
ey S
tag
e 2
En
gli
sh
mar
k
Level 4
Level 5
Level 3
2.27
© Crown Copyright 2005
Basic view
2.28
© Crown Copyright 2005
Multiple attributes
2.29
© Crown Copyright 2005
One attribute highlighted
2.30
© Crown Copyright 2005
Single group
2.31
© Crown Copyright 2005
Optional tests
2.32
© Crown Copyright 2005
Converting levels to points scores
• W = 3• level 1 = 9• level 2 = 15• level 3 = 21• level 4 = 27• level 5 = 33
Average Key Stage 1 points score is the average of reading, writing and mathematics.
2.33
© Crown Copyright 2005
Converting levels to points scores
Level Sublevel Points
1
1c
1b
1a
7
9
11
2
2c
2b
2a
13
15
17
3
3c
3b
3a
19
21
23
4
4c
4b
4a
25
27
29 2.34
© Crown Copyright 2005
Converting levels to points scores
• P scales – possible point scores
(as proposed by PIVATS)
• P3 = 1 P4 = 1.5
• P5 = 2 P6 = 2.5
• P7 = 3 P8 = 5
2.35
© Crown Copyright 2005
Significance of points
• Minimum expected progress of approximately 2 NC levels or 6 sub-levels over Key Stage 2 (12 terms)
– 2 levels or 6 sub-levels = 12 points
– 1 term = 1 point
– average progress roughly = 1 sub-level in two terms
– (actual progress varies in relation to prior attainment – generally across Key Stage 2 it is a little higher than 12 points)
• Higher rate of progress needed for individuals to ‘catch up’
• Higher rate of progress required nationally if we are to raise standards
2.36
© Crown Copyright 2005
School level value-added measure
PupilPupil’s
KS1 APSPupil’s
KS2 APSMedian KS2 APS
for KS1 APSPupil’s
VA score
A 19.7 31 33 -2
B 15 30 27 +3
C 15 25 27 -2
D 21 33 33 0
Total of all pupils’ value added scores = –1Average value added score per pupil = –1/4 (number of pupils)
School’s value added score = –0.25Centred around 100 = 99.75 (rounded to 99.8 for publication)
2.37
© Crown Copyright 2005
Interpreting the value-added measure
• One value-added point is equivalent to one sixth of a level or one term’s progress.
• A value-added measure of 99 means that on average each of the school’s children made a sixth of a level less (or one term less) progress between Key Stage 1 and Key Stage 2 than the median for children with the same Key Stage 1 achievement.
2.38
© Crown Copyright 2005
Cohort sizes
Number of
children
Boundaries within which results are not considered to be statistically different from the average
Lower boundary Upper boundary
10 98.4 101.6
30 99.1 100.9
50 99.3 100.72.39
© Crown Copyright 2005
PANDA Grades
• nationally;• ‘similar’ schools
– free school meal eligibility– prior attainment.
Ofsted performance and assessment reports – schools are benchmarked for comparison with other schools:
2.40
© Crown Copyright 2005
Benchmark groups
Free school mealsPrior attainment
(average point scores)
Up to and including 8% Up to but not including 12
More than 8% and up to 20% At least 12 but less than 14
More than 20% and up to 35% At least 14 but less than 16
More than 35% and up to 50% At least 16 but less than 18
More than 50% Greater than or equal to 18
2.41
© Crown Copyright 2005
Medians and quartiles
95% 75% 60% 50% 40% 25% 5%
Upper quartile
Median Lower quartile
A*
A B C D E E*
2.42
© Crown Copyright 2005
FSM benchmarks
Percentage of children achieving level 4 and above up to and including 8%
95% UQ 60% Median 40% LQ 5%
English A* 100 A 93 B 89 C 86 C 83 D 79 E 64 E*
Maths 100 90 86 83 80 75 59
Science 100 100 97 95 93 90 80
Percentage of children achieving level 4 and above more than 50%
95% UQ 60% Median 40% LQ 5%
English A* 83 A 69 B 62 C 58 C 54 D 46 E 29 E*
Maths 86 70 62 58 53 47 31
Science 95 84 79 76 72 64 45
2.43
© Crown Copyright 2005
Prior attainment benchmarks
Percentage of children achieving level 4 and above up to but not including 12
95% UQ 60% Median 40% LQ 5%
English A* 75 A 59 B 52 C 50 C 46 D 40 E 22 E*
Maths 78 57 51 48 45 37 21
VA 103.1 101.3 100.7 100.3 99.9 99.3 97.9
Percentage of children achieving level 4 and above greater than or equal to 18
95% UQ 60% Median 40% LQ 5%
English A* 100 A 100 B 100 C 98 C 94 D 91 E 78 E*
Maths 100 100 98 94 92 88 73
VA 101.5 100.1 99.7 99.3 99.1 98.8 97.8
2.44
© Crown Copyright 2005
National Transition matrices 2000 (Key Stage 1) – 2004 (Key Stage 2)
Key Stage 2 English Level
KS1 Points D B N 2 3 4 5 A
Less than 7 8 45 17 4 19 5 0 3
7 to less than 9 2 20 17 6 39 13 0 3
9 to less than 12 0 6 7 4 47 32 1 2
12 to less than 14 0 1 1 1 31 61 4 1
14 to less than 16 0 0 0 0 11 73 15 1
16 to less than 18 0 0 0 0 2 59 38 1
18 or more 0 0 0 0 0 29 70 0
2.45
© Crown Copyright 2005
National conversion indicators 2000 (Key Stage 1) – 2004 (Key Stage 2)
ReadingNumber
of children
% of children achieving at KS2
B3 3 4 5 A L4+
KS1
D 1751 75 8 12 4 1 16
W 17216 70 16 10 1 3 11
1 85890 26 31 36 5 2 41
2C 96099 5 20 59 16 1 74
2B 122493 1 8 58 33 1 90
2A 92414 0 3 48 49 1 96
3 168749 0 0 17 83 0 99
4+ 397 0 0 1 98 0 992.46
© Crown Copyright 2005
2004 Key Stage 2 English value-added progress charts
9 <= Key Stage 1 Average Point Score < 12
7% 4%
46%
1%
33%
8%
0%
20%
40%
60%
80%
100%
B N 2 3 4 5
Key Stage 2 ENGLISH Test LevelKey stage 2 English test level
2.47
© Crown Copyright 2005
Value-added
• Shows individual children (in red) against the national median and upper/lower quartiles
www.standards.dfes.gov.uk/ performance/ap/
2.48
© Crown Copyright 2005
Activity
• How well are we doing?
• How well should we be doing?
The value-added graph shows progress of children between Key Stage 1 and Key Stage 2. Consider how this school might answer:
How might the implications of the data be explored further ?
2.49
© Crown Copyright 2005
Questions to discuss
What data is useful to collect? Why?
What action might follow the value-added data analysis undertaken by the school?
2.50
© Crown Copyright 2005
Aims
• developing a strategic approach to the management of inclusion;
• developing their middle management skills;• making effective use of data to improve
teaching and learning and celebrate children’s progress.
To support those who lead on or coordinate inclusion in:
2.51
© Crown Copyright 2005
Session outline
• Brief introduction and recap
• The case study
• Feedback and discussion
• Priorities and action planning
• Conclusions
2.52
© Crown Copyright 2005
A brief recap
• The school improvement framework– how well are we…variance between schools
and groups of children?
• Increasingly rich data about children’s attainment and progress– improvements over time– comparisons with national, local and similar
schools’ data– comparing progress of different groups of children
• PAT, value-added line graphs, PANDAs, transition matrices and progress charts 2.53
© Crown Copyright 2005
• Consider the data you have about the case study school.
• What initial hypotheses or further questions can be drawn from the ‘How well are we…how do we compare…what should we …’ questions?
How well are we?
2.54
© Crown Copyright 2005
One view…
• This is a very caring school that provides well for some children sometimes.
• The use of data is not systematic.
• There is a culture of low expectations.
• Provision and interventions are not well
evaluated.
2.55
© Crown Copyright 2005
…and some proposals
• We will develop our use of data so as to maintain, improve and target high-quality teaching and learning.
• Our leadership team will provide a clear road map for developing systematic use of data, to include– clear role for coordinators;– involvement of all staff.
• We will develop ‘our’ way of tracking progress – for all children, particularly vulnerable groups.
• We will focus teaching priorities by ensuring manageable targets that are understood by all.
2.56
© Crown Copyright 2005
Priorities
• The headteacher wants to ensure a systematic whole-school approach to the use of data to analyse, improve and celebrate children’s progress.
• In relation to lower-attaining or vulnerable children, what three priorities would you suggest?
• What specific actions would help progress the priority areas you have identified?
2.57
© Crown Copyright 2005
Key actions
• A clear, resourced and timed action plan: – specified actions for all staff– training and consultation
• What data would be good to collect and why?
• How and when should we use it?
• What must we do make it happen?
• Track pupil progress: regular assessment points including QCA optional tests.
• A limited, coherent set of targets.• Map our provision.
2.58
© Crown Copyright 2005
Tracking pupil progress
35
30
25
20
15
10
5
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21+
Input score (e.g. Key Stage 1 average point score)
Ou
tco
me
Sco
re (
e.g
. Yea
r 3
op
tio
nal
tes
t)
2.59
© Crown Copyright 2005
We have considered
• the use of data to analyse, improve and celebrate children’s progress;
in order
• to support strategic management of inclusion and personalised learning;
sustained within a
• collaborative and distributed leadership approach.
2.60
© Crown Copyright 2005
Next steps
• For you as an individual or leadership team
• For us as a group
2.61