introduction to cem secondary pre-16 information systems neil defty secondary systems programme...
TRANSCRIPT
Introduction toCEM Secondary Pre-16
Information Systems
Neil Defty
Secondary Systems Programme Manager
Ensuring Fairness
Principles of Fair Analysis :
1. Use an Appropriate Baseline
2. Compare ‘Like’ with ‘Like’
3. Reflect Statistical Uncertainty
CEM Systems
Principle of Fair Analysis No1 : Use an Appropriate Baseline
The Assessments
MidYIS - Year 7, 8 or 9Developed ability assessments:
VocabularyMathsNon-verbal (Cross-sections, Block Counting and Pictures)Skills (Proof Reading and Perceptual Speed & Accuracy)
SOSCA (“INSIGHT Core”) – Year 9Curriculum-based assessments:
Reading (Passage Comprehension, Text Comprehension and Speed Reading)Maths (Number & Algebra, Handling Data and Space, Shapes & Measures)Science (Biology, Chemistry and Physics)
INSIGHT – Year 8* or 9 (* depending on numbers)
Developed ability assessments (‘MidYIS’) plus Curriculum-based assessments (‘SOSCA’)
Yellis - Year 10 or 11Developed ability assessments:
VocabularyMathsPatterns (Matching and Reflections & Rotations)
3 sessions - Computer adaptive
4 sessions - Computer adaptive
1 session – Computer adaptive or paper-based
1 session - Computer adaptive or paper-based
Computer Adaptive Paper-based
Number of Students per Test Session
Limited by number of computers available – multiple testing sessions
Can test all students in a single session (in hall or in form groups) or in more than the one session
Cost Roughly 30% cheaper than
paper-based testStandard cost
Processing of Baseline
Feedback
Baseline feedback available within a couple of hours of testing
Takes around 2-4 weeks for papers to be marked
Preparation
Must be able to install the software or access the internet version of the test
No pre-test set up
Student Experience
“Tailored” assessment All students see all questions, irrespective of suitability
Computer-adaptive vs Paper-based testing
The Analysis
.
Subject X
012345678
70 80 90 100 110 120 130
Baseline
Out
com
e
50 100 150
-ve VA
+ve VA
Regression Line (…Trend Line, Line of Best Fit)
Outcome = gradient x baseline + intercept
Correlation Coefficient (~ 0.7)
Residuals
Subject X
Linear Least Squares Regression
Some Subjects are More Equal than Others….
Principle of Fair Analysis No2 : Compare ‘Like’ with ‘Like’
Trend segments for 2009 GCSEs: drawn over 68% of Yellis intake(mean +/-1 standard deviation)
1
2
3
4
5
6
7
8
20 30 40 50 60 70 80
Yellis Test Score (%)
GC
SE
po
int
sco
re (
A*=
8, A
=7,
B=
6, …
)
English
Maths
Art & Design
Geography
Biology
French
Business Studies
Core Science
Additional Science
Add Appl Science
The Assessments
Maths
Vocabulary
Non-verbal
Skills
Maths
Science
Reading
Baseline Assessment and Predictive Feedback
Individual Pupil Recordsheets (IPRs)
Band Profile Graphs
Predictions
English Language - band D
2
10
23
3124
9
1 0 00
10
20
30
40
50
U G F E D C B A A*
Grade
Perc
ent
English Language - band C
1 26
20
3529
71 0
0
10
20
30
40
50
U G F E D C B A A*
Grade
Perc
ent
English Language - band B
0 0 27
24
40
21
50
0
10
20
30
40
50
U G F E D C B A A*
Grade
Perc
ent
English Language - band A
0 0 0 1
8
26
35
23
7
0
10
20
30
40
50
U G F E D C B A A*
Grade
Perc
ent
Chances Graphs
Value Added Feedback
Burning Question :
What is my Value-Added Score ?
Better Question :
Is it Important ?
Principle of Fair Analysis No3 : Reflect Statistical Uncertainty
The Scatter Plot
Baseline Score
Gra
de
Po
ints
Eq
uiv
alen
t
Look for Patterns…
General under- or over-achievement ?
Do any groups of students stand out ?
– high ability vs low ability ?
– male vs female ?
Value Added Feedback
Year 7 Pupil Level Residuals to GCSE
Standardised Residuals Graph
-0.2 -0.2 -0.1
0.1
-0.6
0.1
-0.2
-0.7 -0.7-0.4 -0.4 -0.2
-2.2
-0.5
2.0
-0.7-0.5
0.10.4
-0.4
-4.0
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
4.0
Art
Bio
logy
Che
mis
try
Des
ign
and
Tec
hnol
ogy
Dra
ma
Eng
lish
Lang
uage
Eng
lish
Lite
ratu
re
Fre
nch
Geo
grap
hy
Ger
man
His
tory
Hom
e E
cono
mic
s
Info
rmat
ion
Tec
hnol
ogy
Mat
hs
Med
ia S
tudi
es
Mus
ic
Phy
sics
Rel
igio
us S
tudi
es
Sin
gle
Sci
ence
Dou
ble
Sci
ence
Av
era
ge
Sta
nd
ard
ise
d R
es
idu
al
Statistical Process Control (SPC) Chart
Attitudinal Surveys
Attitudinal Feedback
Your data is above the Yellis average
Your data is below the Yellis average
Your data is about the same as the Yellis average
Your data is about the same as the Yellis average
Attitudinal Feedback
Secondary Pre-16 Contact Details
Tel: 0191 334 4255
Email: [email protected]@[email protected]@cem.dur.ac.uk
Web: www.cemcentre.org