monitoring sqa qualifications over time dr rob van krieken
TRANSCRIPT
Monitoring SQA Qualifications over time
Monitoring SQA Qualifications over time
Dr Rob van Krieken
Another bridge in Arnhem
ContentContent
Why is SQA monitoring over time?
What are the standards SQA is monitoring?
SQA’s monitoring programme
Implementation
Reports and actions
Why is SQA monitoring over time?Why is SQA monitoring over time?
to avoid
‘creep’ within a stable system
shock when the system changes
Example of “creep”Example of “creep”
Year 1 Year 2
Cut-off % Pass Cut-off % Pass
49 86% 49 87%
50 80 % 50 81 %
51 72% 51 73%
Example of “creep”Example of “creep”
Year 2 Year 3
Cut-off % Pass Cut-off % Pass
49 87% 49 88%
50 81 % 50 82 %
51 73% 51 74%
Example of “creep”Example of “creep”
Year 3 Year 4
Cut-off % Pass Cut-off % Pass
49 88% 49 89%
50 82 % 50 83%
51 74% 51 75%
Example of “creep”Example of “creep”
Year 1 Year 4
Cut-off % Pass Cut-off % Pass
49 86% 49 89%
50 80 % 50 83%
51 72% 51 75%
System changesSystem changes
Vocational qualifications are reviewed systematically at the end of their duration
O-grade to Standard Grade (1984 – 1990s)
SCE Higher – Revised Higher (1992 - )
(Revised) Higher - Higher (Still) (2000)
CSYS – Advanced Higher (2001)
New National Qualifications (2014-2016)
What are the standards SQA is monitoring?What are the standards SQA is monitoring?
‘Minimum level of demand’
– Laid down in specifications (“standards”)
– Exemplified in assessments/marking/boundary
Within the educational context of
– SCQF levels, similar qualifications
– Quality and quantity of teaching and learning
– Cohort
Monitoring programmesMonitoring programmes
Different programmes for National Courses and Vocational Qualifications (HN and SVQ)
Sampling by subject area and level
Cycle depends on qualification / level
Sample Higher National DiplomasSample Higher National Diplomas2014 • 2015 2016 2017 2018 2019
Beauty Therapy Administration and
Information Technology
Early Education and Childcare: Childhood Practice
Social Care Social Service
Hospitality Management
Computing: Software Development
Travel and Tourism
Nautical Science
Early Education and Childcare
Beauty Therapy Administration
and Information Technology
Social Care
Hospitality Management
Computing: Software Development
Social Service
Travel and Tourism
Nautical Science
C&ACollect and archive
C&CCollect and Compare
Sample Higher National QualificationsSample Higher National Qualifications
2013 2014 2015 2016 2017 2018 2019 2020
English C&A C&C C&C
Mathematics C&A C&C C&C
Art and Design C&A C&C
Business Management C&C
Chemistry C&A C&C
Computing C&C
Engineering Science C&C C&C
French C&A C&C
Geography C&A C&C
RMPS C&A C&C
Implementation: Panels, materials, and tasksImplementation: Panels, materials, and tasks
Panel:
– 3 subject experts
Materials:
– Specification
– Assessments, Marking Instructions
– Evidence
– Judgements
Task:
– Questionnaire
QuestionnaireQuestionnaire
1 Educational Context:
Are there any general changes that may have influenced the general ability of candidates in one of the years, in particular the way they meet the demands made by the syllabus and exam?
QuestionnaireQuestionnaire
2 Unit Specification or Standards and Guidance
2.1 Are there differences in the Unit Specifications and Guidance that affect the level of demand?
2.2 Were the Unit Specification Standards in 2011 on the whole more / no more / less demanding than before?
QuestionnaireQuestionnaire
3 Assessment
3.1 Are there differences in Assessment practice that affect the level of demand? Please tick the relevant boxes and add if necessary.
3.2 Was the Assessment in 2011 on the whole
more / no more / less demanding than before?
QuestionnaireQuestionnaire
4 Quality and Judgement of Evidence
4.1 Are there differences in the quality of the evidence and in the way this has been
judged? Please tick the relevant boxes and add if necessary.
QuestionnaireQuestionnaire
5 Overall judgement
A reasoned overall judgement of the level of demand of the qualification as a whole, taking into account standards, assessments, and evidence.
Reports and actionsReports and actions
Overall report
Summary of conclusions
Actions SQA undertakes
Issues and answers 1Issues and answers 1
Too much change
Overall judgement
Quantity and type of evidence
Panel membership
No looking Backward
