grading review sampling
DESCRIPTION
Grading Review Sampling. Report and analysis. Introduction. Our responsibility It is the duty of the Awarding Organisation to put in place measures to safeguard the quality, integrity and value of the qualifications it offers. Background. The drivers - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/1.jpg)
![Page 2: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/2.jpg)
Grading Review Sampling
Report and analysis
![Page 3: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/3.jpg)
3
Introduction
Our responsibility
It is the duty of the Awarding Organisation to put in place measures to safeguard the quality, integrity and value of the qualifications it offers.
![Page 4: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/4.jpg)
4
Background
The drivers
• BTEC benchmarking data shows an increase in higher grades awarded at Level 3 over recent years
• ALPs data shows higher grades awarded compared to “predicted grades” for individual learners– www.alps-va.co.uk
• There are several factors which may have influenced this, but we need to ensure that we have all the facts
• We also need to ensure that our quality assurance processes are supporting accurate and standardised assessment, particularly at the higher grades
• Vocational education in general is subject to increasing scrutiny
![Page 5: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/5.jpg)
5
Claims at “DDD” grade: 2010-11
Sector>=
20%>=
30%>=
40%>=
50%>=
60%>=
70%>=
80%>=
90%=
100%
Applied Science 138 105 66 38 19 9 6 3 1
Business 308 200 132 80 53 41 25 15 13
Construction 84 65 37 22 8 5 5 2 0
Engineering 126 83 51 31 17 6 5 2 2
Health & Social Care 285 234 180 131 79 45 23 14 10
Hospitality 26 17 13 10 7 3 2 2 2
IT 231 156 102 61 36 19 10 4 3
Landbased 84 64 42 23 9 6 5 3 2
Learning & Development 137 116 77 53 38 24 11 4 1
Media , Arts & Design 238 137 67 42 23 17 13 8 8
Performing Arts 311 233 171 119 82 62 45 30 27
Sports 372 284 206 140 90 59 46 31 22
Public Services 147 86 47 25 15 9 6 3 2
Travel & Tourism 94 58 39 27 18 9 5 3 3
Overall 2581 1838 1230 802 494 314 207 124 96
![Page 6: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/6.jpg)
6
Our response
Our response
• Grading Review Sampling formed part of our wider response
• New role of Grade Reviewer, recruited from Grade A and B Standards Verifiers
• A modified version of Confirmation Sampling:– Standards Verification visit, but with more focused sampling
strategy– Audit of assessment records with reference to certificate claims– Involved programmes where 30% or more learners achieved
“DDD” at BTEC Nationals in 2010-11
• Note: cohorts of 5 or fewer were not included– They were subject to normal Confirmation Sampling
![Page 7: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/7.jpg)
7
How it went
Centres
• Mainly FE colleges involved
• Several colleges had a high number of allocations
• Significant concerns expressed by many centres
• More positive after context was explained
• Positive feedback following visits
Resource implications
• Manual allocations
• Increased workload for Deployment, Quality Standards, RQMs, SVs
• Old NQF Sectors – QCF Principal Subject Areas
![Page 8: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/8.jpg)
8
Statistics
Results:
Reasons why reports were blocked:
Number of issues:
Programmes reviewed
Programmes released
1st blocks % BlockedReleased
after action & resample
2nd blocks
2065 1495 570 27.6% 539 31
Certification Claims Procedures 189
Assessment Decisions 489
Internal Verification 402
One issue 182
Two issues 314
All three issues 74
![Page 9: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/9.jpg)
9
Statistics
Sector Blocked Released % Blocked
AEROSPACE NQF1-3 0 1 0.0%
AGRICULTURE NQF1-3 3 13 18.8%
ANIMAL CARE NQF1-3 1 45 2.2%
APPLIED SCIENCE NQF1-3 28 23 54.9%
ART AND DESIGN NQF1-3 32 183 14.9%
BUILDING SERVICES NQF1-3 2 0 100.0%
BUSINESS NQF1-3 39 119 24.7%
CCLD NQF1-3 26 52 33.3%
CIVIL ENGINEERING NQF1-3 0 1 0.0%
CONSTRUCTION NQF1-3 30 22 57.7%
COUNTRYSIDE/FISH NQF1-3 6 24 20.0%
DANCE NQF1-3 8 32 20.0%
![Page 10: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/10.jpg)
10
Statistics
Sector Blocked Released % Blocked
DENTAL TECHNOLOGY NQF1-3 1 0 100.0%
E-BUSINESS NQF1-3 0 2 0.0%
ELEC/ELECTRONIC NQF1-3 7 17 29.2%
E-MEDIA NQF1-3 7 9 43.8%
ENGINEERING NQF1-3 9 27 25.0%
FLORISTRY/HORTICULTURE NQF1-3 2 12 14.3%
FORENSIC SCIENCE NQF1-3 16 26 38.1%
HAIRDRESSING/BEAUTY NQF1-3 6 11 35.3%
HEALTH & SOCIAL CARE NQF1-3 58 179 24.5%
HOSPITALITY NQF1-3 2 10 16.7%
IT PRACTITIONERS NQF1-3 51 129 28.3%
LAW AND LEGAL NQF1-3 0 1 0.0%
![Page 11: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/11.jpg)
11
Statistics
Sector Blocked Released % Blocked
MECH + MAN NQF1-3 10 16 38.5%
MUSIC PRACTICE NQF1-3 22 44 33.3%
MUSIC TECHNOLOGY NQF1-3 11 43 20.4%
OPS + MAIN NQF1-3 0 1 0.0%
PERF. ARTS PERFORMANCE NQF1-3 63 116 35.2%
PERF. ARTS PRODUCTION NQF1-3 12 22 35.3%
PUBLIC SERVICES NQF1-3 22 46 32.4%
SPORT + EX. SCIENCE NQF1-3 8 40 16.7%
SPORT NQF1-3 38 155 19.7%
TRADITIONAL MEDIA NQF1-3 32 48 40.0%
TRAVEL + TOURISM NQF1-3 10 24 29.4%
VEHICLE REPAIR NQF1-3 8 6 57.1%
![Page 12: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/12.jpg)
12
Conclusions
• Plenty of good practice observed
• No deliberate assessment malpractice found
• Sampling strategy identified more issues, but blocks rates on assessment decisions are similar to SV
• Centre issues highlighted:○ Management of staff turnover○ Retention of current learner evidence○ Retention of learner records○ Individual assessors, rather than entire teams assessing
inaccurately
![Page 13: Grading Review Sampling](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56814b8d550346895db8730c/html5/thumbnails/13.jpg)
Next steps
• An initial report from Regulation, Standards & Research has been sent to the regulator
• No repeat of Grading Review Sampling, but it will inform future QA processes○ Upgrading of “Registration & Certification” Objective in Quality
Review & Development○ Overall grade boundaries being looked at○ Standards Verification must sample a range of grades where
possible○ Guidance and requirements being made more explicit○ Stronger recommendations on assessment practice (e.g.
resubmissions)
• Development of next generation BTEC Nationals from 2014○ Consulting FE sector on potential QA models
13