SQA Adapted Digital Question Papers:

The story so far….

Paul D. NisbetPaul D. Nisbet

CALL Scotland CALL Scotland

University of EdinburghUniversity of Edinburgh

Adapted Digital Papers2006 (pilot)

2007 (pilot) 2008 2009 2010

Number of centres 8 12 46 73

Number of candidates 34 80 204 422

Number of requests 65 265 514 1,167 1,962

Number of candidates / centre 4.25 6.67 4.27 5.78

Number of requests / candidates 1.91 2.50 2.51 2.76

Requests for Adapted Digital Papers 2006-2010

2009 requests by local authorityAuthority Requests Authority Requests

Fife CouncilFife Council 149149 The Moray CouncilThe Moray Council 3131

The City of EdinburghThe City of Edinburgh 122122 Highland CouncilHighland Council 2222

FalkirkFalkirk 9797 Shetland Islands CouncilShetland Islands Council 1818

Dumfries and GallowayDumfries and Galloway 8989 North Ayrshire CouncilNorth Ayrshire Council 1616

AberdeenshireAberdeenshire 6565 North Lanarkshire CouncilNorth Lanarkshire Council 1515

Dundee CityDundee City 6262 Renfrewshire CouncilRenfrewshire Council 1313

South AyrshireSouth Ayrshire 6060 East Dunbartonshire CouncilEast Dunbartonshire Council 77

Perth & KinrossPerth & Kinross 5959 East Renfrewshire CouncilEast Renfrewshire Council 44

Aberdeen CityAberdeen City 5252 Midlothian CouncilMidlothian Council 44

West LothianWest Lothian 4545 Western Isles CouncilWestern Isles Council 22

City of GlasgowCity of Glasgow 4242 East Ayrshire CouncilEast Ayrshire Council 11

Scottish BordersScottish Borders 4141 South Lanarkshire CouncilSouth Lanarkshire Council 11

Argyll and ButeArgyll and Bute 3838

StirlingStirling 3636 FE CollegesFE Colleges 2121

InverclydeInverclyde 3333 Independent Secondary Independent Secondary SchoolsSchools

2222

2009 requests by schools

2009 Centres Requests Local Authority

Queen Anne High School 122 Fife

The Royal High School 82 City of Edinburgh

Bo'ness Academy 68

Stranraer Academy 62 Dumfries & Galloway

Prestwick Academy 60 South Ayrshire

Kinross High School 59 Perth & Kinross

Mintlaw Academy 48 Aberdeenshire

Ashcraig School 37 Glasgow City

Campbeltown Grammar School 37 Argyll & Bute

Peebles High School 35 Scottish Borders

Aboyne Academy 34 Aberdeenshire

St Columba's High School (Gourock) 33 Inverclyde

St Paul's RC Academy 32 Dundee City

Holy Rood High School 30 City of Edinburgh

Buckie High School 29 Moray

2009 requests by subject (subjects with requests > 10)

SubjectNumber of requests Subject

Number of requests

EnglishEnglish 284284 FrenchFrench 3939

BiologyBiology 9898 Modern StudiesModern Studies 3737

HistoryHistory 9292 Physical EducationPhysical Education 3636

Computing StudiesComputing Studies 9191 MathematicsMathematics 2929

GeographyGeography 7676 Business ManagementBusiness Management 2828

ChemistryChemistry 5050 DramaDrama 2828

Craft & DesignCraft & Design 5050 Home EconomicsHome Economics 2222

AdministrationAdministration 4848Mathematics: Maths 1, 2 Mathematics: Maths 1, 2 and 3and 3 1212

PhysicsPhysics 4242 GermanGerman 1111

Reasons for requesting ADPs, 2009

Candidates' Difficulties

Physical or Motor Impairment 143

Other Moderate Learning Difficulty

65

Autistic Spectrum Condition 39

Visual Impairment 30

Dyslexia 474

Other Specific Learning Difficulty

244

Physical Health Problems 61

Social Emotional Behavioural

Difficulty 37

Language or Speech Disorder

32

Learning Disability 38

Pupils’ comments 2006 & 2007

Prefer it to using a

scribe. Easier to give

answers.

Easy to do. You can see both the questions and the text at the same time.Enjoyed using

it!

I would prefer to have split screens so that I could see questions and tables together instead of having to refer to another page.

Pupils’ comments 2006 & 2007

Papers are simpler to use. You do not have to fiddle about with different papers. By switching windows, you can see both the question and the text at the same time.

Because it is much easier to use than a reader

Staff evaluations 2006 trials

0

1

2

3

4

5

6

Confidence Independence Motivation Expertise

Traditional papers Digital Papers

Pupils’ results with digital papers

““Candidates’ results from digital papers are Candidates’ results from digital papers are similar to their teachers’ estimates”similar to their teachers’ estimates”

““there appears to be little difference between there appears to be little difference between [marks awarded for] entries using digital papers [marks awarded for] entries using digital papers and the other entries sat by the same and the other entries sat by the same candidates”candidates”

Nisbet, P.D., Shearer, N., Balfour, F., Aitken, S., (2006) Nisbet, P.D., Shearer, N., Balfour, F., Aitken, S., (2006) SQA Adapted SQA Adapted Examination Papers in Digital Format: Feasibility Report 2005-2006Examination Papers in Digital Format: Feasibility Report 2005-2006. .

www.adapteddigitalexams.org.ukwww.adapteddigitalexams.org.uk

Reliability ““Digital exams were the only part of the 2007 Digital exams were the only part of the 2007

diet which did not cause significant problems – diet which did not cause significant problems – they simply worked. Probably this was assisted they simply worked. Probably this was assisted by more stringent attention to detail, adequate by more stringent attention to detail, adequate risk assessment, etc.”risk assessment, etc.”

““I personally would like to see digital exams as I personally would like to see digital exams as the default choice for pupils with physical the default choice for pupils with physical disabilities, and paper, helpers, scribes etc as disabilities, and paper, helpers, scribes etc as options which would need to be specially options which would need to be specially requested, because I believe they are requested, because I believe they are empowering, less difficult to administer and cost-empowering, less difficult to administer and cost-effective.”effective.”

Accommodation““We had 2 rooms set up with We had 2 rooms set up with double sized tables to allow a double sized tables to allow a comfortable amount of room comfortable amount of room for laptop and papers. Each for laptop and papers. Each candidate was allocated a candidate was allocated a table which was the same for table which was the same for each exam. All tables were each exam. All tables were numbered and we made use of numbered and we made use of screen dividers which created screen dividers which created a booth effect and cut down a booth effect and cut down distractions and gave a certain distractions and gave a certain amount of privacy in the small amount of privacy in the small classroom. We had 9 classroom. We had 9 candidates sitting the exam in candidates sitting the exam in digital format and 2 writing in digital format and 2 writing in the same room for the the same room for the Standard Grade English exam. Standard Grade English exam. Candidates names and Candidates names and numbers were on the desks, numbers were on the desks, and times.”and times.”

% of AA candidates using ADPs

(12 schools 2006-08)

Ashcraig

Broughton High

Buckie H

igh

Campeltown H

igh

Graeme High

Holy Rood H

igh

Inverness H

igh

Kinross High

Prestwick

Academy

The Roya

l High

Stranraer A

cademy

Uddingston

2006

2008

0%

10%

20%

30%

40%

50%

60%

70%

2006

2007

2008

2008 requests to support writing (46 centres using ADPs)

Use of ICT

Digital Papers

Scribe

Candidate Signs Responses

Use of tape recorder for responses

Transcription without correctionTranscription with

correction

Calculator

PA Referral

“We really appreciate this format of exam paper. The pupils are generally much happier to be independent rather than depend on scribes and readers. This year all our S3 and S4s sat the SG English examination at the same time. If we had had to provide readers and/or scribes for this we would not have been able to staff it.”

Support for writing 2007-08(12 centres using ADPs)

ADPICT

Scribe

2007

20080%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Support for reading(data from 18 centres in 2008)

Reading support 2008

Reader, 894

Question Paper signed to

candidate, 1

Coloured Paper, 74

Text reading software, 71

Enlarged Print, 63 Braille paper, 2

Text-reading software2007-2008

Ashcraig School

Prestwick Academy

Buckie High School

Campbeltown Grammar School

2007

20080%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Use of text reading software 2007 - 2008

2007

2008

Use of readers 2007-2008

Ashcraig School

Prestwick Academy

Buckie High School

Campbeltown Grammar School

2007

20080%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Use of readers 2007-2008

2007

2008

Cost analysis:School ‘H’ English Standard Grade, 5 pupils using ADPs

instead of reader/scribes

RoomsRooms TimeTime Reader/scribe Reader/scribe timetime

Reading Reading GeneralGeneral

55 1 hour1 hour 5 hours5 hours

Reading Reading CreditCredit

55 1 hour1 hour 5 hours5 hours

Writing F/G/CWriting F/G/C 55 1.5 hours1.5 hours 7.5 hours7.5 hours

TotalTotal 1515 3.5 hours/ pupil3.5 hours/ pupil 17.5 hours17.5 hours

School ‘H’ Standard Grade: digital papers for 5 pupils

5 pupils, 5 computers, 1 room5 pupils, 5 computers, 1 roomPreparation time (SfL): 1 hour?Preparation time (SfL): 1 hour?Technician time: Technician time: 2 hours?2 hours?Running time (SfL): Running time (SfL): 3.15 hours?3.15 hours?Total:Total: 7.15 hours7.15 hours

School ‘H’ readers/scribes vs. digital

Readers / Readers / scribesscribes

Digital Digital PapersPapers

SavingSaving

Staff hoursStaff hours 17.517.5 7.157.15 10 hours10 hours

Staff cost @ £25/half dayStaff cost @ £25/half day £250£250 £143 *£143 * £107£107

Staff cost @ £20/hourStaff cost @ £20/hour £350£350 £143£143 £207£207

Staff cost @ £200/dayStaff cost @ £200/day £1000£1000 £143£143 £857£857

RoomsRooms 55 11 4 rooms4 rooms

School H buys in all readers/scribes at £25 per half day.

The school saved £1000 by using ADPs in 2008.

* Cost calculated at £20/hour

Costs of readers/scribes

16,554 reader requests; 14,197 scribe 16,554 reader requests; 14,197 scribe requestsrequests

~ 30,000 individual exams~ 30,000 individual exams~ 60,000 hours~ 60,000 hoursSay average £20/hour for reader/scribe?Say average £20/hour for reader/scribe?

= £1,200,000= £1,200,000Say £10/hour for invigilator?Say £10/hour for invigilator?

= £600,000= £600,000TOTAL = TOTAL = £1.8 m£1.8 m in Scotland per annum in Scotland per annum