outline teacher malpractice in assessment: the

30/03/2015

1

© UCLES 2015

Teacher Malpractice inAssessment: the InternationalContext

Ardeshir Geranpayeh, Ph.D.

Head of Psychometrics & Data Services

Oxford, 26 March 2015

Outline• Introduction

• Cheating (Malpractice):

- What, Where, How, Why, Who

- Consequences: Threat to Test Validity

• Standards and Regulatory Requirements

• How to avoid Cheating:– Prevention vs Detection

– Policies on punishment

• School Collusion: Teacher Malpractice

Cheating in the News

Driving test fraud:

More learner drivers

caught cheating,

BBC News, 26 Jan 2015


• 2,440 Chinese students caught cheatingin latest high-tech scam

Chinese state television reported on Sundaythat invigilators detected abnormal radiosignals from an illegal frequency duringnational licensing tests for pharmacists inShaanxi province. The independentWednesday, 28 October 2014

30/03/2015

2


According to The Harvard Crimson, a surveyof this year's incoming freshmen revealedthat 42 percent had cheated on high schoolhomework before arriving at the Ivy Leagueinstitution.

The Huffington Post,

18 February 2015


More than 50,000 English language teststaken by overseas students to extend theirBritish visas have been declared invalid orquestionable as a result of an officialinvestigation into cheating on a huge scale.

The Guardian, 24 June 2014


Cheating in Bihar:India arrests

Hundreds over Bihar

Blatant Cheating

In Schools

BBC News

21 March 2015


'Cheating teachers have lost morality':

Dirty tricks on

the rise to boost

exam grades

MailOnline

13 June 2014

30/03/2015

3

abcNews abcNews Poll1 in 3 theythemselveshave cheated.

Rising to 43%of older teens.

Most saycheaters don'tget caught.

What is Cheating?“Any action that violates the rules foradministering a test” Cizek, 1999:3

Where does it happen?

It can take a variety of forms

30/03/2015

4

Pen Scanners

There are YouTubevideos on How to Scan a

Document on the Go

Text Messaging

Examinee can askquestions and getanswers from friendduring test via textmessaging.

Toilet Breaks!

iPodSong names are renamedwith notes or test answersfor viewing on the screen.

Text files can be stored.

Audio notes can bestored.

Video notes can bestored.

Calculator

Notes are enteredinto calculators thathave memory forstoring notes.

30/03/2015

5

Camera Phone

Examinee takespictures of a test witha camera phone andsends picture toanother person whocan text messagecorrect answers back.

New Technology

Google Glass

Share your

adventures instantly

to Google+, Facebook,YouTube …

Apple Watch

Why does it happen?

–Material rewards such as

• Access to life chances

• Competitiveness

• Lack of self confidence

• Publication of league tables(Schools)

–Unintentional Cheating

• Students, material sharing

• Teachers and Schools

Who is involved?

• Maladministration

• Imposters

• Leaked paper / answers

• Prohibited material

• Copying

30/03/2015

6

What is at stake?– Threat to test validity

– Score obtained by fraudulent means is notvalid

– Has negative impact on the validity of scoresobtained by other candidates

– Denying opportunities to others

– Where cheating is seen to be widespread,even honestly obtained test results may losecredibility and certificates become devalued

What is at stake?

• Test Security is now seen as one of the mostimportant aspects of test Validity

• A Test may not be Valid if it is not secure

Standards for Prevention ofCheating

Explicit statements in the Standards for

Educational and Psychological Testing(1999)


1. Protect the security of tests (standard 11.7)

2. Ensure that individuals who administer the tests areproficient in administration procedures and understandthe importance of adhering to directions provided by thetest developer (standard 13.10)

3. Inform examinees that it is inappropriate for them tohave someone else take the test, for them to disclosesecure test materials, or engage in any other form ofcheating (standard 8.7)

30/03/2015

7


4. Ensure that test preparation activities and materialsprovided to students will not adversely affect the validityof test score inferences (standard 13.11) and

5. Maintain the integrity of test results by eliminatingpractices designed to raise test scores withoutimproving students, real knowledge, skills, or abilities inthe area tested (standard 15.9)

Cheating detection:Regulatory requirements

Ofqual’s conditions aroundmalpractice

• Ofqual is a risk-based regulator of qualifications(other than degrees) that sets the standardsthat awarding organisations need to meet whenthey design, deliver and award regulatedqualifications, via a number of conditions ofrecognition.

• Ofqual doesn’t tell AO’s how to managemalpractice. It tells AO’s what must result fromtheir behaviour.

Condition A8: Malpractice andmaladministration

Condition A8 of the General Conditions of Recognitioncovers:

Preventing malpractice and maladministration

Investigating and managing the effect of malpractice andmaladministration

Procedures relating to malpractice and maladministration

Dealing with malpractice and maladministration

30/03/2015

8

Examples of ‘Positive indicators’The awarding organisation: has ways of working that reduce risks and incidents

occurring makes sure that individuals involved in the delivery of its

qualifications understand and routinely follow these waysof working

takes all reasonable steps to provide information whichis not inaccurate or misleading

has policies, practices and/or procedures that reduce therisk of malpractice and maladministration

When Ofqual monitored Cambridge English LanguageAssessment they were happy with how we are managingmalpractice.

Examples of ‘Negative indicators’ Using an individual or Centre to assist with an

investigation when there is a suspicion or allegationthat they were connected to, or responsible for, thecase being investigated

Not demonstrating steps taken to prevent repeatincidents of previous cases

not following its own procedures during investigation not properly evaluating or validating evidence collected

during investigation not keeping records/documents relating to the incident

for an appropriate period not being able to enforce action taken against those

involved, and cannot demonstrate a good reason why ithas been unable to do so

How to avoid Cheating? How to avoid Cheating?

• The Best practice is “Prevention”

• No Statistical magic can replace goodinvigilation

• Tension between Faster/on demand resultsand proper data forensic processes

• Use variety of ID checks

– Candidate ID Verification

– Biomedical checks

– Hand writing

30/03/2015

9

Policies on punishment

• Once a cheating is detected, an action has tobe put in place to

• Stop fraudulent use of test results

• Deter future cheaters

• Punishment is dependent on the level ofcheating, which in turn can depend on 5 levelsof cheating detection

• Individual candidates, Group of candidates,Teacher Malpractice, Test Centre collusionand Widespread cheating

Level of punishment

• Withdrawing results/certificate (individual)

• Re-taking the exam (suspect results)

• Life Ban (if stakes is high or imposters)

• Informing stake holders (regulator)

• Legal action (insider)

What about School collusion:Teacher Malpractice

• Students implicated may not have beeninvolved in the cheating

• Whilst the candidate’s results may becancelled if school was to be blamed, nofurther action will normally be takenagainst candidates

Teacher Malpractice: the Atlanta case

• In 2009, Beverly Hall, Ex-Atlanta schoolsuperintendent, was named nationalsuperintendent of the year largely basedon her district’s improved standardisedtest scores

• She was once credited with turningaround some of the city’s lowest-performing schools

30/03/2015

10


• Biggest US cheating scandal in US History

• Cheating detected on a 2009 standardizedstate test involving 178 teachers andprinciples, 56 schools investigated cheated,43 people were indicted

• Prosecutors set out to reveal a culture of fearand pressure existed under Hall

• Georgia Governor determination to trace itssource

• Cheating traced back to 2001


• The scandal testifies that cheating is nolonger seen as an old-fashioned battlebetween teachers and students

• When the stakes are high, teachers wouldalso be willing to cheat

Final Remarks

• Use new psychometric techniques todetect school collusion

• Erasure analysis detection

• Multifaceted approach to monitor teachers’assessment

• Malpractice awareness among teachers

• Collaborative dialogue with teachers asthe guardians of students’ achievement

30/03/2015

11

Thank you for listening.

[email protected]

outline teacher malpractice in assessment: the

Documents