making your assessments more secure - questionmark · 2018-11-28 · title: making your assessments...

Copyright © 1995-2018 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Welcome

9 Risks to Test Security (and what to do about them)

Copyright © 1995-2018 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

John Kleeman MA MBCS C.Eng CIPP/E

Executive Director and Founder of Questionmark

30 years of experience in the assessment industry

Presenter today

Slide 2


Create stimulus

Participants answer

Dataset created

Make a decision

Assessment Value Proposition

Slide 3

Can these results be trusted


• To make important decisions about people

• Decisions that matter to society, organizations and individuals

We need to be able to trust the

result dataset

• Assessments must be valid, reliable and fair

• Process and systems must be effective

To be able to make good decisions

• Make assessment results less trustable

• Make decisions taken on assessments less reliable

BUT Test security failures and

cheating

Slide 4

Why Test Security Matters


Data Security

System Security

Slide 5

Trustable Results

Planning Assessment

Authoring Items

Assembling assessment

Pilot and Review

Delivery

Analyze Results

Trustable Results Rely on a Trustable & Secure Process


Data Security

System Security

Slide 6

Trustable Results

Planning Assessment

Authoring Items


Pilot and Review

Delivery

Analyze Results

Content theft

Content theft

Test center disruption

Identity fraud

Unauthorized disclosure

Tampering

Some Risks to Trustable Results

Unauthorized aids / help


What keeps you up at night? ▪ Content theft / leakage

▪ Unauthorized aids for cheating during exams

▪ Identity fraud

▪ Results tampering

▪ Personal information breached

Poll Slide

Slide 7

Quick Poll


Cheating on Exams a Problem Worldwide

Slide 8


Meanwhile Data Breaches Rise …

Slide 9


Confidentiality

IntegrityAvailability

Confidentiality

• Keep questions secure -avoid content theft

• Results only available to those who should know

Integrity

• Right person takes assessment

• Assessment process fair and robust

• No cheating

Availability

• Assessments can be taken when needed

• Results are stored safely

First Start with Your Security Objectives

Slide 10


Formal process can identify all threats, less likely to miss key threats

Quantifying risk allows you to prioritize actions that will reduce risk

Recommended by all respected security authorities: ISO 27001, NIST & many more

Example▪ 2017 Verizon data breach report reported 81% of hacking-related

breaches involved weak or stolen passwords.▪ For most organizations, a risk assessment is going to highlight this

and suggest some mitigations

Why Assess Risks?

Slide 11


High probabilityLow impact

High probabilityHigh impact

Low probability.Low impact

Low probability.High impact

LOW HIGH

LOW

HIG

H

Impact

Pro

bab

ility

Then look at Risk Probability and Impact

Prioritize security mitigations based on risk impact and probability


After delivery

Tampering with results Unauthorized disclosure of results

During test delivery

Identity fraudContent theft /

harvestingCopy answers from

another Getting help from

othersUnauthorized test aids

Before delivery

Content theft from item bank Disruption at test center

9 Common Risks to Consider


Risk:

Content theft from item bank

Authoring Items



Content theft from item bank

Slide 15

Author leaks content deliberately to help their students

Author leaks content by mistake or by using poor security

Technical vulnerability in item bank

Potentially very high: requires rewriting all items

Could invalidate test for all test takers

Threats Risk impact Real world example


More real world examples

Slide 16


Use a secure cloud service▪ Avoids local files▪ Avoids email of questions▪ ISO 27001 and good technical security

Good HR practices with authors▪ Training and education▪ Confidentiality agreements▪ Remove access when people leave project▪ Good passwords▪ Extend to translators too if test is translated

Use permissions and roles▪ Authors should only have access to the minimum they need for a project▪ Restrict access to questions/assessments they do not need▪ Single sign-on

Some mitigations to prevent item bank leakage

Slide 17


Just enough capability to get the job done

In assessment management systems:▪ Use roles and permissions to give

people the minimum capabilities they need

▪ Restrict access to questions to those who need

▪ Have a separate user account for privileged (high capability) actions

One person

requests

Another person

approves

Activity authorized

Two important security principles help

Slide 18

Least privilege Segregation of duties


Risk

Disruption at test center

Slide 19

Pilot and Review Delivery


Disruption at a test center

Slide 20

Poor technical security at test center allows questions to be seen

Test center proctors corrupt

Test center proctors / teachers have incentive to help their students

Can result in a batch of test takers at the test center all being helped




Slide 21


Don’t allow download of questions to test center in advance

Consider not using test centers in pilot process

Data analysis / forensics

Using good technical measures▪ High quality TLS encryption

▪ Proctors unable to see questions

Use online proctoring▪ Remote proctors who do not know test takers and cannot receive a bribe less likely

to collude with test takers

Possible mitigations

Slide 22


Traditional Proctoring•Proctor observes test taker physically in

person / in the same room Live Online Proctoring• Proctor observing test taker live via webcam

Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.


Risk

Identity fraud

Slide 24

Delivery


Identity fraud

Slide 25

Candidate gets a friend to take an exam for him/her

Candidate pays someone to take an exam for him/her

At least one result is meaningless

Can threaten acceptance of programme



Proctoring▪ Proctor checks government issued photo-ID▪ If concerned about proctors colluding with candidates, use online

proctoring

Frequent testing▪ Series of tests harder to get someone to take for you than single large test▪ With online assessment and online proctoring, much more realistic to have

more frequent, smaller tests

Use single sign-on (SAML)▪ People less keen to share credentials if they allow access to all their other

data

Reduce the motivation to cheat

Some ways to mitigate candidate impersonation

Slide 26


Fraud needs:▪ Motivation▪ Opportunity▪ Rationalization

For a high stakes test, there may be Motivation

Anti-cheating measures seek to reduce Opportunity

How can we reduce Rationalization?

Fraud triangle

Slide 27


Mitigating Rationalization

Slide 28

Rationalization

“Everyone else cheats so why shouldn't I?”

“I have no alternative but to cheat and I have to pass because …?”

“I can get away with this”

“I didn’t know I was doing anything wrong”

• Explain the facts:• Most people do not cheat• Consequences of cheating

Mitigation

• Provide environments for the candidate to be able to pass the exam honestly

• Explain the security measures in place and the consequences if they’re caught

• Explain and gain positive agreement to theo Honesty Codeo Code of ethicso Non-disclosure


Risk

Content theft / harvesting

Slide 29

Pilot and Review Delivery


Content theft / harvesting

Slide 30

Screenshot or other technical copy of test questions

Group harvesting, everyone remembers a few questions and shares online

Someone pays to take test just to see and remember questions

Slow degradation of integrity of test

Expensive as need to write new items



Use proctoring but don’t allow proctors to see content

Secure browsers like Questionmark Secure

Large and changing item banks with randomized test content

Reduce motivation to cheat / honor code

Use video/audio as part of stimulus to make harder to copy

Prevent harvesters from taking tests (pre-requisites, limit retakes)

Keep time limit as short as sensible

Possible mitigations

Slide 31


A secure browser reduces content theft and cheating

While using a secure browser, it is difficult for participants to▪ Make screenshots▪ Copy content▪ Run other programs▪ Search the Internet or use chat

Questionmark has a Questionmark Secure App and other vendors have other approaches

Secure browsers


Risk

Copy answers from another

Slide 33

Delivery


Copy answers from another

Slide 34

Candidate looks at the screen of the person next door to him/her and copies their answers

In a take-home exam, two people do it together

Organized answer sharing

Invalid test results for a candidate

Devalues credibility for others



Appropriate separation between workstations

Randomization of question order / question selections / choice order

How to mitigate copying answers

Slide 35


Risk

Getting help from others

Slide 36

Delivery


Getting help from others

Slide 37

Use a cellphone or instant messenging to get live help from a friend or coach

Someone sits by test taker to give coaching

Invalidates one result

If prevalent, degrades program




Slide 38


Use a secure browser (e.g. Questionmark Secure) for online tests

Proctoring (especially online)


Frequent testing

Forensics / analysis

Consider other approaches than essays

Some mitigations

Slide 39


Risk

Unauthorized test aids

Slide 40

Delivery


Unauthorized test aids

Slide 41

Cheat sheets / access to notes

Test taker Googles the answers

Calculators / other tools

Makes results less valid




Slide 42


Secure browsers

Proctoring

Make exam open book▪ In the real world, people have access to reference material, is it fair and valid to deny

it during an exam?

Ask higher level questions (e.g. Situational judgement)


Mitigations

Slide 43


Risk

Tampering with results

Slide 44

DeliveryAnalyze Results


Tampering with results

Slide 45

An insider changes someone’s score out of favouritism

Participant bribes someone to change score

Technical vulnerability exploited to adjust scores

Ranges from just one score awry to whole exam invalidated



Robust delivery and results platform

ISO 27001 or similar security management

Store results in an online cloud system

Permissions / roles / segregation of duty

Robust audit trail

Care with results when leave the assessment system

How to mitigate tampering with results

Slide 46


Risk

Unauthorized results disclosure

Slide 47

Analyze Results


Unauthorized results disclosure

Slide 48

Mistake by administrators discloses data

Malware or other technical vulnerability discloses data

Serious embarrassment and negative publicity

GDPR fines

Invasion of privacy



Ensure that your supplier(s) have strong security▪ ISO 27001 is good

▪ Check they remain committed to security

Train your personnel well

Remove access from personnel who leave organization/project

Ensure your systems are well secured if results go there

Strong passwords and single sign on

Permissions / roles / least privilege

Mitigations

Slide 49


After delivery

Tampering with results Unauthorized disclosure of results

During test delivery

Identity fraudContent theft /

harvestingCopy answers from

another Getting help from

othersUnauthorized test aids

Before delivery

Content theft from item bank Disruption at test center

The 9 risks we have looked at


Identify goals of your assessment program

Look at risks that apply to you

Implement mitigation measures based on importance of those risks

Improved security gives better data for better decisions

Improved security also increases trust from stakeholders

Summary

Slide 51


Data Security

System Security

Slide 52

Trustable Results

Planning Assessment

Authoring Items


Pilot and Review

Delivery

Analyze Results

Trustable Results Rely on a Trustable & Secure Process


Your questions

Slide 53


Wednesday, December 5th

• Tips for Writing Good Test Questions

Tuesday, December 18th

• 10 Quick Tips to Improve your Tests and Exams

Weekly

• Introduction to Questionmark (Live product demos)

Upcoming Webinars

Slide 54

Sign up today at www.questionmark.com/webinars

https://www.questionmark.com/webinars


How to Evaluate Questionmark

• Request a one-on-one demoThe Questionmark team will contact you to arrange a demonstration tailored to your needs and questionswww.questionmark.com/go/request-demo-enus

• Request a 30-day trial of Questionmark OnDemandComplimentary technical support during trialwww.questionmark.com/go/od30us

Slide 55

http://www.questionmark.com/go/request-demo-enus

https://www.questionmark.com/go/od30us


REGISTER NOW!

Questionmark Conference 2019Assess for Success

Slide 56

www.questionmark.com/go/conference

February 26th-March 1stHard Rock Hotel

San Diego"My team always walks away with new knowledge -

and we have been using the product for over 12 years!"

Copyright © 1995-2018 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Follow us…

Thank you for attending!

We hope to see you at a future webinarKeep up to date at blog.questionmark.com!

https://www.questionmark.com/go/sn004




http://blog.questionmark.com/

making your assessments more secure - questionmark · 2018-11-28 · title: making your assessments...

Documents