1

Creating and using online assessments to improve training effectiveness

Created byMartin Beltonmartin.belton@qmark.co.uk

June 2001

Tom Blodgett and Allen WoodsTBA Resources, Inc.

2

Introduction Computers have revolutionised

training More people are being trained More subjects to train More methods of delivering training Internet/intranet have

prevented distance becoming the problem

3

Introduction‘...industry is spending billions and

billions on training programs and doing no evaluation of the results’

- Prof. Gary Cheeriness, Rutgers

4

Introduction Too much training is ‘throw it against

the wall and hope some of it sticks’ Ineffective Costly CEOs won’t be impressed by the ‘let’s try it

again’ argument for ever Assessment validates the effectiveness

of training, to standardize best practices

5

Why you need training assessment Assessment enables us to discover:

Whether training is needed What training is needed What an individual has mastered What has been absorbed by a group Whether the investment has payback What can be done to improve the

training

6

Improving training efficiency

Pre courseAssessment

Post courseAssessment 1

Post courseAssessment 2

Learning object 1

Learning object 2

Learning object 3

Learning object 4

LMS Database

7

Today’s training methods Training or instructing by computer is no

different from any other style of training Formal process to help employees learn A coordinated set of activities leading to

change in employee behavior Goals remain the same:

Instruction results in employee learning Employees use new knowledge to address

challenges

8

Perception Software Creates questions, tests, using Windows

applications (Question Manager and Session Manager)

Runs assessments online, in a secure environment (Security Manager)

Creates reports to analyze participant’s data (Enterprise Reporter)

9

Questionmark PerceptionThe Big Picture

1. Create questions and group into quizzes,

tests, exams and surveys using a web browser or

Windows PC

2. Specify security

parameters, schedule

assessments and potentially

link to other systems

3.Deliver assessment using a standard, or secure, browser, or using a Windows PC

4. Report on the results. Reports include coaching reports, item analysis, gap analysis, and others.

10

Enterprise Manager Browser Based

PC Based

Windows Based

Applications in the Big Picture

QuestionManager

AssessmentManager

Browser BasedQuestionManager

AssessmentManager

Enterprise ManagerBrowser Based

ManageParticipants

ManageAdmin Users

Windows PC

Browser BasedIE or Netscape

Secure BrowserPerception PSB

Enterprise Reporter

Windows BasedWindows Reporter

ManageSchedules

11

Databases/TablesUnder the Hood

AssessmentDefinitions

Questions

Self-Registration

Results

Schedules

Names & Passwords

Credit Card

Report Formats

12

Inet InternetIntranet

Internal Network

Perception System Diagram

Widows Based AuthoringQuestion Manager

Assessment Manager

AssessmentDefinitionsQuestions

PerceptionSecure Browser

Internal Explorer

Netscape

Web ServerWBA

AssessmentDefinitionsQuestions

Participants Answering Assessments

ItemResults

ParticipantResults

Authoring Assessments

Enterprise ReporterWindows Reporter

Reporting on Results

Publish

BBA

WindowsRun-Time

Browser Based AuthoringQuestion Manager

Assessment Manager

13

Fits the MeadWestvaco enterprise Perception is AICC compliant

offers integration with leading Learning Management Systems (LMS)

Aggregates assessment results into a single database to demonstrate employee progress

Linked/AICC systems include: WBT Manager, Docent, Registrar, Tecnikal,

SABA, Web CT, Thinq, Fretwell Downing Le

14

Fits the enterprise (cont’d) Perception questions are IMS compatible IMS is a global coalition of around 250

technology vendors/institutions XML-based standards enabling learning

providers to re-use training resources among different online learning systems Questions are exportable and “future-proof” Opportunity to incorporate additional

assessments into your learning solutions