class 13 chapter 9 evaluation of training
TRANSCRIPT
![Page 1: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/1.jpg)
Class 13
Chapter 9
Evaluation of Training
![Page 2: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/2.jpg)
Evaluation Phase
Evaluation
Objectives
Organizational
Constraints
Design
Issues
Evaluation
Strategy and
Design
Process
Measures
Outcome
Measures
• Reaction
• Learning
• Behavior
• Results
Input Process Output
![Page 3: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/3.jpg)
Resistance to Training Evaluation
Argument Counterargument
There is nothing to evaluate. Evaluation is simply measuring
the degree to which training
objectives are achieved.
No really cares. Organization decision-makers
utilize training outcome data in
budgeting and staffing.
Evaluation is a threat to my
job.
Managers use evaluation as a tool
for improving the likelihood of a
successful intervention, not as a
pass/fail grade.
![Page 4: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/4.jpg)
Evaluation = 2 Types
Process Evaluation
Outcome Evaluation
![Page 5: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/5.jpg)
Purpose of Process Evaluation
Identify areas where training process
can be improved
Improved process leads to improved
outcomes
![Page 6: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/6.jpg)
Types of Questions to Answer in a Process
Analysis (Before Implementation)
Were needs diagnosed correctly?
Were needs translated into training objectives?
Was an evaluation system designed to measure
objectives?
Was the training program designed to meet all the training
objectives?
Are the training methods appropriate for the learning
objectives?
Were the guidelines developed in the design phase used
to develop the training content, materials and processes?
![Page 7: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/7.jpg)
Types of Questions to answer in a Process
Analysis (During Implementation)
Was the trainer and techniques well matched to the
objectives?
Were lecture portions of the training effective?
Was involvement encouraged/solicited?
Were questions used effectively?
Did the trainer conduct the various training methodologies
(case study, role-play, etc.) appropriately?
Was enough time allotted and was it used as intended?
Did trainees follow instructions?
Was there effective debriefing following exercises?
Was time allowed for questions?
![Page 8: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/8.jpg)
Who Is Interested in the Process Data
Training
Department
Trainer: Yes, to determine what works well and what
does not.
Other trainers: Yes, to the extent that process is generalizable
Training manager Only if training is not successful or a problem
is shown for a particular trainer.
Customers
Trainees: No
Trainees’ supervisor No
Upper management: No
![Page 9: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/9.jpg)
Outcome Evaluation
What do you evaluate at the
end of training???
Your Training Objectives
![Page 10: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/10.jpg)
Outcome Evaluation
The KEY outcomes achieved by your training
Trainee Reactions
Learning
Transfer to the job
Organizational Results
The same as your objectives
![Page 11: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/11.jpg)
Outcome Evaluation Design
Alignment Between TNA and Outcomes
Training Evaluation
Objectives Outcomes
Measured the
same way
TNA
![Page 12: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/12.jpg)
Design EvaluationDetermines
Develop
Training
Objectives
Evaluation
Plan and
Design
Design Phase Activities Evaluation
Phase
Reactions
Learning
Performance
• Person
• Unit
• Org.
![Page 13: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/13.jpg)
Reaction Questionnaires
Training reaction questionnaires do
not assess learning,
but DO assess the trainee’s
attitudes about and perception of
the training.
![Page 14: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/14.jpg)
Reaction Questionnaires, continued
At a minimum, the following categories
should be assessed in a reaction questionnaire:
Relevance/value of training (to trainees)
Training content and materials
Trainer’s behavior/effectiveness
Facilities, activities and procedures
![Page 15: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/15.jpg)
Reaction Questionnaires, continued
It should also provide an
overall evaluation of the training experience.
![Page 16: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/16.jpg)
Steps to Consider in Developing a
Reaction Questionnaire – Part 1 of 2
1. Determine what you want to find out (consider training
objectives).
2. Develop a written set of questions to obtain the information.
3. Develop a scale to quantify respondents’ data.
4. Make forms anonymous so participants will feel free to respond
honestly.
5. Ask for subgroup information (e.g., young vs. old; minority vs.
non-minority).
• This could be valuable in determining effectiveness of
training for different cultures, for example, which may be
lost in an overall assessment.
![Page 17: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/17.jpg)
Steps to Consider in Developing a
Reaction Questionnaire – Part 2 of 2
6. Allow space for “Additional Comments” in order to allow
participants the opportunity to mention things you might not have
considered.
7. Decide the best time to give the questionnaire to get the
information you want.
a. If right after training, ask someone other than the instructor
to administer and pick up the information.
b. If some time later, develop a mechanism to obtain a high
response rate (e.g., encourage the supervisor to allow
trainees to complete the questionnaire on company time).
![Page 18: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/18.jpg)
Types of Outcomes and Examples of
Factors Affecting those Outcomes
Trainee characteristics
Trainer behaviorReactions
Trainee readiness for the course
Trainee motivation to learn
Design, materials, and content
Learning
Training’s transfer of training effectiveness
Motivational forces in the job setting
Opportunity to apply learned KSAs on the job
Job Behavior
External environment of the organization
Internal environment of the organization
Employee performance, KSAs, and needs
Organizational
Results
![Page 19: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/19.jpg)
Examples of Assessing Objectives
Training objectives are set in the Design
Phase based on “Gaps” identified.
Training objectives are assessed in the
Evaluation Phase
Let’s look at some examples of how this is done for
reaction objectives.
![Page 20: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/20.jpg)
Reaction Objectives - Example
At least 75% of participants will rate the following aspects of training as 4 or higher on a 5 point scale
(with 5 being the highest rating):
1. Quality of the training content
2. Relevance of the training content
3. Effectiveness of each trainer
4. Overall effectiveness
![Page 21: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/21.jpg)
Training Content
Strongly Disagree - Strongly Agree
1 2 3 4 5Rating of
4 or 5Ave %
Quality 0% 0% 13% 40% 33% 73%
78%Relevance 0% 0% 13% 53% 33% 86%
Overall
Rating0% 0% 27% 47% 27% 74%
Table of Reaction Results
![Page 22: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/22.jpg)
Ratings 1 = Low 5 = High
Trainer 1 2 3 4 5Rating of
4 or 5
Average
Rating
A 0% 0% 27% 41% 32% 73% 4.05
B 0% 13% 19% 36% 32% 68% 3.87
C 0% 0% 0% 40% 60% 100% 4.60
Trainer Overall Effectiveness
![Page 23: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/23.jpg)
Financial Evaluation
Actual savings based on outcomes
Cost of performance prior to training
- Cost of performance after training
= Performance savings
- Cost of Training
= Cost Savings
Return on Investment Ratio (ROI)
= Cost Savings / Cost of Training
![Page 24: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/24.jpg)
Cost Savings for
Grievance Reduction Training
Pretraining Post training
Management time
10 hours/grievance
10 hrs. X 63 grievances = 630
hrs.
10 hrs. X 8 grievances = 80
hrs.
Union Rep’s time
7.5 hrs/grievance
7.5 X 63 grievances = 472 ½
hrs 7.5 X 8 grievances = 60 hrs.
Total Cost
Management time
Union rep’s time
Total
630 hrs X $50 = $31,500.00
472 ½ hrs X $.= $11,812.50
$43,312.50
80 hrs. X $50 = $4,000.00
60 hrs. X $25 = $1,500.00
$5,500.00
Reduction in cost of grievances: 43,312.50 – 5,500.00 = $37,812.50
Cost of training: -32,430.00
Cost saving for the first year $ 5,382.50
![Page 25: Class 13 Chapter 9 Evaluation of Training](https://reader033.vdocuments.mx/reader033/viewer/2022061100/629a71c865046d347041caf9/html5/thumbnails/25.jpg)
Who Is Interested in the Outcome Data
Outcome Data
Reaction Learning Behavior Results
Training Department
Trainer Yes Yes Yes No
Other Trainers Perhaps Perhaps Perhaps No
Training Manager Yes Yes Yes Yes
Customers
Trainees Yes Yes Yes Perhaps
Trainees’ Supervisor Not really Only if no
transfer
Yes Yes
Management No No Perhaps Yes