training evaluation
DESCRIPTION
TRANSCRIPT
Training Evaluation
Karishma DhagePGDM – C
Roll No - 131
Training Evaluation
“Any attempt to obtain information (feedback) on the effects of training program and to assess the value of training in the light of that information for improving
further training” – By Hamblin (1970)
Need for Evaluation ?
• Cost Benefit Returns from Training Investment (Major stakeholders like Training Department, Senior Manager, Sponsors, Clients, Trainees)
• Enable Improvements in the assessment of Training Needs (check suitability & feasibility of the objectives set for training)
• Self-correcting feedback (evaluation will help decide area of improvement)
• Feedback on the performance of the Trainees (aims at improving knowledge & attitude of the trainee)
Trading Cycle
Identification of Training Needs
Develop Training Objectives
Measure Training Results
Conduct Training Programme
Design Training Curriculum
Design/Select Training Methods
Design Training Methodology
Models of Evaluation
• Hamblin Model of Evaluation
• Kirkpatrick’s Design of Evaluation
• Warr’s Framework of Evaluation
• Virmani & Premil’a Model of Evaluation
• Peter Bramely’s Model of Evaluation
Hamblin Model of Evaluation
• We can evaluate at any of the 4 levels namely: Reaction, Learning, Job Behavior & Functioning
• Objectives of the training should be set at each of the 4 levels
• According to Hamblin, to be effective, any scheme of evaluation has to be tailor made to fit the needs of the users of training
• These approaches are possible for external training as well as in company training
Approaches to Evaluating External & In–Company Training
Ultimate Value Approach
Trainee Centered Approach
Trainee Centered Approach
Client Centered Approach
Trainee Centered A
Trainee Centered A
Cost Benefit Criteria
Organization Development
criteria
Job Behavior Criteria
Learning Criteria
Reach criteriaTransfer Criteria
In Company Training External Training
Kirkpatricks Design of Evaluation
Warr’s Framework of Evaluation
• Context Evaluation (C) : Obtaining & using information about the current operational context, that is, about individual differences & organisational deficiencies
• Input Evaluation (I) : determining & using facts & opinion about the available human & material training resources in order to choose between alternative training methods
• Process Evaluation (P) : monitoring the training as it is in progress. This involves continuous examination of administrative arrangements & feedback from trainees
• Outcome Evaluation (O) : Measuring the consequences of training
3 levels of outcome evaluation has been distinguished
1. Immediate Outcome : the changes in trainees knowledge, skills & attitudes which can be identified immediately after the completion of training
2. Intermediate Outcome : the changes in trainee’s actual work behavior which result from training assessment, involves monitoring performance on the job
3. Long Term Outcome : the changes in the functioning of part or all of the organization which have resulted from changes in the work behavior initiated through training
Warr’s Framework of Evaluation contd
Vermani & Premila’s Model of Evaluation
3 STAGES
1. Pre-Training Evaluation (Period before training during which trainees have expectations from the course)
2. Context & Input Evaluation ( Teaching & Learning stage)
3. Post-Training Evaluation (the stage after training, where trainee’s are supposed to integrate training with their job performance
Peter Bramely’s Model of Evaluation
• If the intention is to improve performance by changing behavior, some evaluation should be carried out before a learning activity is designed & run
• Knowledge & theory may be necessary but, if behavioral change is expected, some practice of the type of behavior expected will be required
Evaluation Before Designing Learning Event
• Evaluation during a learning activity can be a useful source of information with which we can improve the quality of the event
• A focus on the utility of the learning & action planning for its application in the workplace will reinforce the link between objectives of the event & organizational requirements
Peter Bramely’s Model of Evaluation contd
Evaluation During the Event
• Most difficult stage in the art of evaluation• After the event, analysis would measure changes in
effectiveness at the 1. Organizational Level2. Team Level3. Individual Level• Change in Behavior (to assess whether change has
taken place)• Change in Learning
Evaluation After the Event
Peter Bramely’s Model of Evaluation contd
David Reay’s Approach to Evaluation
STAGE 1 – THE TRAIL PHASE
• Early Tentative phase of development.
1. Development Stage – involves an informal trail of a sample set of materials, exercises or activities
2. Pilot Testing - more refined operation & tells the evaluation whether the training being evaluated is suitable for the target population
STAGE 2 – THE ONGOING PHASE
1. Validation is used to measure the effectiveness of a whole training design
2. Formative evaluation is the monitoring of the effectiveness of different aspects of training on a continuous basis with a view to modification
David Reay’s Approach to Evaluation contd
STAGE 3 – THE FINAL PHASE• Seeks answers to the following questions1. Were the training needs properly identified2. Were the learning objectives relevant3. Were the performance standards identified4. Were the right priorities established5. Did the training meet the objectives set for it6. How did the learners feel about the training7. Were people prepared for training8. Was the training pitched at the right level9. Was the trainee able to transfer the training to do the
job10.Did the training result in some benefit to the
organization
David Reay’s Approach to Evaluation contd
Pre Training Evaluation
IDENTIFICATION OF TRAINING NEEDS• Trainer carries out Training needs Survey• Uses semi structured interviews• Constructs & administers diagnostic questionnaires
EVALUATE PERFORMANCE STANDARDS• Directly linked to training needs• Performance standards identified as KRA’s focus on
Productivity, profit, Achieving deadlines, Machine Down time, coping with external pressure etc
• Method of evaluation would include detailed description of each behavior & method of measuring quality of behavior
EVALUATE TRAINING OBJECTIVES• To assess whether there is a goal congruence
between the trainee’s needs & training objectives
EVALUATE TRAINEE’S PROFILE• Trainee’s level of knowledge, attitude & skill prior
& post training• Not all but some measures of pre training
knowledge, skill & attitudes is desirable to compare them with post training performance
Pre Training Evaluation contd.
INPUT EVALUATION• Committee approach, brainstorming session
would help in situation where feedback on the subject is already available
• Helps to establish the validity of the training design
EVALUATION DURING THE TRAINING• Both evaluation & subsequent modification need
careful handling• Some methods are examined below
Pre Training Evaluation contd.
1. Observation- trainer’s perceptiveness & observation skills cannot be discounted2. Behavior Analysis- relevant for imbibing certain skills & attitudes such as team orientation & attitudes towards learners3. Course Audits- can be conducted at the end of the day or midway through the course4. Session Assessment- done by using semantic differential scale, 3 point or 5 point & trainee is supposed to only tick mark
Pre Training Evaluation contd.
REACTION EVALUATION• Useful for restructuring training programmes, so
long as it is not confused with an evaluation of learning/job behavior
• Learning’s divided into 3 sections
1. Knowledge learning2. Skill learning3. Attitude learning
Post Training Evaluation
EVALUATING THE INDIVIDUAL TRAINER• When responsibilities are grouped under a
common accountability, measurement of performance against accountability becomes the first means of evaluation
• Impact of trainers activities on solving human performance provides means of evaluation
• Elements of the reputation of the trainer includes comments by line manager about the trainer or the results of training
• External reputation is a measure of trainer performance
Evaluating training staff performance
EVALUATING THE TOTAL STAFF1. Accomplishment of departmental objectives 2. Economic accomplishment of the department.
Training should not cost an organization anything. Results of its activities should be worth more than the cost of staffing & running it
3. How effectively the department uses its resources is a third basis of evaluation. One method of measuring this is to have a system for establishing priorities of departmental activities
4. The department’s reputation & the budget request trend can be evaluated
Evaluating training staff performance
•Modify training to meet learner’s needs by altering exercises, changing training methods & reorganizing the sequence of topics•Document any changes you make & explain these changes to the instructional designers•Use the end of course evaluation to generate ideas for making the training better•Keep in mind that most forms of training evaluations are subjective. Hence, it will be important for you to look at the overall picture & watch trends rather than get bogged down with feedback from one source
Evaluation of the effectiveness of Training
•It helps to justify the existence of the training department, showing the unique value added by the training function to the organization•Second reason is to make a “go” or “no go” decision regarding continuation of a training programme in the future•To gather information to make programme improvements•To meet the unique needs of different audiences
Evaluation Reporting Strategy
If in written form it should be organized as follows1. Cover Page: this provides the title of the report, the date,
the company name, and your contact information2. Table of Contents: this provides title of all section & sub
section, with page numbers3. Executive Summary: this is a one to two page summary of
the report that is aimed at busy senior managers who want to skim a short summary & get all the key facts
4. Evaluation Report: provides specific & detailed information about the training & the learner’s performance. It contains a brief background description of the training, purpose of the evaluation, key question answered by the study, any assumptions/limitations on the study, methods used to carry out the evaluation & the results
5. Conclusion & Recommendation: draws conclusion from evaluation & offers recommendation for improving training
Format of the Evaluation Report
Be sure to include in any report such information as:•The needs that the training was intended to meet•The training objectives for the course•Results of end of course reaction surveys, if they were administered•Results of tests taken by learner’s in aggregate or individual form, if tests were administered•Results of on the job follow up studies to assess transfer of training to the job, if they have been conducted•Impact of training outcome study results, if they have been conducted
Contents of the Evaluation Report
•Most difficult evaluating process of all in the review of training•Problem lies in the ability to quantify changes before & after training•It is essential to have evidence of changes in approach, style & adaptability as a result of the learning process•Changes in the affective area of behavior are concerned with experiences of individual persons learning to act in their roles in the concern•A caring company that trains its staff notices the outward & visible signs & is not afraid to comment on non-quantifiable aspects of performance such as motivation, attitudes to work, team building & morale as aresult of its commitment to training
Evaluating attitude change
•During observation information can be difficult to interpret
•During surveys & questionnaires they are more impersonal for participants, & usually not good for getting quotes in participant’s own words
Training Evaluation Challenges
Bibliography
• Enriching Human Capital through T & D by P L Rao
• www.citehr.com
• www. Uwex.edu