2015 WECC Train the Trainer Workshop

1

2

Current Chair of WECC OTS Manager of SMUD Power System Ops Training since 2007

Manager of CAISO Ops Training since 2007

Identify reasons why there is a need to Evaluate Training courses and programs.

Identify Myths and Barriers to conducting Evaluations. Identify Tools that can be used for Evaluations Identify steps in a revised Kirkpatrick model Utilize the Revised Kirkpatrick Steps to develop an

Evaluation plan.

3

Budget Justification Improve Training Design Enhance the Transfer of Learning Satisfy Operational Training Needs Set Priorities for Learning and Development NERC SAT Process

4

Evaluation and measurement are done at the end of training

Measurement only works for skill based programs. It doesn’t

apply to soft skills

Management directed me to implement this training program and hasn’t asked for measurements or evaluation

Measuring the impact of training. Pamela A. Wade 5

#1 Too many Theories and Models

#2 Models are Too Complex

#3 Lack of Understanding of Evaluation

#4 The Search for Statistical Precision

12

6

#5 Evaluation is considered a Post-Program Activity

#6 Failure to see the long-term payoff of evaluation

#7 Lack of support from key stakeholders

12

7

12

#9 Improper use of Evaluation data

#10 Inconsistency

#11 Lack of Standards

#12 Sustainability

8

#8 Evaluations haven’t delivered the data senior managers want.

9

10

Direct Measurement of business goals

Measuring on-the-job behaviors

Measuring Learning

Measuring ‘Did They Like it’

11

12

KIRK PATRICK -

MOTIVATION: What do they need to perceive in order to learn and perform? (Do they see a need for the desired performance?)

LEARNING: What knowledge, skills, and resources do they need in order to perform?

PERFORMANCE: What do the employees have to perform in order to create the desired impact?

RESULTS: What impact (outcome or result) will improve our business?

13

14

Interchange Desk– Curtailments have been problematic for the Operators. Although there is a procedure on curtailment it does

not specifically outline the steps an Operator needs to take to perform the task as these can vary depending on the facilities.

This leads to consistency problems with implementation.

15

Improve the Operator’s accuracy and timeliness when doing curtailments.

Reduce the hours in “After the Fact” accounting making corrections to schedule due to late curtailments and reloads.

16

Recognize a curtailment request.

Perform a curtailment within an allotted timeframe.

Accurately reload a schedule after a curtailment.

17

Utilizing a Job-Aid identifying the Steps required. Utilizing a Curtailment Display screen. Perform a curtailment within an allotted timeframe. Recognize a curtailment request. Accurately reload a schedule after a curtailment How to Verify an Accurate reload.

18

That curtailments are need to be improved. The are an important factor in solving the problem. They are accountable for performing curtailments.

19

Have Management address the class on the problem and importance of the Operator’s role in helping to improve the curtailment process. Email Opening presentation

Explain how the follow process will work after the training.

20

How will This be measured? Instructor Observations of each Operator performing

the task using predetermined criteria using a simulated environment.

Pre/Post Assessments

21

Have each Shift Senior Monitor the Interchange Operator

to ensure . Recognize a curtailment request. Perform a curtailment within an allotted timeframe. Accurately reload a schedule after a curtailment

Operation’s will Monitor the Logs and review Curtailment

for 3 months after course. Training could develop a Difficulty, Importance, and

Frequency survey.

22

Operations can monitor the timeliness of curtailments

and report average timeframes.

After the Fact accounting can track the number of curtailments that need correction.

23

Improve the Operator’s accuracy and timeliness when doing curtailments.

Reduce the hours in “After the Fact” accounting making corrections to schedule due to late curtailments and reloads.

Operations can monitor the timeliness of curtailments and report average timeframes.

After the Fact accounting can track the number of curtailments that need correction.

Recognize a curtailment request.

Perform a curtailment within an allotted timeframe.

Accurately reload a schedule after a curtailment.

Have each Shift Senior Monitor the Interchange Operator to ensure . Recognize a curtailment request. Perform a curtailment within an allotted

timeframe. Accurately reload a schedule after a

curtailment

Operation’s will Monitor the Logs and review Curtailment for 3 months after course.

Training could develop a Difficulty, Importance, and Frequency survey.

24

Utilize a Job-Aid identifying the Steps required.

Utilizing a Curtailment Display screen.

Perform a curtailment within an allotted timeframe.

Recognize a curtailment request.

Accurately reload a schedule after a curtailment

Verify an Accurate reload.

Instructor Observations of each Operator performing the task using predetermined criteria using a simulated environment.

Pre/Post Assessments

That curtailments are need to be improved.

They are an important factor in solving the problem.

They are accountable for performing curtailments.

Have Management address the class on the problem and importance of the Operator’s role in helping to improve the curtailment process. Email Opening presentation

Explain how the follow process will work after the training.

25

Exercise -

Using the Revised Kirkpatrick model answer the question for the training problem described

26

What are the Barriers to conducting meaningful evaluation? 1. Too many Theories and Models 2. Models are Too Complex 3. Lack of Understanding of Evaluation 4. Statistical Precision 5. Evaluation is considered a Post-Program Activity 6. Failure to see the Long-Term Payoff of Evaluation 7. Lack of Support from key Stakeholders 8. Evaluation hasn’t delivered the Data Senior Manager Want. 9. Improper use of Evaluation Data 10. Inconsistency 11. A lack of Standards 12. Sustainability

Make Training Evaluation Work, Jack J Phillips - Patricia Pulliam Phillips 27

28

Identify eight reasons why there is a need to Evaluate Training courses and programs.

Identify Myths and Barriers to conducting Evaluations. Identify Tools that can be used for Evaluations Identify steps in a Revised Kirkpatrick model Utilize the Revised Kirkpatrick Steps to develop an

Evaluation plan.

QUESTIONS?

29

Examples

30

Improve Understanding of Evaluations.

Implement beyond a Level 2 evaluation.

Learners can utilize the reversed Kirkpatrick planning process.

Learners will implement evaluation beyond the level 2.

Level 3 and 4 Evaluation.

Plan an evaluation.

Learner will take a real life training problem and create an evaluation plan.

31

Understand Myth about evaluation.

Recognize Barriers to evaluation.

Understand Kirkpatrick steps.

Understand the reverse model.

Perform a plan using the reverse Model.

Instructor Observations of each learner performing in a group environment discussing and developing an evaluation Plan.

That evaluations goes beyond the classroom.

That there is value in learning about evaluations.

Explain the importance of Evaluation to their program.

32