measuring the impact of elearning: turning kirkpatrick’s four levels of evaluation upside down
TRANSCRIPT
![Page 1: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/1.jpg)
Hosted by Lambda Solutions and Limestone Learning
Measuring the Impact of eLearningTurning Kirkpatrick’s Four Levels of Evaluation Upside Down
E M P O W E R TA L E N T
![Page 2: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/2.jpg)
Your Hosts
Sean HouganMarketing Coordinator
Paula YunkerManaging Partner &
Instructional Designer
![Page 3: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/3.jpg)
Agenda
1. Lambda Solutions and Limestone Learning
2. Why learning evaluation is important.
3. Kirkpatrick’s four levels of evaluation explained.
4. Aligning learning to organizational goals.
5. Typical challenges implementing evaluation in an organization.
6. Practical strategies for implementing learning evaluation.
7. Favorite learning evaluation resources.
8. Q & A
![Page 4: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/4.jpg)
• Our approach
• Clients:
![Page 5: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/5.jpg)
Totara & Moodle are 80% more cost
effective
Managed cloud hosting reduces
your IT operating costs by more than
40%
Over 12 years of experience and 600 customer
implementations
We Reduce The Cost Of Learning
![Page 6: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/6.jpg)
Industry Leading Clients
![Page 7: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/7.jpg)
Poll #1
What do you hope to gain from this webinar?
![Page 8: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/8.jpg)
Why Learning Evaluation Is Important?
![Page 9: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/9.jpg)
A Little History First…
![Page 10: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/10.jpg)
Kirkpatrick’s Four Levels Of Evaluation
![Page 11: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/11.jpg)
Level 1 - Reaction
• Measures to what
degree learners react
favorably to the
learning event.
• The “smile sheet”.
![Page 12: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/12.jpg)
Poll #2
Yes or no? Do you use level 1 learning evaluation?
![Page 13: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/13.jpg)
Level 2 - Learning
• Measures to what degree learners acquire the
intended knowledge, skills and attitudes based
on their participation in a learning event.
![Page 14: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/14.jpg)
Poll #3
What level 2 learning evaluation do you use in your training courses/programs?
• Pre-evaluation (formative)
• During the learning event (formative)
• At the end of the learning event (summative)
• Level 2 isn’t used.
![Page 15: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/15.jpg)
Level 3 - Behaviour
• Measures to what
degree learners apply
what they learned
during training when
they are back on the
job.
![Page 16: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/16.jpg)
Poll #4
Yes or no? Do you or your organization use level 3
learning evaluation?
![Page 17: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/17.jpg)
Level 4 – Results
• Measures to what
degree targeted
business outcomes
occur as a result of the
learning event(s).
![Page 18: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/18.jpg)
Poll #5
Yes or no? Do you or your organization use level 4
learning evaluation?
![Page 19: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/19.jpg)
Level 5 - ROI
![Page 20: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/20.jpg)
Turning The 4 Levels Upside Down
![Page 21: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/21.jpg)
Aligning Learning To Organizational Goals
![Page 22: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/22.jpg)
Strategies For Aligning Learning
• Step 1: Identify the business need.
• Step 2: Create the learning plan
• Step 3: Measure the business results.
![Page 23: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/23.jpg)
Step 1: Identify The Business Need
Ask your key stakeholders and business partners:
• What desired results do you want to realize?
• What are the goals & business outcomes ?
• What needs to change to meet those goals & business outcomes?
![Page 24: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/24.jpg)
Step 1: Identify The Business Need (cont.)
• What KPIs should change?
• What needs to be learned to make those changes happen?
• What will success look like to you?
![Page 25: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/25.jpg)
Step 2: Create The Learning Plan
Develop a learning and evaluation plan to meet the learning requirements based on step 1 answers:
• What needs to be learned to achieve business outcomes?
• What required KPAs are needed?
• What learning experiences and practice are needed to become proficient?
![Page 26: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/26.jpg)
Step 2: Create The Learning Plan (cont.)
• When and how will these learning experiences and practice be best accomplished?
• How and when will the learning be measured?
• How will learning be applied to the job and how will that application be reinforced on the job?
![Page 27: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/27.jpg)
Step 3: Measure The Business Results
To identify how learning aligns to business outcomes and meets stakeholder/business partners needs:
• What changes in business results & key performance indicators have been realized from the training program?
![Page 28: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/28.jpg)
Step 3: Measure The Business Results (cont.)
• For every dollar invested in training how many dollars does the organization get in return? Is the ROI significant enough to justify training?
• What key intangible benefits (soft data) have been gained?
![Page 29: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/29.jpg)
Poll #6
What are some examples of soft data?
![Page 30: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/30.jpg)
Challenges Implementing Evaluation
![Page 31: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/31.jpg)
Typical Challenges
• Lack of buy-in from the top.
• Ownership.
• Unable/unwilling to measure and track evaluation on the job.
• Lack of reinforcement on the job.
![Page 32: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/32.jpg)
Strategies For Implementing Evaluation
![Page 33: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/33.jpg)
Level 4 Implementation Strategies
• Identify what training has the great impact.
• Determine how important the need is to demonstrate value.
• Involve stakeholders/ business partners
• Determine who is responsible?
• Start gathering data early.
![Page 34: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/34.jpg)
Level 3 Implementation Strategies
• Design relevant follow-up questions.
• Administer 3 months post-training event individually, in groups, by survey.
• Ask similar questions to others to get a fuller “picture”.
• Create a Learning Action Plan.
• Learning contract between the learner and the supervisor/manager.
![Page 35: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/35.jpg)
• Create pre-evaluation that aligns to learning content/objectives.
• Design a variety of informal formative evaluation tools; at least one per lesson/objective.
• Design an end-of-course summative evaluation; use a checklist or rating scale to evaluate learning.
Level 2 Implementation Strategies
![Page 36: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/36.jpg)
• Go beyond a “smile sheet”.
• Focus questions on the learning experience.
• Make evaluation statements learner-centered.
• Create questions that align with higher evaluation levels.
• Consider using a 7, 9 or 11 point rating scale; only label each end of the scale.
Level 1 Implementation Strategies
![Page 37: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/37.jpg)
Favorite Learning Evaluation Resources
• Evaluation websites.
• Books/reference material.
• Kirkpatrick Evaluation LinkedIn group.
• Training and certification.
• Industry and best practice research sites.
• Limestone Learning blog.
![Page 38: Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evaluation Upside Down](https://reader030.vdocuments.mx/reader030/viewer/2022032616/55a605d01a28abfb248b45f3/html5/thumbnails/38.jpg)
Questions/Comments
Paula YunkerManaging Partner
Sean HouganMarketing Coordinator