metrics that matter learning analytics the standard in learning measurement
Post on 15-Jan-2016
219 views
TRANSCRIPT
Metrics That Matter Learning Analytics
The standard inLearning Measurement
©2003 KnowledgeAdvisors. All rights reserved.
Metrics That MatterLearning Analytics
Web-enabled automated learning evaluation process Post-class, follow-up and bespoke surveys Detail, summary and management reporting Internal and external benchmark comparisons Analysis tools Gives a better insight into training’s Key Performance Indicators Demonstrate the business value of training Improve performance of the enterprise Better use of training resources and budgets Increase support from across the enterprise for training
©2003 KnowledgeAdvisors. All rights reserved.
MethodKirkpatrick methodology and Phillips ROI layer… model shows an IIP ‘Level 0’ layer
Level 1
Level 2
Level 3
Level 4
Level 5
Did they like it?Course content, instructors, course materials venue, LMS experience, etc
Did they learn?Test knowledge, intentions, relation to own situation, application
Do they use it?Time to job impact, level of job impact, barriers to use
What were the results?Sales, services, quality, cycle time, errors,timescale, etc
What’s the ROI?Financial return, timescale, value, human capital value
Needed by managers > > >
pro
vid
ed t
o m
anag
ers
> >
>
Level 0
Learning objectivesManager completes with delegate
©2003 KnowledgeAdvisors. All rights reserved.
Automated process
Data Collection Data Storage Data Reporting
Post Event SurveyFollow Up SurveyManager SurveyInstructor SurveyScored TestingNeeds AssessmentsPhillips ROI Input TemplatesPaper, web, emailLMS Integration
ASP solutionSQL Server Centralized storageSecure environmentRaw data access
Web-enabled reportsPush email alertsPhillips ROI reportCustom queryCustom reportsAnalytical toolsForecast calculator
Data Processing
Sophisticated queriesOLAP technologyMultiple aggregationsMultiple cuts / views
Because data collection, processing and reporting is automated, you have more time to analyze the data and
turn it into actionable business intelligence
Automated
Valued time
©2003 KnowledgeAdvisors. All rights reserved.
Post event learner survey - corporate providers
©2003 KnowledgeAdvisors. All rights reserved.
Follow up survey - learner
©2003 KnowledgeAdvisors. All rights reserved.
Follow up survey - manager
©2003 KnowledgeAdvisors. All rights reserved.
Comparative performance/standard reports
Client A
Client B
Client C
Client D
Client E
©2003 KnowledgeAdvisors. All rights reserved.
Performance to goals reports
Allows you to establish pre-set goals for each question category on the Post event and Follow up surveys and compare them to actual performance.
©2003 KnowledgeAdvisors. All rights reserved.
Variance reports
©2003 KnowledgeAdvisors. All rights reserved.
Learning levels score card – K1-5
©2003 KnowledgeAdvisors. All rights reserved.
Top/bottom report
Wade MediaTonnell GrpJKT North
Paris TextileQamoliDent Morcam
Customer
NottinghamAberdeenChester
LondonNorwichLiverpool
©2003 KnowledgeAdvisors. All rights reserved.
When is the time to job impact? This lets you know if and when training was applied to the job. It is collected months after the training, automatically by Metrics that Matter. It helps the organization understand when training effects the job, if at all.
How much more productive are employees? You can track specific business results, such as productivity. Asking a manager or a participant to estimate productivity once time has passed on the job is a good indicator of learning’s impact to a specific business objective.
What percent of skills were applied to the job? This indicator provides insight into the amount of new knowledge/skills picked up in the training for each course (or curriculum or program) that the participant directly applied when they went back on the job as the data is collected several months after the training when the participant is back on the job.
©2003 KnowledgeAdvisors. All rights reserved.
How is training aligned with business objectives?
©2003 KnowledgeAdvisors. All rights reserved.
ROI Score cardThe scorecard is a great management tool to showcase value to key stakeholders. This report shows you each level of learning and the Key Performance Indicators within each level.
Level five shows quantitative ROI metrics including:
Benefit to Cost RatioPayback PeriodROI%
©2003 KnowledgeAdvisors. All rights reserved.
Push Email Reporting
Allows you to setup reports to be sent to you via email at anytime you desire so you do not have to go into MTM to retrieve your favorite reports.
©2003 KnowledgeAdvisors. All rights reserved.
Phillips ROI Toolset in MTM
A summary of the ROI Methodology to use as a reading reference
Examples of case studies applying the Phillips ROI Methodology
A list of selected publications of Jack and Patti Phillips
The authorized Phillips End of Program Data collection instrument
The authorized Phillips Follow Up Data collection instrument
The authorized Phillips Cost Calculation Template
The authorized Phillips Follow Up Data collection instrument
KnowledgeAdvisors will be working with Jack and Patti Phillips to continue adding additional tools with each quarterly update to the Metrics that Matter software!
©2003 KnowledgeAdvisors. All rights reserved.
Is training aligned with business results?Survey
©2003 KnowledgeAdvisors. All rights reserved.
Phillips ROI Toolset in MTM
©2003 KnowledgeAdvisors. All rights reserved.
Sample clients
©2003 KnowledgeAdvisors. All rights reserved.
Contact Information
For further information on Metrics that Matter please contact:
New Wave Learning
4a Flemming Court, Glasshoughton
Castleford, West Yorkshire.WF10 5HW
Tel: 01977 628980
Email: [email protected]
Website: www.newwavelearning.co.uk