Click icon to add picture

QUALITY ASSURING TECHNICAL -VOCATIONAL EDUCATION & TRAINING

DEMONSTRATING RETURN ON INVESTMENT (ROI)

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

PRESENTATION OVERVIEW

Definitions

Key Features

Globalization

History of ROI

Calculating ROI

Indicators of ROI

ROI in TVET

Other ROI Indicators

Evaluation Methods

Benefits of ROI

Summary

Conclusion

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

PRINCIPAL OBJECTIVES

• Greater Awareness of ROI in TVET• ROI as strategic tool for HCD• ROI Influences resource allocation• Measures & Indicators of ROI • Evaluation Methodologies of ROI• Benefits of Evaluating ROI

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHAT IS QUALITY ASSURANCE? “Programme for the Systematic

Monitoring and Evaluation of the various aspects of a Project, Service, or Facility to Ensure that Standards of Quality are met”

Merriam – Webster on-line Dictionary

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHAT IS QUALITY ASSURANCE ? “Part of Quality Management,

providing Confidence that Quality requirements (needed or expectations that are stated, generally implied, or obligatory) will be Fulfilled”

International Standards Organization (ISO)

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHY QUALITY ASSURE ?

• Build & Maintain Organizational Brand, Integrity and Reputation

• Minimize mismatch & LM distortions

• Create Competitive Advantage• Provide Superior Utility Value • Gain/retain Customer Affirmation

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

KEY ELEMENTS OF QA SYSTEM• Planning – goal setting • Implementation – guiding SOPs • Evaluation – monitoring

mechanism• Review – analyse/compare results (EQAVET)

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

GLOBALIZATION & TVET

Globalization defines 21st century econ. Skills = currency of LM transactions, certification => instrument of trade, to navigate LM. Productivity determined by competence of employees as well as standardization of production systems.

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

GLOBALIZATION & TVET

• In final analysis, businesses compete for markets in the international arena

• In this environment (NWO), labour becomes internationalized through standardization and portability

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

GLOBALIZATION & TVET

Against this background, modern and efficient TVET Quality Management System becomes an imperative, especially within context of Resource Constraints, Programme Relevance and International Competitiveness

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

QUALITY ASSURING TVET

Competitive Forces Shape Strategy:• Structures, Systems & Processes • Facilities, Standards & Procedures• Monitoring & Evaluation • Technology - Leadership & People • Value Proposition

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

TRAINING & ORG PERFORMANCE Why invest in TVET? Correlation

training performance?• Body of literature that links training

with organizational performance• ILO links skills development (TVET)

with organizational productivity

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

TRAINING & ORG PERFORMANCE• Through training HR is

transformed into HC => greater value-added

• Firms invest in HR, create distinctive competence and build org capacity - even without knowing the ROI

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

MEASURING ROI

Calculating ROI ranges from a simple ratio: [Benefits/Costs] x 100 = %ROI

To more complex means such as; Pay Back - months/years before benefits match costs incurred

Note: shorter the pay back period, more feasible the investment

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

MEASURING ROI

• Sophisticated models - Computer applications, where I/O variables are processed and results generated

• ROI when indirect variables are less quantifiable => challenging exercise

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

INDICATORS OF ROI

(1) Productivity Improvement:• Improved work methods =>

reduced human effort• Higher skill levels => faster work,

reduced Cycle Time• Greater intrinsic motivation =>

improved morale, greater output

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

INDICATORS OF ROI

(2) Labour Savings:• Less duplication of effort• Fewer mistakes => less rework =>

higher quality => greater productivity• Faster access to information• Critical differentiator, provides

competitive advantage

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

INDICATORS OF ROI

(3) Other Cost Savings:• Fewer machine breakdowns =>

lower maintenance costs => greater profit

• Lower Staff T/O => lower HR cost• Fewer customer complaints, greater

loyalty => higher retention rate

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EVALUATING ROI IN TVET

• ROI in TVET means; measuring all economic returns generated from investing in training

• Comparing results with the true cost of training programme

• Determine average annual ROI

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHY MEASURE ROI IN TVET ?

• Evaluating ROI is not an audit of training, neither cost saving

• Rather an objective evaluation and re-engineering of training programmes

• Tool to ensure that future training is targeted and effective

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHY MEASURE ROI IN TVET?

Rationalization of Scarce Resourc

es

Provides Scientifi

c Evidenc

e of Outcom

es

Facilitates Career Advance

ment

Facilitates Comparative Analysis

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

WHY MEASURE ROI IN TVET?

• Like other investments, QA TVET consumes resources (cost), it also produces benefits

• Reasonable rate of return (for risk) is therefore an expectation

• Probably more talked about, than actually done!

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

HISTORY OF ROI IN TRAINING

Kirkpatrick’s 4 - level model1. Reaction – the degree of favourable

reaction to training intervention2. Learning – degree to which intended

competence (KSA) is developed after training intervention

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

HISTORY OF ROI IN TRAINING

3. Behaviour – degree of application,

post training intervention4. Results – degree to which

targeted outcomes are achieved (ROI)

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EQAVET INDICATORS OF ROI

• Programme Participation Rate• Completion Rate (Certification)• Employment Rate (skills utilization)• Productivity Improvement• Unemployment Rate (decline)• Organizational Competitiveness

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

OTHER INDICATORS OF ROI

Individual level:

• Enhanced Reflective Capacity• Critical Thinking & Analytical Skills• Problem Solving Capability• Continuous Improvement• Autonomous & Accountability

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EVALUATION METHODOLOGIES

• Several methods of evaluating ROI

• Method used depends on objectives of ROI evaluation, examples:

• Control Group, Modelling, and Estimating

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

CONTROL GROUPS

• High validity in terms of isolating effects of training programme

• One group participates, other does not (similar in all other respects)

• Random selection techniques• Disadvantage - some staff do not get

benefit of the training programme

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

MODELLING

• Analytical & mathematical, non- linear relationships

• Several variables influence results, calls for sophisticated stats models

• Appropriate data & models available, => accurate predictor of performance variables

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

ESTIMATION

• Participants state how much of their improvement due to training prog.

• Adjusted for level of confidence (60% x 70% con = 42%)

• Validate – supervisors & peers asked

to weight estimates = > 360 -degree

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EVALUATION CRITERIA

• Evaluation criteria classified as either hard or soft data

• Hard: quantitative - easily convertible into monetary terms

• Soft: qualitative - from HR training database, not easily convertible $$

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

EXAMPLE - EVALUATION CRITERIA H A R D D A T A

Production output

Units sold

Products rejected

Scrap produced

Labour productivity

Number of accidents

S O F T D A T A

Work ethics

Absenteeism

Work climate

Social Tension

Work attitude

Staff development

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

BENEFITS OF EVALUATING ROI

• Validates training as a business tool for improving performance & profit

• Justifies training costs incurred by HR• Improvement in training prog design• Rational selection of training delivery

methodologies

SUMMARY & CONCLUSION

• Globalization drives International Competitiveness

• TVET’S Relevance linked to QA• Demonstrating ROI Strategic Tool• Comparative Analysis • Capacity Building & HCD

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

W. C . F L E TC H E R : N O V 3 & 4 , 2 0 1 3

ACKNOWLEDGEMENTS

• European Quality Assurance in Vocational Education & Training (EQAVET)

• International Labour Organization (ILO)

• Naish, Richard: ROI – Building Business case for people development

• Kirkpatrick, Donald: Evaluating Training Prog.

THANKS FOR YOUR ATTENTION !!