![Page 1: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/1.jpg)
Measuring the Success of
Organization Development
Lizette Zuniga, Ph.D.Director, ROI Implementation
ROI Institutewww.roiinstitute.net
![Page 2: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/2.jpg)
Goals of this Session
1. Identify best practices in ROI and Human Capital Analytics
2. Using abbreviated organization culture assessment tool, rate the organization culture in which you work
3. Using an organization culture case study, align OD intervention with business
4. Using a case study, apply ROI methodology to measure the impact of an OD intervention
![Page 3: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/3.jpg)
Maximizing Results: Best Practices for ROI and Human Capital Analytics
1-3 Communicating Results
Descriptive AnalyticsDescribes what happened.
Descriptive AnalyticsDescribes what happened.
Predictive AnalyticsWhat could happen?
Predictive AnalyticsWhat could happen?
Prescriptive AnalyticsWhat should happen? What is
the best course of action?
Prescriptive AnalyticsWhat should happen? What is
the best course of action?
Use analytics to justifyactions.
Building upon aspirational level and use data to make decisions.
Uses advanced rigor in dataanalysis to guide futurestrategy
Business Alignment
Analytics Maturity
Aspirational Experienced Transformed
![Page 4: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/4.jpg)
Predict the BEST outcome
By providing the right intervention in the most effective way, building the necessary capability, and by removing barriers or obstacles from their path, business measures are more likely to improve and ultimately impact the bottom line.
1-4 Communicating Results
![Page 5: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/5.jpg)
OD defined as a planned intervention of change for group, team, or organization
to improve organization.
Organization Development (OD)
Examples – change management, team building, 360 feedback, organization
culture assessment and learning.
![Page 6: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/6.jpg)
What is culture?
![Page 7: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/7.jpg)
“Organizational culture is the way
we do things around here.”
- Burke & Litwin, 1995
![Page 8: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/8.jpg)
Culture is the collective values,
assumptions, beliefs and paradigms
that create the expected and
accepted behaviors of the
organization.
![Page 9: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/9.jpg)
Hierarchical vs. Empowered
Risk Avoiders vs. Risk Takers
Reactive vs. Proactive
Sales Driven vs. Profit Driven
Cautious vs. Innovative
For Example…
![Page 10: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/10.jpg)
How does an organization behave?
•How to make decisions
•How to organize resources
•How to supervise people
•and so on...
![Page 11: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/11.jpg)
Organization Culture Assessment
Complete Capstone Organization Culture Abbreviated Profile
Refer to your handout.
![Page 12: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/12.jpg)
Agua Case Study
• Large manufacturing organization• International customer base• Aggressive growth strategy• Needs– error rates, time to fulfill customer
orders, leadership pool, business change, turnover, customer satisfaction
![Page 13: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/13.jpg)
Diagnostics
• Quantitative– *Organization Culture Assessment– *Leadership Awareness Index– Key Performance Metrics
• Qualitative – Interviews
*Results are as strong as your instrument.
![Page 14: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/14.jpg)
Agua Culture Analysis
• Leadership – autocratic
• Planning – unclear priorities
• Decision Making – slow process
![Page 15: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/15.jpg)
Agua Culture Analysis
• Leadership – autocratic
• Planning – unclear priorities
• Decision Making – slow process
![Page 16: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/16.jpg)
Agua OD Interventions
• Formed a Guiding Coalition • Held Master Action Planning Sessions– 3 sub-groups
1. Focused on business needs2. Succession planning3. Communication and change
• Provided Leadership Development – leaders tasked with leading teams in new way
![Page 17: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/17.jpg)
Levels of Evaluation
Measurement Focus Typical Measures
![Page 18: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/18.jpg)
OD InterventionBusiness Alignment and Forecasting The ROI Process Model
Learning Needs
Need to learn new high performing behaviors
Preference Needs
Relevant to their job, time spent in planning sessions was value add
Measurement and Evaluation
Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Participants satisfied with intervention, find it relevant
Learning Objectives
Targets for learning organization behaviors
Application Objectives
Targets for leadership and organization behaviors Performance Needs
Organization behaviors not aligned with high performing
Impact Objectives
Identify targets for each metric
Business Needs
Quality, Costs, Time, Revenue, HR and Customer Metrics
Payoff Needs
investment for OD InterventionROI Objectives
Target ROI5
4
3
2
5
4
3
2
1 1
Initial Analysis
Align OD Intervention Using V-Model
![Page 19: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/19.jpg)
OD InterventionBusiness Alignment and Forecasting The ROI Process Model
Learning Needs
Need to learn new high performing behaviors
Preference Needs
Relevant to their job, time spent in planning sessions was value add
Measurement and Evaluation
Reaction
Survey
Learning
Increase Knowledge/skills item on Course Evaluation
Application
360° Assessment, Master Action Plan
Impact
Pre versus post measures
ROI
Calculate ROI
Reaction Objectives
Participants satisfied with intervention, find it relevant
Learning Objectives
Targets for learning organization behaviors
Application Objectives
Targets for leadership and organization behaviors Performance Needs
Organization behaviors not aligned with high performing
Impact Objectives
Identify targets for each metric
Business Needs
Quality, Costs, Time, Revenue, HR and Customer Metrics
Payoff Needs
investment for OD InterventionROI Objectives
Target ROI5
4
3
2
5
4
3
2
1 1
Initial Analysis
Align OD Intervention Using V-Model
![Page 20: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/20.jpg)
Agua Data Collection Method
MeasureMeasure ToolTool
SatisfactionSatisfaction QuestionnaireQuestionnaire
LearningLearning QuestionnaireQuestionnaire
Behavior ChangeBehavior Change Master Action Plan, 360Master Action Plan, 360°° Feedback Feedback
Business ImpactBusiness Impact Quality, Customer Satisfaction and HR MetricsQuality, Customer Satisfaction and HR Metrics
Return on Investment Return on Investment Monetary benefits and project costsMonetary benefits and project costs
![Page 21: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/21.jpg)
Impact MeasuresLevel 4 Metric Baseline Six Months Later One Year Later
Time to Order 31 18 6
Error Rates 35 per 100 24 per 100 7 per 100
Customer Satisfaction
60% - meet needs45% - committed
N/A 82% - meet needs68% - committed
Turnover 655 299 275
Promotions 5% 8% 18%
![Page 22: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/22.jpg)
Monetary Value for Error Rates
• $750 per error x 28 (change) = $21,000 (monthly)• $21,000 x 12 (months) = $252,000 annual• $252,000 x 92% (estimate %) = $231,840
Isolation estimates collected from Guiding Coalition:1.Estimates of contributing factors2.Confidence in estimate3.Values are then averaged
![Page 23: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/23.jpg)
Monetary Value for Turnover
• $3,200 per turnover x 380 (change) = $1,216,000
• $1,216,000 x 85% estimate = $1,033,600
![Page 24: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/24.jpg)
Combined Monetized Benefits
Measure Annual Change
Unit Value Annual ImprovementValue
Confidence %
Adjustments based on Confidence
Error Rates 336 $750 $252,000 92% $231,840
Turnover 380 $3,200 $1,216,000 85% $1,033,600
![Page 25: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/25.jpg)
Project CostsCost of Item Cost
Diagnostics $12,000
Strategic Planning Sessions $112,000
Leadership Development $550,000
360° Feedback $25,500
Program Materials $22,000
Overhead $1,500
Facilities, food and refreshments $8,000
Communication $5,000
Participant salaries (time involved) $95,000
Evaluation Costs $18,500
TOTAL $849,000
![Page 26: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/26.jpg)
ROI % =
ROI Results ROI Results ROI Results ROI Results
*Net Benefits
Costs*Net = Benefits - Costs
Costs = $849,000Benefits = $1,265,440
x 100
Now it’s
your tu
rn!
Calculat
e the RO
I.
![Page 27: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/27.jpg)
ROI =
ROI Results ROI Results ROI Results ROI Results
$1,265,440 - $849,000
$849,000
49% ROI
![Page 28: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/28.jpg)
AssumptionsAssumptionsAssumptionsAssumptions
• High turnover in tight labor market impacts required hiring numbers
• People leave their jobs because of supervisor• Estimates used in isolating the effects are
credible way to determine contribution from intervention
![Page 29: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/29.jpg)
Conclusions
• Preliminary findings indicate that the intervention appears to be impacting turnover and error rates in favorable way
• Marked decrease in problems with filling customer orders on time
• Customer satisfaction increased• Improvement observed in communication and
problem solving
![Page 30: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/30.jpg)
• Provided with the case study, participants were instructed to brainstorm
![Page 31: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/31.jpg)
Group 1 Smile Sheet
— Why are you here?— Was this helpful / useful?— Are you motivated to take action?— Will you use what you learned?— Rate the importance of your current work?
![Page 32: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/32.jpg)
Group 2 Learning
— Pre-training vs. Post-training scoring Time on (Learning curve)
— Assessment during training Confidence Readiness Competencies
— Understanding key resources— Self knowledge
Knowing when I need help Knowing enough to self correct
![Page 33: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/33.jpg)
Group 3 Leadership Behavior
— Before and after— Decision making process
Time # ppi involved Implementation success?
— Change metrics from cultural assessment— Before and after interviews / survey
Managers and up Managers and down
— GAPS! Map (R&R)
![Page 34: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/34.jpg)
Internal Measures
– Customer Satisfaction• On time orders / deliveries• Reduced returns / re-do’s• Surveys• Referrals
– Quality– Staff morale
• Impact of engagement• How they are engaged
External Measures
– What is most important to the customer?
– What are you willing to do if you are satisfied?
Group 4
![Page 35: OCA Sept 2014: Measuring Organization Development Interventions](https://reader033.vdocuments.mx/reader033/viewer/2022061212/5495490bac79593b2e8b4dcf/html5/thumbnails/35.jpg)
Group 5 R/I
— R = Savings Profit Productivity Personnel Turnover Staff and Customer satisfaction
— Intangibles Goodwill Morale Identity Loyalty
— I = Direct cost – consultant Participation Supplies Facilities Opportunities