program evaluation handouts
DESCRIPTION
Program Evaluation Material. Incorporated into the City of Chicago's Intergovernmental Executive Development Program.TRANSCRIPT
![Page 1: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/1.jpg)
Program Evaluation
George Vukotich, Ph.D. Copyright © 2004
Intergovernmental Executive Development Program
![Page 2: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/2.jpg)
Program EvaluationMeasuring Success
It all begins with setting goals and knowing where you are going.
![Page 3: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/3.jpg)
SMART GOALS
Specific
Measurable
Agreed On
Realistic
Timed
![Page 4: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/4.jpg)
Relevant Measurement
Activity vs. Productivity
Effectiveness vs. Efficiency
Direct vs. Indirect
Are we measuring the right thing.
![Page 5: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/5.jpg)
Price / Quality / Innovation
What is your organization known for.
What is your approach to providing services.
![Page 6: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/6.jpg)
What to Evaluatee.g. Evaluating IEDP
The course material
The instructors
The facility
Quality of interaction with others
What was learned
![Page 7: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/7.jpg)
Evaluation ProcessADDIE Model
Analyze
Design
Develop
Implement
Evaluate
![Page 8: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/8.jpg)
Evaluate to What Standard
Benchmarking
Based on pre-set goals
Compare to other similar projects Similar industries, organizations,
products
Past / Historical standards
![Page 9: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/9.jpg)
How to evaluate
Forms Interviews Focus Groups Self Analysis Grades Supervisors Review Work Productivity
![Page 10: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/10.jpg)
Degree of Success
To what extent does something
need to be effective to.
![Page 11: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/11.jpg)
Formative vs. Summative
Conducting the evaluation during the process and making adjustment along the way.
Waiting until the process is complete and evaluating the results.
![Page 12: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/12.jpg)
Test – Retest Methodology
![Page 13: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/13.jpg)
Sampling
![Page 14: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/14.jpg)
Data – Not just getting it, but using it.
![Page 15: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/15.jpg)
Reliability and Validity
VALIDITY:
Is what we are evaluating the right thing to evaluate to help us be successful.
RELIABILITY:Will we get consistent measures over time.
![Page 16: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/16.jpg)
Kirkpatrick’s 4 - Levels
Level 1 – Reaction
Level 2 – Learning
Level 3 – Behavior
Level 4 -- Results
![Page 17: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/17.jpg)
Balanced Scorecard
![Page 18: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/18.jpg)
Cost / Benefit Analysis
Employee job satisfaction
Students completing a program
Reduction in processing errors
Customer service ratings
![Page 19: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/19.jpg)
Unintended Consequences
What is the cost of making an error in our evaluation.
![Page 20: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/20.jpg)
♦ Turnover
♦ Time to Hire, Cost to Hire
♦ Reduced Processing Costs
♦ Improved Safety
♦ Reduced Insurance Costs
CORRELATIONS
Return on Investment (ROI)
![Page 21: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/21.jpg)
Theory into practice
How can your team use “program evaluation” to help you be more successful.
![Page 22: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/22.jpg)
![Page 23: Program Evaluation Handouts](https://reader033.vdocuments.mx/reader033/viewer/2022051611/54b833454a79598b178b4655/html5/thumbnails/23.jpg)