2011 distinguished evaluator: the health foundation of greater cincinnati
DESCRIPTION
Dr. Ann McCracken's presentation from the Distinguished Evaluator workshop at the Health Foundation of Greater Cincinnati 6/14/2011TRANSCRIPT
![Page 1: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/1.jpg)
Distinguished Evaluator SeriesBuilding Evaluation Capacity
June 14, 2011
![Page 2: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/2.jpg)
An Invitation to tell it Like it is
![Page 3: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/3.jpg)
Evaluation
HOOP? HELP?
NOT-FOR PROFIT
![Page 4: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/4.jpg)
In what ways is evaluation a hoop to jump through for not-for-profits?
![Page 5: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/5.jpg)
In what ways is evaluation a help for not-for-profits?
![Page 6: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/6.jpg)
How did HFGC evaluation get from 1998 to 2011?
![Page 7: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/7.jpg)
We had some lofty evaluation thoughts
Data Driven Decision Making
Empowering
Funder and NFP on the same page
Increase Sustainability
![Page 8: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/8.jpg)
How Do For-Profits and Not-For-Profits Differ?
For-Profits Not-For-Profits
Cost Center
Care Center
Care Center
Cost Center
![Page 9: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/9.jpg)
How are Research and Evaluation Designs Different?
Basic Research
Applied Research
Evidence Based -Practice
Evaluation
![Page 10: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/10.jpg)
Who should do the evaluating?
• External Evaluator• Internal Evaluator (build internal capabilities)
![Page 11: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/11.jpg)
Lots of Reading on Evaluation
State-of the Art
![Page 12: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/12.jpg)
Learned from Grantees
![Page 13: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/13.jpg)
KaZammmm
![Page 14: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/14.jpg)
• Logic Model• Process Evaluations (What you do)– Formative Stuff
• Outcome Evaluations (What happens because you do it)– Summative Stuff
![Page 15: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/15.jpg)
Logic Model (Logical Framework)
• Activities and Outcomes Approach • Project theory– Story Approach– Story of why and how the project works– Demonstrates connections between program
science, the program operations, and expected outcomes
![Page 16: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/16.jpg)
![Page 17: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/17.jpg)
What We Sort of Knew and Where We Didn’t Know Much
Problem
Intervention
Outcomes
![Page 18: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/18.jpg)
Problem
Intervention
Outcomes
Goal
Objectives•Infrastructure•Implementation•Sustainability
![Page 19: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/19.jpg)
Created a mutually agreeable evaluation plan
• HFGC created the boxes (evaluation forms) • Grantees filled boxes in a way that worked for
them• HFGC shared the wisdom learned over time
from successful and unsuccessful projects• Evaluation plan part of mutual expectations in
contract
![Page 20: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/20.jpg)
Evaluation Use
• Grantees asked to review plan quarterly• Make changes if needed• Review next quarter• Get assistance from Program Officer if needed• HFGC learned that groups of similar grantees
helped one another established grantee groups
• Grantees provided input and plan simplified
![Page 21: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/21.jpg)
We Learned that to truly be useful that grantmaking was more than $$$
• Workshops– Evaluation– Communications– Business Plan– Many others
• Policy and advocacy work• Individual and Group consultation• Technical Assistance
![Page 22: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/22.jpg)
We evaluated our approaches to evaluation
• Grantee Perception Study• Sustainability Study• Workshop Evaluations
![Page 23: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/23.jpg)
Revisit Use of Evaluation (2011-????)
![Page 24: 2011 Distinguished Evaluator: The Health Foundation of Greater Cincinnati](https://reader033.vdocuments.mx/reader033/viewer/2022061221/54bd62914a79594a758b4577/html5/thumbnails/24.jpg)
So for the rest of the day we will
• Look at how others around the country are evaluating (PBS film)
• Reflect on evaluation in our organizations• Hear a panel of experts and the wisdom of the
audience