Evaluation BasicsPrinciples of Evaluation

Keeping in mind the basic principles for program and evaluation success, leaders of youth programs can begin learning and using evaluation practices…

Evaluation BasicsPractices of Evaluation

Practices of evaluation include tools and procedures that help leaders…

implement evaluation processesaccomplish evaluation purposesrecognize evaluation principles

Practices are the “nuts and bolts” of doing evaluation

Evaluation BasicsPractices of Evaluation

Sample practices for each process step include…

Planning: SMART ObjectivesFocusing: Evaluability/Standards ChecklistDesign: TimetableMethods: Survey Template, Observation rubricData Collection: Excel Spreadsheet 1Analysis and Interpretation: Excel Spreadsheet

2Communicating Results: Reporting Template

Evaluation BasicsPractices of Evaluation

Planning: The SMART acronym describes steps to turn a general goal into a detailed roadmap for change. SMART objectives are…

Specific about the type, amount, timing of change

Measurable in setting criteria for observing change

Achievable, recognizing program limits, possibilities

Relevant to larger goals and purposesTime-bound in the program scope and sequence

Evaluation BasicsPractices of Evaluation

Focusing: An Evaluability Checklist helps leaders decide whether and how to evaluate their program. Two key evaluation processes are…

Formative, assessing the operation of the program

Summative, determining the results of the program

Evaluation BasicsPractices of Evaluation

Focusing: Formative Evaluability Indicators describe elements to check to see if the program is on track. Key elements of program operation include…

Participants, assessing the operation of the program

Staff determining the results of the programResources, materials and equipment

identified, if not procuredEvaluation, plan and procedures in place

Evaluation BasicsPractices of Evaluation

Focusing: Evaluation Standards help program leaders determine evaluability, or when the program is ready to evaluate. Evaluation Standards include…

Utility, usefulness of the evaluation to stakeholders

Feasibility, potential for effectiveness and efficiency

Propriety, potential for fair and ethical assessment

Accuracy, dependability of findings for a setting

Evaluation BasicsPractices of Evaluation

Designing: A Timeline enables program leaders to plan and manage each stage of evaluation. Key elements of program operation include…

Time 1 Time 2 Time 3 Time 4

Task 1

Task 2

Task 3

Evaluation BasicsPractices of Evaluation

Selecting Methods: Adaptable data collection tools that , enable program leaders to gather fixed-choice and open-ended data from program participants and/or observers include…

Survey, which asks questions about program objectives and experiences; and

Observation Rubric, which provides a framework for judging conditions or performance

Evaluation BasicsPractices of Evaluation

Data Collection: Simple spreadsheets provide excellent ways to organize and store data as follows:

Quantitative (numerical) data typically documents participant data by row, each question category by column; and

Qualitative (verbal) comments can be entered one-per-row, usually without respondent characteristics

Evaluation BasicsPractices of Evaluation

Analysis and Interpretation: Most spreadsheets can be used to make simple counts or comparisons, or to organize responses by category:

Quantitative (numerical) data can be added or averaged by column, with items totaled by frequency (count) or percentage (ratio); and

Qualitative (verbal) comments can be grouped by theme, set either before or after data collection

Evaluation BasicsPractices of Evaluation

Communicating Results: Although different stakeholders or persons interested in a program, appreciate different descriptions, all persons benefit from understanding:

Summary Information: program title, purpose, audience, timing and setting; evaluation findings simply stated, followed by conclusions, implications, and credits for ideas and funding.

Evaluation BasicsPerformance in Evaluation Glossary

Evaluation as PYDEvaluationProgram

Development Taking Stock Implementing Evaluating

(Valued) Outcomes(Valued) ProcessesOrganizational

MissionAssessed Needs

YD Outcomes: Caring, Competence, Confidence, Contributing, Character

Eval. Process Steps Planning Focusing Designing Selecting Methods Data Collecting Analyzing & Interp. Communic. Results

Evaluation BasicsPerformance in Evaluation Glossary

Evaluation Purposes Show accountability Grow capacity Know how to improve

programs Bestow relationships

Evaluation BasicsPerformance in Evaluation Glossary

Evaluation Principles

Develop programs with strength and focus

Intensity Frequency Duration Evidence-based Research-based Program fidelity Readiness for change Stages of change

Develop programs for long-term impact

Programs activities

Develop & evaluate in stages (maturation)

Start-up/pilot Functioning/

Established Mature Formative evaluation Summative

evaluation

Evaluation BasicsPerformance in Evaluation Glossary

Evaluation PracticesPlanning/SMART objFocusing/EvaluabilityDesign/TimetableMethods/Survey &

Observation Rubric Fixed choice Open-ended

Data Collection/Spreadsheet

Quantitative Qualitative

Analysis & Interpretation/Spreadsheet

Communicating Results/Template:

program title, purpose, audience, timing and setting; evaluation findings simply stated, followed by conclusions, implications, and credits for ideas and funding.