1
Chapter 3 - SAEOPP Priority 1 TrainingChapter 3 - SAEOPP Priority 1 Training
SHOW & TELL:SHOW & TELL:
TRIO PROJECT EVALUATIONTRIO PROJECT EVALUATION
Presenter: Elizabeth C. RetamozoPresenter: Elizabeth C. Retamozo
2
PresenterPresenter
Elizabeth C. Retamozo
Executive Director NOSOTROS Education Center
www.nosotrosedu.org Phone: 469-450-7137
3
Session SummarySession Summary
This session will focus on providing TRIO This session will focus on providing TRIO personnel with an overview of:personnel with an overview of:
Program EvaluationProgram Evaluation
Types of Program Evaluations Types of Program Evaluations
Methods & Types of Data Collection Methods & Types of Data Collection
Quantitative & Qualitative Data Quantitative & Qualitative Data
Formative & Summative Methods Formative & Summative Methods
4
EvaluationEvaluation Evaluation -Evaluation - Systematic method for collecting, Systematic method for collecting,
analyzing, and using information to answer basic analyzing, and using information to answer basic questions about your program. questions about your program.
Helps to identify effective & ineffective services, practices, Helps to identify effective & ineffective services, practices, & approaches.& approaches.
Evaluation PlanEvaluation Plan - A written document describing - A written document describing the overall approach or design you anticipate using the overall approach or design you anticipate using to guide your evaluation. to guide your evaluation.
Includes what you plan to do, how you plan to do it, who Includes what you plan to do, how you plan to do it, who will do it, when it will be done, & why the evaluation is will do it, when it will be done, & why the evaluation is being conducted.being conducted.
The evaluation plan serves as a guide for the evaluation.The evaluation plan serves as a guide for the evaluation.
5
EvaluationEvaluation Plan Plan Comprehensive plan to determine the Comprehensive plan to determine the
effectiveness of the program in meeting stated effectiveness of the program in meeting stated objectives.objectives.
Includes the criteria that will determine Includes the criteria that will determine effectiveness & the documentation that objectives effectiveness & the documentation that objectives have been met.have been met.
Describes the methodologyDescribes the methodology
Provides the information about the delivery of Provides the information about the delivery of services & use of resources services & use of resources
Informs & assists program staff to make changes Informs & assists program staff to make changes to improve program effectiveness.to improve program effectiveness.
6
The GRPA ConnectionThe GRPA Connection
Government Performance and Results Act of 1993 Government Performance and Results Act of 1993 (GPRA) (GPRA)
DOE Response to GPRA Requirements: Goals 1-6DOE Response to GPRA Requirements: Goals 1-6
TRIO Programs Performance Indicator: Goal 5TRIO Programs Performance Indicator: Goal 5
TRIO Programs’ Specific Program Measures:TRIO Programs’ Specific Program Measures: Upward Bound, Talent Search, Educational Opportunity Upward Bound, Talent Search, Educational Opportunity
CentersCenters Student Support Services & Ronald E. McNairStudent Support Services & Ronald E. McNair
Prior Experience Criteria & Mandatory ObjectivesPrior Experience Criteria & Mandatory Objectives Project Evaluation PlanProject Evaluation Plan
Activities & Services Specific to Each ProjectActivities & Services Specific to Each Project
7
Types of TRIO Evaluation PlansTypes of TRIO Evaluation Plans
Goals-Based Evaluation Goals-Based Evaluation Goal-based evaluations are evaluating the extent to Goal-based evaluations are evaluating the extent to
which programs are meeting predetermined goals or which programs are meeting predetermined goals or objectives. objectives.
Outcomes-Based Evaluation Outcomes-Based Evaluation Outcomes evaluation looks at impacts/benefits to Outcomes evaluation looks at impacts/benefits to
clients during & after participation in your programs.clients during & after participation in your programs.
Process-Based EvaluationProcess-Based Evaluation Process-based evaluations are geared to fully Process-based evaluations are geared to fully
understanding how a program works -- how does it understanding how a program works -- how does it produce that results that it does. produce that results that it does.
8
TRIO Program Evaluation TRIO Program Evaluation
Shapes the development of the project Shapes the development of the project from the beginning of the grant period. from the beginning of the grant period.
Comprehensive plan to determine the Comprehensive plan to determine the effectiveness of the program in meeting effectiveness of the program in meeting stated objectives.stated objectives.
Includes criteria that will determine Includes criteria that will determine effectiveness & document that objectives effectiveness & document that objectives have been met.have been met.
Provides information about the delivery of Provides information about the delivery of services & their impact & effectivenessservices & their impact & effectiveness
9
TRIO Program EvaluationTRIO Program Evaluation
Provides a mechanism for evaluating & Provides a mechanism for evaluating & implementing programmatic changes to implementing programmatic changes to improve program effectivenessimprove program effectiveness
Provides accountability information about Provides accountability information about the success at the initial site & effective the success at the initial site & effective strategies for replication in other settingsstrategies for replication in other settings
Relates your project to GPRA & the TRIO Relates your project to GPRA & the TRIO Programs Performance IndicatorsPrograms Performance Indicators
10
UB Proposal Evaluation Criteria UB Proposal Evaluation Criteria The Secretary evaluates the quality of the evaluation The Secretary evaluates the quality of the evaluation
plan for the project on the basis of the extent to plan for the project on the basis of the extent to which the applicant’s methods of evaluation - - which the applicant’s methods of evaluation - -
(1) Are appropriate to the project and include both (1) Are appropriate to the project and include both quantitative and qualitative evaluation measures; and quantitative and qualitative evaluation measures; and
(2) Examine in specific and measurable ways the (2) Examine in specific and measurable ways the success of the project in making progress toward success of the project in making progress toward achieving its process and outcome objectives. achieving its process and outcome objectives.
11
TS/EOC Proposal Evaluation CriteriaTS/EOC Proposal Evaluation Criteria The Secretary evaluates the quality of the evaluation The Secretary evaluates the quality of the evaluation
plan for the project on the basis of the extent to which plan for the project on the basis of the extent to which the applicant’s methods of evaluation - - the applicant’s methods of evaluation - -
(1) Are appropriate to the project’s objectives; (1) Are appropriate to the project’s objectives; (2) Provide for the applicant to determine, using (2) Provide for the applicant to determine, using
specific and quantifiable measures, the success of the specific and quantifiable measures, the success of the project in - project in -
(i) Making progress toward achieving its objectives (a (i) Making progress toward achieving its objectives (a formative evaluation); and formative evaluation); and
(ii) Achieving its objectives at the end of the project (ii) Achieving its objectives at the end of the project period (a summative evaluation); and period (a summative evaluation); and
(3) Provide for the disclosure of unanticipated project (3) Provide for the disclosure of unanticipated project outcomes, using quantifable measures if appropriate. outcomes, using quantifable measures if appropriate.
12
SSS Proposal Evaluation Criteria SSS Proposal Evaluation Criteria
The Secretary evaluates the quality of the evaluation The Secretary evaluates the quality of the evaluation plan for the project on the basis of the extent to which plan for the project on the basis of the extent to which - -- -
(1) The applicant’s methods of evaluation - - (1) The applicant’s methods of evaluation - - (i) Are appropriate to the project and include both (i) Are appropriate to the project and include both
quantitative and qualitative evaluation measures; and quantitative and qualitative evaluation measures; and (ii) Examine in specific and measurable ways, using (ii) Examine in specific and measurable ways, using
appropriate baseline data, the success of the project appropriate baseline data, the success of the project in improving academic achievement, retention and in improving academic achievement, retention and graduation of project participants; and graduation of project participants; and
(2) The applicant intends to use the results of an (2) The applicant intends to use the results of an evaluation to make a programmatic changes based evaluation to make a programmatic changes based upon the results of project evaluation. upon the results of project evaluation.
13
McNair Proposal Evaluation CriteriaMcNair Proposal Evaluation Criteria The Secretary evaluates the quality of the evaluation plan for The Secretary evaluates the quality of the evaluation plan for
the project on the basis of the extent to which the the project on the basis of the extent to which the applicant's methods of evaluation– applicant's methods of evaluation–
(1) Are appropriate to the project's objectives; (1) Are appropriate to the project's objectives; (2) Provide for the applicant to determine, in specific and (2) Provide for the applicant to determine, in specific and
measurable ways, the success of the project in—measurable ways, the success of the project in—
(i) Making progress toward achieving its objectives (a (i) Making progress toward achieving its objectives (a formative evaluation); and formative evaluation); and
(ii) Achieving its objectives at the end of the project period (ii) Achieving its objectives at the end of the project period (a summative evaluation); and (a summative evaluation); and
(3) Provide for a description of other project outcomes, (3) Provide for a description of other project outcomes, including the use of quantifiable measures, if appropriate. including the use of quantifiable measures, if appropriate.
14
Common Words & TerminologyCommon Words & Terminology
Appropriate to Project’s ObjectivesAppropriate to Project’s Objectives
Specific & MeasurableSpecific & Measurable
Qualitative & QuantitativeQualitative & Quantitative
Formative & SummativeFormative & Summative
15
Appropriate to Project’s ObjectivesAppropriate to Project’s Objectives
Goals-Based Evaluation - InternalGoals-Based Evaluation - Internal Goal-based evaluations are evaluating the extent Goal-based evaluations are evaluating the extent
to which programs are meeting predetermined to which programs are meeting predetermined goals or objectives. goals or objectives.
Outcomes-Based Evaluation – Outcomes-Based Evaluation – Internal/ExternalInternal/External
Outcomes evaluation looks at impacts/benefits to Outcomes evaluation looks at impacts/benefits to clients during & after participation in your clients during & after participation in your programs.programs.
Process-Based EvaluationProcess-Based Evaluation - - ExternalExternal Process-based evaluations are geared to fully Process-based evaluations are geared to fully
understanding how a program works -- how does it understanding how a program works -- how does it produce that results that it does. produce that results that it does.
16
Specific & MeasurableSpecific & Measurable
Quantify description of program effectiveness for Quantify description of program effectiveness for each objectiveeach objective
Be specific & use quantifiable measuresBe specific & use quantifiable measures
Measures outcomes related to your mandatory Measures outcomes related to your mandatory objectivesobjectives
Measurable: %, time lines, aggregate data, Measurable: %, time lines, aggregate data, descriptive numbersdescriptive numbers
Data examples: Demographics, GPAs, Year to Data examples: Demographics, GPAs, Year to Year Retention, Graduation Rates, College Year Retention, Graduation Rates, College Enrollment & Graduation Rates, & Enrollment in Enrollment & Graduation Rates, & Enrollment in Continuing Education Continuing Education
17
Quantitative DataQuantitative Data Information that can be expressed in Information that can be expressed in
numerical terms, counted or compared on numerical terms, counted or compared on a scale. For example, improvement in a a scale. For example, improvement in a student’ test scores as measured pre and student’ test scores as measured pre and post test scores.post test scores.
Quantifiable – specifically measure (GPA Quantifiable – specifically measure (GPA increase, Retention & Graduation rates, increase, Retention & Graduation rates, academic progress, credit hours earned).academic progress, credit hours earned).
Methods: data collection, test results & Methods: data collection, test results & documentsdocuments
18
Quantitative DataQuantitative Data Grade Point Averages Grade Point Averages Project Retention RatesProject Retention Rates Grade Reports & Test ScoresGrade Reports & Test Scores Secondary/Postsecondary Persistence Secondary/Postsecondary Persistence
StatusStatus Secondary/Postsecondary Graduation Secondary/Postsecondary Graduation
Rates Rates Credit Hour CompletionCredit Hour Completion Good Standing StatusGood Standing Status
19
Qualitative MethodsQualitative Methods May include May include
“meaning”, “meaning”, experience, opinions experience, opinions
Ways to describe Ways to describe program program effectiveness that are effectiveness that are measurable, but may measurable, but may not aggregate in not aggregate in traditional, traditional, quantifiable methodsquantifiable methods
Examples:Examples:
- Focus groups- Focus groups
- Case studies - Case studies
- Open ended interviews- Open ended interviews
- Surveys- Surveys
20
Qualitative DataQualitative Data Information that is difficult to measure, Information that is difficult to measure,
count, or express in numerical terms. For count, or express in numerical terms. For example, a participant's impression about example, a participant's impression about the fairness of a program rule/requirement the fairness of a program rule/requirement is qualitative data. is qualitative data.
Interviews & Focus GroupsInterviews & Focus Groups Used to fully understand someone's Used to fully understand someone's
impressions or experiences, or learn more impressions or experiences, or learn more about their answers to questionnairesabout their answers to questionnaires
21
Qualitative DataQualitative Data ObservationsObservations
Used to gather accurate information about Used to gather accurate information about how a program actually operates, how a program actually operates, particularly about processesparticularly about processes
Questionnaires, Surveys, ChecklistsQuestionnaires, Surveys, Checklists
Used to quickly and/or easily get lots of Used to quickly and/or easily get lots of information from people in a non-information from people in a non-threatening way threatening way
Examples: Sample Evaluation FormsExamples: Sample Evaluation Forms
Pre & Post Test to Measure LearningPre & Post Test to Measure Learning
22
Formative EvaluationFormative Evaluation
Type of process evaluation of programs or Type of process evaluation of programs or services that focuses on collecting data on services that focuses on collecting data on program operations so that needed program operations so that needed changes or modifications can be made to changes or modifications can be made to the program in its early stages. the program in its early stages.
Used to provide feedback to staff about the Used to provide feedback to staff about the program components that are working & program components that are working & those that need to be changed.those that need to be changed.
Example: Changes to Delivery of Services Example: Changes to Delivery of Services
23
Formative EvaluationFormative Evaluation Continuous (ex. monthly) assessment of Continuous (ex. monthly) assessment of
program effectiveness & progress towards program effectiveness & progress towards achieving objectivesachieving objectives
Allows for changes in activities so final Allows for changes in activities so final objectives/outcomes will be met – changes objectives/outcomes will be met – changes in delivery of servicesin delivery of services
Ex: Monthly report of student grades; student/client Ex: Monthly report of student grades; student/client progress reports; monthly student/client contact progress reports; monthly student/client contact reports; client/student evaluation of program; reports; client/student evaluation of program; monthly staff meeting progress reporting; monthly staff meeting progress reporting; student/client attendance & feedback; student pre-student/client attendance & feedback; student pre-post test results post test results
24
Summative EvaluationSummative Evaluation
A type of outcome evaluation that assesses A type of outcome evaluation that assesses the results or outcomes of a program. the results or outcomes of a program.
Concerned with a program's overall Concerned with a program's overall effectivenesseffectiveness
Measurement of objectives or outcomes at Measurement of objectives or outcomes at the end of a pre-defined periodthe end of a pre-defined period
Quantitative data is analyzed for a given Quantitative data is analyzed for a given periodperiod
Qualitative evaluation supplements the Qualitative evaluation supplements the quantitative dataquantitative data
25
Summative Evaluation QuestionsSummative Evaluation Questions
Did you meet program objectives?Did you meet program objectives?
How did you accomplish them?How did you accomplish them?
What is your documentation?What is your documentation?
What changes do you need to make to the What changes do you need to make to the activities or objectives? activities or objectives?
Examples: Student year-end program Examples: Student year-end program evaluations; annual performance reportevaluations; annual performance report
26
Goals-Based Evaluation - InternalGoals-Based Evaluation - Internal
State Mandatory ObjectivesState Mandatory Objectives
State Evaluation CriteriaState Evaluation Criteria
Summative & Formative Evaluation Summative & Formative Evaluation MethodsMethods
Qualitative & Quantitative MeasuresQualitative & Quantitative Measures
Assign Staff ResponsibilityAssign Staff Responsibility
State Time Delineation State Time Delineation
Analyze & Report Findings Analyze & Report Findings
27
Goals-Based Evaluation - Internal Goals-Based Evaluation - Internal
Data Collection ChartData Collection Chart
(1) What types of data will be collected;(1) What types of data will be collected;
(2) When various types of data will be collected; (2) When various types of data will be collected;
(3) What methods will be used; (3) What methods will be used;
(4) What instruments will be developed & when; (4) What instruments will be developed & when;
(5) How the data will be analyzed; (5) How the data will be analyzed;
(6) When reports & outcomes will be available; and(6) When reports & outcomes will be available; and
(7) How the grantee will use the information (7) How the grantee will use the information collected through evaluation to monitor progress of collected through evaluation to monitor progress of the funded project & to provide information about the funded project & to provide information about success at the initial site & effective strategies for success at the initial site & effective strategies for replication in other settings. replication in other settings.
28
Goals-Based Evaluation - Internal Goals-Based Evaluation - Internal
Sample Data Collection ChartSample Data Collection Chart
Talent Search Talent Search Educational Opportunity Educational Opportunity
CentersCenters Upward Bound Upward Bound Student Support Services Student Support Services McNairMcNair
29
Making Progress Toward Achieving its Objectives -Making Progress Toward Achieving its Objectives -
Formative Evaluation:Formative Evaluation:
Project Director will conduct the following Project Director will conduct the following formative evaluation methods of all project formative evaluation methods of all project activities and services that correspond to activities and services that correspond to keep the project running smoothly during keep the project running smoothly during the year.the year.
Project Director will evaluate the results of Project Director will evaluate the results of all daily contacts & services provided all daily contacts & services provided weekly and monthly as it relates to weekly and monthly as it relates to progress to meet project objectives. progress to meet project objectives.
30
Making Progress Toward Achieving its Objectives - Making Progress Toward Achieving its Objectives -
Formative Evaluation:Formative Evaluation:
Progress towards the accomplishment of all Progress towards the accomplishment of all objectives will be tracked through objectives will be tracked through quantitative reports generated by the quantitative reports generated by the project database on a daily, weekly, & project database on a daily, weekly, & monthly basis. monthly basis.
Quantitative reports collect specific Quantitative reports collect specific information documenting what participants information documenting what participants have been served, the types of services have been served, the types of services provided, the frequency of services provided, the frequency of services provided, & by which project staff member. provided, & by which project staff member.
31
Making Progress Toward Achieving its Objectives - Making Progress Toward Achieving its Objectives -
Formative Evaluation:Formative Evaluation:
Project database generates weekly, monthly, Project database generates weekly, monthly, & quarterly performance reports & generates & quarterly performance reports & generates individual participant outcome & contact individual participant outcome & contact reports documenting services provided & the reports documenting services provided & the impact of these services on each project impact of these services on each project participant. participant.
Participants & parents will be asked to Participants & parents will be asked to evaluate specific group activities such as evaluate specific group activities such as workshops, field trips to colleges and career workshops, field trips to colleges and career sites to rate their level of satisfaction with the sites to rate their level of satisfaction with the quality of services they receive – qualitative. quality of services they receive – qualitative.
32
Achieving Objectives at the End of the Project Period - Achieving Objectives at the End of the Project Period -
Summative Evaluation:Summative Evaluation:
Project database will combine all monthly Project database will combine all monthly quantitative reports to produce a report quantitative reports to produce a report documenting & confirming the achievement documenting & confirming the achievement of project objectives. of project objectives.
Participants & parents will complete end-of-Participants & parents will complete end-of-year surveys to measure their level of year surveys to measure their level of satisfaction with the overall quality of satisfaction with the overall quality of project services. project services.
33
Achieving Objectives at the End of the Project Period - Achieving Objectives at the End of the Project Period -
Summative Evaluation:Summative Evaluation:
Target school personnel & project staff will be Target school personnel & project staff will be asked to evaluate specific activities & services asked to evaluate specific activities & services on a numeric rating system to measure their on a numeric rating system to measure their level of satisfaction with the overall quality of level of satisfaction with the overall quality of project services - qualitative. project services - qualitative.
Project Director will address the results of the Project Director will address the results of the summative evaluation to implement any summative evaluation to implement any corrective measures to eliminate potential corrective measures to eliminate potential problems by appropriately modifying & problems by appropriately modifying & improving project activities & services for the improving project activities & services for the next project year. next project year.
34
Sample TRIO Evaluation PlansSample TRIO Evaluation Plans
Upward Bound Upward Bound
Talent Search Talent Search
Educational Opportunity Centers Educational Opportunity Centers
Student Support Services Student Support Services
McNair McNair
35
External EvaluationExternal Evaluation
Recommended in last grant proposal Recommended in last grant proposal competitionscompetitions
Must be conducted if included in grant proposalsMust be conducted if included in grant proposals
Allowable cost for the project Allowable cost for the project
May be paid by the grantee agency/institution May be paid by the grantee agency/institution
May be provided free of charge – May be provided free of charge – in-kind/donationin-kind/donation
36
External EvaluationExternal Evaluation
May be conducted by person May be conducted by person working/employed by agency/institution working/employed by agency/institution BUT not for the project BUT not for the project
Must have experience conducting Must have experience conducting evaluations for educational opportunity evaluations for educational opportunity programs & TRIO like programsprograms & TRIO like programs
Must state what type of evaluation will be Must state what type of evaluation will be conducted by the evaluator.conducted by the evaluator.
37
External EvaluationExternal Evaluation Process-Based EvaluationProcess-Based Evaluation
Process-based evaluations are geared to Process-based evaluations are geared to fully understanding how a program works -- fully understanding how a program works -- how does it produce that results that it does. how does it produce that results that it does.
Useful if programs are long-standing and Useful if programs are long-standing and have changed over the years.have changed over the years.
Useful for accurately portraying to outside Useful for accurately portraying to outside parties how a program truly operates (e.g., parties how a program truly operates (e.g., for replication elsewhere).for replication elsewhere).
38
Process-Based EvaluationProcess-Based Evaluation
Examines the extent to which a program Examines the extent to which a program is operating as intended by assessing is operating as intended by assessing ongoing program operations and whether ongoing program operations and whether the targeted population is being served. the targeted population is being served.
Helps program staff identify needed Helps program staff identify needed interventions and/or change program interventions and/or change program components to improve service delivery. components to improve service delivery.
39
Process-Based EvaluationProcess-Based Evaluation
Involves collecting data that describes Involves collecting data that describes program operations in detail, including:program operations in detail, including:
Types & levels of services providedTypes & levels of services provided
Staffing & location of service deliveryStaffing & location of service delivery
Sociodemographic characteristics of Sociodemographic characteristics of participantsparticipants
Community in which services are providedCommunity in which services are provided
Linkages with collaborating agencies. Linkages with collaborating agencies.
40
External Evaluation PlanExternal Evaluation Plan
Outcomes-Based EvaluationOutcomes-Based Evaluation Outcomes-based evaluation helps Outcomes-based evaluation helps
determine if your organization is really determine if your organization is really doing the right program activities to bring doing the right program activities to bring about the outcomes you believe to be about the outcomes you believe to be needed by your clients/students.needed by your clients/students.
Outcomes are benefits to clients from Outcomes are benefits to clients from participation in the program. Example: participation in the program. Example: Enhanced learning (academic knowledge, Enhanced learning (academic knowledge, perceptions/attitudes or skills, etc). perceptions/attitudes or skills, etc).
41
External Evaluation PlanExternal Evaluation Plan Outcome-Based EvaluationOutcome-Based Evaluation
1.1. Identify the major outcomes that you want to Identify the major outcomes that you want to examine or verify for the program under examine or verify for the program under evaluation. evaluation.
2. Choose the outcomes that you want to examine, 2. Choose the outcomes that you want to examine, prioritize the outcomes.prioritize the outcomes.
3. For each outcome, specify what observable 3. For each outcome, specify what observable measures, or indicators, will suggest that you're measures, or indicators, will suggest that you're achieving that key outcome with your clients. achieving that key outcome with your clients.
4. Specify a "target" goal of clients, i.e., what 4. Specify a "target" goal of clients, i.e., what number or percent of clients you commit to number or percent of clients you commit to achieving specific outcomes achieving specific outcomes
42
External Evaluation PlanExternal Evaluation Plan
Outcome-Based EvaluationOutcome-Based Evaluation
5.5. Identify what information is needed to show Identify what information is needed to show these indicators.these indicators.
6. Decide how can that information be efficiently 6. Decide how can that information be efficiently and realistically gathered & which methods to and realistically gathered & which methods to use (ex. program documentation, observation of use (ex. program documentation, observation of program personnel & clients in the program, program personnel & clients in the program, questionnaires & interviews about clients questionnaires & interviews about clients perceived benefits from the program, case perceived benefits from the program, case studies of program failures & successes, etc.)studies of program failures & successes, etc.)
7. Analyze & report the findings7. Analyze & report the findings
43
•Promotes standards in student affairs, student services, and student development programs since its inception in 1979
• Develops a Book of Professional Standards and Guidelines and Self-Assessment Guides (SAG) that are designed to lead to a host of quality-controlled programs and services.
CAS TRIO/EOP Standards & Guidelines & Self-Assessment Guide (SAG) for TRIO & EOP Programs
Provides a vehicle and opportunity for projects to plan for formative & summative evaluation, to participate in project self-assessment, and plan for project improvements.
The Council for the Advancement of Standards in The Council for the Advancement of Standards in Higher Education (CAS)Higher Education (CAS)
44
Benefits of Program Evaluation Benefits of Program Evaluation
Understand, verify or increase the impact Understand, verify or increase the impact of services on participants or clients of services on participants or clients
Improve delivery mechanisms to be more Improve delivery mechanisms to be more efficient and less costly efficient and less costly
Produce data or verify results that can be Produce data or verify results that can be used for public relations & promoting used for public relations & promoting services in the communityservices in the community
Fully examine & describe effective Fully examine & describe effective programs for duplication elsewhere. programs for duplication elsewhere.
45
Basic Guide to Program Evaluation
Written by Carter McNamara, MBA, PhD, Authenticity Consulting, LLC. Copyright 1997-2007.Adapted from the Field Guide to Nonprofit Program Design, Marketing and Evaluation.
Online Guide: http://www.tgci.com/magazine/A%20Basic%20Guide%20to%20Program%20Evaluation.pdf
Bibliography & ResourcesBibliography & Resources
46
Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources
Written by Carter McNamara, MBA, PhD, Authenticity Consulting, LLC. Copyright 1997-2007.Adapted from the Field Guide to Nonprofit Program Design, Marketing and Evaluation.
Bibliography & ResourcesBibliography & Resources
47
US Department of Health & Human Services Administration for Children & Families - Office of Planning, Research & Development
The Program Manager's Guide to Evaluation
http://www.acf.hhs.gov/programs/opre/other_resrch/pm_guide_eval/reports/pmguide/pmguide_toc.html
This guide explains program evaluation -what it is, how to understand it, and how to do it. It answers your questions about evaluation and explains how to use evaluation to improve programs and benefit students, families, and staff.
Bibliography & ResourcesBibliography & Resources
48
The Council for the Advancement of Standards The Council for the Advancement of Standards in Higher Education (CAS)in Higher Education (CAS)CAS TRIO/EOP Standards & Guidelines & Self-Assessment Guide (SAG) for TRIO & EOP Programs
Provides a vehicle and opportunity for projects to plan for formative & summative evaluation, to participate in project self-assessment, and plan for project improvements.
http://www.cas.edu/http://www.cas.edu/
• Resources • Basics PowerPoint• Handout• Assessment Model• Online Store• Self-Assessment Guide
Bibliography & ResourcesBibliography & Resources