healthy futures performance measures session. session overview combination of presentation and...
TRANSCRIPT
Healthy FuturesPerformance Measures Session
Session Overview
• Combination of presentation and interactive components
• Time at the end of the session for Q&A• Use notecards on table to write questions
Objectives
• Strengthen understanding of the three elements integral to a strong performance measurement plan:– Data Collection
– Data Management
– Data Reporting
• Experience planning for the data collection process
Agenda
Activity Duration
Review Data Quality and Data Collection 30 minutes
Activity 45 minutes
Review Data Management and Reporting 10 minutes
Additional resources and Q&A 5 minutes
How does my program demonstrate impact?
What evidence do I need to collect to show I achieved my selected outcome?
Community Need
Demonstrated by facts presented in application
Service Activity
What service, how often and for how long the volunteers provide
Anticipated Output
Number of individuals who receive service
Anticipated Outcome
Number of individuals who experience change
6
Amount of service provided (i.e. people served).
Reflect the changes or benefits that occur.
How does my program demonstrate impact?
Outputs
Outcomes
Work Plan Refresher
• The Service Activity Description should explain what the Senior Corps volunteers are doing in a way that shows how they will achieve an output and outcome.
• It should include:• who the beneficiaries are;• what the volunteers will be doing with the beneficiaries; • how often volunteers will provide the service;• for how long, and; • where the service will take place.
Work Plan Refresher
• 50 Senior volunteers will provide respite services to 150 primary family caregivers of older, homebound adults. They will serve a minimum of 8 hours per week for at least 12 weeks. They will serve each client a minimum of 2 hours per week for 12 weeks. Service activities may include companionship/friendly visiting and reminiscing, light meal preparation, letter writing/correspondence, going for short walks, and reading stories together.
Work Plan Refresher
• The Instrument Description should:• give the name of the instrument;• briefly describe who will collect the data; • from whom the data will be collected, and;• when it will be collected.
Work Plan Refresher
• Station supervisors will conduct the Respite Care Survey with caregivers every 6 months at the middle and end of the program year.
Five Data Quality Elements
A Standard Process to Consider Data
Validity Completeness Consistency
Accuracy Verifiability
Validity: Data Relevance• Ask yourself:
– Is the data relevant?
– Are your reported items consistent with the approved goals of the current grant?
– Are you measuring what you intended to measure?
ExampleH14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support.
Data Instrument: Survey
Questions in Survey:• Did you have a long wait for a
volunteer? • Are you happy with the service?
Data Completeness
• Ask yourself: – Does the reported data
contain enough information to represent performance measure activities?
– Did the reported data come from all stations and/or a random sampling of volunteers/ service recipients?
ExampleH14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support.
Data Instrument: Survey
Survey Distribution: Jane provides the survey to 7 of 12 stations because she is running behind on her schedule
Consistency: Standard Protocols and Procedures
• Ask yourself: – Do you clearly document data
collection methods?
– Do you use the same data collection method for all sites?
– Do you train data collectors?
– Have you taken steps to eliminate bias?
ExampleH14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support.
Data Instrument: Survey
Survey Distribution: Jane provides the same survey to all stations at the same time
Accuracy: Data Free From Error
• Ask yourself: – Do you have a plan to
collect and review data?
– Do you have documentation on your data collection plan?
– Does the data vary significantly in unexpected ways?
ExampleData Collection Approach: Collect data from all 12 stations
Data Received: 150 surveys
Data Check: Jane’s colleague reviews the data to check her calculations, as per her organization's Data Collection Handbook
Verifiability: Data Collected According to Plan
• Ask yourself: – Did the grantee double
check that the data collected followed the plans and procedures?
– Did the grantee use controls throughout data collection?
– Does the grantee regularly review and update its data quality control procedures?
Example
Situation: Jane submits her PPR and leaves for vacation; her colleagues cannot validate the data she input into it.
Doing Data Collection: Identifying Your Source
• List pros and cons of possible data sources
• Identify a preferred data source
• Consider back-ups
Doing Data Collection: Considering Methods
• Survey
• Interviews
• Observation
• Standardized tests
• Tracking sheets
• Focus groups
• Diaries or journals
• Secondary data
Method Requirements
Sample questions are provided for the required survey
Doing Data Collection: Creating a Schedule
• Identifies who will collect data, using which instrument, and when
– Include stakeholders in planning
– Include time for collecting, analyzing, and reporting data
• Share with team to keep everyone informed
Doing Data Collection: Selecting a Final Measurement Instrument
• For CNCS priorities and performance measures, look for instruments by program– Go to http://
www.nationalservice.gov/resources/performance-measurement/senior-corps
– Programs and projects can also look anywhere they like to find instruments:• Use Internet search engines• Leverage experience and known best practices in your field• Talk to others within you professional network to find out what they
are using
Doing Data Collection: Selecting a Final Measurement Instrument
• You may have a pre-existing tool or want to modify a resource found on the Knowledge Network
• Consider: • How else might you use this data? • What may other stakeholders want?• How will this affect your data collection and
instrument design?• Keep in mind the 5 data quality elements when
developing or modifying a measurement instrument.
Learning Activity
Data Management
• Now what? • Consider:
– Potential usage
– Access
– Storage and maintenance
Data Management
• Data collection procedures and guidelines
• Source documents for data (e.g. sign-in sheets, activity logs, attendance logs, client tracking database, tracking forms, surveys, etc.)
• Formalize data sharing agreements in MOUs
Data Reporting
Data Reporting
Q&A
Thank you
Stay current by visiting the Knowledge Network! http://www.nationalservice.gov/resources/performance-measurement/senior-corps