“advanced” data collection january 27, 2005
DESCRIPTION
“Advanced” Data Collection January 27, 2005. Who We Are: Innovation Network. National nonprofit organization Committed to evaluation as a tool for empowerment Build evaluation capacity of nonprofits and funders so they can better serve their communities Practice a participatory approach. - PowerPoint PPT PresentationTRANSCRIPT
“Advanced” Data CollectionJanuary 27, 2005
Slide 2Innovation Network, Inc.
Who We Are: Innovation Network
National nonprofit organization Committed to evaluation as a tool for empowerment Build evaluation capacity of nonprofits and funders so
they can better serve their communities Practice a participatory approach
Slide 3Innovation Network, Inc.
Who We Are: Innovation Network
www.innonet.org/services/LSC_Train
Slide 4Innovation Network, Inc.
Objectives of This Session
Review the five major groups of data collection methods
Discuss some of the subtleties of web-based data collection
Slide 5Innovation Network, Inc.
Data Collection
Program Plan
Evaluation Plan
Data Collection Strategy
Slide 6Innovation Network, Inc.
Data Collection: 3 Steps
Choose the method Decide which people or records will be the source of
the information Determine the level of effort involved in using that
method with that population
Slide 7Innovation Network, Inc.
Data Collection Steps 1 & 2:Choose Method, Identify Source
Review documents Observe Talk to people Collect written responses Pictorial/multimedia
Slide 8Innovation Network, Inc.
Data Collection Step 3:Level of Effort
Instrument development Cost/practicality of actually collecting data Cost of analyzing and presenting data
Also consider: Communication Power Proxy Power
Slide 9Innovation Network, Inc.
Document Review
Examples: Attendance records Application information Feedback forms Web statistics Curricula or content
Slide 10Innovation Network, Inc.
Document Review
Efficient and cost-effective
Dependent on the existence of documentation Better for process evaluation than outcome evaluation
Slide 11Innovation Network, Inc.
Observation
When conducting and evaluation the observer: Does not intervene in any way Does not stand out or draw attention Does not record names or any identifying information
It is important that observations be held over time (and at different places/days/times)
Slide 12Innovation Network, Inc.
Observation
Inexpensive way to obtain an outside perspective
Highly susceptible to bias Subjective; relies on anecdotal data
Slide 13Innovation Network, Inc.
Interviews
Structured Follow a predetermined interview protocol Identify broad evaluation questions and specific interview
questions in advance
Unstructured Begin with one or two open-ended questions Proceed based on the responses received
Slide 14Innovation Network, Inc.
Interviews
Provides the story behind someone’s experiences Useful as a follow-up to a survey to further investigate
responses
Time consuming (both in terms of collection and analysis)
Respondents susceptible to “leading” by interviewers
Slide 15Innovation Network, Inc.
Interview Tips
Make the respondent feel comfortable Introduce yourself and the evaluation Start with easy, short answer questions – end with more
revealing, long answer questions Record the interviews
Important for capturing great quotes
Slide 16Innovation Network, Inc.
Focus Groups
A small group of people (who have something in common) who are led through a conversation by a moderator and discuss their opinions and provide suggestions on a topic.
Slide 17Innovation Network, Inc.
Focus Groups
Efficient and cost-effective way to gather qualitative data
Requires a skilled moderator/facilitator Collects consensus rather than individual perspectives
Slide 18Innovation Network, Inc.
Focus Group Tips
Ideally 8-10 person groups Maximizes amount of information you can gather without
the suppression of voices Don’t get bogged down Rewards
Breakfast/Lunch/Dinner Gift certificates
Slide 19Innovation Network, Inc.
Surveys
Delivery options: By mail In person Online By telephone
Slide 20Innovation Network, Inc.
Surveys
Collect quantitative information Efficient
Open-ended questions (qualitative data) can be cumbersome for respondents and data entry
Slide 21Innovation Network, Inc.
Survey Tips
Introduce the survey Describe the purpose and what will be done with the
results Reinforce that it is confidential and that you request they
respond candidly Lead with easy, closed-ended questions Group categories of questions together Consider how long it will take for someone to respond
rather than the length of pages
Slide 22Innovation Network, Inc.
Web Statistics
# of hits Most popular pages Most popular downloads Common “drop off” points Where are they coming from?
Slide 23Innovation Network, Inc.
Don’t Be Afraid to Ask More!
Why are they there? Who are they? What do they do? What are they looking for?
Did they find it? What are they going to do with it?
Are they willing to help you with future evaluation projects?
Slide 25Innovation Network, Inc.
Next Steps
We can do more! Online, instructor-led trainings Online, self-paced modules Individual technical assistance
www.innonet.org/services/LSC_Train
Slide 26Innovation Network, Inc.
Next Steps
We can do more! Survey Builder Survey Library Data Analysis Tool
Slide 27Innovation Network, Inc.
Thanks for Your Participation!
Measure results. Make informed decisions. Create lasting change.
Innovation Network, Inc.1625 K St. NW, 11th Floor
Washington, DC 20006(202) 728-0727
www.innonet.orgEhren Reed: [email protected]
www.innonet.org/services/LSC_Train