DATA ANALYSIS AND INTERPRETATION

Allison Nichols, Ed.D.

Evaluation Specialist, West Virginia University

Martha Garton, M.A.

Grant County, WV, Extension Educator

Evaluation Community of Practice Webinar

October 20, 2010

IS YOUR DATA MOSTLY TEXT FROM . . .?

Interviews Focus groups Meeting minutes Sign-in sheets Emails and other correspondence Evidence of completed projects such as a new

curriculum Media outlets such as newspaper articles, TV and

radio clips, Internet sites and Internet hits

OR IS IT MOSTLY NUMBERS FROM….?

Surveys Tests Assessments Observations Logs

POLL

Is your data mostly A = text B = numbers C = both

CHOOSING A METHOD

If the data is text or words, you will probably choose a qualitative method.

Content analysis Look for reoccurring themes Categorize themes into larger and smaller themes Look for the number of times a theme is mentioned Illustrate findings with quotes

Unlike quantitative analysis, findings cannot be generalized beyond the group being studied.

CHOOSING A METHOD

If your data consists of numbers, you will probably choose a quantitative method to analyze. Most commonly used are frequencies

Frequencies are summary data of the number of attendees, meetings, workshops,

dissemination methods, items that are disseminated.

the number of survey respondents who choose a particular answer.

Descriptive data including Mean (average) Mode (most common answer) Median (answer right in the middle)

POLL

How comfortable are you analyzing quantitative data? A = not at all comfortable B = somewhat comfortable C = very comfortable

How comfortable are you analyzing qualitative data? A = not at all comfortable B = somewhat comfortable C = very comfortable

STATISTICAL TESTS T-tests – if you want to measure the difference between

the mean for answers pre- and post-intervention. There are two types: unmatched tests matched tests

Chi Square – if you want to look at the difference between categories of people, i.e. men and women

Regression – if you want to look at the association of one variable with another variable; does one go up when the other goes down? i.e. are respondents more likely to react positively the longer they have been in the program.

Factor analysis – when you want to see if questions are correlated with each other and thus fall within factors that can be named.

COMBINING STATISTICAL AND QUALITATIVE DATA Strong evaluation plans combine statistical

and qualitative data. Example 1

If by using statistical tests such as Chi Squares, we know that women are more likely to adopt healthy behaviors than men, but ---

We don’t know why - - We can use qualitative methodologies such as

focus groups and interviews to understand why. Example 2

If we conduct focus groups and know ways that participants benefit from a program

We can conduct a survey to learn how many program participants benefit in a particular way.

WHAT TO DO WITH THE DATA ANALYSIS

What are the important findings?

What are the implications of the findings?

What are the recommendations?

CASE STUDY: EXAMPLE FROM WV 4-H CAMPING

The National 4-H Camping Consortium created a series of tools for evaluating camp. They included

Three logic models General camping Life skills at camp Camp Context – Essential Elements of Youth

Development Two corresponding questionnaires

Life Skills Camp Context

EVALUATION PROTOCOL

West Virginia University Extension has used both questionnaires since 2007: In its 60 summer residential camps (county and

state) With 2,000 – 3,000 youth each summer

FACTORS/DOMAINS USED IN THE ANALYSIS

Life skills developed at camp Accepting Self and Others Accomplishing Goals Taking Responsibility

Essential Elements provided in camp setting Opportunity to Build a Relationship with a

Caring Adult Opportunity for Independent Learning and

Mastery Emotionally Safe and Inclusive Environment Physically Safe Environment

DATA SUMMARIES

Each camp director/Extension agent receives the following information

Frequencies for all questions Descriptive statistics for all questions Descriptive statistics for each of the factors:

life skills and essential elements All of the above for the whole state

EXAMPLE OF DATA SUMMARY

PURPOSE OF SUMMARIES

Each camp director receives directions on how to use the statistics for: Improving programs Reporting to stakeholders Writing narratives for the faculty file

INSTRUCTIONS

The directions state that camp directors should: Discuss significant findings for their camp in the

current year Compare findings for their camp with findings

from previous years and discuss improvements Compare findings for their camp with findings

from the whole state and discuss why their camp is different and how they should adjust their programming to meet needs of their county

EXAMPLE OF INSTRUCTION SHEET

INTERESTING FINDINGS

For the past several years we have found that boys feel significantly less safe (both emotionally and physically) than girls at camp.

Girls have consistently more positive responses on all items.

As the number of years at camp and the number of years in 4-H increase, responses on all item are more positive.

COMBINING QUANTITATIVE DATA WITH QUALITATIVE DATA

To answer the question “why do boys feel less safe, we held three focus groups at state camp this year (Older Member Camp)

After analyzing the data from the focus group, we plan to revamp the questions and conduct focus groups (using counselors as facilitators) at county-based camps next summer.

PRELIMINARY ANALYSIS OF FOCUS GROUP DATA

COUNTY-BASED FACULTY MEMBER (EXTENSION AGENT) WILL DISCUSS HER EXPERIENCE THE 4-H CAMP EVALUATION PROCESS

QUESTIONS, DISCUSSION