Indicators of Progress: Objective and Subjective Indicators

Constructing Indicators of Progress/Well-being with

Citizens/CommunitiesCouncil of Europe- July 2008

Susan Brutschy, Applied Survey Research

2

Applied Survey Research

Non-profit social research firm in California with over 25 years of experience.

ASR conducts: Community indicator projects Health assessments Children’s reports Homeless census and surveys Program evaluations Strategic planning processes

3

Santa Cruz County Community Assessment

Project

4 4

Community Indicator Projects

5

Agenda

Applied Survey Research (ASR’s ) Community Indicator Cycle

Working with communities to choose outcomes and indicators of progress

Criteria for Choosing Indicators Blending Objective and Subjective Indicators A Review of other Methods: Mark Friedman,

JCCI, Redefining Progress, and Dr. Joe Sirgy

6

Community Indicator Process

Step 1

Form a Steering

Committee

Establish Broad-based Community Involvement

CreateWorking Committees

Create an Outcomes Framework / Reach Consensus on Outcomes & Indicators

Collect Secondary DataConduct Community Surveys

Analyze Results

Publish andPromote the

Report

Develop Community

Goals and Benchmarks

Update the Report

Regularly and Sustain

Your Project

Encourage Community

Action

Step 2

Step 7

Step 6

Step 5

Step 4

Step 3

Step 10

Step 9

Step 8

7

Commitment to a Results Framework

In Your Community!

Turning

the

Curve

8

ASR and RBA in Community Action

RBA Step 1: Select a community goal

RBA Step 2: How are we doing on the most important indicators?

RBA Step 3:Who has a part to play in doing better? Who is missing from the table?

RBA Step 4: What works to do better?

Low cost and no-cost solutions?

RBA Step 5: What are we going to do? (Do something!)

RBA Step 6: Now…how are we doing? (Measure progress)

9

ASR: Criteria for Choosing Indicators

1. Indicators should be understandable to the general user and the public

2. Indicators should respond quickly and noticeably to change.

3. Indicators should be relevant for policy decisions

4. Indicators should be available annually.

10

Asset Based Indicators Strength based approach

Highlight assets in the community Natural beauty Youth resiliency Supportive teachers Adult mentors to youth Families with extended

support networks

11

Secondary/ Administrative Data Collection

United States Census

Federal data sources

State and local government agencies

Health organizations

Libraries

Schools

On-line databases

The Internet

12

Primary Data Collection

Telephone surveys (random digit dial of a representative sample).

Face to face surveys in communities, especially with low-income, people of color and native populations.

13

Key Indicators: Objective and Subjective The Economy: Median family income

Percentage of respondents indicating they are better off economically this year than last year.

Percent of respondents reporting spending ½ or more of their take-home pay on rent/housing costs.

Health: Health insurance coverage Percent of respondents who needed health care

and were unable to receive it.

Public Safety: Crime rate Percent of respondents who reported concern

about crime. Percent of respondents who felt their child had a safe

place to play.

14

Mark Friedman

Criteria for selecting indicators: Communication Power: The indicator must be

understandable to a broad and diverse audience.

Proxy Power: The indicator must say something of central importance about the result the community would like to achieve.

Data Power: Are the data reliable and consistent?

Criteria for moving from data to action: Specificity, Leverage, Values, and Reach

15

Jacksonville Community Council Inc: JCCI

Criteria for selecting indicators:

Validity Availability and timeliness Reliability and stability Responsiveness Understandability Policy relevance Representativeness

Redefining Progress

Relevant Valid Credible Measurable Consistent/reliable Comparable Understandable Leading Compelling Interesting

Exciting Engaging to the

media Accessible/affordable Relate to the whole

community Reveal linkages Balance resources

with needs Be creative and

action oriented

17

Redefining Progress: Subjective Indicators

Measure Perceptions of the Community to Complement Objective/Secondary Data:

Internet Web Surveys

Scientific Survey Techniques

Focus Groups

Town Hall Meetings

18

Joe Sirgy, “The Science of Community Indicators Research” (ISQOLS)

Criteria for selecting indicators:1.Face Validity2.Policy Relevance

1. Source Credibility2. Consistency and Reliability3. Geographic Comparability4. Comprehension and Excitement5. Accessibility and Affordability

19

Joe Sirgy, “The Science of Community Indicators Research”

Validating Objective Indicators with Subjective Ones:

1.Ensure that the objective reality is not divorced from subjective experience

2.Provide data for indicators where there are no other sources

3.Collect information about people’s perceptions of their community and services

4.Allow policy makers to use both objective and subjective data for recommendations for change

20

Joe Sirgy: Types of Survey Research

1. Door to Door Surveys2. Telephone Surveys3. Mail Surveys4. E-mail/ Internet Surveys

21

Summary

Outcomes Framework Community driven indicators

Asset Based Criteria for Selecting Indicators Key Indicators

Secondary/Administrative Indicators Primary/Subjective Indicators

Telephone Surveys Face to Face Surveys Focus Groups Town Halls

Susan Brutschy, Presidentsusan@appliedsurveyresearch.org

Applied Survey ResearchWatsonville Office

P.O. Box 1927Watsonville, CA 95077

(831) 728-1356

San Jose Office991 West Hedding Street, Suite 102

San Jose, CA 95126(408) 247-8319

Many of the reports referenced in this presentation can be downloaded at www.appliedsurveyresearch.org

Questions and Discussion