evaluating the effectiveness of citizens review panels blake l. jones, msw, lcsw program coordinator...

24
Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels [email protected] www.uky.edu/socialwork/trc

Upload: dominick-hicks

Post on 18-Dec-2015

216 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Evaluating the Effectiveness of Citizens Review Panels

Blake L. Jones, MSW, LCSW

Program Coordinator

Kentucky Citizens Review Panels

[email protected]

www.uky.edu/socialwork/trc

Page 2: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

"Never doubt that a small group of thoughtful, committed citizens can change the world; indeed it is the

only thing that ever has."  

Margaret Meade

Page 3: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

“Service is the rent we pay forliving. It is the very purpose of lifeand not something you do in your spare time."

~ Marian Wright Edelman

Page 4: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

“This whole citizens review thing is nothing but a public relations ploy”

~Anonymous CRP member

Page 5: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

“We’re not really helping kids. All we’re doing is just generating another report that CPS won’t use!”

~Survey Respondent

Page 6: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Objectives

National Update

Dissertation proposal

What are YOUR thoughts one measuring CRP effectiveness?

Page 7: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

How are state’s using CRP’s?

Great Variability> Numbers (Alaska: 5 volunteers, New Mexico: several

hundred)

> Staff Support

> Tasks

Recruitment vs. Appointment

Page 8: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Examples of Models

Created new panels (KY, Tenn.) > contract with Universities, other governmental agencies

Using existing panels (I.e., Child fatality review boards, regional or county QA teams, Governor’s task force teams). This appears common.

Hybrid (create new panels, but coordinate with larger group of existing panels)

Some states—Maryland, for example--have long history of “citizen review panels.”

Page 9: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

What are Panel’s Doing?

Working on legislation issues (I.e, dual track response)

Examining community collaboration with CPS (through surveys, focus groups)

Evaluating state budget cuts on social services

Case reviews (I.e, looking for family involvement in case planning)

Employee satisfaction

Mandated reporters Administrative tasks

(by-laws) CPS’ involvement with

immigrant populations CPS’ relationship with

schools, law enforcement, mental health providers

Policy and Procedure (I.e., exit interviews)

Page 10: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

CRPs used in the CFSR process

Minnesota Alaska (informal) Kentucky Wyoming South Carolina California (citizens reading PIP) Maryland (maybe?) Oregon Others?

Page 11: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Is it Working? Mixed results

Citizens having difficulty in defining their role, staying on track (easier with foster care review boards)

Seem to have changes on the local level (I.e., examining local policy and procedure)

Difficulty in recruiting diverse membership

Budget considerations (some recommendations not financially feasible)

Citizens want more feedback

Page 12: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Comments

“There is very little feedback from CPS so these panels rightly feel like their work is unappreciated.”

“ . . . Some team members are concerned that (their report) becomes so much paper in some big building”

“Some members do not have the basic skills to research a problem, develop a plan, begin working on it and track its progress”

“It makes no sense to me that an ‘objective’ body that is supposed to be evaluating a government agency would be housed within that agency”

Page 13: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Evaluation of Citizen Boards

Mostly focused on “outcomes” (i.e., Litzelfelner and CASA research)

Most researchers have found positive results from citizens boards

Perceptions of outside reviewers generally positive, though some distrust

reviewers (Leashore)

Page 14: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

“Unauthentic” vs “Authentic” participation

King, Felty & Susel (1998)

Unauthentic• Conflictual• Input sought “after

the fact”• Reactive• Citizen treated as

necessary evil• Mistrust

Authentic• Collaborative• Input sought before

decisions are made• Proactive• Citizen treated as

partner• Trust

Page 15: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Difficulties in doing traditional program evaluations on CRPs

Wide variability in groups

Hard to measure impact of CRPs on child welfare systems (need longitudinal study)

What are “outcomes”? (reports?)

Shifting child welfare priorities (federal and state)

Page 16: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Dissertation Study

Page 17: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

States Involved

Wisconsin Florida Alaska New York Alabama Ohio Tennessee Maryland Georgia

New Hampshire New Mexico South Carolina Arkansas Wisconsin North Carolina Wyoming West Virginia Nevada Minnesota

Page 18: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

CRP Members and CPS Staff surveyed

Looking for:

> timely access to info

> training

> chairperson

> frequent contact with liaison

> ability to impact policy decisions

> Feedback from CPS

Page 19: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Arnstein’s Ladder

Where does YOUR citizens review panel fall on the “ladder”

Hypothesis: there will be a significant difference

Page 20: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Delegated Power: Citizens on your panel have the needed power to actively change the child protective system Partnership: The child protection system allows citizens to share in decision making, but retains all the power to change things  Placation: The system just “tells you want they want you to hear” about child protection in your state  Consultation: The child welfare system consults with citizens but does not give them any power Informing: The child welfare administrators in your state engage in “one way” communication with citizens (i.e., by providing superficial answers or discouraging questions) Manipulation: The child welfare system uses the citizens review panels to push its own agenda 

Page 21: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Other Variables

Previous leadership/volunteer experience

Gender, ethnicity (“representative” of community?)

Content of report/response to report

Page 22: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

Limitations

Studying perceptions only

Point in time study (need longitudinal study to assess organizational change)

Generalizable nationwide?

Page 23: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu
Page 24: Evaluating the Effectiveness of Citizens Review Panels Blake L. Jones, MSW, LCSW Program Coordinator Kentucky Citizens Review Panels bljone00@uky.edu

What are

YOUR

Thoughts?