Washington State’sWashington State’sExperience With Research-Based Experience With Research-Based

Juvenile Justice ProgramsJuvenile Justice Programs

May 19, 2005

Robert Barnoski, Ph.D.

(Barney)

Washington State Institute for Public Policy

www.wsipp.wa.gov

2

Washington State Institute for Public PolicyWashington State Institute for Public Policy

Created in 1983 by the state Legislature

Mission: Carry out non-partisan research on projects assigned either by the legislature or the Institute’s Board of Directors

8 legislators 4 higher education provosts or presidents 4 state agency directors

3

What We Will Talk About Today

This presentation describes Washington State's experiences implementing research-based programs

Some History

Outcome Evaluations

Quality Assurance

Cost Benefit

Cost Benefit Meta-Analysis

4

Juvenile Justice in Washington State

• Determinant sentencing since 1977; based on prior record and current offense

• Two systems: county vs. state– Each county runs juvenile court, detention, and probation– State runs juvenile correctional institutions, parole, and

distributes state funds to counties (Juvenile Rehabilitation Administration – JRA)

• Juvenile courts also manage non-offender cases: dependency, youth-at-risk, truancy, etc.

5

History of Legislation1995 Funding of intensive probation program.1996 Preliminary outcomes are not good.

1997 Determine if research-based programs exist and can be implemented in Washington State juvenile courts.Eliminate parole but establish intensive parole in JRA.

1998 Switch funding from intensive probation to research-based programs in juvenile courts.

Re-establish parole in JRA.

2002 Preliminary research-based program outcomes are positive and point to competent delivery.Parole and intensive parole outcomes are not good.

2003 Develop adherence and outcome standards for research-based programs.JRA moves toward research-based and Functional Family Parole (FFP).

Cost-benefit analyses of prevention and intervention programs beyond juvenile offenders.

6

1997: Community Juvenile Accountability Act (CJAA) Legislature said it would provide funding if research-based

programs could be identified

State, juvenile court administrators, and the Institute met regularly to identify “research-based” programs

Six programs were identified, four were recommended

1998: Legislature funded the programs ~$3.3 million/year

1999: Programs started

2001: Evaluation groups filled

2003: Evaluation completed

Initiation of Research-Based Effort in Washington State

7

1. Find research-based programs scientifically shown to work

2. Develop an assessment to identify the most appropriate program for each youth

3. Implement quality assurance to ensure services are delivered as designed

4. Conduct a valid outcome evaluation

Four Things Were Needed to Implement Research-Based Programs

8

Functional Family Therapy (FFT)

Aggression Replacement Training (ART)

Coordination of Services

Multi-Systemic Therapy (MST)

Multidimensional Therapeutic Foster Care

Adolescent Diversion

First Thing: Research-Based Program List

9

Functional Family Therapy($2,100 per youth)

• Blueprint Program: University of Colorado’s Center for the Study and Prevention of Violence

• Structured family-based intervention to enhance protective and reduce risk factors in the family

• FFT is a three-phase program Motivate family toward change Teach family how to change a specific critical problem Help family generalize their problem-solving skills

• Trained therapists have caseloads of 10 to 12 families

• Involves about 12 visits during a 90-day period

10

Aggression Replacement Training($700 per youth)

• Group training of 10 youth by two instructors

• Classes meet for one hour three times per week for 10 weeksAggression cycle Skill developmentMoral reasoning

• Training, not therapy, that uses guided discussion, modeling, role play, and homework

11

Coordination of Services($400 per youth)

• Educational program for low-risk juvenile offenders and their parents. The goals: Describe the consequences of delinquent behavior Stimulate goal setting Review the strengths of youth and family Explain what resources are available

• Participants given vehicle to open lines of communication and make shifts in thinking

• Community groups present participants with information concerning services they provide

12

Pre-screen for level of risk – more intensive service to higher risk youth and families; minimal for low risk

Comprehensive assessment only for higher risk youth – 10 life domains of risk and protective factors

Motivational interview to engage youth and family

Re-assess dynamic factors for progress – individual and groups

Common language for talking about youth and families

Second Thing: Build the Washington Juvenile Court Assessment

13

Mapping Problem to Intervention for Moderate to High Risk Youth

Problem Domain Intervention

Family (Primary) Functional Family Therapy (FFT), Multi-Systemic Therapy (MST), and Mentoring

No Family Multi-Dimensional Treatment Foster Care

Aggression Aggression Replacement Training (ART)

Impulsive Dialectical Behavior Therapy (DBT) or ART

Mental Health DBT or Family Integrated Treatment (FIT)

Sexual Behavior ? - ART, DBT, FIT

Alcohol/Drugs ? - ART, DBT, FIT

Attitudes ART, DBT

Skills ART, DBT

School, Employment, Free Time, Relationships

14

Case Management:Sequence to Self Reliance

If family has problems:Family =>

Engage & motivate family => Specific Problems

=> GeneralizationIf no family problems:Engage & motivate youth

=> Attitudes & Skills => Specific Problems

=> Generalization

15

Role of Probation in RehabilitationRole of Probation in Rehabilitation• Assess youth’s strengths and weaknesses

Assign youth to right caseload (low risk caseload)Assign youth to right program(s)Engage and motivate youth and family

– Courts: motivational interview and change cycle – JRA: Functional Family Probation

Support and help youth and family generalize what is learned during research-based program

Deflect enabling undesirable behaviorMonitor changes in risk and protective factors to see if approach is

making a difference and adjust

• Does not make job easier, but might make it more effective

16

Assessment Proliferation

• Must be something to WSJCA, since it is being used in at least a dozen states (including New York, Illinois, Florida, Utah, N. Dakota) and multiple courts – some in California

• Service providers particularly seem to like it

• We encourage continual refinement and improvement of the assessment; capitalizing on what has been done

• It is evolving to be a systematic collection of relevant social file information

17

Third Thing: Quality Assurance Infrastructure

• Steering committee

• Contract with program designers

• Statewide program experts

• Regional consultants

• Initial training

• On-going consultation, feedback, and training

• Assessment of competent program delivery

18

• A “rigorous” test Not enough funding for everyone, so “Waiting List” youth

assigned to a control group by court staff Includes completers and non-completers

• Youth’s eligibility for program is based on risk assessment Level of risk for re-offending Profile of risk factors

• Tests if probation plus a research-based program reduces recidivism more than probation alone

Fourth Thing: Evaluation Design

19

Control Group (N=313) Individual Therapists (N=387) Group Mean (Average)

18-M

ont

h U

nadj

uste

d F

elon

y R

ecid

ivis

m P

erc

ent

age

17

12

55

47

42

14

34

18

23

31 28

14

26

17

33

23

14

33

22

17 17

11

0

C 1 2 3 4 5 6 7 M 8 9 10 11 12 13 M 14 15 16 17 18 M 19 20 21 22 23 24 25 M

Not Competent

Borderline

Competent

Highly Competent

8

18 20

43

47

63

26

FFT ResultsFFT Results

20

6%

25%

49%

27%

7%

50%

7%

19%

45%

Misdemeanor and Felony Recidivism

Felony Recidivism Violent Felony Recidivism

18-M

onth

Rec

idiv

ism

Per

cent

age

Control (N=525) Not Competent (N=203) Competent (N=501)

ART ResultsART Results

21

19%

3%

17%

1%

Felony

12-M

onth

Rec

idiv

ism

Per

cent

age

Control (N=171) COS (N=171)

Adjusted 12-Month Recidivism

Misdemeanor and Felony

Coordination of Services Results

22

• FFT delivered competently Reduces 18-month felony recidivism by 38 percent Benefit to cost ratio of $7.69

• ART delivered competently Reduces 18-month felony recidivism by 24 percent Benefit to cost ratio of $12.60

• Coordination of Services Reduces 12-month felony recidivism by 57 percent Benefit to cost ratio of $13.58

• MST No findings because of problems implementing the

Institute’s evaluation design

Findings From Initial Studies

23

A Clear Lesson…

The key to reducing recidivism with research-based programs is

competently delivering the service.

24

Parole

Intensive Parole

Local Parole

Local Commitment

Group Homes

------------------------------------------

Basic Training Camp

Dialectic Behavior Therapy

Mentoring Program

Family Integrated Transitions

Additional JRA Programs We Were Asked to Evaluate

25

-$12,478-$4.95$2,098-$10,379Regular Parole (v. not having parole)

-$5,992$0.00$5,992$0Intensive Parole

-$1,500$0.00$1,500$0Intensive Probation Supervision

$5,073$1.78$6,471$11,544Mentoring

$5,128$13.58$408$5,536Interagency Coordination

$8,805$12.60$759$9,564Aggression Replacement Training

$14,315$7.69$2,140$16,455Functional Family Therapy

$19,247$3.15$8,968$28,215Family Integrated Transitions

$22,364n/a-$7,586$14,778Washington Basic Training Camp

$31,243$38.05$843$32,087Dialectical Behavior Therapy

Benefits Minus Costs

Benefits per

Dollar of Cost

CostsBenefits

Measured Benefits and Costs Per YouthCost Benefit Estimates for Washington State Programs as of September 17, 2004

26

Benefits and Costs of Prevention and Early Intervention Programs for Youth

September 2004

Steve Aos et al. Washington State Institute for Public Policy

www.wsipp.wa.gov

2003 Legislature Starts to Repeat Process for Other Intervention and

Prevention Programs

27

• Question: Are there Question: Are there “research-based”“research-based” programs programs or policies with a or policies with a “real world”“real world” ability to: ability to:

• Reduce crime,• Lower substance abuse,• Improve educational outcomes,• Decrease teen pregnancy,

• Reduce teen suicides?• Lower child abuse or neglect, or

Research Questions & MethodsResearch Questions & Methods

• The six outcomes• The quality of a study’s research design• Whether the program is “real world”

• We screened evaluations written in English for:We screened evaluations written in English for:

• We computed effects (meta-analytically)We computed effects (meta-analytically)

• We then calculated We then calculated monetarymonetary benefits and costs benefits and costs

28

Summary of Benefits and Costs (2003 Dollars)Summary of Benefits and Costs (2003 Dollars)Dollars Per Youth (PV lifecycle) BenefitsBenefits CostsCosts B B -- C C

Early Childhood Education $17,202 $7,301 $9,901

Nurse Family Partnership $26,298 $9,118 $17,180

Functional Family Therapy $16,455 $2,140 $14,315

Life Skills Training $746 $29 $717

Seattle Soc. Dev. Project $14,246 $4,590 $9,837

Guiding Good Choices $7,605 $687 $6,918

Multi-D Treat. Foster Care $26,748 $2,459 $24,290

Intensive Juv. Supervision $0 $1,482 -$1,482

Big Brothers/Sisters (all costs) $4,058 $4,010 $48 (taxpayer costs only) $4,058 $1,283 $2,775

Selected FindingsSelected Findings

Multi-Systemic Therapy $14,996 $5,681 $9,316

29

Managing Competent Program Delivery

Invest in increasing skills, not accountability

• Pick good people

• Provide required training

• Enhance skills

• Retain only competent providers

30

• Written knowledge test and feedback after initial training

• Follow-up skill training and consultations

• Certification at end of probationary period

Initial Provider Training

31

• Form regional support teams

• On-site direct observation by an expert, including audio- or video-taping to improve competence

• Case-by-case consultations

• Give feedback

Continually Enhance Provider Skills

32

Please place a check in the box that best describes what happened in this session. YES▼

NO ▼

1. Was a positive climate established through welcoming students?

2. Were group norms reviewed, emphasizing positive participation?

3. Were all ACT concepts covered to this point reviewed?

4. Were issues from the last anger control training session reviewed?

5. Did all youth complete the hassle log(s)?

Skill Feedback

33

• Establish process for corrective action

• Corrective action taken when needed

• Building morale; taking pride that we are good!

Ensure Competent Providers Are Retained

34

• Youth eligible, assigned, and completing programs (75%)

• Changes in targeted risk and protective factors

• Program provider competency ratings

• Meeting expected recidivism rates

Annual Legislative Accountability

35

“Let’s get going on one of these programs.”

Some of You May Be Thinking…

-or-

“We already do a great job, so why would we be interested in this?”

36

• Understand what is currently being done

• Understand what is being done in research-based programs

• Examine the differences Are you already doing itDoes it seem like an improvementWhat needs to be tested

• Implement quality assurance for current practice

• Have competent researcher do valid outcome evaluations

If You Are Not Comfortable With the Research Approach, a Possibility…

37

Essential Ingredients

• Infrastructure for program fidelity• Funding and de-funding contingent

on continued outcomes

• Leadership• Legislation• Funding pilot research-based programs • De-funding ineffective programs

• Juvenile court involvement (administrators, supervisors, and staff)

• Local provider involvement• Skilled independent

researchers

• Assessment • Research-based programs

• Valid outcome evaluations

• Cost-benefit analysis