Application of GTO to Implementation (Including

Training) and Evaluation of a Child Welfare Practice Model

Anita P. Barbee, MSSW, Ph.D.University of Louisville

anita.barbee@louisville.edu

Other Colleagues that Contribute to this Work

• Dana Christensen, Ph.D.• Becky Antle, MSSW, Ph.D.• Riaan van Zyl, Ph.D.• Abe Wandersman, Ph.D.• Katherine Cahn, MSW, Ph.D.

Implementation of Casework Practice Models

• Many state child welfare agencies are adopting or creating casework practice models

• The purpose is to shape the thinking and behavior of front line child welfare workers in order to improve safety, permanency and well-being.

• This is a huge implementation undertaking and a good one with which to apply the GTO model and EBSIS.

Past Experience

• Most states rely primarily on training to execute a practice model.

• We conducted a number of studies on the impact of training SBC to supervisors and their teams on reactions, learning, transfer and outcomes.

• Thus, we found that training can have an impact on practice and outcomes, particularly with reinforcement (TA).

• But, while training is a necessary activity in implementing a practice model it is not sufficient

Accountability Questions Relevant Literatures

1. What are the underlying needs and conditions that must be addressed? (NEEDS/RESOURCES)

1. Needs/Resource Assessment

2. What are the goals, target population, and objectives? (i.e., desired outcomes)? (GOALS)

2. Goal Setting

3. What science (evidence) based models and best practice programs can be used in reaching the goals (BEST PRACTICE)?

3. Consult Literature on Science Based and Best Practice Programs

4. What actions need to be taken so the selected program “fits” the community context? (FIT)

4. Feedback on Comprehensiveness and Fit of Program

5. What organizational capacities are needed to implement the program? (CAPACITIES)

5. Assessment of Organizational Capacities

6. What is the plan for this program (PLAN) 6. Planning

7. Is the program being implemented with quality (PROCESS)

7. Process evaluation

8. How well is the program working? (OUTCOME EVALUATION)

8. Outcome and Impact Evaluation

9. How will continuous quality improvement strategies be included? (IMPROVE)

9. Total Quality Management; Continuous Quality Improvement

10. If the program is successful, how will it be sustained? (SUSTAIN)

10. Sustainability and Institutionalization

GTO as a Painter’s Palette

#1Needs/

Resources

#2Goals

#3Best

Practices

#9Improve/

CQI#8

OutcomeEvaluation

#7Implementation

Process Evaluation

#4Fit

#5Capacities

#6Plan

#10Sustain

RESULTS

Kentucky Example• Early in the development of SBC agency leadership

determined that there was a need for a unifying model from which to practice child welfare, goals were set, a model was developed based on best practices and fit to meet the needs of Kentucky clients.

• The biggest facilitator of the adoption of the SBC practice model was developing capacities- of tools to execute the practice model, policies to support the practice model, including the model in the SACWIS system, and extensive training and coaching (TA) of managers and supervisors as well as workers in the model (Three of the Supports needed for Implementation: Tools, Training, TA)

• Furthermore, process and outcome evaluations were conducted as the model rolled out and as supervisors and their teams were trained and then 33 items of the CQI case review tool measured adherence to the SBC model (4th support).

• All of these supports helped sustain the model through the first administration (8 years) and into the second (4 years). By the third administration, however, adherence had dropped to 30%.

• However, adherence led to successful completion of all CFSR measures of safety, permanency and well-being.

Support System Model

=+To

AchieveDesired

Outcomes

Actual Outcomes Achieved

Current Level of Capacity

Innovation

Training

TA

QI/QA

Tool

TOWARD AN EVIDENCE BASED SYSTEM FOR INNOVATION SUPPORT (EBSIS)

To AchieveDesired

Outcomes

Training +

QI/QA +

Tools +

TA +

=Current Level of Capacity

+

GTO Steps: (1) Needs & Resources; (2) Goals & Desired Outcomes; (3) Science-based practices; (4) Fit; (5) Capacity ; (6) Plan; (7) Implementation & Process Evaluation; (8) Outcome evaluation; (9) Continuous Quality Improvement; and (10) Sustainability

Actual Outcomes Achieved

Levels & AccountabilityAccountability Question Children’s

BureauState Region Supervisors Workers

1. NEEDS/ RESOURCES Funding Funding Leadership More hired and getting MSWs

More hired, pay increased, PCWCP

2. GOALS Laws, PIP SP, PIP, CD SP, PIP Help define goals Help define

3. EVIDENCE-BASED PRACTICES

Family Centered, Strengths Based..

Funded development

Piloted Helped develop, piloted, first trained

Gave feedback

4. FIT CB T/TA Network can help states

Since we created it fit

Slight variations across state

Provided leadership Helped in pilot

5. CAPACITY Systemic factors Assess readiness to change, infrastructure

Share good news as spread across Regions

Change agents, champions of change

Change agents, champions of change, EBP readiness

6. PLAN PIP, Community partners

Training PlanSupport Plan

Leaders, coaches, pilot

Train early and then with workers, coach

Train veteran then new workers

7. IMPLEMENTATION CQI, Fidelity Change policy, tech, tools

Ensure fidelity

Track data to assure fidelity in practice

Hold accountable, give support

8. OUTCOME EVALUATION

Funding Include eval in train contract

Exp vs Comparison

Sup training evaluated

Examined Sup worker perform

9. CQI CFSR processes, outcomes and systemic factors

CQI tool included 33 SBC items

CQI specialists, CQI teams

Conduct CQI case reviews, on CQI teams

On CQI teams

10. SUSTAINABILITY CFSR keeps coming, Funding

Build deep bench

Engage community

Refreshers, coaching consults

New worker training, tools

REFERENCES• Martin, M. H., Barbee, A. P., Antle, B., & Sar, B. (2002). Expedited permanency planning: Evaluation of the Kentucky Adoptions Opportunities Project (KAOP). Child Welfare: Special Issue on Permanency Planning, 81, 203- 224.• Barbee, A. P., Antle, B. F., & Martin, M. (2003). Supervisor- Team Training: Issues in Evaluation (pp. 136-137). B Johnson, K. Keitzman & K. Ringuette (Eds.). Proceedings of the Fifth Annual National Human Services Training Evaluation Symposium: 2002. Berkeley, CA: CalSWEC.

• Antle, B. F., Christensen, D., Barbee, A. P., Martin, M. (2008). Solution-based casework: A paradigm shift to effective, strengths-based practice for child protection. Special Issue on Evidence Based Practice. Journal of Public Child Welfare, 2, 197-227.

• Antle, B. F., Barbee, A. P., van Zyl, M. A. (2008). A Comprehensive Model for Child Welfare Training Evaluation. Children and Youth Services Review, 9, 1063- 1080. • Antle, B.A., Barbee, A.P., Sullivan, D.J., & Christensen, D. (2009). The Prevention of Child Maltreatment Recidivism through the Solution-Based Casework Model of Child Welfare Practice. Children and Youth Services Review 31,1346-1351.

• Antle, B. F., Sullivan, D. J., Barbee, A. P., Christensen, D. N. (2010). The Effects of Training Methodology on Training Transfer. Child Welfare.

• van Zyl, M. A., Antle, B. F. & Barbee, A. P. (2010). Organizational change in child welfare agencies. In S. Fogel, M. Roberts-DeGennero (Eds). Empirically Supported Interventions for Community and Organizational Change. New York: Lyceum Books.

• Barbee, A. P., Christensen, D., Antle, B., Wandersman, A., Cahn, K. (2011). System, organizational, team and individual changes that need to accompany adoption and implementation of a comprehensive practice model into a public child welfare agency. Children and Youth Services Review, 33, 622-633.

• Antle, B.F., Christensen, D.N., van Zyl, M.A., & Barbee, A.P. (in press). The Impact of the Solution Based Casework (SBC) Practice Model on Federal Outcomes in Public Child Welfare. Child Abuse and Neglect.

• Barbee, A. P., & Liz Winter (in press). Skill Based Training and Transfer of Learning. In H. Cahalane (Ed). Clinical Social Work Practice. Springer.