Putting CQI Into ActionStarting Early TogetherAllegheny County, PA

(Pittsburgh)

Sarah Thurston, Quality Assurance Team

Brief Background

• Allegheny County has received three system of care grants since 1998– Through the grants, have served 0-25 yr olds

in nine neighborhoods– First two grants have graduated from funding

and have transitioned to other models (i.e. billing)

– Current grant is Starting Early TogetherStarting Early Together, early childhood focus serving 0-6 yr olds

1998 Grant

2001 Grant

CQI plan built around values-based service delivery process

2005 Grant SET

Pursued High Fidelity Wraparound service delivery model

2009Discontinued credentialing for HiFi

Intermediate Outcomes (Years 2-4)

Partner Community LevelCommunity Network Teams are fully developed and active

Program staff are skilled and well-trained Children from diverse settings and racial/ethnic backgrounds are referred to and enrolled in local programsLocal programs enroll 25 children/yearCommunity Network Teams identify and develop local services/supports/resourcesService planning efforts engage all key partners and result in sustainable plansLocal programs develop and begin implementing sustainability plansParticipation in evaluation activities is strong

Practice/System LevelSocial marketing plan is implemented

Provider trainings are well-attendedUse of evidence-based practices increasesInteragency transition plans are in placeSOCI staff provide regular updates to key representatives from the PA Office of Child Development and Commission on Children and FamiliesSOCI staff actively participate in local early childhood collaborative meetings SOCI staff actively participate in federal system of care training/TA opportunitiesSOCI staff actively support local system integration initiatives

IMPACT(Years 5&6)

Partner Community LevelCaregiver/child/family relationships are strengthened and functioning improves

Family/consumer satisfaction with the system of care is highChildren/families receive appropriate and effective services, treatments and supportsDisparities in access to mental health services decline among racial minorities and children in foster careFamilies are empowered and support each otherLocal infrastructure for SOCI program is fully developed and self-sustaining

Practice/System LevelPublic awareness of early childhood mental health issues and effective mental health promotion strategies increase

Provider capacity to identify and treat/address early childhood mental health issues in an effective and culturally-competent manner increaseFamily/provider awareness of formal/informal services and how to access them increaseService/system integration and transition practices improveFederal/state/local partnerships are developed

MEAR

CART & National Outcomes Study: YSS-F, MSSC

National & Local Outcomes Study: CGSQ, PSI

National Outcomes Study: MSSC, CCSP

Local MIS

Local MIS

Tracking of committee memberships & activities

Family & provider focus groups

Informal agency self-report

“Stuck” in the middle with you…

CQI Process

So what next?

Intermediate Outcomes (Years 2-4)

Partner Community LevelCommunity Network Teams are fully developed and active

Program staff are skilled and well-trained Children from diverse settings and racial/ethnic backgrounds are referred to and enrolled in local programsLocal programs enroll 25 children/yearCommunity Network Teams identify and develop local services/supports/resourcesService planning efforts engage all key partners and result in sustainable plansLocal programs develop and begin implementing sustainability plansParticipation in evaluation activities is strong

Practice/System LevelSocial marketing plan is implemented

Provider trainings are well-attendedUse of evidence-based practices increasesInteragency transition plans are in placeSOCI staff provide regular updates to key representatives from the PA Office of Child Development and Commission on Children and FamiliesSOCI staff actively participate in local early childhood collaborative meetings SOCI staff actively participate in federal system of care training/TA opportunitiesSOCI staff actively support local system integration initiatives

We already

regularly view &

discuss data from the local

MIS

How do we convert more

referrals to enrollments?

Build new partnerships or utilize existing ones to

accomplish data collection – you don’t have to do it all

yourselves!

Local academic institutions

Agencies/providers

Community groups

So we called in our university partner…

And we built a plan.

ACTION: Tweaking

data collection

MIS DATA: # of Referrals, # of Enrollments

PHASE 1: Focus groups and interviews

RESULTS: Problems with

communication

ACTION: Spokesperson

training

MIS DATA: # of Referrals, # of Enrollments

PHASE 2: Phone

interviews

RESULTS: Problems with # of contacts

Shared with

admin staff

Shared

with

Team

Lea

der

staf

f

Ongoing PHASE 2: Phone interviews

with MIS data

And so on, and so on…

Are staff implementing the process appropriately?

Are we reaching families where they are?

SET Stoplight

Report

SET ReferralReport

Build in regular and consistent opportunities for data sharing,

feedback from stakeholder groups, and creation of action

plans.Community Evaluation Team Meeting

Operations Meeting

Advisory Committee Meeting

23%

9%

6%

5%

5%

5%5%

38%

Caregiver, 38%

Early Intervention, 23%

CYF, 9%

MH provider, 6%

Head Start, 5%

Phys Health Provider, 5%

Family Support Ctr, 5%

Other (school, childcare, etc),5%

0

2

4

6

8

10

12

14

Jan 08 Feb 08 Mar 08 Apr 08 May 08 Jun 08 Jul 08 Aug 08 Sep 08 Oct 08 Nov 08 Dec 08

Referred Enrolled Ever Enrolled

REFERRAL

REPORT

ST

OP

LIG

HT R

EP

OR

T

Team Leaders use reports as a staff supervision tool

Administrators use reports as program

monitoring and systems

discussion tool

IMPACT(Years 5&6)

Partner Community LevelCaregiver/child/family relationships are strengthened and functioning improves

Family/consumer satisfaction with the system of care is highChildren/families receive appropriate and effective services, treatments and supportsDisparities in access to mental health services decline among racial minorities and children in foster careFamilies are empowered and support each otherLocal infrastructure for SOCI program is fully developed and self-sustaining

Practice/System LevelPublic awareness of early childhood mental health issues and effective mental health promotion strategies increase

Provider capacity to identify and treat/address early childhood mental health issues in an effective and culturally-competent manner increaseFamily/provider awareness of formal/informal services and how to access them increaseService/system integration and transition practices improveFederal/state/local partnerships are developed

CART & National Outcomes Study: YSS-F, MSSC

Based on National

Outcomes data, families were reporting a

high level of satisfaction

with services. But were the study tools capturing

enough about family/SET staff

interactions?

Are families satisfied with SET staff??

So we partnered with an existing Allegheny County project of NAMI

(National Alliance on Mental Illness) …

And we built a plan.

CART Consumer Action Response Team

Phone interviews with

enrolled & recently

graduated families

RESULTS: Confusion about

‘graduation’

ACTION 1: Training for

families about graduation

Shared with

admin &

office staff

And so on, and so on…

Develop survey with CART staff

ACTION 2: Each office

wrote a plan of correction for

any other issues

Revise survey with CART staff

JULY 2010: Phone interviews with

enrolled & recently graduated families

Referrals to Enrollments?

Referral Details?

Service Delivery?

Service Satisfaction?

What is the SET experience?

as often as they will come.

Short term = more

energy, more time

Long term = more

efficient, less time

Bring as many people to the table as possible…

Make your efforts serve multiple purposes…

be the “Swiss Army Knife” of CQI / Evaluation!