ccs 2015 monitoring report › publications › p01224-15.pdf · ccs programs are designed to be...

Comprehensive Community Services 2015 Monitoring Report

P-01224-15 (06/2018)

i

Table of Contents List of Figures ............................................................................................................................................ ii List of Tables ............................................................................................................................................. ii Executive Summary ................................................................................................................................... 1 Introduction .............................................................................................................................................. 2

Data Sources .................................................................................................................................................... 3 CCS Regional Models ................................................................................................................................. 4 The Program .............................................................................................................................................. 7

CCS Utilization .................................................................................................................................................. 7 Medicaid Recipients and Expenditures .......................................................................................................... 11

Participant Characteristics ....................................................................................................................... 13 Gender ........................................................................................................................................................... 13 Age ................................................................................................................................................................. 14 Race and Ethnicity .......................................................................................................................................... 16 Veteran Status................................................................................................................................................ 16 Substance Use ................................................................................................................................................ 17 Medical Conditions ........................................................................................................................................ 18 Primary Diagnoses ......................................................................................................................................... 19

Program Characteristics ........................................................................................................................... 21 Program Staffing ............................................................................................................................................ 21 Evidence-Based Practices............................................................................................................................... 21 Waiting Lists ................................................................................................................................................... 29 Suicide Risk Assessment and Protocol ........................................................................................................... 31

Participant Transitions and Length-of-Stay ............................................................................................... 32 Participant Transition Reasons ...................................................................................................................... 32 Participant Transition Destinations ............................................................................................................... 33 LOS for Participants in CCS ............................................................................................................................. 34

Participant Hospitalization Rates ............................................................................................................. 38 Participant Satisfaction ............................................................................................................................ 39

Survey Responses........................................................................................................................................... 39 Adult Satisfaction ........................................................................................................................................... 41 Youth Satisfaction .......................................................................................................................................... 42 Family Satisfaction ......................................................................................................................................... 44 Satisfaction Survey Summary ........................................................................................................................ 45

Appendix I – Sources for National Health Estimates ................................................................................. 46 Appendix II – 2015 CCS Program Survey ................................................................................................... 47 Appendix III – Participant Satisfaction Surveys History ............................................................................ 75

Methods and Scale Calculation ...................................................................................................................... 75 Satisfaction Survey Scales .............................................................................................................................. 75 Instruments .................................................................................................................................................... 81

References .............................................................................................................................................. 89

ii

List of Figures Figure 1: Map of CCS Regions as of October 31, 2015 ........................................................................................... 6 Figure 2: Number of DQA-Certified CCS Programs, 2005-2015.............................................................................. 7 Figure 3: Age Composition of CCS Participants Served, 2015 .............................................................................. 14 Figure 4: Substance Use in Wisconsin and among CCS Participants (2012-2015) ............................................... 17 Figure 5: Rates of Various Health Conditions among CCS Participants, 2012-2015 (With National Adult Prevalence Rate) ................................................................................................................................................... 19 Figure 6: Percent of CCS Programs Offering and Percent of Adult Participants Receiving Each Evidence-based Practice (EBP), 2015 .............................................................................................................................................. 22 Figure 7: Percent of CCS Programs Offering Each EBP and Percent Difference, 2014-2015................................ 23 Figure 8: Percent Adult Participants Served Under Fully Implemented EBP (versus Partially Implemented), 2015............................................................................................................................................................................... 24 Figure 9: Percent of CCS Adult EBPs Implemented with Fidelity Checks ............................................................. 25 Figure 10: Percent CCS Programs Offering each EBP and Percent of Youth Participants Receiving Each EBP .... 27 Figure 11: Percent Youth Participants Served Under Fully Implemented EBP (versus Partially Implemented), 2015 ...................................................................................................................................................................... 28 Figure 12: Percent of CCS Youth EBPs Implemented with Fidelity Checks, 2015 ................................................. 29 Figure 13: Reasons for CCS Participant Transition, 2015 ...................................................................................... 33 Figure 14: Count of CCS Program Participants by Length-of-Stay (LOS) in Months, Statewide 2015 (Any LOS) . 36 Figure 15: Count of CCS Program Participants by Length of Stay (LOS) in Months, Statewide 2015 .................. 37 Figure 16: CCS Persons Hospitalized Rate among 2015 Transitions ..................................................................... 38

List of Tables Table 1: CCS Point-in-Time End of Year Counts - 2013 to 2015 ............................................................................. 7 Table 2: CCS Referrals, Transitions, and Point-in-Time Caseload – 2015 ............................................................... 8 Table 3: CCS and Community Support Programs: Counts Served and Transitions from Community Support Programs to CCS, 2013-2015 ................................................................................................................................ 10 Table 4: CCS Medicaid Count of Recipients and Expenditures Paid to Counties and Tribes, 2012-2015 ............ 12 Table 5: CCS Participants Served by Age, by County, 2015 .................................................................................. 15 Table 6 (A. and B.): Most Common Primary Diagnosis among CCS Participants .................................................. 20 Table 7: Other Evidence-Based Practices (EBPs) Used by CCS Programs, 2015 ................................................... 23 Table 8: Waitlist Status among CCS Programs with a Waitlist, 2013-2015 .......................................................... 30 Table 9: Interim Services used by CCS Programs with Waitlists in 2014-2015 .................................................... 30 Table 10: CCS Programs Overall Quality of Suicide Assessment and Protocol, 2015 ........................................... 31 Table 11: CCS Programs Reporting on Suicide Risk Assessment Tools, 2015 ....................................................... 31 Table 12: CCS Participant Transition Destinations by Transition Reasons, 2015. ................................................ 34 Table 13: Description of CCS Length-of-Stay (LOS) in Months, Statewide 2015 .................................................. 35 Table 14: Description of CCS Length of Stay (LOS) in Months, Statewide 2015 ................................................... 36 Table 15: Number of CCS Participant Satisfaction Surveys Returned, 2015 ........................................................ 40 Table 16: Average Scale Scores and Percent of Adult Participants Reporting a Mostly, Mixed, and Less Recovery-Oriented Experience in CCS, 2015 ........................................................................................................ 41 Table 17: Average Scale Scores and Percent of Youth Participants Reporting a More, Mixed, and Less Positive Experience in CCS, 2015 ........................................................................................................................................ 43 Table 18: Average Scale Scores and Percent of Family Participants Reporting a More, Mixed, and Less Positive Experience in CCS, 2015 ........................................................................................................................................ 44

1

Executive Summary Comprehensive Community Services (CCS) is one of the supportive programs counties and tribes utilize to provide mental health and substance use services. Available to Wisconsin counties and tribes since 2005, CCS is for individuals who need ongoing services beyond occasional outpatient care, but less than the intensive care provided in a hospital setting. CCS is unique in that services include both mental health and substance use disorder services and that they are available to people of all ages across their lifespan. Since July 2014, counties and tribes have been encouraged to form regions to provide CCS. These arrangements allow the counties and tribes to receive state funds to pay for the nonfederal share of Medicaid. The number of CCS certified programs and people served increased in 2015. In 2015, the Wisconsin Department of Health Services (DHS), Division of Quality Assurance (DQA), certified 37 CCS programs. At the end of 2015, there were 64 certified programs spread across 22 regions. Close to 1,500 more people were served and $19.5 million more was billed to Medicaid in 2015 compared to 2014, driven largely by the state initiative to provide funding to counties that offered CCS in a regional model. CCS can exist in collaboration with Community Support Programs. Concerns about the impact of the CCS expansion and funding structure on the declining participation in Community Support Programs (6 percent from 2013 to 2015) were smaller than expected in most CCS regions. In fact, several CCS regions continued to see increases in people served by Community Support Programs. Most participants benefit from CCS participation as shown by low hospitalization rates. The reported incidence of hospitalizations was low among CCS participants. Five percent of participants were hospitalized while in CCS. Only 3 percent were hospitalized following their transition from CCS. Nearly one-third transitioned out of CCS because they recovered. Among the 795 participants for whom a transition reason was reported in 2015, 30 percent left because they had recovered to the extent CCS was no longer needed, 20 percent withdrew from CCS, 20 percent moved, and 10 percent needed additional services beyond what CCS could offer. Many transitioned CCS participants continue services at a lower level of care. In 2015, 43 percent of those transitioned from CCS for any reason sought services in outpatient therapy and psychiatry. A transition to lower-level care is an expected outcome of CCS. Many transitioned CCS participants were enrolled in the program for at least a year. Over one-third of CCS participants continued in services anywhere from one to 3 1/2 years, even though over half chose to leave after relatively short periods of time in the program (less than one year). Participants are satisfied with their services, but there is still room for improvement. Satisfaction surveys showed most adult, youth, and families (that is, parents or caregivers of children under 13) were satisfied with CCS, though adults had somewhat lower levels of overall satisfaction when compared to youth and families.

2

Introduction CCS is a recovery-focused, integrated program that provides individualized, community-based rehabilitation services for adults with serious mental illness and children with severe emotional disturbance, as well as for individuals with substance use and co-occurring mental health and substance use disorders. To the greatest extent possible, CCS intends to restore participants’ levels of functioning and wellness. The majority of CCS services are provided in the participant’s home and local community by a team of professionals, peer specialists, and natural supports, all coordinated by a CCS service facilitator. CCS programs are certified services governed per the requirements of Wis. Admin. Code ch. DHS 36. CCS services are eligible for Medicaid reimbursement for those who qualify for Medicaid and whose services fall within the federal definition of rehabilitative services under 42 CFS § 440.130(d). Eligibility for CCS is based on a person’s level of need determined by a DHS-approved Functional Eligibility Screen for Mental Health and Alcohol and Other Drug Abuse Services. Under Wis. Admin. Code § DHS 36.15, certified programs are authorized to provide mental health and substance abuse services to participants across the lifespan who require more than outpatient counseling but less than an intensive wraparound psychosocial rehabilitation program. In addition to a mental health, substance use, or co-occurring mental health and substance use disorder, participants must have an impairment that limits one or more major life activities, which results in the need for services. Health care providers may refer potential participants to their county or tribe. Each county or tribe determines its access point for CCS and has policies and procedures on referral and screening for the program. Once potential participants are evaluated through a screening and eligibility process, they are notified of their eligibility and services available through CCS. Following their agreement to participate, they are then provided appropriate services. CCS programming is individualized in that it must be tailored to each participant’s needs and recovery goals as identified through a comprehensive assessment and service plan. CCS programs are designed to be community-based, enhance participant’s recovery and satisfaction, and continuously seek quality improvement. First, CCS programs develop the service array to interface with and enhance available behavioral health and crisis services in their community. Second, CCS programs focus on quality improvement by monitoring participant satisfaction and progress toward their desired outcomes. Third, CCS programs appoint a coordination committee comprised of various stakeholders and develop and implement a quality improvement plan to evaluate the effectiveness of CCS and incorporate the feedback of participants and the committee into this plan. This CCS 2015 Monitoring Report will describe: • The structure, growth, and utilization of the program across Wisconsin and in comparison to Community

Support Programs. • Numbers and characteristics of participants enrolled in and served by CCS. • CCS Medicaid recipients and expenditures. • Services offered through CCS, including evidence-based practices (EBPs) and suicide risk assessment. • Participant transition reasons and destinations, hospitalization rates, and length-of-stay (LOS). • Satisfaction with CCS services among adult, youth, and parents or caregivers of child participants.

3

Data Sources Program Survey In 2016, the DHS Division of Mental Health and Substance Abuse Services (now known as the Division of Care and Treatment Services) administered the 2015 CCS Program Survey resulting in 59 returned surveys, one for each eligible CCS program.i Each year, all certified CCS programs are provided with, and asked to complete this program-level, self-report survey.ii Programs report on program characteristics, services provided to their participants, and characteristics of participants served during the previous year among various other data points described in this report. The CCS program survey covers similar topics and, for many survey items, collects the same information using identical survey language as the longstanding survey administered to Community Support Programs providers, allowing for comparison of data across these programs. While CCS and Community Support Programs survey data is available as far back as 2011 and 1994, respectively, data in this report highlights CCS data from 2012-2015 and Community Support Programs data from 2013-2015.iii CCS Quarterly Enrollment Numbers Starting in December 2013, DQA-certified CCS programs and regions were asked to report enrollment data quarterly as specified in Wis. Stat. § 51.42(7)(d). This requirement, combined with the higher frequency and self-report nature of these data, means that it is one of the most reliable sources of admissions, transitions, and people served in CCS. Participant Satisfaction Surveys See Appendix III – Participant Satisfaction Surveys for a full description of these instruments and their data. Program Participation System The Program Participation System (PPS) is the mental health and substance use disorder client data tracking system used by DHS. Counties and tribes submit demographic and service data for services they authorize. This data system also allows program providers to enter required status update data every six months for clients considered to have an ongoing need for services. Data collected through status updates includes information on criminal justice involvement, housing, employment, and suicide risk. Medicaid Fee-for-Service Claims Medicaid Fee-for-Service Claims data are gathered through extractions from interChange, Wisconsin’s online Medicaid fee-for-service claims analysis data universe. Individuals served in a CCS program who are also members of a Medicaid health maintenance organization (HMO) plan are also covered under a fee-for-service arrangement and thus will be shown in these data. Because of the payment structure of Medicaid data, CCS regions operating on the 51.42 Model (described below in CCS Regional Models) cannot be further described by county. These include The Human Service Center (Forest, Oneida, and Vilas counties) and North Central Health Care (Langlade, Lincoln, and Marathon counties). Claims expenditure figures presented also do not include county cost settlement amounts which, if included, would slightly increase or decrease overall expenditures to individual counties. Data presented includes the years 2011 through 2015.

i Some analyses in this report will not include pertinent data from all of the 59 CCS programs who submitted the CCS 2015 Program Survey. In most instances this is due to a non-applicable question, such as not having any discharge data due to no discharges occurring in 2015. Other reasons for incomplete data may include incomplete or suspect data for question items. This latter circumstance was rare. ii In some circumstances an annual program survey may not be completed for all CCS programs certified within that same year. Reasons for this include program having a certification date later in the year, not having a program fully up and running, and not having participants enrolled during the pertinent year for the survey. iii See Appendix II – 2015 CCS Program Survey for a copy of the entire survey

4

Division of Quality Assurance Certification Data DHS houses a SharePoint website that it uses to track certain characteristics of CCS program providers, including regional model type and program certification date as granted by DQA. The timeframe for this data spans the life of CCS dating back to 2005. NIATx Hospital Readmission Rate Reduction Project Hospitalization Data See the Participant Hospitalization Rates section of this report for a description of this data.

CCS Regional Models This report covers the first full year of CCS regionalization. Counties and tribes were encouraged to form regions to provide CCS as part of a 2013-2015 state budget initiative. Effective July 1, 2014, this initiative allocated state funds to pay for the nonfederal share of allowable Medicaid program costs to counties and tribes that operate under a regional program structure. By the end of 2015, a total of 22 regions had been approved by the Division of Mental Health and Substance Abuse Services (now known as the Division of Care and Treatment Services) and certified by DQA to provide CCS. CCS counties and tribes are organized into one of four regional models: population-based, shared services, multi-county, and 51.42. Counties and tribes operating under these regional models are considered a regional service-delivery system eligible for state CCS expansion funding.

Population-Based Model Individual counties with a population over 350,000, or sovereign tribal nations, can be certified to provide CCS as a population-based region. These counties and tribes are not required to collaborate with other counties or tribes, but are required to maintain DQA and Medicaid certifications for CCS. In 2015, the four population-based regions were the Lac du Flambeau Band of Lake Superior Chippewa Indians and Milwaukee, Waukesha, and Dane counties.

Shared Services Model Counties or tribes in this model agree to share significant resources, expenses, and components of CCS programming (such as providers, supervision, training, and administrative operations) while being responsible for maintaining individual county or tribal CCS certifications for DQA and Medicaid billing. Counties or tribes with existing or new CCS certifications are allowed to share services and expenses to create efficiencies. In 2015, the shared services regions were: • Central Wisconsin Health Partnership (CWHP): Adams, Green Lake, Juneau, Waushara, Marquette and

Waupaca counties • JRW Tri-County Region: Jefferson, Rock, and Walworth counties • Northeast Wisconsin Behavioral Health Consortium: Brown, Calumet, Manitowoc, Outagamie, and

Winnebago counties • Portage-Wood Partnership: Portage and Wood counties • Wisconsin River CCS Collaboration: Columbia, Richland, and Sauk counties • New Horizons North Comprehensive Community Services: Ashland and Bayfield counties • Lakeshore Recovery Collaborative: Dodge, Ozaukee, Sheboygan, and Washington counties • CCS of Clark & Trempealeau Counties: Clark and Trempealeau counties • Kenosha Racine Regional Consortium: Kenosha, and Racine counties • Lakeshore Recovery Consortium: Door, Kewaunee, and Shawano counties

5

Multi-County Model In this model, two or more counties partner under a single CCS program certification. A multi-county region is certified as a program, with one county identified as the lead administrative agency with responsibility to maintain CCS certification. Counties collaborate to create a single CCS service area, encouraging the expansion of CCS programming to larger geographic areas. In 2015, the six multi-county regions were: • Western Region Integrated Care (WRIC): La Crosse (lead), Monroe, and Jackson counties • Western Region Recovery and Wellness Consortium (WRRWC): Barron, Buffalo, Chippewa (lead), Dunn,

Pepin, Pierce, Polk, and Rusk counties • Green/Lafayette Regional Comprehensive Community Services Program: Green (lead) and Lafayette

counties • North Central Region CCS Program: Iron and Taylor (lead) counties • Marinette/Oconto County Comprehensive Community Services Consortium: Marinette (lead) and Oconto

counties • Strive Comprehensive Community Services: Crawford and Vernon (lead) counties

51.42 Model In this model, two or more counties join as a legal entity to provide a number of human service programs, one of which is CCS. The region is required to have DQA and Medicaid CCS certification and is statutorily based. These entities serve an area and provide regional programming, including CCS. In 2015, there were two 51.42 regions: • The Human Service Center (HSC): Forest, Oneida, and Vilas counties • North Central Health Care (NCHC): Langlade, Lincoln, and Marathon counties.

Figure 1 illustrates the counties and tribes across Wisconsin certified to provide CCS by the end of 2015. Note: Fond du Lac County did not operate CCS in a regional model in 2015.

6

Figure 1: Map of CCS Counties and Tribes (as of December 31, 2015)

Source: DQA certification data

Lac du Flambeau Band of Lake Superior Chippewa Indians

7

The Program In the first 17 months of the CCS expansion initiative (July 2014-December 2015), the number of CCS programs certified more than doubled. This included 27 new CCS programs in 2015 alone (Figure 2). Figure 2: Number of DQA-Certified CCS Programs, 2005-2015

Source: DQA Certification Data Table 1 presents a point-in-time count of the number of persons served each year from 2013 to 2015 from the CCS quarterly enrollment data and demonstrates that the number of persons served by CCS increased at a similar rate to the count of CCS programs, increasing 65 percent from 2014 to 2015 as also displayed in Figure 2. Table 1: CCS Point-in-Time End of Year Counts - 2013 to 2015

December 2013 December 2014 December 2015

CCS Participants Served 1,519 1,947 3,214*

Number of DQA-Certified CCS Programs 31 37 64

Source: CCS quarterly enrollment numbers * = Revised: This number may differ from the annual total listed in previously published quarterly reports due to updates submitted by counties and tribes.

CCS Utilization

Participants Admitted, Served, and Transitioned The number of persons enrolled in CCS at a given time, or caseload, substantially increased from 2014 to 2015 across the state, but this rate of increase varied noticeably across CCS regions suggesting that some regions were expanding enrollment at higher rates when compared to others.

8 15

17 20

26 28 28 28 31

37

64

0

10

20

30

40

50

60

70

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

CCS

Cert

ified

Pro

gram

s

Year

8

Table 2 presents CCS quarterly data across regions on admissions and transitions in addition to point-in-time counts of persons served. This table also uses the same data to portray admission and transition volume, as a percent of year-end counts of persons served in 2015, and demonstrates that, in addition to increases in caseload, rates of admission and transition were variable in 2015. For example, New Horizons North CCS had one of the highest admission rates at 138 percent of its 2015 end-of-year caseload, well above the state average of 64 percent, while Human Services Center had one of the highest rates of transition at 55 percent of its end-of-year caseload, well above the statewide average of 23 percent. Table 2: CCS Referrals, Transitions, and Point-in-Time Caseload – 2015

*= Summation of individual counties transitioning to a new regionalized model in 2015. ** = Revised: This number may differ from the annual total listed in previously published quarterly reports due to updates submitted by counties and tribes.

Participants Served by CCS and Community Support Programs Community Support Programs is the intensive psychosocial rehabilitation program modeled after Assertive Community Treatment (ACT). Community Support Programs is designed to serve adults living with a serious and persistent mental health disorder (typically a psychotic spectrum disorder) and, in general, is designed to serve participants with more serious needs than CCS programs. Similar to CCS, Community Support Programs provide professional care and treatment in the community that includes a broad range of services to meet an individual's unique personal needs, reduce symptoms, and promote recovery. Community Support Programs differ from CCS in that the program is designed to be capable of providing services that can be tailored to the individual’s needs at any given time, ranging from minimal to intensive, or a level that might otherwise require

Number Admitted

Percent of Client Caseload (Number

served - end of 2015)

Number Discharged

Percent of Client Caseload (Number

served - end of 2015)

Point-in-Time

Number Served

Point-in-Time

Number Served

(During 2015)

that were Admissions

(During 2015)

that were Discharges

(End of 2014)

(End of 2015)

CCS of Clark & Trempealeau Counties 9 113% 1 13% - 8Central Wisconsin Health Partnership (CWHP) 76 80% 41 43% 68 103Dane County Human Services CCS 78 99% 0 0% - 78Green-Lafayette Regional CCS Programs* 21 45% 8 17% 34 47Human Service Center (HSC) 36 86% 23 55% 30 43JRW Tri-County Region CCS 129 75% 53 31% 98 174Kenosha-Racine Regional Consortium (KRRC)* 209 74% 28 10% 98 279Lac du Flambeau Tribe 17 155% 6 55% - 11Lakeshore Recovery Collaborative* 103 42% 37 15% 171 237Lakeshore Recovery Consortium* 68 81% 24 29% 32 76Marinette/Oconto County CCS Consortium* 19 50% 34 89% 56 41Milwaukee County 225 113% 52 26% 69 242New Horizons North CCS 112 138% 31 38% - 81North Central Health Care (NCHC) 193 41% 114 24% 401 480North East Wisconsin Behavioral Health Consortium 209 50% 125 30% 316 400Portage-Wood Partnership 84 53% 44 28% 113 153Strive CCS 25 104% 1 4% - 24Taylor/Iron County CCS 6 100% 0 0% - 6Waukesha County 31 28% 19 17% 100 112Western Region Integrated Care (WRIC) 134 46% 41 14% 198 291Westerm Region Recovery and Wellness Consortium (WRRWC) 132 113% 12 10% - 120Wisconsin River CCS Collaboration 95 48% 50 25% 147 192Eau Claire/St. Croix Shared CCS - 0% - 0% - -Fond du Lac County 6 38% 6 38% 16 16

Statewide Total Number of Participants 2,017 63% 750 23% 1,947 3,214**

CCS Region

Referral Volume Discharge Volume Caseload

9

care in a hospital setting. Community Support Programs also differs from CCS in that it emphasizes a strong team component to aid in the transferring of knowledge and building of a relationship between providers and participants. Community Support Programs are also eligible for Medicaid reimbursement and have been available since 1989. A 2013-2015 state budget initiative provided state general purpose revenue funding for the nonfederal share of Medicaid for regionalized CCS initiatives. However, for Community Support Programs, the nonfederal match for federal Medicaid must come from county revenue sources, as was the case for CCS prior to 2014. While it may be too early to tell what the impact of increased access CCS had on the number of individuals served by Community Support Programs, the differences in funding structures and the fact that both are offered to populations with varying degrees of mental health and substance use disorder needs warrants close monitoring of changes in enrollment across these two programs. The number of participants served by CCS has increased substantially since 2005. However, Community Support Programs serve a substantially higher number of people across the state. There have been modest declines in caseload of Community Support Programs since 2008, which have become slightly more pronounced in recent years when CCS expansion took place. Table 3 presents the abbreviated results of an analysis of Community Support Programs and CCS program survey data and is intended to help each of these programs monitor potential changes in enrollment and ensure the ultimate goal of providing the greatest access to care among people eligible for Community Support Programs and CCS. Among the 20 CCS regions with Community Support Programs during 2013-2015, only two (10 percent) had substantial (that is, greater than 25 percent) decreases in Community Support Programs enrollment. Community Support Programs and CCS program survey data also track, among those transitioned, whether a transfer occurs between the two programs. As shown in Table 3, the percent of transitions from Community Support Programs to CCS was relatively modest with the exception of two CCS regions. A large percentage of participants in Community Support Programs transitioning to CCS could signify an inappropriate move of a higher, more intensive-need client into a lower intensity, non-Assertive Community Treatment-based program. However, there are many explanations for this movement. For example, participants in Community Support Programs who originally may have had lower level needs among people in the program may have been able to be transferred to CCS while still preserving the same level of care and services. Also, one would expect that even after a movement of potentially appropriate people from Community Support Programs to CCS, more resources or “slots” would open up for new people to enroll in Community Support Programs since substantial proportions of people with serious mental illness in the general population continue to not receive needed services statewide and nationally.1 The number of people served in Community Support Programs would quickly rebound or increase following the temporary decrease in persons served following this migration. Table 3 reveals both of these potential scenarios are not the case for the two regions that had a large proportion of their participants in Community Support Programs transitioned to CCS between 2013 and 2015. Their Community Support Programs and CCS have been in place since at least 2013 and the number of people served in their Community Support Programs decreased during 2013 and 2015 more than any other CCS region in the state.

10

Table 3: CCS and Community Support Programs: Counts Served and Transitions from Community Support Programs to CCS, 2013-2015

Count Served

% Change in Count Served

% Transitioned to CCS

Program CCS

Community Support

Programs

Community Support Programs

Year 2013 2015 2013 2015 2013 - 2015

CCS

Regi

on

CCS of Clark & Trempealeau Counties - 9 92 85 -8% 0% Central Wisconsin Health Partnership (CWHP) 69 139 127 120 -6% 0% Dane County Human Services CCS - 79 537 541 1% 2% Fond du Lac 14 19 - - - - Green-Lafayette Regional CCS Programs 70 59 107 106 -1% 10% Human Service Center (HSC) 11 57 34 40 18% 18% JRW Tri-County Region CCS 116 223 483 467 -3% 5% Kenosha-Racine Regional Consortium (KRRC) 88 307 284 278 -2% 14% Lac du Flambeau Tribe - 15 - - - - Lakeshore Recovery Collaborative 131 314 294 206 -30% 66% Lakeshore Recovery Consortium 22 107 114 122 7% 10% Marinette/Oconto County CCS Consortium 71 71 82 88 7% 5% Milwaukee County - 239 1136 1271 12% 0% New Horizons North CCS - 113 111 113 2% 34% North Central Health Care (NCHC) 425 582 263 40 -85% 77% Northeast Wisconsin Behavioral Health Consortium 355 558 373 371 -1% 18% Portage-Wood Partnership 133 195 - - - - Strive CCS - 25 92 73 -21% 4% Taylor/Iron County CCS - 6 28 20 -29% 0% Waukesha County 117 133 186 182 -2% 0% Western Reg. Integrated Care (WRIC) 163 305 198 159 -20% 19% Western Region Recovery and Wellness Consortium (WRRWC) 137 175 193 10% 8% Wisconsin River CCS Collaboration 162 247 231 225 -3% 5% Counties without a CCS - - 630 562 -11% 0% Statewide 1947 3939 5577 5262 -6% 19%

Key Red = Regions with either a greater than 25% transition from CCS to Community Support Programs or a greater than 25% decline in people served in Community Support Programs during 2013-2015. Orange = Taylor/Iron: Numbers served are very low. Conclusions should be carefully interpreted; New Horizons North: A substantial proportion of transitions from Community Support Programs to CCS occurred, but Community Support Programs numbers have not declined. Green = Counties that have increased the number of participants in Community Support Programs across years.

Source: 2013-2015 Community Support Programs and CCS program surveys.

11

Medicaid Recipients and Expenditures An individual does not need to be Medicaid eligible to receive CCS. However, county and tribal agencies have funding constraints that prohibit or limit their ability to offer CCS apart from Medicaid. Some counties serve a small number of people in CCS who are not on Medicaid or who are awaiting Medicaid eligibility. In order for Medicaid to reimburse for CCS, the CCS member must be Medicaid eligible. Members enrolled in Medicaid who are determined to need CCS are eligible for CCS enrollment. CCS regions receive state funds, administered through the Medicaid payment system, for the nonfederal share of Medicaid expenses for CCS. All allowable services provided under the CCS benefit are reimbursed fee-for-service regardless of whether the member is enrolled in a BadgerCare Plus HMO, a Medicaid Supplemental Security Income (SSI) HMO, or a special managed care program (such as Family Care and the Family Care Partnership Program).iv

Medicaid Recipients and Expenditures, by Countyv Table 4 presents the total count of Medicaid CCS recipients (number of persons served) and claims expenditures (amount paid by Medicaid to each county) by county in 2012-2015 for CCS programming provided under the standard Medicaid fee-for-service system. As expected, the number of CCS Medicaid recipients increased. There was a 123 percent or $19.5 million increase in expenditures from 2014 to 2015, whereas this figure was only 49 percent or $5.1 million from 2013 to 2014.vi With the exception of Fond du Lac County, that chose not to operate under a CCS regional model and consequently did not receive increased Medicaid funds for CCS following expansion, all counties operating CCS across 2014 and 2015 saw substantial increases in expenditures paid from Medicaid. For example, Rock County saw a 665 percent increase in expenditures paid while some counties had more modest increases such as Winnebago and Walworth counties (23 percent).

iv It is important to note that in 2015 CCS was not available to members who receive services under the Medicaid managed care benefit for either Wraparound Milwaukee or Dane County Children Come First. These two programs included case management as a covered service and, as a result, additional care coordination may have not been billed separately at the same time through Medicaid. v Although the Lac du Flambeau Band of Lake Superior Chippewa Indians offered CCS and billed for Medicaid services in 2015, no data was available in the Medicaid fee-for-service claims analysis universe for Wisconsin tribes. vi Since counties paid the nonfederal share of CCS costs prior to July 2014, the expenditure totals (in Table 3) represent only the federal share paid through the state’s Medicaid program through June 2014. Expenditures beginning July 2014 include both the federal and nonfederal share of Medicaid CCS costs.

12

Table 4: CCS Medicaid Count of Recipients and Expenditures Paid to Counties and Tribes, 2012-2015

Source: interChange, Wisconsin’s online Medicaid fee-for-service claims analysis universe Notes: Human Services Center includes Forest, Oneida, and Vilas counties; North Central Health Care includes Langlade, Lincoln, and Marathon counties; A dash (-) indicates the program was not DQA-certified during the specified period or the statistic does not apply due to incomplete data for calculation.

Count Paid Count Paid Count Paid to 2014 Count Paid to 2015Adams 31 $173,487.76 33 $128,852.72 37 $164,885 28% 46 $311,376 89%Ashland - - - - - - - 74 $378,057 -Barron - - - - - - - 24 $87,591 -Bayfield - - - - - - 32 $277,985 -Brown 130 $863,237.01 86 $862,583.14 108 $944,441 9% 134 $1,678,973 78%Calumet 44 $128,954.33 33 $94,691.97 41 $215,239 127% 40 $297,065 38%Chippewa - - - - - - - 50 $236,872 -Clark - - - - - - - 7 $9,455 -Columbia 12 $79,835.06 20 $147,040.27 37 $160,525 9% 63 $597,374 272%Crawford - - - - - - - 12 $29,580 -Dane - - - - - - - 77 $221,841 -Dodge 34 $72,974.64 31 $67,323.84 50 $105,478 57% 61 $251,803 139%Door - - - - - - - 14 $58,714 -Dunn - - - - - - - 18 $144,869 -Fond du Lac 16 $81,646.56 16 $102,777.67 23 $204,445 99% 22 $175,678 -14%Green 37 $100,496.36 41 $112,884.03 40 $145,569 29% 55 $230,318 58%Green Lake 5 $18,933.70 10 $38,967.84 17 $46,083 18% 15 $46,072 0%Human Services Center 25 $166,491.31 17 $72,349.55 37 $204,691 183% 61 $492,713 141%Jackson - - - - - - - 29 $213,740 -Jefferson 81 $395,362.87 79 $356,871.08 79 $358,784 1% 101 $752,189 110%Juneau - - - - 9 $61,941 - 35 $241,947 -Kenosha 73 $177,990.51 87 $249,979.45 107 $487,598 95% 236 $2,657,896 445%Kewaunee 17 $36,897.87 22 $85,203.63 30 $116,027 36% 44 $566,345 388%Lac du Flambeau Tribe - - - - - - - 4 $127,090 -La Crosse 140 $1,007,000.08 160 $1,242,971.45 242 $2,311,172 86% 305 $5,002,140 116%Lafayette - - - - - - - 9 $14,092 -Manitowoc 22 $156,894.05 20 $89,389.16 24 $113,493 27% 29 $162,618 43%Marinette 24 $47,508.57 70 $370,202.76 75 $475,280 28% 73 $569,240 20%North Central Health Care 224 $970,185.17 364 $1,124,808.62 423 $2,541,711 126% 506 $4,500,883 77%Milwaukee - - - - - - - 236 $1,743,468 -Monroe - - - - - - - 25 $245,272 -Outagamie 138 $1,188,499.77 134 $1,016,604.20 139 $1,310,073 29% 178 $2,388,678 82%Ozaukee - - - - - - - 15 $118,266 -Pepin - - - - - - - 7 $45,309 -Pierce - - - - - - - 27 $171,081 -Polk - - - - - - - 15 $32,493 -Portage 44 $217,690.93 38 $154,579.72 53 $290,415 88% 60 $565,335 95%Racine - - - - - - - 63 $356,199 -Richland 74 $351,481.40 70 $276,281.46 66 $392,995 42% 60 $641,144 63%Rock - - - - 13 $43,776 - 62 $334,760 665%Rusk - - - - - - - 2 $4,146 -Sauk 59 $521,110.61 70 $761,406.29 94 $1,248,686 64% 125 $1,968,264 58%Shawano - - - - 8 $34,542 - 40 $555,551 1508%Sheboygan 31 $129,186.69 58 $213,345.38 99 $351,673 65% 120 $831,825 137%Taylor - - - - - - - 13 $38,255 -Trempealeau - - - - - - - 3 $3,950 -Vernon - - - - - - - 13 $52,207 -Walworth 32 $296,494.73 32 $205,972.12 29 $271,930 32% 39 $333,314 23%Washington 54 $424,569.72 55 $455,160.79 67 $460,791 1% 105 $1,145,273 149%Waukesha 103 $1,141,232.15 108 $1,113,528.44 107 $1,253,985 13% 120 $1,239,539 -1%Waushara 34 $209,129.73 29 $207,798.47 35 $210,935 2% 37 $417,809 98%Winnebago 96 $452,013.46 97 $716,307.98 111 $668,621 -7% 134 $821,572 23%Wood 69 $401,363.78 69 $338,494.50 90 $568,973 68% 134 $826,975 45%Total 1,649 $9,810,668.82 1,849 $10,606,377 2,290 $15,764,757 49% 3,809 $35,215,199 123%

2015Paid %

Change 2014 2012County

2013 2014Paid %

Change 2013

13

Participant Characteristics With regard to participant characteristics, the CCS program survey asks counties and tribes to report on the demographics (that is, gender, age, race, ethnicity, and veteran status), substance use, and medical conditions of individuals they served in CCS. Together, these data illustrate who CCS is serving across Wisconsin. While 59 CCS programs completed the 2015 CCS Program Survey for a total of 3,939 respondents, some programs may have not submitted all data requested.

Gender Based on 2015 CCS Program Survey data, the gender composition of CCS participants during the year was fairly evenly divided between males and females, with slightly more participants being male (51 percent) than female (49 percent). While the proportions change slightly each year, there has been a relatively equitable gender breakdown over time among CCS participants.

14

Age County and tribal agencies are required to offer CCS to eligible participants of all ages, including children and elderly adults.

Statewide Nearly one-third of CCS participants in 2015 were youth as shown below in Figure 3. This proportion has remained nearly the same since 2012. CCS serves 15 percent more youth than the broader county and tribal mental health system, according to PPS data. While the proportions change slightly each year, there has been a relatively equitable age breakdown over time among CCS participants with only a three percent increase in children being served across 2013 and 2014. Also, in 2015, only three percent of those served in CCS were adults age 65 and older whereas eight percent of those served by county and tribal mental health systems were in this same age group suggesting that more elderly adults may have been in CCS than were served. Figure 3: Age Composition of CCS Participants Served, 2015

Source: 2015 CCS Program Surveys (n=3,915).

County and Regional Levelvii Table 5 presents the count and percent of Medicaid participants served. The number of youth, adults, and elderly adults served is shown for each county or 51.42 region providing CCS in 2015. It appears most programs focused on providing services to adults. They vary in their support of children, ranging from close to three-fourths of participants in Fond du Lac and Shawano counties to no participants in some smaller counties (for example, Taylor and Vernon counties) and counties with supplementary youth mental health services such as Children Come First in Dane County and Wraparound Milwaukee in Milwaukee County.

vii Although the Lac du Flambeau Band of Lake Superior Chippewa Indians offered CCS and billed for Medicaid services in 2015, no data was available in the Medicaid fee-for-service claims analysis universe for Wisconsin tribes on age.

17 Years and Under (32%)

18-20 Years (6%)

21-64 Years (59%)

65-74 Years (3%) 75 Years and Over

(0%)

15

Table 5: CCS Participants Served by Age, by County, 2015

County or Region

Children Age 17 and Under

Adults Age 18 - 64

Older Adults Age 65 and Over Total

Count Count Percent Count Percent Count Percent Adams 20 43 24 52 2 4 46 Ashland 7 10 63 86 3 4 73 Bayfield 2 7 28 93 0 0 30 Brown 15 11 105 78 14 10 134 Calumet 17 43 23 58 0 0 40 Chippewa 43 32 91 68 0 0 134 Clark 5 71 1 14 1 14 7 Columbia 36 57 27 43 0 0 63 Dane 10 13 62 81 5 6 77 Dodge 31 51 30 49 0 0 61 Door 9 64 5 36 0 0 14

Fond du Lac 16 76 5 24 0 0 21

Green 3 5 52 93 1 2 56

Green Lake 11 73 4 27 0 0 15 Human Services Center 20 31 44 68 1 2 65 Jefferson 52 53 45 45 2 2 99 Juneau 9 27 24 73 0 0 33 Kenosha 124 52 99 42 14 6 237 Kewaunee 18 41 26 59 0 0 44 La Crosse 144 44 176 53 10 3 330 Manitowoc 16 55 13 45 0 0 29 Marinette 39 54 33 46 0 0 72 North Central Health Care 170 34 309 61 27 5 506 Milwaukee 0 0 233 99 3 1 236

Outagamie 37 21 136 77 4 2 177

Ozaukee 9 60 6 40 0 0 15 Portage 41 68 19 32 0 0 60 Racine 10 16 48 79 3 5 61 Richland 12 20 46 77 2 3 60 Rock 23 38 37 62 0 0 60 Sauk 64 52 59 48 1 1 124 Shawano 30 75 10 25 0 0 40 Sheboygan 20 17 95 79 5 4 120 Taylor 0 0 13 100 0 0 13 Trempealeau 2 67 1 33 0 0 3 Vernon 0 0 26 100 0 0 26 Walworth 17 45 21 55 0 0 38 Washington 31 30 71 68 3 3 105 Waukesha 7 6 111 93 2 2 120 Waushara 17 47 19 53 0 0 36 Winnebago 68 52 61 47 2 2 131 Wood 41 31 87 66 3 2 131 Total 1246 33 2388 64 108 3 3742 County or Region Count Percent Count Percent Count Percent Total

Count Children Adults Older Adults Source: InterChange, the online Medicaid fee-for-service claims analysis universe. Notes: Human Services Center includes Forest, Oneida, and Vilas counties; North Central Health Care includes Langlade, Lincoln, and Marathon counties.

16

Race and Ethnicity The distribution of racial and ethnic groups among CCS participants has had modest changes from previous years.viii The great majority of participants (83.8 percent) were reported as white (6.2 percent lower than in 2014) and some minority racial groups increased as a proportion of CCS participants served including black/African American (8.7 percent or 3.7 percent higher than in 2014) and American Indian/Alaskan Native (3.4 percent or 1.4 percent higher than in 2014). Asian (1.0 percent), Hawaiian/Pacific Islander (less than 0.1 percent), and those participants identifying as more than one race (3.0 percent) remained nearly steady as a proportion of CCS participants across these years. The ethnic composition of CCS participants in 2015 also changed modestly from 2014. The percentage of known Hispanic/Latino participants increased slightly (from 2.0 to 3.2 percent). This is half the rate one would expect given the distribution persons in Wisconsin identifying as Hispanic or Latino (6.6 percent).2 It should be noted that the CCS service area in 2015 included a greater number of more racially and ethnically diverse Wisconsin counties as well as the new Lac du Flambeau Band of Lake Superior Chippewa Indians CCS. Nearly 90 percent of Wisconsin's black or African American population lives in the following six counties: Milwaukee, Dane, Racine, Kenosha, Rock, and Waukesha. When looking at black or African American people as a percent of the total county or tribal population, Milwaukee County tops this list, with 25.6 percent. With the exception of Dane County that became CCS certified in 2015, all of these counties became CCS certified in 2014. The higher rate of racial and ethnic minorities among CCS participants is likely at least partially explained by this more diverse pool of potential CCS eligible participants.

Veteran Status Only one percent of those served in CCS in 2015 with known veteran status were recorded as veterans.ix This percent is quite low, given that veterans represent approximately seven percent of Wisconsin’s total population.3 However, identified veterans also are underrepresented in the county and tribal mental health systems as a whole, with only 805 total participants recorded as being veterans in PPS between 2008 and 2015. Whether veterans are truly underrepresented among CCS participants (or merely not identified as such) is unknown at this time.

viii Racial groups were reported for 3,774 participants on the CCS program survey (96 percent of the 3,939). ix Thirty-one percent of participants were recorded as having an unknown veteran status.

17

Substance Use Tobacco use, alcohol abuse, and illicit drug use among CCS participants from 2012 to 2015 is presented in Figure 4 along with substance use prevalence statistics for the general Wisconsin population in 2015. CCS programs were asked to count participants in each category that applied, so categories likely include overlap between participants. For example, a participant who smokes and abuses alcohol will appear in both categories. National research indicates mental health participants are more likely than other adults to have a variety of co-occurring substance use concerns.4 Indeed, the results from the 2015 CCS Program Survey showed a comparable or higher incidence of substance use than the Wisconsin population in general. According to the survey results, tobacco use among CCS participants was somewhat higher than the state population and has been increasing over the years despite decreasing trends in tobacco use among the general population (from 20 percent in 2012 to 17 percent in 2015). CCS participants also were more likely to be dependent on or abuse alcohol, a trend that has increased somewhat since 2012 and is higher than estimates for alcohol use among the general population (at seven percent in 2012 and eight percent in 2015). Similar trends are shown among CCS participants who used illicit substances (at 2 percent in 2012 and 3 percent in 2015).4 Given the high, but conservative, estimated rates of co-occurring diagnoses and tobacco use among people with any mental health disorder and the rather steep increases in people using substances identified across 2012 and 2015 in the program survey data, it is likely that CCS programs are improving the tracking of participants substance use. In addition, CCS also serves people with a substance use disorder only. Although historical data suggests people with substance use disorders are a small proportion of enrollees, CCS programs have been placing more effort into enrolling people with substance use disorder as a primary or sole diagnosis. Regardless, these data reinforce the importance of addressing co-occurring substance use among persons in CCS and add further weight to the stance that evidence-based practices (EBPs) such as Integrated Dual Disorder Treatment (IDDT) or integrated treatment for co-occurring disorders and the Tobacco Cessation Bucket Approach may be helpful if widely applied to CCS participant population. Figure 4: Substance Use in Wisconsin and among CCS Participants (2012-2015)

Sources: 2012-2015 CCS program surveys; Tobacco Use: CDC, Behavioral Risk Factor Surveillance System (WI 2015, age 18+ years); Alcohol Abuse: National Survey on Drug Use and Health (NSDUH) (WI 2013-2014, ages 12+ years); Illicit Drug Use: National Survey on Drug Use and Health (NSDUH) (WI 2013-2014, ages 12+ years).

26%

12% 11%

25%

10% 8%

26%

18%

11%

32%

16% 16% 17%

8%

3%

0%

10%

20%

30%

40%

Tobacco Use Alcohol Abuse Illicit Drugs Use

CCS 2012CCS 2013CCS 2014CCS 2015Wisconsin

18

Medical Conditions Physical health problems and mental health disorders have a high incidence of co-occurrence putting participants at risk for health complications and early death.5, 6 National studies have found that more than 68 percent of adults with a mental disorder reported having one or more general medical disorders, and 29 percent of these people with a medical disorder had a co-occurring mental health condition.7 Regardless, the data presented below suggests that data on the co-occurrence of physical health disorders and medical problems is largely underreported among CCS participants and thus may be under addressed. Results from the CCS program survey dating back to 2012 are displayed below in Figure 5 along with national adult prevalence rates in parentheses under each health condition listed on the x-axis. x One plausible explanation for these unexpected results is CCS participants include more youth (age 17 and younger) and fewer elderly (age 65 and older) than the general adult populations used to calculate national prevalence rates of physical health conditions. Moreover, because health problems generally accompany those older in age, CCS participants are expected to have overall lower rates of health problems.xi These data also suggest that certain ailments such as obesity and high cholesterol may be decreasing in incidence among CCS participants through time. While this effect may potentially show CCS participants overall health may benefit while they are in the program, the absence of individual-level, time-series data prevents causal explanations to be established and does not rule out the potential for a selection of healthier participants into CCS. Despite these possibilities, it remains unlikely age and the CCS program’s ability to treat other medical conditions can explain away the well-established, strong, and positive association of physical health conditions with mental health disorders. Instead it is more likely CCS programs do not systematically ask participants for detailed information about their physical health, so the rates of these ailments for CCS participants are likely underreported. If the assumption is correct that garnering knowledge of a problem or need is a necessary in identifying, assessing, and treating those with health problems, then CCS program providers need to move the monitoring of medical conditions in CCS participants to the forefront of their program planning in the immediate future.

x National health estimates were drawn from a variety of sources. See Appendix I - Sources for National Health Estimates. xiThe CCS program survey collects aggregate data (not individual participant records), so it is not possible to make a direct comparison between CCS and U.S. rates (by controlling for this age effect).

19

Figure 5: Rates of Various Health Conditions among CCS Participants, 2012-2015 (With National Adult Prevalence Rate)

Source: 2012-2015 CCS program surveys. Note: See References for references for adult health condition prevalence rates.

Primary Diagnoses Tables 6.A. and 6.B. present distributions of the most common mental health disorder diagnoses among adult and youth CCS participants transitioned in 2015, respectively. Overall results from this PPS data on primary diagnoses are generally in-line with national estimates on mental health disorder prevalence.xii, xiii The National Institute of Mental Health (NIMH) has identified the most common mental health disorders as anxiety and depressive disorders for adults and attention deficit hyperactivity disorder (ADHD), depressive disorder, and conduct disorder among adolescents.

xii Diagnoses are the primary mental health diagnosis (Axis I or II) reported for receiving treatment for mental health or substance use disorder services upon for reception of services. Diagnoses remain largely unchanged from that determined upon first entry into services, but may not necessarily reflect diagnosis/es upon discharge if circumstances change or an additional diagnoses is discovered. This also means that many younger adults continuing to receive CCS in 2015 may have received their diagnoses when they were youth and thus some adult diagnoses may also contain youth diagnoses. xiii Only one CCS participant was listed in the PPS substance use or AODA module with a primary substance use disorder diagnosis and therefore substance use disorder diagnoses are not included in this data. Future improvements in PPS diagnosis reporting along with potential options for reporting integrated services , or services that are designed and delivered to fully address both a client’s mental health and substance use disorder needs by qualified mental health, addiction, or dually-credentialed staff, are being explored.

11.0% 9.1%

7.4% 6.7% 6.6% 5.7%

2.8% 2.3% 1.7%

0.0%

4.0%

8.0%

12.0%

16.0%

20.0%

2012 2013 2014 2015

20

Table 6 (A. and B.): Most Common Primary Diagnosis among CCS Participants

A. Adults B. Youth

Primary Diagnosis** Frequency Percent

Depressive Disorder 37 21.8

Anxiety Disorder 30 17.6

Schizophrenia and/or Psychotic Disorder 25 14.7

Bipolar Disorder 24 14.1

Other Mood disorder 12 7.1

Unspecified Mental Health Disorder

12 7.1

Source: PPS mental health and alcohol and other drug abuse modules. Note: Defined as present among five percent or more of CCS adults (N=170) or five percent or more of CCS youth (N=117) transitions in 2015.

Primary Diagnosis** Frequency Percent Attention Deficit Disorder - Attention Deficit Hyperactivity Disorder

25 21.4

Conduct or Oppositional Defiant Disorder 24 20.5

Unspecified Mental Health Disorder 15 12.8

Other Mood Disorder 14 12.0

Depressive Disorder 13 11.1

Anxiety Disorder 9 7.7

21

Program Characteristics

With regard to program characteristics and services provided, the CCS program survey asks programs to report on program staffing, program utilization (numbers of participants served during the year, the number of newly enrolled, and transitioned participants), participant transition status (reasons participants left CCS and their destinations), as well as the availability and use of EBPs across adults and youth. Taken together, these data help paint a picture of how CCS programs function and what services they provide.

Program Staffing CCS programs may differ in their staff composition. Programs can be staffed either entirely by county or tribal employees or by a mix of these employees and contractors. In 2015, just over three-quarters of the programs, 46 out of 59 (78 percent) fall into the latter category. This is a slight increase from 2014 when 27 out of 36 (75 percent) CCS programs were staffed by a mix of county or tribal workers and contractors.

Evidence-Based Practices A main goal of the CCS program survey is to determine the extent to which CCS programs incorporate the use of EBPs into the services they provide and deliver these appropriately to adults or youth. Using EBPs is a way for CCS programs to enhance recovery process of participants. However, CCS programs are not required to use EBPs. Rather, programs are provided with the information and encouraged to incorporate EBPs to the best of their ability.

Much of the CCS program survey is devoted to questions around the use of EBPs. Programs are asked not only which EBPs they offered and which EBPs their participants received to track trends and identify potential disparities in EBP usage, but also a series of questions around EBP training and monitoring EBP fidelity. In responding to all of the EBP questions, programs are asked to adhere to the strict definitions of the EBP as laid out in a guiding document. Thus, many CCS programs report they follow many of the guiding principles or practices of a given EBP, but don’t strictly qualify as providing that EBP. For that reason, it can be assumed that more programs utilize some variation of an EBP other than what is presented here.

Evidence Based Practices (EBPs) for Adults The CCS program survey asks about specific adult EBPs that have been either supported by the Substance Abuse and Mental Health Services Administration (SAMHSA) or have been supported by evidence and materials for dissemination and technical assistance are available to CCS providers. For adults these EBPs include: • Integrated Dual Disorder Treatment (IDDT) or Integrated Treatment for Co-Occurring Disorders: A

program that supports individuals with co-occurring mental illness and substance use disorder. • Family Psychoeducation: A developed partnership among participants, families, practitioners, and

supporters. • Illness Management and Recovery (IMR): A program that focuses on education of the participant

regarding the illness and symptoms and management of both in the journey of recovery. • MedTEAM (Medication Management): A program that uses best practice coupled with patient input to

make medication management decisions. • Supported Employment: An emphasis on the importance of work with relation to recovery and assists the

participant in addressing symptoms that interfere with finding and securing employment. • Permanent Supportive Housing: A program that helps individuals secure and maintain safe housing. • Tobacco Cessation Bucket Approach: A set of tobacco reduction interventions tailored toward the user’s

willingness to move toward quitting or stage of change.

22

Figure 6 presents within EBP-type data across both number of CCS programs offering each EBP (n=59) and number of adult participants served within each EBP (n=2,661). Supported Employment is offered by the largest proportion of CCS programs although, similar to other EBPs, it serves a relatively small proportion of CCS participants. Also notable are the IMR and EBPs falling under the Other EBP category (for example, Motivational Interviewing) are serving a relatively larger proportion of CCS participants relative to the number of programs that are offering these respective EBPs.xiv

Figure 6: Percent of CCS Programs Offering and Percent of Adult Participants Receiving Each Evidence-based Practice (EBP), 2015

Source: 2015 CCS Program Survey. Notes: While some youth are reported as being served under adult EBPs, percentages for Adults Received use the total adults reported as served as the baseline for this calculation. Some participants may have received more than one EBP in 2015.

xiv Cross-year comparisons of this data have been omitted because of many changes that accompanied CCS services during program expansion in 2014 and 2015.

34% 32%

25%

12%

58%

15%

5%

20%

5% 2%

13%

5%

17%

2% 0%

18%

0%

10%

20%

30%

40%

50%

60%

IDDT FamilyPsychoeduc.

IMR MedTEAM SupportedEmployment

Perm.Supportive

Housing

TCBA Other EBP

Programs Offered (N=59)

Adults Received (N= 2,661)

23

Figure 7 presents the percent change in the number of CCS program offering an EBP from 2014 (n=36) to 2015 (n=59). Most EBPs saw some drop in the percent of total CCS programs offering adult EBPs with the exception of Supported Employment, which saw a substantial increase in the proportion of programs offering this EBP. Cross-year comparisons of this data should be interpreted carefully because of many changes that accompanied CCS services during program expansion in 2014 and 2015.

Figure 7: Percent of CCS Programs Offering Each EBP and Percent Difference, 2014-2015

Source: 2014-2015 CCS program surveys. Table 7 lists the EBPs most commonly listed among CCS programs as part of their EBP options, but not listed on the survey. Less commonly reported programs included person-centered planning, Project YES! Healthy Transitions Grant Program, Wellness Recovery Action Plan (WRAP), participant run services, Moral Reconation Therapy, and Motivational Enhancement therapy. Table 7: Other Evidence-Based Practices (EBPs) Used by CCS Programs, 2015

Other Evidence-Based Practices Number of Programs Using Motivational Interviewing 7 Dialectical Behavioral Therapy (DBT) 3 Cognitive Behavioral Therapy (CBT) 3 Peer Support 2

Source: 2015 CCS Program Survey. Figure 8 suggests that many CCS participants are receiving only partially implemented EBPs, or EBPs that only include some of the core elements as part of their EBP model. Supported Employment, Tobacco Cessation Bucket Approach, and IDDT are the exceptions to this trend.

0%

20%

40%

60%

80%

100%

IDDT FamilyPsychoeduc

IMR MedTEAM SupportedEmployment

PermanentSupportive

Housing

TCBA Other

2014 (N=36)

2015 (N=59)

-7%

+16%

-2%

-6% -10%

N/A

-2%

-5%

24

Figure 8: Percent Adult Participants Served Under Fully Implemented EBP (versus Partially Implemented), 2015

Source: 2014-2015 CCS program surveys. Note: Some participants may have received more than one EBP in 2015.

64%

5%

30%

1%

83%

11%

71%

38%

0%

20%

40%

60%

80%

100%

IDDT(N=129)

FamilyPsychoeduc.

(N=56)

IMR (N=351) MedTEAM(N=146)

SupportedEmployment

(N=445)

PermanentSupportive

Housing(N=44)

TCBA (N=7) Other EBP(N=476)

Fully Implemented Partially Implemented

25

As shown in Figure 9, a sizable proportion of CCS programs offering EBPs have staff trained to implement their respective EBPs, though very few monitor their fidelity and almost none use an outside monitor to review their fidelity. Out of all EBPs offered by CCS programs, Supported Employment has the highest proportion of fidelity checks in place.

Figure 9: Percent of CCS Adult EBPs Implemented with Fidelity Checks

Source: 2014-2015 CCS program surveys. In summary, the higher than average rates of CCS participation and higher likelihood for high fidelity services to be delivered to CCS participants in the Supported Employment EBP suggests that devotion of additional staff and training towards an EBP may substantially impact the proportion of persons receiving a fully implemented and high fidelity EBP. This, in turn, may mean that more people may have more positive outcomes as a result of their CCS services. Notwithstanding this likelihood, it is also important to note that an EBP should only be delivered to people who have a need in an area that the EBP addresses; something DHS currently does not have data to measure. As may be the case for Supported Employment, there is a greater likelihood that more CCS participants are in need of employment support than some of the more specific areas that other EBPs address thus suggesting that low participation in an EBP does not necessarily imply underutilization. Regardless, it is important that CCS participants have the ability to access these EBPs if they do have a need and therefore low rates of EBPs being offered is an area to be further addressed by CCS programs in the future.

70%

47%

93%

71%

94%

22%

67%

15% 16%

60%

43%

74%

11%

67%

15%

5%

27%

14%

68%

11%

0% 5%

0%

13%

0%

59%

0% 0%

IDDT (N=20) FamilyPsychoeduc.

(N=19)

IMR (N=15) MedTEAM (N=7) SupportedEmployment

(N=34)

PermanentSupportive

Housing (N=9)

TCBA (N=3)

Have CCS staff been specifically trained toimplement?Did you use the EBP toolkit to guide yourimplementation?

26

Evidence Based Practices for Youth The CCS program survey also asks about specific youth EBPs with the same criteria mentioned for adult EBPs. These EBPs include: • Multisystemic therapy (MST): An intensive family- and community-based treatment that addresses the

multiple determinants of serious antisocial behavior. • Therapeutic Foster Care (TFC) and Multidimensional Treatment Foster Care: A clinically-based

intervention for youth in foster care with severe mental, emotional, or behavioral health needs that includes placing youth in an individual foster parent home with parents trained to address the needs of these youth.

• Functional Family Therapy (FFT): An outcome-driven prevention and intervention program for youth who have demonstrated the entire range of maladaptive, acting out behaviors and related syndromes. It uses phased treatment to maximize protective factors among youth and family.

• Parent-Child Interaction Therapy (PCIT): A treatment program for young children with disruptive behavior disorders that places emphasis on improving the quality of the parent-child relationship and changing parent-child interaction patterns.

• Trauma-Focused Cognitive Behavioral Therapy (TF-CBT): A psychosocial treatment model designed to treat posttraumatic stress and related emotional and behavioral problems in children and adolescents.

• Trauma-Informed Child-Parent Psychotherapy (TI-CPP): An evidence-based, parent-child therapeutic treatment for children from birth to age 6 who have experienced trauma and, as a result, are experiencing emotional, behavior, attachment, and/or mental health problems.

27

Similar analyses of data are provided below for CCS programs (n=59) offering EBPs to youth (n= 1,254).xv As shown in Figure 10, Trauma-Focused Cognitive Behavioral Therapy followed closely by Other EBPs, and then Multisystemic Therapy are offered by the largest proportion of CCS programs although, similar to adult EBPs, all youth EBPs are offered by a relatively small proportion of CCS programs and are received by an even smaller proportions of CCS participants. Also somewhat similar to adults, CCS youth are served by a relatively high proportion of EBPs listed under Other EBPs (for example, Motivational Interviewing). xvi

Figure 10: Percent CCS Programs Offering each EBP and Percent of Youth Participants Receiving Each EBP

Source: 2015 CCS Program Survey. Note: While some youth are reported as being served under adult EBPs, percentages for Youth Received use the total youth reported as served as the baseline for this calculation. Some participants may have received more than one EBP in 2015.

xv Cross-year comparisons of this data have been omitted because data on youth EBPs was not collected on the CCS program survey until 2015. xvi EBPs most commonly listed among CCS programs, but not listed on the survey were Motivational Interviewing (n=3), cognitive behavioral treatment or cognitive behavioral social skills training (n=3), and dialectical behavior therapy (n=2).

17%

7%

12%

5%

27%

5% 12% 6%

1% 4%

3%

11%

1%

18%

0%

10%

20%

30%

40%

50%

60%

MultisystemicTherapy (MST)

TherapeuticFoster Care (TFC)

Functional FamilyTherapy (FFT)

Parent-ChildInteraction

Therapy (PCIT)

Trauma-FocusedCognitive

Behavior Therapy(TF-CBT)

Trauma-InformedChild-Parent

Psychotherapy (TI-CPP)

Other EBP(SAMHSA)

Programs Offered (N=59)

Youth Consumers Received (N=1,254)

28

Similar to adults, many CCS youth participants are receiving only partially implemented EBPs or EBPs that only include some of the core elements as part of their EBP model as shown in Figure 11. The exception to this is that nearly one-third of CCS participants are served under fully implemented Trauma-Focused Cognitive Behavioral Therapy and Multisystemic therapy programs, respectively. Figure 11: Percent Youth Participants Served Under Fully Implemented EBP (versus Partially Implemented), 2015

Source: 2015 CCS Program Survey. Note: Some participants may have received more than one EBP in 2015. Figure 12 demonstrates that, unlike adult EBPs, all CCS programs offering youth EBPs have staff trained to implement these respective EBPs though, similar to adult EBPs, a much lower proportion of these EBPs use EBP toolkits for their implementation, monitor their fidelity, or use an outside monitor to review their fidelity. The exception to this is that Multisystemic therapy has the highest proportion of fidelity checks in place with over two-thirds of CCS programs having all respective fidelity checks in place.

64%

0%

31% 6%

66%

11%

45%

0%

20%

40%

60%

80%

100%

MultisystemicTherapy (MST)

TherapeuticFoster Care (TFC)

Functional FamilyTherapy (FFT)

Parent-ChildInteraction

Therapy (PCIT)

Trauma-FocusedCognitive

Behavior Therapy(TF-CBT)

Trauma-InformedChild-Parent

Psychotherapy(TI-CPP)

Other EBP(SAMHSA)

Partially Implemented

Fully Implemented

29

Figure 12: Percent of CCS Youth EBPs Implemented with Fidelity Checks, 2015

Source: 2015 CCS Program Survey. In summary, this analysis suggests youth in CCS are more likely to receive a variety of EBPs including several outside of the standard recommendations offered by SAMHSA. Similar to adult EBPs a wide degree of variation exists in the level of fidelity and completeness of implementation of these EBPs. The implementation model used by Multisystemic therapy appears to be the most promising as CCS youth in it are most likely to receive services under a high fidelity model and thus have the greatest improvement in outcomes resulting from CCS services.

Waiting Lists With the expansion of CCS came the potential for readily available financial resources to offer services to people who qualify for the program and are in need. However, for a county of tribe to provide CCS, it must have a sufficient network of service providers and program staff. With factors such as staff turnover, a limited labor pool of qualified clinical mental health and substance use disorder services professionals, and limited networks of providers (particularly in rural areas), some CCS programs end up placing eligible participants on waitlists. Information on waitlists has been gathered through the CCS program survey since 2013 and is presented below (Table 8). In general, waitlists are uncommon, but continue to be an issue for some CCS programs. Similar to 2014, only 10 (18 percent) of the 57 CCS programs with complete waitlist data said they had a waitlist for CCS programming during 2015. These 10 programs added an additional 200 people to their waitlists during 2015 and still had 54 individuals waiting for services at the end of 2015 at an average of just over five participants per CCS program with a waitlist. This is a drop from 2014 when 123 individuals were

100% 100%

86%

100% 100% 100%

90%

25%

43%

67%

44%

33%

80%

50% 57%

33%

56%

33%

70%

25%

43%

0%

25% 33%

MST (N=10) TFC (N=4) FFT (N=7) PCIT (N=3) TF-CBT (N=16) TI-CPP (N=3)

Have CCS staff been specifically trained to implement?Did you use the EBP toolkit to guide your implementation?Did you monitor fidelity for this EBP?Did you use an outside monitor to review fidelity?

30

waiting for services at the end of that year at an average of just under 18 participants per CCS program with a waitlist. The maximum and average time spent on waitlists by CCS participants in 2015 was substantially lower than those experienced by some CCS participants in 2014 with maximum waitlist times of three and 10 months, respectively, and average wait times of 2.2 and 4.9 months, respectively. Table 8: Waitlist Status among CCS Programs with a Waitlist, 2013-2015

CCS Region - County or Tribe Waitlist Status

2013 2014 2015 WRIC - La Crosse county JRW Tri-County CCS - Walworth county Milwaukee CCS - Milwaukee county Portage-Wood Partnership - Portage Wisconsin River CCS Collaboration - Sauk county WRIC - Monroe county HSC - Oneida and Vilas counties JRW Tri-County CCS - Rock county

Key Red/Shaded = CCS had waitlist in this year

Green/Unshaded = CCS had NO waitlist in this year

Source: CCS program surveys.

Programs offered various interim services to those on the waitlist (Table 9). Across 2014 and 2015, there were very few differences in the distribution of and variety of services offered. Once again, CCS programs with waitlists were more likely to connect waitlist participants with clinical services rather than peer-based resources, such as clubhouses and drop-in centers. While the linkages to clinical services are indeed crucial, programs may also want to consider strengthening their ties with less formal resources such as peer-run respites. Table 9: Interim Services used by CCS Programs with Waitlists in 2014-2015

Interim Service Number of Programs Using Interim Service

2014 2015 Outpatient mental health services 7 9 Assist with locating community resources 7 9 Crisis intervention services 6 9 Psychiatric services 7 8 Outpatient substance abuse services 6 7 Case management services 5 6 Medication management services 4 4 Drop-in center 1 3 Clubhouse 0 1

Note: Multiple interim services were used by each CCS program within their respective years of data reported. Source: CCS program surveys.

31

Suicide Risk Assessment and Protocol The 2015 CCS Program Survey asked programs to report whether they had a policy or standard practice for assessing suicide risk among their participants (Table 10) and, if so, what tools they used (Table 11).xvii Of the 58 programs submitting data on suicide risk assessment and protocol in 2015, 84 percent said they did assess their participants for suicide risk, a two percent decrease from 2014. Of these 49 programs, 47 percent indicated they used some sort of tool or instrument to assess this risk although only 25 programs or 44 percent identified a protocol for assessing suicide risk suggesting that quality of the tools and practices around suicide risk assessment may vary across contexts. Beyond the nine percent of programs with no tool or protocol identified, 14 percent of programs reporting using only limited assessments such as the Patient Health Questionnaire (PHQ)-2 or PHQ -9, which conduct a rather cursory review of the correlates of suicide rather than the actual suicidal ideations or thoughts, or refer CCS participants at risk for suicide to a crisis unit, or only address suicide risk “as needed.”

Table 10: CCS Programs Overall Quality of Suicide Assessment and Protocol, 2015

Any Protocol Identified?

Limited Assessment (for example, PHQ-2,

PHQ -9, refers to a crisis unit, or only 'as

needed')

Both Tool and Protocol Identified

No Tool or Protocol Identified

Count 25 14 4 9

Percent 44% 25% 7% 16% Source: 2015 CCS Program Survey Table 11: CCS Programs Reporting on Suicide Risk Assessment Tools, 2015

Any Tool C-SSRS (Columbia

Suicide Severity Rating Scale)

Other Non-Evidence or Research Based

Suicide Assessment

Other Research-Based Suicide Assessment

Count 27 17 2 8

Percent 47% 30% 4% 14% Source: 2015 CCS Program Survey.

xvii These data are summary of qualitative data from this single question and, due to its open-ended format, may not exhaust all information on a CCS program’s suicide risk assessment and protocol (that is, the data may be incomplete).

32

Participant Transitions and Length-of-Stay The following three sections of this report present a descriptive analysis of information on transition reasons and post-transition destinations for CCS participants as well as length-of-stay (LOS) or duration enrolled in CCS. Data on transition destination and reason can be used for a preliminary analysis of the effectiveness of CCS as it identifies the proportion of CCS transitions from 2015 who program staff report as being recovered and not needing additional CCS services. Data on LOS can be useful for program staff to begin to understand what may be the appropriate duration of CCS for their participants, but, as shown below, may also demonstrate that CCS programming does not necessarily fall into a one-size-fits-all model. While CCS has been around for several years in some locations, results on transitions and participant LOS should be interpreted with caution as CCS has undergone a great deal of change since 2014. Accordingly all results are intended for descriptive analysis only and should not necessarily be used to extrapolate data points for future estimates or predictive modeling, set baselines for performance measurement, or establish causal linkages in program services and outcomes.

Participant Transition Reasons In general, the services offered by CCS programs are considered to be fairly long-term, medium-intensity services though rates of transition (in both 2014 and 2015) have remained steady at a rate of about one in five CCS participants served being transitioned each year. Although the rate of transitions may seem higher than expected, the fact that transition rates vary substantially by CCS region (for an example, see Table 2: CCS Referrals, Transitions, and Point-in-Time Caseload – 2015) and the continued expansion of CCS across the state suggests it may be too early to conclude what an acceptable transition distribution is for CCS as a whole. Figure 13 reflects the distribution of transition reasons for the 795 CCS participants with reported reasons for transition in the 2015 CCS Program Survey. Across 2014 and 2015 the proportion of CCS transitions due to participants moving or needing additional services remained nearly the same, while participants discharging because they decided to withdraw or because funding or authorization ended increased modestly (2 and 3 percentage points, respectively). Among the less cited reasons for transitions across 2014 and 2015, dramatic increases were seen in the percent persons reported as transitioned due to being incarcerated (one to five percent of all transitions, respectively), death (two to five percent, respectively).

33

Figure 13: Reasons for CCS Participant Transition, 2015

Source: 2015 CCS Program Survey. While some reasons for transition may be categorized into positive, neutral, and negative categories (for example, recovered, moved, and incarcerated, respectively), for reasons mentioned above, it may be too soon to judge program effectiveness with these outcomes. Rather, it may be of some use to understanding where persons go after they transition, especially in the circumstance of so many potentially neutral transitions (for example, moved and withdrew make up 42 percent of transitions in 2015). The following section provides further breakdown of these transition reasons by post-transition destination.

Participant Transition Destinations Table 12 displays the percent of participants going into a transition destination by each transition reason listed above, where applicable. Among all transition reasons listed, the top transition destination was outpatient therapy and psychiatry. This is the top destination for CCS participants regardless of if they recovered, withdrew, or moved. This suggests many CCS participants continue to seek general mental health services, but that these services are at a lower-level of care, which is an expected outcome for CCS. Unfortunately, there is still a sizable proportion (nearly one in every five) of people transitioned from CCS who did not transfer to other services. This suggests that these participants, although considered to have high enough needs to qualify for CCS after taking the mental health or AODA Functional Screen, may not be viewing the mental health or substance use disorder services immediately available to them as sufficient or effective for helping them in their recovery.xviii

xviii For an additional analysis of this same type of data across both CCS and CSP programs and across 2013, 2014, and 2015, see the CCS Program – CCS Utilization – Participants Served by CCS and CSP section of this report.

Moved 20%

Recovered 30%

Funding or Authorization

Ended 5%

Needed Additional

Services 10%

Consumer Withdrew

22%

Incarcerated 5%

Death 4%

All Other 4%

34

Table 12: CCS Participant Transition Destinations by Transition Reasons, 2015.

Transition Reason

Moved Recovered

Funding or Authorization Ended

Needed Additional Services

Participant Withdrew

Total - Transition Destination

Tran

sitio

n D

estin

atio

n

Outpatient therapy/ psychiatry 10% 20% 2% N/A 11% 43%

Did not transfer to other services 3% 6% 1% 0% 8% 18%

Unknown 5% 1% 0% 0% 5% 11%

Targeted Case Management (TCM) or other CM program

1% 5% 2% N/A 2% 10%

Other 0% 0% 0% 4% 0% 5%

Another CCS 5% N/A N/A N/A N/A 5%

Community Support Programs (CSP) 0% N/A 0% 3% 0% 4%

Nursing Home 0% N/A N/A 2% N/A 2%

Inpatient/IMD 0% N/A N/A 1% N/A 1%

Group Home/CBRF N/A 0% 0% N/A 0% 0%

Source: 2015 CCS Program Survey. Note: Out of 697 participants eligible for a transition destination.

LOS for Participants in CCS This analysis of CCS participant LOS focuses on a conservative estimate of CCS transitions in 2015xix. It became apparent after conducting a preliminary analyses of LOS that a large number of CCS participants are transitioned after rather short periods of time (that is, less than three months). While this may be interpreted as inappropriate placement in CCS, there may be an explanation. One of the strengths of CCS is that it is a voluntary program. Some participants may decide the services offered do not coincide with their vision of recovery. Therefore, they withdraw after a short period of time in the program, which is an acceptable outcome for some CCS participants. There is no recommended LOS for CCS. This allows the program to serve people for a few months to several years and still be potentially effective. Arguments could be made that participants should stay within CCS for a certain minimum amount of time to actually receive a treatment effect from services. Others have suggested

xixThis analysis included people who had CCS services and the associated episode (that is, a set of services that are used to address the participant’s current needs with no more than 90 days of service inactivity) for those services that concluded in 2015 and no additional CCS services in 2016. In addition, only the participant’s latest consecutive set of services (that is, those services with less than one month between services) are used to calculate LOS. There is a small group of CCS participants in this analysis that may have been in CCS prior to their latest set of consecutive services, but had services that are part of an earlier, separate episode of mental health care that included CCS.

35

that there are a sub-set of CCS participants who only need CCS programming to stabilize and get them “back up on their feet.” Regardless, neither of these possibilities have been evaluated as effective approaches with CCS participants. Only after additional time has passed will further analyses be able to link up similar but more comprehensive LOS data with participant outcome data available through PPS or the mental health or AODA functional screen, which is used to screen for eligible participants and annually assess the functional status of those enrolled in the program. To provide the most general picture across these and other possibilities, LOS is presented across two groups of CCS participants: 1) those who spent any amount of time in CCS and 2) those who spent three or more months in CCS.

LOS among Participants who spent Any Amount of Time in CCS Table 13 presents descriptive characteristics of the durations CCS 2015 transitions spent in the program no matter the duration of their LOS. While the average LOS is just under 1 1/2 years, the median LOS, which takes into further account skew and outliers in the data, suggests more CCS participants are in the program closer to just under one year. Also notable is the large standard deviation of nearly two years. This suggests there is a lot of variation in LOS and coupled with the large maximum LOS (at nearly 9 1/2 years) suggests that some CCS participants stay in the program for substantially longer periods of time than others. Table 13: Description of CCS Length-of-Stay (LOS) in Months, Statewide 2015

Average 17.3 Median 10.0 Minimum 0.0 Maximum 113.0 Std. Deviation 22.4

Source: PPS mental health and AODA modules Note: n=266

36

Figure 14 presents the same data as in the descriptive table but in a histogram format. Clearly, there are a large number of CCS participants in this group who transitioned from the program after a relatively short stay (that is, five or less months). Regardless, a substantial proportion of these CCS participants stayed in the program for one to 3 1/2 years and a small but sizable group stayed in CCS longer than eight years, thus explaining the wide variation and skew in the data. Figure 14: Count of CCS Program Participants by Length-of-Stay (LOS) in Months, Statewide 2015 (Any LOS)


LOS among Participants who spent Three or More Months CCS If it is assumed CCS participants should have at least three months in the program to receive any appreciable effect from its services, then a timeframe of roughly 1 1/2 to two years seems more reasonable for a common LOS in CCS (Table 14). It is important to note that focusing this LOS analysis to participants with longer periods of time in the program removes a considerable proportion of people transitioned from this analysis (that is, 24 percent or 64 people) and, as expected, substantially increases average and median LOS by 5.2 and 4.0 months, respectively. The standard deviation and maximum LOS remains very high. This suggests there is still wide variation in LOS among participants, regardless of the minimum amount of time spent in CCS. Table 14: Description of CCS Length of Stay (LOS) in Months, Statewide 2015

Average 22.5 Median 14.0 Minimum 3.0 Maximum 113.0 Std. Deviation 23.4


0

10

20

30

40

50

60

70

80

90

100

Coun

t Peo

ple

Count Months

37

Figure 15 parallels Figure 14, but focusing instead on the group of CCS participants with at least three months of service in CCS. By focusing on this group of CCS participants, the distribution of LOS becomes more rounded, slightly putting the mode, or most commonly occurring value for LOS, in the six to 11 month range. These CCS participants tended to stay around for at least five to six months before discharging and slightly larger proportion of these participants fell within the one to 3 1/2 year LOS group. A sizable proportion of these participants also stayed in the program longer than eight years. Figure 15: Count of CCS Program Participants by Length of Stay (LOS) in Months, Statewide 2015


0

10

20

30

40

50

60

70

80

90

100

Coun

t Peo

ple

Count Months

38

Participant Hospitalization Rates

A general measure of success of CCS is the rate of hospitalizations among those who transition from CCS (Figure 16).xx As mentioned above, CCS is intended to assist those who are in need of care outside of inpatient and hospital contexts, but who may have ongoing needs that if left unaddressed could result in hospitalizations during times of crisis. Similar to participant LOS, this analysis uses CCS program enrollment data from PPS and links it to data used for calculating inpatient readmission rates originally intended for a NIATx hospital rate reduction project. An admission to inpatient care includes voluntary and involuntary civil inpatient admissions and emergency detentions and excludes forensic admissions, managed inpatient admissions, and residential treatment admissions. The time period of this project focused on 2013 through the first half of 2016, thus the follow-up period for readmission rates is defined by the date of transition for a CCS participant in 2015 (that is, as early as January 1, 2015) through June 30, 2016. This was a person-level analysis so people readmitted multiple times within the follow-up period and within one of the three time-period categories presented were counted only once even if they had more than one hospitalization in these categorized time periods (although these cases were very uncommon). Overall, CCS has a low hospitalization rate. While some CCS participants had a hospitalization event during their enrollment in the program and following their transition from CCS, those transitioned from CCS for any reason from the program had fewer hospitalizations than while in the program. Current data limitations prevent a pre-enrollment calculation of hospitalization rate from CCS, though this analysis clearly suggests that there may be some effect on reducing hospitalizations at least following transition from the program. Figure 16: CCS Persons Hospitalized Rate among 2015 Transitions

Sources: PPS mental health and AODA modules and Insight. Note: n=246

xx For a more conservative estimate of hospitalizations during and after CCS enrollment, this analysis focused on a population of CCS discharges similar to that used in the earlier length-of-stay (LOS) analysis. This analysis includes people who had CCS services and the associated episode (that is, a set of services that are used to address the participant’s current needs with no more than 90 days of service inactivity) for those services that concluded in 2015 and no additional CCS services in 2016.

5%

3%

1%

While in CCS Following Discharge fromCCS

Both (During enrollmentand after discharge)

39

Participant Satisfaction The CCS Administrative Rule prescribes that programs have some way of gauging participant satisfaction. Specifically, Wis. Admin. Code § DHS 36.08 states: “The CCS shall develop and implement a quality improvement plan to assess participant satisfaction and progress toward desired outcomes identified through the assessment process.” Starting in the fall of 2014, all regionalized CCS programs were required to annually administer one of three satisfaction surveys to participants who have had at least six months of CCS services during the previous calendar year. Ideally, each eligible CCS participant receives one of three age-group specific surveys that ask questions about their own experience with CCS services and interactions with staff. The first is the Recovery Oriented System Indicators (ROSI) Adult Satisfaction Survey and is completed by participants 18 years and older and measures the extent to which participants experience CCS as recovery-oriented. The second is the Mental Health Statistical Improvement Project (MHSIP) Youth Satisfaction Survey, filled out by participants 13-17 years old. The third is the MHSIP Family Satisfaction Survey, filled out by the parent or guardian (caregiver) on behalf of child participants who are 12 years and younger and measures the extent to which participants experience CCS positively.xxi

Survey Responses By the fall of 2015, 33 counties had enrolled participants for at least six months. These counties administered satisfaction surveys to their participants.xxii Total counts of survey returned by county are presented in Table 15. As expected with program expansion continuing in 2015, several (60 percent) more surveys were collected in 2015 than in 2014.

xxi See Appendix III – Consumer Satisfaction Surveys for further description of the origin of these consumer satisfaction instruments, survey methodology, and copies of the 2014 ROSI Adult Satisfaction Survey, 2014 MHSIP Youth Satisfaction Survey, and 2014 MHSIP Family Satisfaction Survey. xxii Although the Lac du Flambeau Band of Lake Superior Chippewa Indians offered CCS in 2015, none of their participants meet the survey eligibility criteria of enrollment in CCS for six or more months.

40

Table 15: Number of CCS Participant Satisfaction Surveys Returned, 2015

Source: CCS participant satisfaction surveys, 2015 HSC = Human Services Center (includes Forest, Oneida, and Vilas counties) NCHC = North Central Health Care (includes Langlade, Lincoln, and Marathon counties) WRIC =Western Region Integrated Care (includes La Crosse, Monroe, and Jackson Counties) The following three sections summarize adult, youth, and family (that is, parents or caregivers of children under 13) satisfaction with CCS programming during 2015 for all CCS services in Wisconsin. These three tables present several statewide summary statistics that can be compared by individual CCS programs with their own gathered responses and data. Scale scores were calculated across all respondents for six specific scales of interest covered by various questions in each of the three surveys (described in their respective sections, below).xxiii It is important to note that although these scales (and the overarching themes they represent) correspond across each of the youth and family MHSIP surveys, they do not necessarily correspond across the adult ROSI survey. Results presented in the row labeled “Average Score for All Respondents” provide a summary measure of satisfaction (calculated for all respondents combined). This measure does not take into account the percent of respondents having various categories of recovery oriented positive experiences listed across the left side of the table. For each of the three surveys, “Overall Mean” represents the average level of satisfaction (recovery-oriented or positive experience) reported across all survey questions for all respondents. This measure does not take

xxiii Each survey contains 26 questions about youth or family satisfaction with the mental health and substance use services received in the past six months. Using factor analysis, a statistical technique that identifies groups of related items based on their high correlation (or association) with each other, researchers reduced the number of measures needed to understand participant responses by combining items together into six scales. See Appendix III – Consumer Satisfaction Surveys- Methods and Scale Calculation section of this report for a further description of these scales.

County/Region Family Youth AdultAdams 3 0 4Ashland 0 0 7Bayfield 0 0 6Brown 1 0 32Calumet 0 3 17Columbia 6 2 1Dodge 2 1 5Fond du Lac 2 0 1HSC 0 1 5Green 0 0 12Green Lake 0 5 3Jefferson 4 15 19Juneau 0 2 4Kenosha 9 10 58Kewaunee 4 2 10WRIC 4 0 25NCHC 7 17 56Manitowoc 2 5 6

County/Region Family Youth AdultMarinette 4 3 1Milwaukee 0 0 55Outagamie 0 9 56Ozaukee 2 2 3Portage 14 4 8Racine 0 2 28Richland 0 2 8Rock 0 2 9Sauk 10 9 18Shawano 3 3 0Sheboygan 10 5 40Walworth 4 8 9Washington 3 4 33Waukesha 0 0 57Waushara 3 3 7Winnebago 10 13 45Wood 3 3 21Total 110 135 669

41

into account the six different scales specific items on the surveys fall into which are listed across the top of the table. The lower three rows (below the dotted line) show the satisfaction distribution among respondents (the percent who are more, moderately, and less satisfied with their CCS services). For these three rows, “Overall Mean” reflects the percent of all respondents who had a more positive, mixed, and less positive experience. The percentages for each of the six scales reflect how many respondents reported high, medium, or low scores using predetermined score cutoffs.

Adult Satisfaction

Overall Adult Mean On average, similar to 2014, most adult respondents to the ROSI survey agreed or strongly agreed the services they received were recovery-oriented indicated by an overall satisfaction with CCS of 3.3 out of 4.0. This coincides with the largest percent of respondents with a mostly recovery-oriented experience (that is, were more satisfied) with CCS, which increased by less than a percent from 2014. In addition, very few respondents were less satisfied with CCS services (1.4 percent). Table 16: Average Scale Scores and Percent of Adult Participants Reporting a Mostly, Mixed, and Less Recovery-Oriented Experience in CCS, 2015

Scale

ROSI Overall Mean

Person

Centered Barriers* Empower Employ Staff

Approach* Basic Needs

Average for All Respondents 3.3 3.5 3.2 3.5 2.9 3.6 3.0

% w/ Mostly Recovery-Oriented Experience

79.4% 85.2% 70.9% 90.8% 53.1% 90.0% 66.3%

% w/ Mixed Experience

19.3% 12.5 % 24.8% 7.2% 26.8% 6.6% 17.2%

% w/ Less Recovery-Oriented Experience

1.4% 2.3% 4.2% 2.0% 20.2% 3.4% 16.5%

Key (Excluding Percent w/ Mixed Experience)

Gray = Reverse coded scales* Color Coded = Greater than two percent change across 2014 to 2015 - Green = Favorable change - Red/Shaded = Unfavorable change

Source: CCS participant satisfaction surveys, 2015 Note: *Scales 1, 3, 4 and 6 are positively worded, with scores ranging from 1.0 (Strongly Disagree) to 4.0 (Strongly Agree). Questions used to calculate Scales 2 and 5 were negatively worded but these two scales have been recoded, so for this analysis, higher values (closer to 4.0) reflect a more positive, recovery-oriented experience in CCS.

42

Adult Scales For ease in interpretation, scale descriptions are provided below: • Person-Centered: These items describe whether clinical staff have a person-centered focus and allow for

person-centered decision-making. • Barriers: These items describe passive barriers to recovery that respondents may experience. • Empower: These items describe the degree to which respondents feel empowered by staff and others. • Employ: These items describe the degree to which educational and employment opportunities are

available to the individual respondent or respondents in general. • Staff Approach: These items describe the degree to which agency staff use a paternalistic and/or coercive

approach working with respondents. • Basic Needs: These items describe the respondent’s current financial ability to meet his or her basic needs.

In 2015 nine of 10 adult respondents felt empowered by their CCS staff (Empower) and that agency staff most often do not use a paternalistic or coercive approach working with participants (Negative Staff Approach). A somewhat smaller proportion felt this same way about their CCS staff having a person-centered focus and allowing for person-centered decision-making (Person Centered). Just over two-thirds felt this same way about passive barriers to recovery that they have experienced (Barriers) followed closely by satisfaction with CCS helping to increase their financial ability to meet their basic needs (Basic Needs). Creating employment and education opportunities (Employ) scored rather low relative to other scales at just over half of respondents being at least satisfied. Table 16 also uses a color coding scheme to identify values that have changed (by at least a 2 percentage points) in a favorable or unfavorable direction from 2014 to 2015. Favorable changes were seen for the Barriers and Staff Approach scales with nearly three and four percentage point increases in the proportion of adults feeling a mostly recovery-oriented experience for these scales, respectively. In contrast unfavorable changes were seen for with the Employ and Basic Needs scales with close to six percentage point decreases in the proportion of adults feeling a mostly recovery-oriented experience for each scale.

Youth Satisfaction

Overall Youth Mean In general, similar to 2014, youth respondents (that is, adolescents age 12 to 17) were very satisfied with CCS programming they had received during the past six months with an average overall score of 4.2 out of five. Many youth (86.6 percent) expressed that most of the time they had a more positive experience in CCS, though there was a four percent decrease from 2014 to 2015. This also was accompanied by modest increases in the percent of youth having mixed or less positive experiences.

43

Table 17: Average Scale Scores and Percent of Youth Participants Reporting a More, Mixed, and Less Positive Experience in CCS, 2015

MHSIP Mean

Scale 1 - Overall

Satisfaction Scale 2 -

Participation Scale 3 - Access

Scale 4 - Culture

Scale 5 - Outcomes

Scale 6 - Social

Connectedness Average Score for All Participants

4.2 4.2 4.2 4.3 4.5 3.9 4.2

Percent with More Positive Experience

86.6% 81.3% 83.5% 80.3% 95.2% 70.7% 83.1%

Percent with Mixed Experience

11.9% 14.9% 15.0% 18.2% 4.8% 23.3% 14.6%

Percent with Less Positive Experience

1.5% 3.7% 1.5% 1.5% 0.0% 6.0% 2.3%

Source: CCS participant satisfaction surveys, 2015 Note: All Youth Scale scores range from 1.0 (Strongly Disagree) to 5.0 (Strongly Agree) with a higher value (closer to 5.0) reflecting a more positive experience in CCS.

Youth Scales For ease in interpretation, scale descriptions are provided below (with the Family Satisfaction Survey language in parentheses): • Satisfaction: These items describe a youth’s (caregiver’s) overall level of satisfaction with their (child’s)

services. • Participation: These items describe how well a youth (caregiver) was integrated into their (child’s)

treatment planning. • Access: These items describe the ease with which a youth (child) obtained their mental health and/or

substance abuse services. • Culture: These items describe the cultural sensitivity of providers. • Outcomes: These items describe a youth’s (caregiver’s) life improvements that occur as a direct result of

their (child’s) CCS services. • Social Connectedness: These items describe the extent to which youth (caregivers) are socially connected,

have “natural supports” in place—family, friends, and acquaintances—to help bolster and sustain their (child’s) recovery.

In 2015 a high percent of youth respondents expressed being satisfied or very satisfied with the ability of their CCS providers to be culturally sensitive (Culture), integrate them into their treatment plan (Participation), to promote social connections or have natural supports in place to help bolster and sustain recovery (Social Connectedness), and foster ease in obtaining their mental health and/or substance abuse services (Access), and . A smaller proportion of youth felt this same way about their perceived treatment-related improvements

Key (Excluding Percent w/ Mixed Experience) Color Coded = Greater than two percent change across 2014 to 2015 - Green = Favorable change - Red/Shaded = Unfavorable change

44

as a direct result of the mental health or substance abuse services (Outcomes), but more than four out of five youth were satisfied, overall with their services (Satisfaction). Table 17 uses the same color coding scheme as the adult satisfaction table (Table 16) to identify values that have changed (by at least a 2 percentage points) in a favorable or unfavorable direction from 2014 to 2015. Unfavorable changes were seen across the Participation and Access scales (down six and 2.6 percentage points, respectively) while the Culture scale saw just over a three percentage increase. The MHSIP mean also saw a four percent decrease across 2014 to 2015. In addition, Overall Satisfaction and Outcomes saw increases in people with a less positive experience (down 3.7 and six percent, respectively).

Family Satisfaction

Overall Family Mean In general, similar to 2014, caregivers were very satisfied with the CCS programming their child (age 12 or younger) had received during the past six months with an average overall score of 4.3 out of five and over nine out of 10 (91.7 percent) expressed that they had a more positive experience with CCS. Trends across 2014 and 2015 are most positive for family of caregivers of children in CCS when compared with their youth and adult counterparts. There was over a four percent increase in youth expressing their experience in CCS was more positive accompanied by modest decreases in the percent of youth having mixed or less positive experiences from 2014 to 2015.

Table 18: Average Scale Scores and Percent of Family Participants Reporting a More, Mixed, and Less Positive Experience in CCS, 2015

Scale

MHSIP Scale 1 - Overall

Satisfaction Scale 2 -

Participation Scale 3 - Access

Scale 4 - Culture

Scale 5 - Outcomes

Scale 6 - Social

Connectedness Mean

Average for All Participants 4.3 4.4 4.6 4.6 4.8 3.8 4.4

Percent w/ More Positive Experience

91.7% 84.5% 94.4% 92.6% 97.8% 61.1% 90.0%

Percent w/ Mixed Experience

7.3% 13.6% 5.6% 7.4% 2.2% 30.6% 8.2%

Percent w/ Less Positive Experience

0.9% 1.8% 0.0% 0.0% 0.0% 8.3% 1.8%

Source: CCS participant satisfaction surveys, 2015Note: All Family Scale scores range from 1.0 (Strongly Disagree) to 5.0 (Strongly Agree) with a higher value (closer to 5.0) reflecting a more positive experience in CCS.

Key (Excluding Percent w/ Mixed Experience) Color Coded = Greater than two percent change across 2014 to 2015 - Green = Favorable change - Red/Shaded = Unfavorable change

45

Family Scales For ease in interpretation, please see the scale descriptions provided earlier in this report. Similar to youth in CCS, 98 percent of caregivers of children in CCS expressed being satisfied or very satisfied with the ability of their providers to be culturally sensitive (Culture). More than nine in 10 caregivers felt the same about being integrated into their child’s treatment planning (Participation) and their ease in obtaining their mental health and substance use services for their child (Access), followed by being socially connected or having their own natural supports in place to help them bolster and sustain recovery for their child (Social Connectedness). Despite these extremely high levels of satisfaction with different components of their child’s CCS program, a substantially lower proportion (just under two-thirds) parents or caregivers felt this same way about their child’s treatment-related improvements as a direct result of the mental health or substance abuse services (Outcomes). Table 18 (page 44) uses the same color coding scheme as the adult and youth satisfaction tables (Tables 16 and 17) to identify values that have changed (by at least a 2 percentage points) in a favorable or unfavorable direction from 2014 to 2015. Compared to adults and youth, the results of the family survey demonstrated the only sizable (0.2 points each) changes in overall scale scores across Access, Outcomes, and Social Connectedness and a similar increase for the entire survey. Favorable changes in the experiences of parents or caregivers of children respondents were seen across Social Connectedness (nearly 12 percent), Outcomes (6.4 percent), Access (just over five percent), Participation (just over two percent), and the overall score (just over four percent).

Satisfaction Survey Summary

Overall, adult respondents reported being mostly satisfied with the CCS programming they received. While there is no one-to-one comparison with the youth and family survey results, overall adults average scale scores were just slightly lower than the youth and family when taking into account its four, versus five point continuous scale. In addition, approximately two of every five adults were more satisfied with their services, indicated by the ROSI overall mean, which is less than the nearly nine of 10 youth and caregivers of children respondents, indicated by the MHSIP overall mean. For adults, improvements were seen with agency staff most often not using a paternalistic or coercive approach working with participants (negative Staff Approach) and respondents not experiencing passive barriers to recovery (Barriers), but results suggest that CCS programs may especially benefit their participants by further investing in efforts to create employment and education opportunities (Employ) as well as increasing the financial ability of their adult participants to meet their basic needs (Basic Needs). Overall youth respondents reported being rather satisfied with the CCS programming they received. However, they also had substantial decreases in their satisfaction with CCS programming (indicated by the MHSIP mean) and scored lower in 2015 than in 2014 on levels of satisfaction across various domains. This was especially the case when they considered the ability of their CCS programming to integrate them into their treatment plan (Participation). Overall, family respondents reported being very satisfied with the CCS programming their child received and saw increases across 2014 and 2015 with this satisfaction. One major exception was lower satisfaction with their perceptions that their children saw improvements in their outcomes as a result of CCS programming they received (Outcomes). On a positive note, satisfaction in this area did improve substantially in 2015 over 2014. Satisfaction with their own social connectedness also improved substantially over this same time period.

46

Appendix I – Sources for National Health Estimates National rates of various health issues for the analysis of physical health conditions were drawn from the following sources: • Asthma: Asthma and Allergy Foundation of America, “Asthma Facts and Figures,”

http://www.aafa.org/display.cfm?id=9&sub=42. • Cardiovascular Problems: Centers for Disease Control and Prevention, “FastStats: Heart Disease,”

http://www.cdc.gov/nchs/fastats/heart.htm. • Chronic Obstructive Pulmonary Disease (COPD): Centers for Disease Control and Prevention, “CDC

Features: 6.3% of Adults Report Having COPD,” http://www.cdc.gov/Features/copdadults/index.html. • Diabetes, Type I: National Diabetes Education Program. “The Facts About Diabetes: A Leading Cause of

Death in the U.S.” http://ndep.nih.gov/diabetes-facts/. Note that the prevalence rate was calculated based on the fact that diabetes overall affects 8 percent of the adult population, and Type I makes up 5 percent of those cases.

• Diabetes, Type II: Gardner, Amanda. “One in eight Americans diagnosed with Type II Diabetes: Poll.” Health Day, February 20, 2013. http://health.usnews.com/health-news/news/articles/2013/02/20/1-in-8-americans-diagnosed-with-type-2-diabetes-poll.

• High Blood Pressure: Centers for Disease Control and Prevention, “High Blood Pressure Facts,” http://www.cdc.gov/bloodpressure/facts.htm

• High Cholesterol: Centers for Disease Control and Prevention, “Cholesterol,” http://www.cdc.gov/cholesterol/facts.htm.

• Metabolic Syndrome: Norton, Amy. “Metabolic Syndrome Continues to Climb in U.S.,” Reuters, October 15, 2010. http://www.reuters.com/article/2010/10/15/us-metabolic-syndrome-idUSTRE69E5FL20101015

• Obesity: Centers for Disease Control and Prevention, “Overweight and Obesity,” http://www.cdc.gov/obesity/data/adult.html

http://www.aafa.org/display.cfm?id=9&sub=42

http://www.cdc.gov/nchs/fastats/heart.htm

http://www.cdc.gov/Features/copdadults/index.html

http://ndep.nih.gov/diabetes-facts/

http://health.usnews.com/health-news/news/articles/2013/02/20/1-in-8-americans-diagnosed-with-type-2-diabetes-poll

http://health.usnews.com/health-news/news/articles/2013/02/20/1-in-8-americans-diagnosed-with-type-2-diabetes-poll

http://www.cdc.gov/bloodpressure/facts.htm

http://www.cdc.gov/cholesterol/facts.htm

http://www.reuters.com/article/2010/10/15/us-metabolic-syndrome-idUSTRE69E5FL20101015

http://www.cdc.gov/obesity/data/adult.html

47

Appendix II – 2015 CCS Program Survey This worksheet is provided to assist you in completing the 2015 CCS Program Survey. You can collect the information you need and record your answers on this worksheet, then use it to enter your responses into the online survey. Please do not submit copies of this worksheet with your responses. For us to receive your program data, you will need to complete the online survey. If your CCS is part of a certified region (containing more than one county), please complete a separate survey for each county. Also, if you manage more than one CCS, please complete a separate survey for each program. Questions with asterisks (*) are required to complete the survey. Dashed lines on this worksheet indicate a page break in the online survey. Please complete the online survey by Friday, April 29. If you have any questions or difficulties with the survey, please contact Laura Blakeslee at [email protected]. Thank you!

Administrative Information 1. Please enter the name of the county contracting for or directly operating your CCS.*

2. Please enter the formal name of the county agency or contracted private agency

that operates your CCS.*

3. Does your CCS employ county employees only, or a mixture of county employees

and private contractors? * _________ County employees only _________ County employees and private contractors

4. Please enter the DQA program certification number for your CCS.* 5. Please enter the name of the person responsible for completing this survey.*

page 1

----------------------------------------------------------------------------------------------------

48

Program Information 6. How many active CCS consumers did you have on 12/31/2014? * 7. How many new admissions to your CCS did you have in 2015? *

page 2

---------------------------------------------------------------------------------------------------- 8. Total number of consumers served by your CCS in 2015:

[This number is calculated automatically by the online survey = #6 + #7] 9. How many discharges from your CCS did you have in 2015? *

page 3

---------------------------------------------------------------------------------------------------- 10. Number of active CCS consumers you had on 12/31/2015: [This number is calculated automatically by the online survey = #8 - #9] 11. How many of the continuing 2014 CCS enrollees plus new 2015 CCS enrollees being served

were concurrently enrolled in Family Care? * 12. How many of the CCS consumers discharged in 2015 were enrolled in Family Care? *

page 4 ---------------------------------------------------------------------------------------------------------------------

49

Discharge Reasons In this section, please provide information on reasons why consumers were discharged from your CCS in 2015 and where they went after discharge. In Question 13, please indicate whether or not consumers were discharged from your CCS for each reason listed. In Question 14, please enter the number of consumers discharged for each reason: if zero consumers were discharged for a particular reason, enter “0” for your answer to that reason; if there was more than one reason for a consumer’s discharge, please choose the primary reason. Your total number of discharges in Question 14 must match the number of discharges during 2015 (as reported in Question 9). 13. Were consumers discharged from your program in 2015 because ...* [If you answer "No" to any of these reasons for discharge, the online survey will automatically skip further questions about that particular reason.]

YES NO they moved from your geographic service area?

they recovered to the extent that CCS-level services were no longer needed?

funding or authorization ended for the consumer?

the consumer needed services beyond what CCS can offer (inpatient, etc.)?

the consumer decided to withdraw?

they were sent to jail?

they were sent to prison?

of death?

of unknown reasons?

of other reasons not listed above?

14. How many consumers were discharged in 2015 because ...* [The sum of the numbers entered for this question must equal the total number of 2015 discharges (reported in Question 9). Please enter “0” if no consumers were discharged for a particular reason.]

# of Consumers they moved from your geographic service area? they recovered to the extent that CCS-level services were no longer needed? funding or authorization ended for the consumer? the consumer needed services beyond what CCS can offer (inpatient, etc.)? the consumer decided to withdraw? they were sent to jail? they were sent to prison? of death? of unknown reasons? of other reasons not listed above?

page 5

----------------------------------------------------------------------------------------------------

50

Discharge Destinations For all CCS consumers discharged in 2015 for each reason listed in this section, please provide the number of consumers who transitioned to each of the following destinations. The total number of consumers discharged for each reason (across all transition destinations) will automatically appear in the final row for each question in the online survey. Please double-check that these totals match the number of consumers you reported (in Question 14) as being discharged for each reason. Each of the following questions on discharge transition destinations will be on a separate page of the online survey. If you reported that no consumers were discharged for a particular reason in Question 13, you will not see any further questions about that discharge reason. 15. For all consumers discharged in 2015 because they moved from your geographic service area, how many went

to each of the following destinations? *

# of Consumers Another CCS Outpatient therapy/psychiatry

Targeted Case Management (TCM) or other case management program

Community Support Programs (CSP) Nursing Home Inpatient/IMD Consumer did not transfer to other services Unknown Other 16. If answering "Other" in the question above (about consumers who moved from your geographic service area), please describe where these consumers went.

page 6 ----------------------------------------------------------------------------------------------------

51

17. For all consumers discharged in 2015 because they recovered to the extent that CCS-level services were no longer needed, how many went to each of the following destinations? *

# of Consumers Outpatient therapy/psychiatry


Group Home/CBRF Consumer did not transfer to other services Unknown Other 18. If answering "Other" in the question above (about consumers who recovered to the extent that CCS-level services were no longer needed), please describe where these consumers went.

page 7 ---------------------------------------------------------------------------------------------------- 19. For all consumers discharged in 2015 because funding or authorization ended for the consumer, how many went to each of the following destinations? *

# of Consumers Outpatient therapy/psychiatry


Community Support Programs (CSP) Group Home/CBRF Consumer did not transfer to other services Unknown Other 20. If answering "Other" in the question above (about consumers for whom funding or authorization ended), please describe where these consumers went.

page 8 ----------------------------------------------------------------------------------------------------

52

21. For all consumers discharged in 2015 discharged because they needed services beyond what CCS can offer, how many went to each of the following destinations? *

# of Consumers Nursing Home Community Support Programs (CSP) Inpatient/IMD Consumer did not transfer to other services Unknown Other 22. If answering "Other" in the question above (about consumers who needed services beyond what CCS can offer), please describe where these consumers went.

page 9 ---------------------------------------------------------------------------------------------------- 23. For all consumers discharged in 2015 because the consumer decided to withdraw, how many went to each of the following destinations? *

# of Consumers Outpatient therapy / psychiatry Targeted Case Management (TCM) or other CM program Community Support Programs (CSP) Group Home / CBRF Consumer did not transfer to other services Unknown Other 24. If answering "Other" in the question above (about consumers discharged because the consumer decided to withdraw), please describe where these consumers went.

page 10 ----------------------------------------------------------------------------------------------------

53

25. For all consumers discharged in 2015 for other reasons not listed in Question 14, please describe the reasons these consumers were discharged.

Other Reason 1: ______________________________________________________________________ Other Reason 2: ______________________________________________________________________ Other Reason 3: ______________________________________________________________________

page 11 ----------------------------------------------------------------------------------------------------

54

Demographic Information Please provide the following information about all CCS consumers you served in 2015. The sum of the numbers entered for each question in this section must equal the total number of consumers you served in 2015 (as calculated in #8). 26. Please enter the number of 2015 consumers of each gender.*

# of Consumers Female Male Unknown 27. Please enter the number of 2015 consumers in each age group.*

# of Consumers 17 and under 18-20 21-64 65-74 75+ Unknown 28. Please enter the number of 2015 consumers of each race.*

# of Consumers American Indian/Alaskan Native Asian Black/African American Hawaiian/Pacific Islander White More Than One Race Unknown 29. Please enter the number of 2015 consumers of each ethnicity.*

# of Consumers Hispanic/Latino Not Hispanic/Latino Unknown 30. Please enter the number of 2015 consumers who are veterans and non-veterans.*

# of Consumers Veterans Non-Veterans Unknown

page 12 ----------------------------------------------------------------------------------------------------

55

Medical Conditions and Substance Use 31. Please enter the number of 2015 consumers with the following substance use patterns. * [Count a consumer multiple times if they qualify for more than one category on the list.] # of Consumers

Use Tobacco

Abuse Alcohol

Abuse Other Drugs

32. Please enter the number of 2015 consumers with the following medical conditions. * [Count a consumer multiple times if they have more than one medical condition on the list.]

# of Consumers

Metabolic Syndrome (consumer has all of the following: high blood pressure/ hypertension, high cholesterol, and obesity around the midsection)

High blood pressure/Hypertension (exclude those with Metabolic Syndrome)

High cholesterol (exclude those with Metabolic Syndrome)

Obesity (exclude those with Metabolic Syndrome)

Type I Diabetes

Type II Diabetes

Asthma

COPD (Chronic Obstructive Pulmonary Disease)

Cardiovascular problems (angina / coronary artery disease, heart attack, or stroke)

page 13 ----------------------------------------------------------------------------------------------------

56

Evidence-Based Practices (EBPs) – Adults This section asks about EBPs your program offered adult consumers in 2015. Using the “EBP Definitions” document (sent with the email invitation for this survey), please report whether you fully offered, partially offered, or did not offer each EBP last year: being “Fully” implemented means the EBP you offered had all the critical elements; being “Partially” implemented means the EBP you offered had some of the critical elements; reporting “No” means you did not offer the EBP at all last year. If you answer "No" to an EBP in Question 33, please report “0” for that EBP in Question 34 instead of leaving it blank; the online survey will automatically skip other questions about that EBP on the following pages. 33. Did you offer the following Evidence-Based Practices (EBPs) in 2015? * [Please answer "Yes Fully," “Yes Partially,” or "No" for each EBP.]

EBP Yes Fully

Yes Partially No

Integrated Treatment for Co-Occurring Disorders (IDDT) Family Psychoeducation

Illness Management and Recovery (IMR)

MedTEAM

Supported Employment

Permanent Supportive Housing

Tobacco Cessation Bucket Approach

Other EBP (not listed here, but found on the SAMHSA website)

34. How many adult consumers received each of the following EBPs in 2015? * [Please count a consumer multiple times if they received more than one EBP last year.]

EBP # of Consumers

Integrated Treatment for Co-Occurring Disorders (IDDT)

Family Psychoeducation


MedTEAM




Other EBP (not listed here, but found on the SAMHSA website) page 14

57

35. How many consumers of each gender received each of the following EBPs in 2015? * Please count a consumer multiple times if they received more than one EBP last year. [On the online survey, the total number of consumers receiving each EBP across gender will automatically appear in the final column for each EBP listed. Please check that this total matches the number of consumers you reported (in Question 34) as receiving that EBP.]

EBP Female Male Unknown




MedTEAM




36. How many consumers in each age group received each of the following EBPs in 2015? * Please count a consumer multiple times if they received more than one EBP last year. [On the online survey, the total number of consumers receiving each EBP across all age groups will automatically appear in the final column for each EBP listed. Please check that this total matches the number of consumers you reported (in Question 34) as receiving that EBP.]

EBP 17

and under

18-20 21-64 65-74 75+ Un-known




MedTEAM



Tobacco Cessation Bucket Approach page 14, cont.

58

37. How many consumers of each race received each of the following EBPs in 2015? * Please count a consumer multiple times if they received more than one EBP last year. [On the online survey, the total number of consumers receiving each EBP across all races will automatically appear in the final column for each EBP listed. Please check that this total matches the number of consumers you reported (in Question 34) as receiving that EBP.]

EBP

Amer. Indian/ Alaskan Native

Asian

Black/ African American

Hawaiian/ Pacific Islander

White

More than One Race

Un-known




MedTEAM




page 14, cont.

59

38. How many consumers of each ethnicity received each of the following EBPs in 2015? * Please count a consumer multiple times if they received more than one EBP last year. [On the online survey, the total number of consumers receiving each EBP across ethnicity will automatically appear in the final column for each EBP listed. Please check that this total matches the number of consumers you reported (in Question 34) as receiving that EBP.]

EBP Hispanic/ Latino

Not Hispanic/ Latino

Unknown




MedTEAM




page 14, cont. ----------------------------------------------------------------------------------------------------

60

Evidence-Based Practices (EBPs) – Adults, Continued Please answer the following set of questions on your use of each EBP that your program offered to consumers during 2015. Please check that you have answered "Yes" or "No" for all questions. Refer to the “EBP Definitions” document to guide your answers to these questions. [If you answered "No" to any of the EBPs in Question 33 (to indicate your program did not use that EBP with any clients in 2015), the online survey will automatically skip the following questions about that EBP.] 39. Integrated Treatment for Co-Occurring Disorders (IDDT) * Yes No

Have CCS staff been specifically trained to implement this EBP?

Did you use the EBP toolkit to guide your implementation?

Did you monitor fidelity for this EBP?

Did you use an outside monitor to review fidelity for this EBP?

40. If you monitored fidelity for Integrated Treatment for Co-Occurring Disorders (IDDT), what fidelity measure did you use? ___________________________________________________________________

page 15 ---------------------------------------------------------------------------------------------------------------------

41. Family Psychoeducation * Yes No





42. If you monitored fidelity for Family Psychoeducation, what fidelity measure did you use? ___________________________________________________________________

page 16 ----------------------------------------------------------------------------------------------------

61

43. Illness Management and Recovery (IMR) * Yes No





44. If you monitored fidelity for Illness Management and Recovery (IMR), what fidelity measure did you use? ___________________________________________________________________

page 17 --------------------------------------------------------------------------------------------------------------------- 45. MedTEAM * Yes No





46. If you monitored fidelity for MedTEAM, what fidelity measure did you use? ______________________________________________________________

page 18 ---------------------------------------------------------------------------------------------------------------------

62

47. Supported Employment * Yes No





48. If you monitored fidelity for Supported Employment, what fidelity measure did you use? ___________________________________________________________________

Page 19 --------------------------------------------------------------------------------------------------------------------- 49. Permanent Supportive Housing * Yes No





50. If you monitored fidelity for Permanent Supportive Housing, what fidelity measure did you use? ___________________________________________________________________

page 20 ---------------------------------------------------------------------------------------------------------------------

63

51. Tobacco Cessation Bucket Approach * Yes No





52. If you monitored fidelity for Tobacco Cessation Bucket Approach, what fidelity measure did you use? ___________________________________________________________________

Page 21 --------------------------------------------------------------------------------------------------------------------- 53. What other Adult EBPs (not listed above, but found on the SAMHSA website) did you use in 2015?

Other Adult EBP 1: ______________________________________________________________________

Other Adult EBP 2: ______________________________________________________________________

Other Adult EBP 3: ______________________________________________________________________

page 22

---------------------------------------------------------------------------------------------------------------------

64

Evidence-Based Practices (EBPs) – Children This section asks about EBPs your program offered child consumers in 2015. Using the “EBP Definitions” document (sent with the email invitation for this survey), please report whether you fully offered, partially offered, or did not offer each EBP last year: being “Fully” implemented means the EBP you offered had all the critical elements; being “Partially” implemented means the EBP you offered had some of the critical elements; reporting “No” means you did not offer the EBP at all last year. [If you answer "No" to an EBP in Question 54, please report “0” for that EBP in Question 55 instead of leaving it blank; the online survey will automatically skip other questions about that EBP on the following pages.] 54. Did you offer the following Evidence-Based Practices (EBPs) in 2015? * [Please answer "Yes Fully," “Yes Partially,” or "No" for each EBP.]

Yes Fully

Yes Partially No

Multisystemic Therapy (MST)

Therapeutic Foster Care (TFC)

Functional Family Therapy (FFT)

Parent-Child Interaction Therapy (PCIT)

Trauma-Focused Cognitive Behavior Therapy (TF-CBT)

Trauma-Informed Child-Parent Psychotherapy (TI-CPP)

Other EBP (not listed here, but found on the SAMHSA website) 55. How many child consumers received each of the following EBPs in 2015? * [Please count a consumer multiple times if they received more than one EBP last year.]

# of Consumers







Other EBP (not listed here, but found on the SAMHSA website) page 23

65

56. How many consumers of each gender received each of the following EBPs in 2015? * Please count a consumer multiple times if they received more than one EBP last year. [On the online survey, the total number of consumers receiving each EBP across gender will automatically appear in the final column for each EBP listed. Please check that this total matches the number of consumers you reported (in Question 55) as receiving that EBP.]

EBP Female Male Unknown






Trauma-Informed Child-Parent Psychotherapy (TI-CPP) 57. How many consumers in each age group received each of the following EBPs in 2015? * Please count a consumer multiple times if they received more than one EBP last year. [On the online survey, the total number of consumers receiving each EBP across all age groups will automatically appear in the final column for each EBP listed. Please check that this total matches the number of consumers you reported (in Question 55) as receiving that EBP.]

EBP 0-12 13-17 18-20 Un-known






Trauma-Informed Child-Parent Psychotherapy (TI-CPP) page 23, cont.

66

58. How many consumers of each race received each of the following EBPs in 2015? * Please count a consumer multiple times if they received more than one EBP last year. [On the online survey, the total number of consumers receiving each EBP across all races will automatically appear in the final column for each EBP listed. Please check that this total matches the number of consumers you reported (in Question 55) as receiving that EBP.]

EBP

Amer. Indian/ Alaskan Native

Asian

Black/ African American

Hawaiian/ Pacific Islander

White

More than One Race

Un-known







page 23, cont.

67

59. How many consumers of each ethnicity received each of the following EBPs in 2015? * Please count a consumer multiple times if they received more than one EBP last year. [On the online survey, the total number of consumers receiving each EBP across ethnicity will automatically appear in the final column for each EBP listed. Please check that this total matches the number of consumers you reported (in Question 55) as receiving that EBP.]

EBP Hispanic/ Latino

Not Hispanic/ Latino

Unknown







page 23, cont. ---------------------------------------------------------------------------------------------------------------------

68

Evidence-Based Practices (EBPs) – Children, Continued Please answer the following set of questions on your use of each EBP that your program offered to consumers during 2015. Please check that you have answered "Yes" or "No" for all questions. Refer to the “EBP Definitions” document to guide your answers to these questions. [If you answered "No" to any of the EBPs in Question 54 (to indicate your program did not use that EBP with any clients in 2015), the online survey will automatically skip the following questions about that EBP.] 60. Multisystemic Therapy (MST) * Yes No





61. If you monitored fidelity for Multisystemic Therapy (MST), what fidelity measure did you use? ___________________________________________________________________

page 24 --------------------------------------------------------------------------------------------------------------------- 62. Therapeutic Foster Care (TFC) * Yes No





63. If you monitored fidelity for Therapeutic Foster Care (TFC), what fidelity measure did you use? ___________________________________________________________________

page 25 ---------------------------------------------------------------------------------------------------------------------

69

64. Functional Family Therapy (FFT) * Yes No





65. If you monitored fidelity for Functional Family Therapy (FFT), what fidelity measure did you use? ___________________________________________________________________

page 26 --------------------------------------------------------------------------------------------------------------------- 66. Parent-Child Interaction Therapy (PCIT) * Yes No





67. If you monitored fidelity for Parent-Child Interaction Therapy (PCIT), what fidelity measure did you use? ___________________________________________________________________

page 27 ---------------------------------------------------------------------------------------------------------------------

70

68. Trauma-Focused Cognitive Behavior Therapy (TF-CBT) * Yes No





69. If you monitored fidelity for Trauma-Focused Cognitive Behavior Therapy (TF-CBT), what fidelity measure did you use? ___________________________________________________________________

page 28 --------------------------------------------------------------------------------------------------------------------- 70. Trauma-Informed Child-Parent Psychotherapy (TI-CPP) * Yes No





71. If you monitored fidelity for Trauma-Informed Child-Parent Psychotherapy (TI-CPP), what fidelity measure did you use? ___________________________________________________________________

page 29 ---------------------------------------------------------------------------------------------------------------------

71

72. What other Child EBPs (not listed above, but found on the SAMHSA website) did you use in 2015?

Other Child EBP 1: ______________________________________________________________________

Other Child EBP 2: ______________________________________________________________________

Other Child EBP 3: ______________________________________________________________________

page 30

---------------------------------------------------------------------------------------------------------------------

72

Consumer Satisfaction 73. Did your CCS use a survey or other tool to measure consumer satisfaction in 2015? * [If you answer “No” to this question, the online survey will automatically skip other questions about consumer satisfaction.] ____Yes ____No

page 31 --------------------------------------------------------------------------------------------------------------------- 74. Which surveys or tools did you use to measure consumer satisfaction in 2015? [Please check all that apply.]

The instrument in my Evidence-Based Practice (EBP) toolkit

Recovery-Oriented Systems Inventory (ROSI) survey

Mental Health Statistical Improvement Project (MHSIP) survey

Other tool (please describe): ___________________________________________________________________

page 32 ---------------------------------------------------------------------------------------------------------------------

73

CCS Waiting List Information 75. Were there times during 2015 when there was a waiting list for CCS services from your program? * [If you answer “No” to this question, the online survey will automatically skip further questions about waiting lists.]

____Yes ____No

page 33 ---------------------------------------------------------------------------------------------------------------------

76. How many individuals were on the CCS waiting list on 12/31/2014? * ___________

77. How many additional individuals were placed on the CCS waiting list during 2015? * ___________

78. How many individuals were on the CCS waiting list on 12/31/2015? * ___________

79. How long was the average wait time (in months) during 2015 before individuals on your waiting list received CCS services? * [Please provide an average number of months, not a range of months.] ___________

80. Which of the following interim services did individuals receive while they were on your CCS waiting list? * [Please check all that apply.]

None

Case management services

Outpatient mental health services

Psychiatric services

Assistance with locating community resources

Medication management services

Outpatient substance abuse services

Crisis intervention services

Clubhouse

Drop-in center

Other services (please describe): ____________________________________________________________________________

page 34 ---------------------------------------------------------------------------------------------------------------------

74

Suicide Risk Assessment 81. Does your CCS have a policy or standard practice for assessing and managing suicide risk?

If yes, is your program using any particular tools? (If so, please list them here.)

page 35 --------------------------------------------------------------------------------------------------------------------- Final Comments 82. Do you have any clarifications about your answers, additional comments, or suggestions about how to improve this CCS Program Survey next year? 83. Please record your email address below so that we may send you an email confirmation of your survey completion and a copy of your survey responses.* [If you do not receive an email confirmation after you complete the survey, it means that we did not receive your survey and you may need to submit it again.]

page 36 Thank you for completing this survey!

75

Appendix III – Participant Satisfaction Surveys History These consumer satisfactions surveys were chosen by CCS state program and evaluation staff with the help of a CCS Advisory Committee with the intent to standardize the collection and analysis of consumer satisfaction data for CCS participants and allow for more meaningful comparisons across place and time for individual CCS programs. The Recovery Oriented System Indicators (ROSI) adult satisfaction survey evolved from collaborative efforts among a number of state mental health authorities and national organizations through a project called Mental Health Recovery: What Helps and What Hinders? Through an extensive process that included the use of participant focus groups followed by pilot testing the survey, the instrument was developed as one means to assess the performance of state and local mental health systems and providers. Initial use of this instrument in Wisconsin was among five counties who began using the ROSI Adult Survey with grants from DHS Bureau of Prevention Treatment and Recovery to implement recovery principles within evidence-based practices for mental health participants.

Methods and Scale Calculation The CCS consumer satisfaction surveys ask participants to provide feedback on their level of satisfaction with the mental health, substance use, or co-occurring disorder services they received through CCS during the past six months. Accordingly, for each of 2014 and 2015 surveys were administered to CCS participants who had been served in each respective calendar year and had a period of at least six months of enrollment in CCS. Along with technical assistance documents, conference calls, and correspondence with state evaluation staff, these surveys also are accompanied by data entry and analysis tools that allow CCS program providers to organize and use their data on consumer satisfaction to improve the CCS programming and ensure equitable outcomes for all program participants. Both youth and family Mental Health Statistical Improvement Project (MHSIP) satisfaction surveys used here are variations of the standardized MHSIP survey used by the Substance Abuse and Mental Health Services Administration for state-by-state comparisons. These surveys are aimed at younger participants with serious mental health conditions (serious emotional disorders) or substance abuse concerns. CCS programs collect satisfaction data using one (or some combination) of three methods of survey administration: in-person interview, phone interview, and self-administered survey. After collecting adult, youth and family satisfaction surveys from as many participants as possible and entering all survey responses into data entry and reporting workbooks (developed and provided by DHS, one for each of the three surveys), programs returned completed adult, youth and family workbooks to DHS for analysis. CCS programs also are encouraged to use these workbooks themselves by evaluating automatically generated scales and summary statistics (calculated across all survey items and respondents based on data entered into the workbooks) to gauge and improve satisfaction among their own participants.

Satisfaction Survey Scales ROSI Adult Scales For each question on the ROSI Adult Survey, the adult respondent was given a range of response options (from “1”=Strongly Disagree to “4”=Strongly Agree). The adult’s responses then were summarized across six

76

satisfaction domains: whether services are person-centered (Person-Centered), whether respondents experience barriers to recovery (Barriers), the degree to which respondents feel empowered by staff (Empower), the degree to which the respondent has educational/employment opportunities (Employ), the degree to which agency staff are paternalistic and/or coercive (Staff Approach), and the respondent’s financial ability to meet basic needs (Basic Needs). All the questions on the ROSI Adult Survey either fall into one of these six domains (with all of the items correlated with each scale grouped in lists below) or are not correlated with any of the scales (so are not included in any scale). Scale 1 – Person-Centered: These items describe whether clinical staff have a person-centered focus and allow for person-centered decision-making. The person-centered scale was constructed for all individuals who responded to at least six of the following items, identified by question number (q#): q23. Staff see me as an equal partner in my treatment program. q24. Mental health staff support my self-care or wellness. q30. Staff give me complete information in words I understand before I consent to treatment or medication. q31. Staff encourage me to do things that are meaningful to me. q32. Staff stood up for me to get the services and resources I needed. q33. Staff treat me with respect regarding my cultural background (think of race, ethnicity, religion, language, age, sexual orientation, etc.). q34. Staff listen carefully to what I say. q37. Mental health staff help me build on my strengths. q38. My right to refuse treatment is respected. Scale 2 – Barriers: These items describe passive barriers to recovery that respondents may experience. The barriers scale was constructed for all individuals who responded to at least four of these items: q4. I do not have the support I need to function in the roles I want in my community. q5. I do not have enough good service options to choose from. q7. Staff do not understand my experience as a person with mental health problems. q8. The mental health staff ignore my physical health. q11. I cannot get the services I need when I need them. q14. I lack the information or resources I need to uphold my client rights and basic human rights. Scale 3 – Empower: These items describe the degree to which respondents feel empowered by staff and others. The empower scale was constructed for all individuals who responded to at least two of these items: q1. There is at least one person who believes in me. q3. I am encouraged to use participant-run programs (support groups, drop-in centers, etc.). q9. Staff respect me as a whole person. Scale 4 – Employ: These items describe the degree to which educational and employment opportunities are available to the individual respondent or respondents in general. The employ scale was constructed for all individuals who responded to at least three of these items: q20. I have a chance to advance my education if I want to. q22. Mental health services helped me get or keep employment. q28. There was a participant peer advocate to turn to when I needed one. q29. There are participants working as paid employees in the mental health agency where I receive services. Scale 5 – Staff Approach: These items describe the degree to which agency staff use a paternalistic or coercive approach working with respondents. The staff approach scale was constructed for all individuals who responded to at least three of these items:

77

q27. Staff use pressure, threats, or force in my treatment. q35. Staff lack up-to-date knowledge on the most effective treatments. q36. Mental health staff interfere with my personal relationships. q41. I am treated as a psychiatric label rather than as a person. Scale 6 – Basic Needs: These items describe the respondent’s current financial ability to meet his/her basic needs. The basic needs scale was constructed for all individuals who responded to both of these items: q15. I have enough income to live on. q19. I have housing that I can afford. Other ROSI Items Not Included in Scales: These items were not strongly correlated with any of the above items in the six scales, but could be important recovery indicators on their own: q2. I have a place to live that feels like a comfortable home to me. q6. Mental health services helped me get housing in a place I feel safe. q10. Mental health services have caused me emotional or physical harm. q12. Mental health services helped me get medical benefits that meet my needs. q13. Mental health services led me to be more dependent, not independent. q16. Services help me develop the skills I need. q17. Substance abuse services help me be better able to deal with my alcohol or drug problem. q18. Substance abuse services help me have a better understanding of my addiction. q21. I have reliable transportation to get where I need to go. q25. I have a say in what happens to me when I am in crisis. q26. Staff believe that I can grow, change and recover. q39. My treatment plan goals are stated in my own words. q40. The doctor worked with me to get on medications that were most helpful for me. q42. I can see a therapist when I need to. q43. My family gets the education or supports they need to be helpful to me. q44. I have information or guidance to get the services and supports I need, both inside and outside my mental health agency. Because the ROSI Adult Survey allows responses on a scale from 1 (Strongly Disagree) to 4 (Strongly Agree), with no middle response option for being “Undecided” (as the MHSIP surveys have), the Overall Mean and six Scale values range from 1.0 to 4.0. In the Adult Survey, wording on statements used to create Scales 1, 3, 4 and 6 are positively phrased, so a value closer to 4.0 on these scales represents a more positive experience (meaning the respondent felt services were more recovery-oriented); meanwhile, wording on statements used to create Scales 2 and 5 are negatively phrased, so a value closer to 1.0 on these scales represents a more positive experience (meaning the respondent felt services were more recovery-oriented). Note: The percent of adult respondents with a mostly, mixed and less recovery-oriented experience have been adjusted for the negative scales (2 and 5), so the percent reported for these measures have the same meaning as the other (positive) scales.

MHSIP Youth Scales For each question on the MHSIP Youth Survey, the adolescent was given a range of response options (from “1”=Strongly Agree to “5”=Strongly Disagree). The youth’s responses then were summarized across six satisfaction scales (different from the six ROSI Adult Scales): general satisfaction with services (Satisfaction), satisfaction with participation in treatment planning (Participation), satisfaction with access to services (Access), satisfaction with the cultural sensitivity of providers (Culture), satisfaction with treatment outcomes

78

(Outcomes), and their level of social connectedness (Connectedness). All the questions on the MHSIP Youth Survey fall into one of these six scales. Scale 1 – Satisfaction: These items describe a youth’s overall level of satisfaction with their services. The satisfaction scale was constructed for all individuals who responded to at least four of the following items, identified by question number (q#): q1. Overall, I am satisfied with the services I received. q4. The people helping me stuck with me no matter what. q5. I felt I had someone to talk to when I was troubled. q7. The services I received were right for me. q10. I got the help I wanted. q11. I got as much help as I needed. Scale 2 – Participation: These items describe how well a youth was integrated into treatment planning. The participation scale was constructed for all individuals who responded to at least two of these items: q2. I helped to choose my services. q3. I helped to choose my treatment goals. q6. I participated in my own treatment. Scale 3 – Access: These items describe the perceived ease with which a youth obtained their mental health and substance use services. The access scale was constructed for all individuals who responded to both of these items: q8. The location of services was convenient for me. q9. Services were available at times that were convenient for me. Scale 4 – Culture: These items describe the perceived cultural sensitivity of providers. The culture scale was constructed for all individuals who responded to at least three of these items: q12. Staff treated me with respect. q13. Staff respected my family’s religious or spiritual beliefs. q14. Staff spoke with me in a way that I understood. q15. Staff were sensitive to my cultural or ethnic background. Scale 5 – Outcomes: These items are prefaced with the following phrase: “As a direct result of the mental health or substance abuse services I received,…” and describe the perceived treatment-related improvements in a youth’s life. The outcomes scale was constructed for all individuals who responded to at least five of these items: q16. I am better at handling daily life. q17. I get along better with family members. q18. I get along better with friends and other people. q19. I am doing better in school and/or work. q20. I am better able to cope when things go wrong. q21. I am satisfied with my family life right now. q22. I am better able to do things I want to do. Scale 6 – Connectedness: These items describe the extent to which youth are socially connected, have natural supports in place—family, friends, and acquaintances—to help bolster and sustain recovery. The Connectedness scale was constructed for all individuals who responded to at least three of these items: q23. I know people who will listen and understand me when I need to talk. q24. I have people that I am comfortable talking with about my child’s problems.

79

q25. In a crisis, I would have the support I need from family or friends. q26. I have people with whom I can do enjoyable things. Because the Youth Survey allows responses on a scale from 1 (Strongly Agree) to 5 (Strongly Disagree), the Overall Mean and six Scale values range from 1.0 to 5.0. Wording on all statements in the Youth Survey are positively phrased, so a value closer to 1.0 represents a more positive experience (the respondent was more satisfied) while a value closer to 5.0 represents a less positive experience (the respondent was less satisfied).

MHSIP Family Scales For each question on the MHSIP Family Survey, the caregiver (parent or guardian) was given a range of response options (from “1”=Strongly Agree to “5”=Strongly Disagree). The caregiver’s responses then were summarized across the same six scale domains used in the Youth Survey (Satisfaction, Participation, Access, Culture, Outcomes, and Connectedness), but from the perspective of the caregiver. Again, all of the questions on the MHSIP Family Survey fall into one of these six scales. Scale 1 – Satisfaction: These items describe a caregiver’s overall level of satisfaction with their child’s services. The satisfaction scale was constructed for all individuals who responded to at least four of the following items, identified by question number (q#): q1. Overall, I am satisfied with the services my child received. q4. The people helping my child stuck with us no matter what. q5. I felt my child had someone to talk to when he or she was troubled. q7. The services my child and/or family received were right for us. q10. My family got the help we wanted for my child. q11. My family got as much help as we needed for my child. Scale 2 – Participation: These items describe how well a respondent’s family members were integrated into treatment planning. The participation scale was constructed for all individuals who responded to at least two of these items. q2. I helped to choose my child’s services. q3. I helped to choose my child’s treatment goals. q6. I participated in my child’s treatment. Scale 3 – Access: These items describe the perceived ease with which mental health and/or substance abuse services were obtained. The access scale was constructed for all individuals who responded to both of these items. q8. The location of services was convenient for us. q9. Services were available at times that were convenient for us. Scale 4 – Culture: These items describe the cultural sensitivity of providers. The culture scale was constructed for all individuals who responded to at least three of these items. q12. Staff treated me with respect. q13. Staff respected my family’s religious or spiritual beliefs. q14. Staff spoke with me in a way that I understood. q15. Staff were sensitive to my cultural or ethnic background. Scale 5 – Outcomes: These items are prefaced with the following phrase: “As a direct result of the mental health or substance abuse services my child received,…” and describe the perceived treatment-related

80

improvements in respondents’ lives. The outcomes scale was constructed for all individuals who responded to at least five of these items. q16. My child is better at handling daily life. q17. My child gets along better with family members. q18. My child gets along better with friends and other people. q19. My child is doing better in school and/or work. q20. My child is better able to cope when things go wrong. q21. I am satisfied with our family life right now. q22. My child is better able to do things he/she wants to do. Scale 6 – Connectedness: These items describe the extent to which respondents’ family members are socially connected, have natural supports in place —family, friends, and acquaintances—to help bolster and sustain recovery. The connectedness scale was constructed for all individuals who responded to at least three of these items. q23. I know people who will listen and understand me when I need to talk. q24. I have people that I am comfortable talking with about my child’s problems. q25. In a crisis, I would have the support I need from family or friends. q26. I have people with whom I can do enjoyable things. Because the Family Survey allows responses on a scale from 1 (Strongly Agree) to 5 (Strongly Disagree), the Overall Mean and six Scale values range from 1.0 to 5.0. Wording on all statements in the family survey are positively phrased, so a value closer to 1.0 represents a more positive (more satisfied) experience while a value closer to 5.0 represents a less positive (less satisfied) experience.

81

Instruments Recovery Oriented System Indicators (ROSI)

2015 ADULT SATISFACTION SURVEY

To provide the best possible mental health and substance abuse services, we want to know what you think about the services you received in the last 6 months. Do not write your name or address on this survey. Your answers are confidential and will not be linked to any of the mental health and/or substance abuse services you receive.

Section One: Please indicate how much you disagree or agree with each of the following statements. Read each statement and circle the number (from “1”=Strongly Disagree to “4”=Strongly Agree) that best represents your situation in the past 6 months. If the statement is about something you did not experience, circle the last response “N/A” to indicate this item does not apply to you.

Strongly Disagree

Disagree Agree Strongly Agree

N/A

1. There is at least one person who believes in me. 1 2 3 4 N/A

2. I have a place to live that feels like a comfortable home to me. 1 2 3 4 N/A

3. I am encouraged to use consumer-run programs (support groups, drop-in centers, etc). 1 2 3 4 N/A

4. I do not have the support I need to function in the roles I want in my community. 1 2 3 4 N/A

5. I do not have enough good service options to choose from. 1 2 3 4 N/A

6. Mental health and/or substance abuse services helped me get housing in a place I feel safe. 1 2 3 4 N/A

7. Staff do not understand my experience as a person with mental health and/or substance abuse problems. 1 2 3 4 N/A

8. The mental health/substance abuse staff ignore my physical health. 1 2 3 4 N/A

9. Staff respect me as a whole person. 1 2 3 4 N/A

10. Mental health and/or substance abuse services have caused me emotional or physical harm. 1 2 3 4 N/A

11. I cannot get the mental health/substance abuse services I need when I need them. 1 2 3 4 N/A

12. Mental health/substance abuse services helped me get medical benefits that meet my needs. 1 2 3 4 N/A

13. Mental health/substance abuse services led me to be more dependent, not independent. 1 2 3 4 N/A

14. I lack the information or resources I need to uphold my client rights and basic human rights. 1 2 3 4 N/A

15. I have enough income to live on. 1 2 3 4 N/A

16. Services help me develop the skills I need. 1 2 3 4 N/A

17. Substance abuse services help me be better able to deal with my alcohol or drug problem. 1 2 3 4 N/A

18. Substance abuse services help me have a better understanding of my addiction. 1 2 3 4 N/A

82

Section Two: Please read each statement and circle the number that best represents your situation during the past 6 months. The responses range from “1”=Never/Rarely to “4”=Almost Always/Always. If the statement is about something that you did not experience, circle the last response “N/A” to indicate this item does not apply to you.

Never/ Rarely

Sometimes Often Almost Always/ Always

N/A

19. I have housing that I can afford. 1 2 3 4 N/A

20. I have a chance to advance my education if I want to. 1 2 3 4 N/A

21. I have reliable transportation to get where I need to go. 1 2 3 4 N/A

22. Mental health/substance abuse services helped me get or keep employment. 1 2 3 4 N/A

23. Staff see me as an equal partner in my treatment program. 1 2 3 4 N/A

24. Mental health/substance abuse staff support my self-care or wellness. 1 2 3 4 N/A

25. I have a say in what happens to me when I am in crisis. 1 2 3 4 N/A

26. Staff believe that I can grow, change, and recover. 1 2 3 4 N/A

27. Staff use pressure, threats, or force in my treatment. 1 2 3 4 N/A

28. There was a consumer peer advocate to turn to when I needed one. 1 2 3 4 N/A

29. There are consumers working as paid employees in the mental health/substance abuse agency where I receive services.

1 2 3 4 N/A

30. Staff give me complete information in words I understand before I consent to treatment or medication. 1 2 3 4 N/A

31. Staff encourage me to do things that are meaningful to me. 1 2 3 4 N/A

32. Staff stood up for me to get the services and resources I needed. 1 2 3 4 N/A

33. Staff treat me with respect regarding my cultural background (think of race, ethnicity, religion, language, age, sexual orientation, etc)

1 2 3 4 N/A

34. Staff listen carefully to what I say. 1 2 3 4 N/A

35. Staff lack up-to-date knowledge on the most effective treatments. 1 2 3 4 N/A

36. Mental health/substance abuse staff interfere with my personal relationships. 1 2 3 4 N/A

83

Please circle the number that best represents your situation during the past 6 months. Never/

Rarely Sometimes Often Almost Always/

Always N/A

37. Mental health/substance abuse staff help me build on my strengths. 1 2 3 4 N/A

38. My right to refuse treatment is respected. 1 2 3 4 N/A

39. My treatment plan goals are stated in my own words. 1 2 3 4 N/A

40. The doctor worked with me to get on medications that were most helpful for me. 1 2 3 4 N/A

41. I am treated as a psychiatric label rather than as a person. 1 2 3 4 N/A

42. I can see a therapist when I need to. 1 2 3 4 N/A

43. My family gets the education or supports they need to be helpful to me. 1 2 3 4 N/A

44. I have information or guidance to get the services and supports I need, both inside and outside my mental health/substance abuse agency.

1 2 3 4 N/A

Section Three: Are there other issues related to how services help or hinder your recovery? Please explain.

Section Four: Please answer the following questions to let us know a little about you. We are asking you to provide this information so we are able to have a general description of the participants taking this survey. Please circle the answer that best fits your response to the question or write in your answer on the line provided.

45. What is your gender?

1 = Female 2 = Male

46. What is your age? years

47. What is your race or ethnic background? (Circle the one that applies best.)

1 = American Indian/Alaska Native 5 = White/Caucasian

2 = Asian 6 = More than one race or ethnic group

3 = Black/African American 7 = Other (describe: )

4 = Native Hawaiian/Pacific Islander

48. Do you consider yourself Hispanic or Latino/a?

1 = Yes 2 = No

49. What is your level of education? (Circle the highest level you reached or currently are in.)

1 = Less than high school 3 = College/Technical Training 5 = Other (describe: )

2 = High school/GED 4 = Graduate School

84

50. Are you currently receiving mental health and/or substance abuse services?

1 = Mental Health only 2 = Substance Abuse only 3 = Mental Health and Substance Abuse services

51. How long have you been receiving mental health and/or substance abuse services?

1 = Less than 1 year 3 = 3 to 5 years

2 = 1 to 2 years 4 = More than 5 years

52. Which services or program have you used in the past six months? (Circle all that apply.)

1 = Community Support Program (CSP) 2 = Comprehensive Community Services (CCS)

53. What type of place do you live in? (Circle the one that applies best.)

1 = My own home or apartment 4 = Boarding house

2 = Supervised/supported apartment 5 = Homeless or homeless shelter

3 = Residential facility 6 = Other (describe: )

54. What Wisconsin county do you currently live in? County, WI

55. Do you have any other comments about the services you received in the last 6 months?

Thank you for your time and cooperation in completing this survey! [This survey was adapted from the instrument developed by Dumont JM, Ridgway P, Onken SJ, Dornan DH & Ralph RO.]

85

Mental Health Statistics Improvement Program (MHSIP) 2015 YOUTH SATISFACTION SURVEY

To provide the best possible mental health and substance abuse services, we want to know what you think about the services you received in the past 6 months. Do not write your name on this survey. Your answers are confidential and will not be linked to the mental health and/or substance abuse services you receive.

Please indicate how much you agree or disagree with each of the following statements by circling the number (from “1”=Strongly Agree to “5”=Strongly Disagree) that best represents your opinion. If the statement is about something you have not experienced, circle “N/A” to indicate that this item does not apply to you.

Strongly Agree

Agree Undecided Disagree Strongly Disagree

N/A

1. Overall, I am satisfied with the services I received. 1 2 3 4 5 N/A

2. I helped to choose my services. 1 2 3 4 5 N/A

3. I helped to choose my treatment goals. 1 2 3 4 5 N/A

4. The people helping me stuck with me no matter what. 1 2 3 4 5 N/A

5. I felt I had someone to talk to when I was troubled. 1 2 3 4 5 N/A

6. I participated in my own treatment. 1 2 3 4 5 N/A

7. The services I received were right for me. 1 2 3 4 5 N/A

8. The location of services was convenient for me. 1 2 3 4 5 N/A

9. Services were available at times that were convenient for me. 1 2 3 4 5 N/A

10. I got the help I wanted. 1 2 3 4 5 N/A

11. I got as much help as I needed. 1 2 3 4 5 N/A

12. Staff treated me with respect. 1 2 3 4 5 N/A

13. Staff respected my family’s religious or spiritual beliefs. 1 2 3 4 5 N/A

14. Staff spoke with me in a way that I understood. 1 2 3 4 5 N/A

15. Staff were sensitive to my cultural or ethnic background. 1 2 3 4 5 N/A

Questions 16-22: As a direct result of the mental health and/or substance abuse services I received … Strongly

Agree Agree Undecided Disagree Strongly

Disagree N/A

16. I am better at handling daily life. 1 2 3 4 5 N/A

17. I get along better with family members. 1 2 3 4 5 N/A

18. I get along better with friends and other people. 1 2 3 4 5 N/A

19. I am doing better in school and/or work. 1 2 3 4 5 N/A

20. I am better able to cope when things go wrong. 1 2 3 4 5 N/A

21. I am satisfied with my family life right now. 1 2 3 4 5 N/A

22. I am better able to do things I want to do. 1 2 3 4 5 N/A

Questions 23-26: Please tell us about the current relationships you have with persons other than your

86

mental health and/or substance abuse provider(s). Strongly


Disagree N/A

23. I know people who will listen and understand me when I need to talk. 1 2 3 4 5 N/A

24. I have people that I am comfortable talking with about my problems. 1 2 3 4 5 N/A

25. In a crisis, I would have the support I need from family or friends. 1 2 3 4 5 N/A

26. I have people with whom I can do enjoyable things. 1 2 3 4 5 N/A

Questions 27-35: Please answer the following questions to let us know a little about you.

27. Are you currently receiving mental health and/or substance abuse services?


28. How long have you received these services?

1 = Less than 6 months 3 = 1 year to 2 years

2 = 6 months to 1 year 4 = More than 2 years

29. Do you currently live with one or both parents?

1 = Yes 2 = No

30. What is your gender?

1 = Female 2 = Male

31. What is your age? years

32. What is your race or ethnic background? (Circle the one that applies best.)





33. Do you consider yourself Hispanic or Latino/a?

1 = Yes 2 = No

34. What Wisconsin county do you currently live in? County, WI

35. Do you have any other comments about the services you received in the last 6 months?

Thank you for your time and cooperation in completing this survey! [This survey was created by the Mental Health Statistics Improvement Program (MHSIP) in 2001.]

87

Mental Health Statistics Improvement Program (MHSIP) 2015 FAMILY SATISFACTION SURVEY

To provide the best possible mental health and substance abuse services, we want to know what you think about the services your child received in the past 6 months. The parent or guardian who had the most contact with your child’s service provider(s) should fill out this survey. Do not write your name on this survey. Your answers are confidential and will not be linked to the mental health and/or substance abuse services your child receives.

Please indicate how much you agree or disagree with each of the following statements by circling the number (from “1”=Strongly Agree to “5”=Strongly Disagree) that best represents your opinion. If the statement is about something you or your child have not experienced, circle “N/A” to indicate that this item does not apply to you.

Strongly Agree


N/A

1. Overall, I am satisfied with the services my child received. 1 2 3 4 5 N/A

2. I helped to choose my child’s services. 1 2 3 4 5 N/A

3. I helped to choose my child’s treatment goals. 1 2 3 4 5 N/A

4. The people helping my child stuck with us no matter what. 1 2 3 4 5 N/A

5. I felt my child had someone to talk to when he or she was troubled. 1 2 3 4 5 N/A

6. I participated in my child’s treatment. 1 2 3 4 5 N/A

7. The services my child and/or family received were right for us. 1 2 3 4 5 N/A

8. The location of services was convenient for us. 1 2 3 4 5 N/A

9. Services were available at times that were convenient for us. 1 2 3 4 5 N/A

10. My family got the help we wanted for my child. 1 2 3 4 5 N/A

11. My family got as much help as we needed for my child. 1 2 3 4 5 N/A

12. Staff treated me with respect. 1 2 3 4 5 N/A

13. Staff respected my family’s religious or spiritual beliefs. 1 2 3 4 5 N/A

14. Staff spoke with me in a way that I understood. 1 2 3 4 5 N/A

15. Staff were sensitive to my cultural or ethnic background. 1 2 3 4 5 N/A

Questions 16-22: As a direct result of the mental health and/or substance abuse services my child received … Strongly


Disagree N/A

16. My child is better at handling daily life. 1 2 3 4 5 N/A

17. My child gets along better with family members. 1 2 3 4 5 N/A

18. My child gets along better with friends and other people. 1 2 3 4 5 N/A

19. My child is doing better in school and/or work. 1 2 3 4 5 N/A

20. My child is better able to cope when things go wrong. 1 2 3 4 5 N/A

21. I am satisfied with our family life right now. 1 2 3 4 5 N/A

22. My child is better able to do things he/she wants to do. 1 2 3 4 5 N/A

88

Questions 23-26: Please tell us about the current relationships you have with persons other than your child’s mental health and/or substance abuse provider(s).

Strongly Agree


N/A

23. I know people who will listen and understand me when I need to talk. 1 2 3 4 5 N/A

24. I have people that I am comfortable talking with about my child’s problems. 1 2 3 4 5 N/A

25. In a crisis, I would have the support I need from family or friends. 1 2 3 4 5 N/A

26. I have people with whom I can do enjoyable things. 1 2 3 4 5 N/A

Questions 27-35: Please answer the following questions to let us know a little about your child.

27. Is your child currently receiving mental health and/or substance abuse services?


28. How long has your child received these services?

1 = Less than 6 months 3 = 1 year to 2 years

2 = 6 months to 1 year 4 = More than 2 years

29. Does your child currently live with one or both parents?

1 = Yes 2 = No

30. What is your child’s gender?

1 = Female 2 = Male

31. What is your child’s age? years

32. What is your child’s race or ethnic background? (Circle the one that applies best.)





33. Do you consider your child Hispanic or Latino/a?

1 = Yes 2 = No

34. What Wisconsin county does your child currently live in? County, WI

35. Do you have any other comments about the services your child received in the last 6 months?

Thank you for your time and cooperation in completing this survey! [This survey was created by the Mental Health Statistics Improvement Program (MHSIP) in 2001.]

89

References 1DHS, Division of Care and Treatment Services, Bureau of Prevention Treatment and Recovery. (2017). 2017 Wisconsin Mental Health and Substance Use Needs Assessment. Retrieved from: https://www.dhs.wisconsin.gov/publications/p00613-17.pdf 2 Wisconsin ethnicity statistics were taken from the United States Census Bureau QuickFacts website. (Accessed on January 26, 2017.) Available at: http://www.census.gov/quickfacts/table/PST045216/55,00. 3 Wisconsin veteran statistics were taken from the United States Census Bureau QuickFacts website. (Accessed on September 16, 2015.) Available at: http://www.census.gov/quickfacts/table/PST045214/00,55. 4 Alcohol abuse and illicit drug use data taken from Substance Abuse and Mental Health Services Administration, Center for Behavioral Health Statistics and Quality (CBHSQ), National Survey on Drug Use and Health (NSDUH), 2010 to 2014. (Accessed on February 1, 2017.) Available at: https://www.samhsa.gov/data/sites/default/files/2015_Wisconsin_BHBarometer.pdf. 5 Druss, B.G., & Walker, E.R. (2011). Mental disorders and Medical Comorbidity. Research Synthesis Report, 21. Retrieved from: http://www.ibhpartners.org/wp-content/uploads/2015/12/co-occurring-disorders-Druss.pdf 6 Ziege, Anne and Tim Connor. (2009). “Linking Mental and Physical Health: Results from the Wisconsin Behavioral Risk Factor Survey.” Wisconsin DHS, Division of Public Health, Bureau of Health Information and Policy. 7 Schulte, MT and Y-I Hser. (2014). “Substance use and associated health conditions throughout the lifespan.” Public Health Reviews, 35(2): epub: www.publichealthreviews.eu.

https://www.dhs.wisconsin.gov/publications/p00613-17.pdf

http://www.census.gov/quickfacts/table/PST045216/55,00

http://www.census.gov/quickfacts/table/PST045214/00,55

https://www.samhsa.gov/data/sites/default/files/2015_Wisconsin_BHBarometer.pdf

http://www.ibhpartners.org/wp-content/uploads/2015/12/co-occurring-disorders-Druss.pdf

ccs 2015 monitoring report › publications › p01224-15.pdf · ccs programs are designed to be...

Documents