outcomes of residential treatment: a study of the adolescent clients of girls and boys town

Outcomes of Residential Treatment:A Study of the Adolescent Clients ofGirls and Boys Town

Robert E. LarzelereKatherine DingesM. Diane SchmidtDouglas F. SpellmanThomas R. CristePatrick Connell

Girls and Boys Town

ABSTRACT: Pre-treatment and post-treatment data was collected from adolescentclients of a new residential treatment center at Girls and Boys Town. Youth who receivedtreatment improved significantly on the Child Behavior Checklist and the Children’sGlobal Assessment Scale and were maintaining their treatment gains at follow-up. Tenmonths following discharge, the majority were stabilized and functioning adequately inschool and with their primary caregiver. For these youth, residential treatment succeededwhere other interventions failed.

KEY WORDS: residential treatment—evaluation; youth in care; Boys Town.

Local communities are reluctant to fund long-term inpatient psychi-atric care or treatment (Chang, Sanacora, & Sanchez, 1996), and familypreservation is not a panacea for all youth in need of help (Gelles,1991). The gap between these two choices includes intermediate staytreatment options for many youth. Residential treatment centers havebecome a last-resort treatment for the most difficult youth, with psychi-atric units being used mostly for short-term crisis management.

Residential treatment centers are defined here as out-of-home facili-ties for mental health treatment that are more treatment-oriented thangroup homes, but less restrictive than inpatient psychiatric units. Thisstudy of youth outcomes is from one such treatment center, located at

Correspondence should be directed to Dr. Robert E. Larzelere, Director of BehavioralHealth Research, Youth Care Building, Father Flanagan’s Boys’ Home, Boys Town, NE68010.

Child & Youth Care Forum, 30(3), June 2001, 2001 Human Sciences Press, Inc. 175

Child & Youth Care Forum176

Girls and Boys Town and designed for severely disturbed children andadolescents.

Residential treatment centers are less costly than inpatient psychiat-ric units, yet they have the potential for treating severely disturbedchildren who have not been successfully treated in less restrictive treat-ment settings (e.g., foster care, out-patient treatment, day treatment,group homes). However, there has been relatively little evidence ofthe effectiveness of residential treatment centers, especially for newprograms designed to be less restrictive and less costly than inpatienttreatment.

The Residential Treatment Center (RTC) at Girls and Boys Townimplements a psychoeducational treatment model. It is a coordinatedcombination of cognitive-behavioral and educational treatment compo-nents designed to supplement and support the more traditional psychi-atric and clinical treatment modalities for children and adolescents(Daly et al., 1998; Sterba, Davis, & Criste, 2000). Similar to manyother residential treatment centers, the RTC combines psychiatricsupervision of psychotropic medication with an integrated treatmentplan developed by a multi-disciplinary team. The treatment incorpo-rates individual, group, and family psychotherapy, and special educa-tion.

Minimal treatment components for each youth include weekly meet-ings with the medical director (a child psychiatrist), weekly familypsychotherapy whenever feasible, and daily work on academic subjectsin a therapeutic special education atmosphere. Distinctive aspects ofthe RTC treatment include a contingency-based system of managingbehavior and a behaviorally specified social skills curriculum that isused to develop replacement skills for previously problematic behaviors.The youth practice their prescribed social skills and self-control strate-gies (e.g., calming skills) during daily interactions initiated by direct-care staff. Staff are trained in the classroom and on the job to implementtreatment and build relationships. The staff-youth ratio is 1 :2.5 duringthe day and 1:6 at night.

The contingency management techniques and social skills curriculawere developed from the Teaching Family Model (Dowd, Herron, Hy-land, & Sterba, 1998; Friman, 1999). The current treatment modelexpands on the older Teaching Family Model by teaching positive socialskills to replace dysfunctional ways of dealing with stress or anger, byemphasizing the daily use of self-control skills, and by emphasizingbetter generalization of treatment gains to a post-discharge environ-ment, as suggested by Lyman and Campbell (1996). Generalization isalso promoted through initial and ongoing discharge planning—includ-ing anticipated aftercare services by incorporating family members and

Larzelere, Dinges, Schmidt, Spellman, Criste, and Connell 177

community representatives into the treatment team process wheneverpossible—and through support of appropriate behavior and academicprogress in a school-like setting for each youth. More details are pre-sented in Daly et al. (1998) and in Sterba et al. (2000).

Previous research reviews on traditional residential treatment, morebroadly defined, have found only a few studies documenting its effec-tiveness (Bates, English, & Kouidou-Giles, 1997; Pfeiffer & Strzelecki,1990). Pfeiffer and Strzelecki’s review (1990) concluded that residentialtreatment was effective overall, but their only evidence was that certainaspects of treatment predicted better outcomes in four studies (e.g.,having a planned discharge, a good therapeutic alliance).

Similarly, Bates, English, and Kouidou-Giles (1997) noted thatthe evidence for treatment effectiveness was weak not only for residen-tial treatment (broadly defined), but also for less restrictive alterna-tives such as treatment foster care and intensive family preservationservices. They found no evidence of differential effectiveness favoringresidential treatments over other options such as therapeutic grouphomes and long-term psychiatric inpatient treatment. The failure tofind differential effectiveness may be due to several factors. Histori-cally, residential treatment has been securely funded and had littlemotivation to document its effectiveness. Ethical constraints limitedthe most conclusive types of research evaluations, because it was uneth-ical to randomly assign such needy youth to no-treatment controlgroups.

This study attempted to overcome many of the methodological prob-lems of previous evaluations of residential treatment centers. Pfeiffer(1989) noted 16 research design issues, only four of which were handledappropriately in the majority of treatment outcome studies of residen-tial treatment in his review. This study deals adequately with 11 ofPfeiffer’s 16 methodological issues. These included psychiatric diagno-ses, a description of the intervention, follow-up information, an ade-quate response rate, and multiple measures from multiple sources atintake, discharge, and follow-up. The measures included standardizedmeasures as well as indicators of post-discharge functioning. The studyalso included information on prior treatment, used uniform criteria fordiagnoses, and used appropriate inferential statistics.

The five methodological issues that we were not able to deal withadequately included subdivision of the sample into different subgroups,exploration of the role of specific treatment components, inclusion of acontrol group, blind data collection, and a uniformly set time for thefollow-up interview. We aimed for follow-up interviews 6 months afterdischarge, but many follow-up interviews were completed substantiallylater to increase our response rate.


Method

Participants

This study incorporates all available data on the first 43 youth whowere treated at the Residential Treatment Center. Their average agewas 13.0 years at intake, with a range of 6 to 17 years. They included21 boys and 22 girls. Twenty-six percent were Black, one youth wasNative American, and the remaining 72% were White. Primary diagno-ses at intake were depressive disorders (51%), with smaller percentagesof disruptive disorders (21%), adjustment disorders (9%), post-trau-matic stress disorder (9%), attention deficit hyperactivity disorder (7%),and schizo-affective disorder (2%). Many youth had secondary diagno-ses as well. On average, they had experienced four different placementsduring the 6 months before coming to the RTC. They stayed in treat-ment at the RTC from 18 to 505 days, with a mean treatment durationof 181 days (median = 165 days).

Procedures

Most measures were administered at intake, at discharge, and/or aspart of a follow-up survey. The follow-up surveys occurred at an averageof 10 months after discharge, with a range from 6 to 21 months. Thefollow-up response rate was 65% (n = 28), although there were addi-tional missing data for some items, partly due to changes in the follow-up interview. Thus sample sizes are noted for specific follow-up results.

Measures

The Child Behavior Checklist (CBCL) is a widely used measure ofproblem behaviors, with well documented reliability and validity (Ach-enbach, 1991). It provides three broad-band scales (Internalizing, Ex-ternalizing, and Total Problems) and eight more specific narrow-bandscales (e.g., Anxious/Depressed, Attention Problems, Aggressive Behav-ior). The Internalizing scale represents a range of internally directedsymptoms (anxiety, depressive, withdrawal, and somatic), whereas theExternalizing scale represents externally directed symptoms (aggres-sion, delinquency).

The goal was to have the CBCL completed by the adult most knowl-edgeable about the youth’s behavior in the preceding months. Mostintake CBCLs were completed by a parent or other caregiver (85% ofintake CBCLs). A few were completed by an RTC therapist after theyouth had been in treatment for about 1 month (15%). Most dischargeCBCLs were completed by the youth’s primary therapist at the RTCnear their discharge time (77%). Other discharge CBCLs were com-


pleted by a subsequent caregiver about 30 days after they were dis-charged from the RTC (23%). The follow-up CBCLs were completed bya parent or other caregiver responsible for the youth at that time.

The Children’s Global Assessment Scale (C-GAS) (Shaffer et al.,1983) is an overall assessment of a youth’s level of functioning. It isequivalent to Axis V on the DSM-IV. It was administered at intakeand discharge.

The Restrictiveness of Living Environments Scale (ROLES) (Hawk-ins, Almeida, Fabry, & Reitz, 1992) indicates the relative restrictive-ness of 25 settings youth might be living in, ranging from their parentalhome to jail or a psychiatric hospital. It was completed from case recordsfor all placements during the 6 months prior to coming to the RTC. Itwas also completed for the discharge placement. The follow-up surveyalso used this scale for all placements during the 6 months after dis-charge from the RTC.

The Youth Satisfaction Survey is a 12-item measure of the youth’ssatisfaction with various aspects of the treatment program. It wasdeveloped at Girls and Boys Town, based on previous measures ofconsumer satisfaction (Wolf, Kirigin, Fixsen, Blase, & Braukmann,1995). The 12 items ask about satisfaction with the treatment program(5 items), direct-care staff’s relationship with the youth (6 items), andthe safety of the treatment residence (1 item). Youth rated, on a 7-point scale, the extent to which they were satisfied. Its coefficient alphawas .57 in this sample. It was administered to the youth just beforetheir discharge.

A telephone follow-up survey was administered to the youth’s care-giver. Although planned for six months after discharge, it was notcompleted until substantially later for some youth. The follow-up sur-vey was modified midway through the study, resulting in missing datafor some questions. Items on all versions of the follow-up survey askedabout whether they were in school and/or employed, whether they hadused psychological or psychiatric services after discharge, and whethertheir quality of life had improved as compared to prior to their RTCtreatment. Other questions for most of the follow-up cases includeditems about doing better or worse in school, the quality of their relation-ship with their primary caregiver, the frequency of serious conflictswith their primary caregiver, relationship problems with others, anddelinquent behavior.

Results

Youth showed improvement on most outcome variables, improve-ments that were generally maintained at the follow-up. Standardized


outcome assessments will be summarized first, followed by satisfactionmeasures, and then by specific questions from the follow-up survey.

Youth improved significantly on the Child Behavior Checklist, andmaintained those gains at the follow-up. Table 1 presents the mean T-scores on the CBCL scales at admission, discharge, and follow-up forthe 18 youth with completed CBCLs at all three occasions. Althoughthe response rate on the CBCL was 95% at intake, 61% at departure,and 63% at the follow-up, only 41% had completed CBCLs at all threeoccasions. This subsample did not differ significantly from the other59% on admission scores on the C-GAS, t (41) = −46, n.s; the intakeCBCL, t (39) = −.68 for Total Problems, n.s.; the restrictiveness of theprior placement, χ2 (1) = .00, n.s.; or the number of placements in theprior 6 months, t (40) = .25, n.s.

The youth improved significantly on most narrow-band scales andon all broad-band scales. The broad-band mean scores improved aboutone standard deviation, from high in the clinical range at intake (e.g.,Total Problems T-score = 74.4) to in or near the borderline clinical rangeat discharge (e.g., 64.3), an improvement maintained at the time of the

Table 1Mean CBCL T-Scores at Admission, Discharge, & Follow-Up

Scale Admission Discharge Follow-Up F-value

Withdrawn 66.3 60.7 60.9 4.18*Somatic Complaints 60.4 55.3 57.3 3.08b

Anxious/Depressed 71.1 64.6 60.1 11.20***Social Problemsa 67.1 63.3 66.5 2.00Thought Problemsa 65.1 60.7 60.7 1.16Attention Problemsa 72.8 63.0 64.0 10.82***Delinquent Behavior 76.1 66.7 66.6 10.54***Aggressive Behavior 75.9 62.3 64.8 10.01***Internalizing

Problems 69.4 62.7 59.8 9.87***Externalizing

Problems 75.8 63.8 65.4 11.27***Total Problemsa 74.4 64.3 65.1 11.96***

Note. N = 18, except where otherwise noted.aN = 16.bp < .10.*p < .05. **p < .01. ***p < .001.


follow-up (65.1). The four narrow-band scales with the highest pre-treatment means all showed similar improvement, p < .001. The otherfour narrow-band scales showed smaller gains, which significantly de-creased only for the Withdrawn subscale, p < .05.

Pairwise comparisons with larger sample sizes had generally similarresults on the CBCL, with the following exceptions: Improvements fromintake to discharge did not reach significance for three internalizingscales: Internalizing, Anxious/Depressed, or Withdrawn, n = 24. In con-trast, improvements from intake to follow-up were significant in So-matic Complaints as well as all the scales with significant gains inTable 1, ns of 22 or 25.

The Children’s Global Assessment Scale (C-GAS) mirrored thosechanges, showing that a more complete sample improved significantlyfrom intake to discharge, t (39) = 9.20, p < .001. The mean score atintake (36.8) reflected “Major impairment in functioning in severalareas and unable to function in one of these areas.” In contrast, themean C-GAS score at discharge (55.6) indicated that youth then dis-played “Variable functioning with sporadic difficulties or symptoms inseveral but not all social areas.”

The Restrictiveness of Living Environment Scale (ROLES) indicatedthat most youth were in a less restrictive environment following dis-charge. Although 67% came from a more restrictive setting into theRTC treatment, only 9% were discharged to a more restrictive setting,χ2 = 25.14, n = 43, p < .001. Placement stability also improved. Duringthe six months before coming to the RTC treatment, the youth averaged3.0 placement changes, compared to an average of 0.5 placementchanges during the six months following discharge for the 28 youthwith that information, t (27) = 4.97, p < .001. Further, most placementchanges after discharge from the RTC were to even less restrictivesettings (8 of the 13 changes).

The mean satisfaction score in the Youth Satisfaction Scale at dis-charge was 6.4 on a 7-point scale (n = 29). This mean score is between“satisfied” and “completely satisfied.”

Consistent with these results, 96% were either going to school (89%)or had graduated and were working (7%) at the time of the follow-upinterview (out of n = 28). Of those asked (n = 19), 79% were reportedas doing the same or better in school than they had been before theRTC treatment.

The youth also tended to be relating fairly well to their adult caregiverat follow-up even though such relationships were not problem-free.Over half (60%) of the youths’ caregivers said that the relationshipbetween the youth and them was going “fairly well,” whereas another20% said it was going “extremely well.” Most of the other cases (15%)said it was going “Neither good nor bad,” and only 5% said the relation-


ship was “fairly poor” (out of n = 20). Although most of these relation-ships were going fairly well, 43% of the caregivers reported some inci-dents of “serious conflict” occurring monthly or more often (n = 21).Similarly, 52% reported that the youth had had relationship problemsin multiple settings in the time since discharge (n = 21).

The follow-up survey indicated that specific delinquency problemswere less likely or equally likely after the RTC treatment compared tothe six months prior to admission. The same questions were not askedat admission, but similar items on the CBCL for the same time period(6 months) permitted the following comparisons (ns from 18 to 22): Afterthe RTC treatment, youth were less likely to be involved in propertydestruction (26% afterwards, compared to 74% prior to admission, χ2

= 9.00, p < .01), running away (14% vs. 77%, χ2 = 14.00, p < .01), theft(33% vs. 61%, χ2 = 2.78, p < .10) or physical assault (37% vs. 68%, χ2 =6.00, p < .05). They were involved in other problems at the same rateas they were during the 6 months prior to admission: substance use(alcohol or drugs: 22% vs. 33%), fire setting (11% vs. 11%), truancy(37% vs. 42%), and lying (89% vs. 89%). The percentage involved inany of those behaviors was never greater after discharge than it wasprior to admission.

One key to the maintenance of these treatment gains was utilizingseveral kinds of outpatient treatment after discharge. The majority(86%) had received psychological or psychiatric outpatient treatmentafter discharge (n = 28). The most common types of treatment wereindividual treatment (83% of those receiving treatment) and medicationchecks (46%).

Overall, caregivers reported that 76% of the youth now had a qualityof life that was better than what they were experiencing prior to theirRTC treatment. Another 16% had a life quality about the same asbefore, leaving only 8% whose lives were rated as worse than beforethe RTC treatment (n = 25).

Discussion

As a whole, these findings indicate that youth tended to improve ona range of outcomes during their treatment at the Girls and Boys TownRTC. The youth improved about one standard deviation on most CBCLscales. This improvement was also reflected on the C-GAS and on theirlevel and stability of placement. After discharge, they were placed ina less restrictive treatment setting (home in 44% of the cases) and weremuch less likely to move to other treatment settings.

Consistent with those gains, almost all youth were attending schooland getting along at least fairly well with their current adult caregiver


at the time of the follow-up. In general, the youth were functioningwell, considering the high clinical symptom levels they evidenced uponintake to the RTC.

Implications for Residential Treatment Centers

This study suggests that a residential treatment center can servean important function in treating severely disturbed children. It isimportant to note that the youth not only improved on a variety ofscales, but their lives stabilized. This combination of outcomes is impor-tant in current efforts to find the most effective, cost-efficient treat-ments for a wide range of needy youth. One role of residential treatmentcenters in emerging behavioral health systems is to help youth whosesymptom severity has resulted in numerous placements, none of whichhave successfully helped the youth return to an adequate level of func-tioning at home and at school. For such youth, residential treatmentcenters may be optimal for preparing them to return to reasonablyadequate functioning in a less restrictive setting.

What features of this particular residential treatment program mightaccount for its success? It is difficult to identify crucial treatment compo-nents, since the results apply to the entire treatment package. Thecrucial components could be those shared with other residential treat-ments. In addition, some distinctive features of this treatment mighthave enhanced its effectiveness. Possible distinctives include a range ofpositively oriented behavioral strategies, including replacement socialskills such as calming skills for coping with stressful situations moreappropriately. The full range of social skills is implemented by thedirect-care staff at neutral practice times as well as on occasions whenproblems arise. Thus treatment is being implemented during all wakinghours to supplement therapy hours with the psychiatrist, family thera-pist, and individual and group counselors. Further, most youth contin-ued in some form of outpatient treatment after discharge, which mayhave accounted for their placement stability and improved functioning.

Limitations

Along with its strengths, some limitations of this study should benoted. First, there was no control group, so we cannot be sure that theseyouth would not have made these gains without the RTC treatment.

It should also be noted that we did not have follow-up data on about1/3 of the youth and we had incomplete CBCL data on 59% of them.Youth with incomplete data were not significantly different on any pre-treatment measures, and they improved significantly on three of fouravailable outcomes, but they did not improve as much as those with


complete CBCL data. The 24 youth with incomplete CBCLs improvedfrom 36.9 to 50.8 on the C-GAS, a significant increase, t (23) = −4.73,p < .001, but significantly lower than the complete-CBCL group atdischarge, t (39) = −3.39, p < .01. They also reduced their restrictivenessof placement. Fifteen of the 25 youth with incomplete CBCLs had comefrom more restrictive settings than the RTC, but only 4 youth were dis-charged to a more restrictive setting, a significant decrease, χ2 (1) =12.3, p < .001. Moreover, this was not a significantly higher probabilityof placement to a more restrictive setting than in the subgroup withcomplete CBCL data, p = .13, Fisher’s exact test.

Other evaluations of the group with incomplete CBCL data werehampered by other missing data. Placement stability improved for 11youth with that information, from 3.3 moves before the RTC to 0.8moves afterwards, in equivalent 6-month periods, t (10) = 2.36, p < .05.However, the number of post-RTC moves was more than the completeCBCL group, M = .2, t (26) = 2.16, p < .05. Finally, 6 youth who lackedfollow-up CBCLs had higher mean CBCL scores at discharge than atintake, although not significantly so, ps > .05. They scored significantlyhigher on Externalizing, Internalizing, and Total Problems after theRTC than did the 18 with complete CBCL data, with t-values from2.62 to 3.17 (p < .05).

Thus more complete follow-up information would have identified afew more youth who, on average, did not improve as much. This concernis mitigated, however, by the fact that we also documented significantimprovement on the C-GAS and the ROLES for the full group, whichwere completed for over 95% of the youth. It is likely then that morecomplete data would have reduced the improvements reported herein,but not eliminated or reversed them.

Conclusions

In conclusion, the youths’ symptoms decreased and they functionedfairly well in a less restrictive setting after participating in this RTCtreatment. Overall their placement was more stable than it had beenbeforehand.

This suggests that residential treatment centers can effectively treatthe more difficult youth and prepare most of them to function fairlyadequately in a stable, less restrictive setting. However, better researchis needed on residential treatment centers, especially using an equiva-lent comparison group. We also need to know what treatment compo-nents are optimal for treating youth at this level of difficulty. How canwe optimally match treatment setting and treatment strategies withthe treatment needs of these youth? We need to evaluate each level of


treatment in systems of care, before we are ready to evaluate the systemof care as a whole (Weisz, Han, & Valeri, 1997).

References

Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 Profile.Burlington, VT: University of Vermont Department of Psychiatry.

Bates, B. C., English, D. J., & Kouidou-Giles, S. (1997). Residential treatment and itsalternatives: A review of the literature. Child & Youth Care Forum, 26, 7–51.

Chang, R., Sanacora, G., & Sanchez, J. R. (1996). The need for outcome studies [letter].Journal of the American Academy of Child and Adolescent Psychiatry, 35, 557.

Daly, D. L., Schmidt, M. D., Spellman, D. F., Criste, T. R., Dinges, K., & Teare, J. (1998).The Boys Town Residential Treatment Center: Treatment implementation and pre-liminary outcomes. Child & Youth Care Forum, 17(4), 267–279.

Dowd, T., Herron, R., Hyland, T., & Sterba, M. (Eds.). (1998). Issues in quality childcare. Boys Town, NE: Boys Town Press.

Friman, P. C. (1999). Behavioral family-style residential care for troubled out-of-homeadolescents: Recent findings. In J. E. Carr & J. Austin (Eds.), Handbook of appliedbehavior analysis. Reno, NV: Context Press.

Gelles, R. J. (1991). Physical violence, child abuse, and child homicide: A continuum ofviolence or distinct behavior? Human Nature, 2(1), 59–72.

Hawkins, R. P., Almeida, M. C., Fabry, B., & Reitz, A. L. (1992). A scale to measurerestrictiveness of living environments for troubled children and youths. Hospitaland Community Psychiatry, 43(1), 54–58.

Lyman, R. D., & Campbell, N. R. (1996). Treating children and adolescents in residentialand inpatient settings. Thousand Oaks, CA: Sage.

Pfeiffer, S. I. (1989). Follow-up of children and adolescents treated in psychiatric facilities:A methodology review. The Psychiatric Hospital, 20, 15–20.

Pfeiffer, S. I., & Strzelecki, S. C. (1990). Inpatient psychiatric treatment of children andadolescents: A review of outcome studies. Journal of the American Academy of Childand Adolescent Psychiatry, 29(6), 847–853.

Shaffer, D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., & Aluwahlia,S. A. (1983). A children’s global assessment scale (CGAS). Archives of General Psychi-atry, 40, 1228–1231.

Sterba, M., Davis, J., & Criste, T. R. (2000). Boys Town’s psychoeducational treatmentmodel: A training manual for helping behavioral health professionals care for andtreat troubled children and adolescents. Boys Town, NE: Boys Town Press.

Sterba, M., & Dowd, T. (1998). Treating youth with DSM-IV disorders. Boys Town, NE:Boys Town Press.

Weisz, J. R., Han, S. S., & Valeri, S. M. (1997). More of what? Issues raised by the FortBragg Study. American Psychologist, 52, 541–545.

Wolf, M. M., Kirigin, K. A., Fixsen, D. L., Blase, K. A., & Braukmann, C. J. (1995).The teaching-family model: A case study in data-based program development andrefinement (and dragon wrestling). Journal of Organizational Behavior Manage-ment, 15, 11–68.

outcomes of residential treatment: a study of the adolescent clients of girls and boys town

Documents