the role of therapeutic alliance in therapy outcomes for youth in residential care

This article was downloaded by: [Universiteit Twente]On: 29 November 2014, At: 15:16Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Residential Treatment forChildren & YouthPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/wrtc20

The Role of TherapeuticAlliance in Therapy Outcomesfor Youth in Residential CareMichael L. Handwerk PhD a , Jonathan C. HuefnerPhD , Jay L. Ringle MA , Brigid K. Howard MA ,Stephen H. Soper MS , Julie K. Almquist MS , M. BethChmelka BS & Father Flanagan's Boys' Homea National Research Institute , 13603 Flanagan Blvd.,Boys Town, NE, 68010Published online: 12 Dec 2008.

To cite this article: Michael L. Handwerk PhD , Jonathan C. Huefner PhD , Jay L.Ringle MA , Brigid K. Howard MA , Stephen H. Soper MS , Julie K. Almquist MS , M.Beth Chmelka BS & Father Flanagan's Boys' Home (2008) The Role of TherapeuticAlliance in Therapy Outcomes for Youth in Residential Care, Residential Treatment forChildren & Youth, 25:2, 145-165

To link to this article: http://dx.doi.org/10.1080/08865710802310152

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, and

http://www.tandfonline.com/loi/wrtc20

http://dx.doi.org/10.1080/08865710802310152

are not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

http://www.tandfonline.com/page/terms-and-conditions

Residential Treatment for Children & Youth, Vol. 25(2) 2008Available online at http://www.haworthpress.com© 2008 by The Haworth Press. All rights reserved.

doi:10.1080/08865710802310152 145

WRTC0886-571X1541-0358Residential Treatment for Children & Youth, Vol. 25, No. 2, July 2008: pp. 1–29Residential Treatment for Children & Youth

The Role of Therapeutic Alliance in Therapy Outcomes for Youth in Residential Care

Handwerk et al.RESIDENTIAL TREATMENT FOR CHILDREN & YOUTH Michael L. Handwerk, PhDJonathan C. Huefner, PhD

Jay L. Ringle, MABrigid K. Howard, MAStephen H. Soper, MSJulie K. Almquist, MSM. Beth Chmelka, BS

Father Flanagan’s Boys’ Home

SUMMARY. This study examined the impact of therapeutic alliance(TA) on therapy outcomes for youth with behavioral and emotional prob-lems residing in residential care. Study participants were 71 youth in anout-of-home family-style residential treatment facility who were referred toan onsite psychotherapy clinic. A therapeutic alliance scale was completedindependently after each session by the youth and their therapist. Two out-come measures were used: a symptom scale that was completed by theyouth before each therapy session, and a daily observational measure madeby direct care staff of all significant events for each youth. Youth symp-toms improved significantly over the course of therapy and their behaviorsdecreased to a rate similar to their peers. TA ratings, however, wereonly marginally related to therapy outcomes. Implications for assessingtherapeutic alliance in child therapy are discussed.

KEYWORDS. Therapy, therapeutic alliance, therapy outcome, children &adolescents

Jonathan C. Huefner may be written at the National Research Institute, 13603Flanagan Blvd., Boys Town, NE 68010 (E-mail: [email protected]).

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

146 RESIDENTIAL TREATMENT FOR CHILDREN & YOUTH

Therapy is an established intervention for ameliorating emotional andbehavioral problems of children and adolescents (Weiss & Weisz, 1995;Weisz, Weiss, Han, Granger, & Morton, 1995). Meta-analyses indicate anaverage treatment effect size for youth participating in therapy that isvirtually identical to the effect size seen for adults (Casey & Berman,1985; Kazdin, Bass, Ayers, & Rodgers, 1990; Weiss & Weisz, 1995;Weisz, Weiss, Alicke, & Klotz, 1987).

Although not unequivocal, some research indicates that regardless ofthe specific form or type of therapy (e.g., cognitive behavioral, psychody-namic, interpersonal, etc.) adults experience similarly positive outcomes(Wampold et al., 1997), prompting some to deemphasize the importanceof specific techniques and focus instead on factors common to all formsof therapy (Lambert & Barley, 2001). Therapeutic alliance (TA) has beenhypothesized to be an important common factor in producing improve-ments for therapy participants. TA has been conceptualized in a variety ofdifferent schemes (Bordin, 1979; Horvath & Greenberg, 1989), thoughprominent conceptualizations include most of the following constructs:client-therapist relationship/bond, agreement on goals, collaboration ontasks, and client resistance. Extant research examining therapy outcomesfor adults has found a modest but consistent relationship between TA andtherapy outcomes (Horvath & Symonds, 1991). A meta-analysis of 79studies with adults reported an average correlation between TA and ther-apy outcome to be r = .22 (Martin, Garske, & Davis, 2000).

TA may be even more important in child therapy than in adult therapy(Bickman et al., 2004). Adults usually voluntarily seek professional helpfor their problems, while children rarely seek or initiate professional help.Rather, children often attend therapy “involuntarily” at the request (orinsistence) of a parent or other adult because of perceived problems. Thismay place more importance on the relationship between youth and thera-pist, as youth may be more reluctant to share and disclose with, as well aslisten and attend to, an adult therapist. For example, several studiesreported low TA to be related to high premature termination rates forchild psychotherapy (Garcia & Weisz, 2002; Kendall & Sugarman, 1997;but see Robbins, Turner, Alexander, & Perez, 2005). Johnson, Wright,and Ketring (2002) found that TA accounted for 19%–55% of the vari-ance in changes on ratings of symptom distress for 43 families participat-ing in family therapy. Similarly, TA has been found to correlate withtherapy outcome for youth with posttraumatic stress disorder (PTSD;Cloitre, Stovall-McClough, Miranda, & Chemtob, 2004), adolescentdelinquent males (Florsheim, Shotorbani, Guest-Warnick, Barratt, &

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 147

Hwang, 2000), children referred for oppositional behavior (Kazdin,Marciano, & Whitley, 2005), and anxious youth (Johnson, Wright, &Ketring, 2002).

Conversely, unlike the adult psychotherapy outcome literature, meta-analysis examining therapy outcomes for children and adolescents indicatesbehavioral therapies tend to produce better outcomes than nonbehavioraltherapies (Weiss & Weisz, 1995). This suggests that specific techniquesmay be relatively more important in therapeutic interventions with childrenthan the therapeutic relationship. For example, studies of youth with anxietydisorders (Kendall et al., 1997; McLeod & Weisz, 2005), with depression(Kaufman, Rohde, Seeley, Clarke, & Stice, 2005), in a partial hospitaliza-tion program (Bickman et al., 2004), and from low socioeconomic status(SES) families (DeVet, Kim, & Charlot-Swilley, 2003) found no or littlerelationship between TA and therapy outcome for some or all of the par-ticipants/raters.

Despite the importance of understanding the relationship between TAand therapy outcomes for children and adolescents, the role of TA in childtherapy remains murky. Compared to the adult literature on therapeutic alli-ance, relatively few studies have investigated TA in child psychotherapy,and those that have produce inconsistent results both within and acrossstudies (Jensen, Weersing, Hoagwood, & Goldman, 2005; Kazdin, 1982).

A recent meta-analysis (Shirk & Karver, 2003) examined 23 studiesthat investigated the impact of the relationship between children and ther-apists on therapy outcomes. Across all studies, the results indicated anaverage correlation between TA and therapy outcome to be r = 0.24,similar to correlations found between TA and outcome in adult meta-analyses. Several mediating variables impacted the strength of the rela-tionship between TA and outcome for youth. Of interest, measures of TAcollected later in therapy were more strongly associated with therapy out-come than those collected during the initial phases of therapy, and mea-sures of TA collected from therapists and parents were more stronglyrelated to outcomes than children’s’ ratings. Many studies reported thatchildren’s ratings showed limited variability as children’s TA ratingstended to be highly positive. There was also a trend for studies demon-strating treatments effects (i.e., therapy was effective) to find larger rela-tionships between TA and outcome. Youth in treatment for externalizingproblems were found to have higher associations between TA andoutcome than for youth in treatment for internalizing problems.

Most studies analyzed by Shirk and Karver (2003) did not collect multiplemeasures of TA over time. In fact, the majority of studies examined TA

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


towards the end of therapy, either after or near the point at which outcomeratings were collected. This likely minimized the conclusions that couldbe made regarding the predictive value of TA for therapy outcome, astherapy outcome may have been confounded with ratings of relationship(i.e., “I liked my therapist, so therapy must have been effective”; or viceversa). Also, few of the studies examined traditionally defined TA.Although all examined some element of the relationship between clientand therapist, most measures were not alliance measures per se thatincluded content aimed at measuring bonds, task collaboration, and goalagreement.

One question of significant concern to the present investigation iswhether therapy conducted in residential care settings is substantially dif-ferent than therapy conducted in other settings (i.e., primarily in outpa-tient settings). Although conclusions regarding this matter need to betempered by the lack of extant studies, available evidence indicates verylittle difference in the outcomes for youth in therapy regardless of setting.For example, Weiss and Weisz (1995) examined 10 studies that reportedthe effectiveness of therapy in residential care settings. The effect sizewas essentially equivalent to that found in other settings.

Similarly, initial evidence suggests that the role of TA in therapy out-come is not substantially affected by setting. Shirk and Karver (2003)reported that therapeutic setting (outpatient vs. all other settings) did notsignificantly impact the strength of the relationship between TA and ther-apy outcome. Surprisingly, however, few studies have specifically exam-ined the role of TA for youth in residential care. Of the 32 studiesreviewed, only 2 were conducted in a residential care environment, andthese included only 134 total participants. This is unfortunate as mostmodels of residential care emphasize the importance of the relationshipbetween therapeutic staff and child residents in facilitating the therapeuticprocess.

This study presents data from the first year of an ongoing project at alarge residential treatment facility for youth with behavioral and emo-tional problems to evaluate the impact of TA on therapy outcomes foryouth in residential care. This study addresses many of the limitations ofexisting TA-outcome research outlined above. Specifically, we used a TAmeasure based on traditional concepts of TA, collected TA ratings fromboth youth and therapist, collected measures of TA on multiple occasionsthroughout the course of therapy, and used two outcome measures, includ-ing one that was relatively independent of participant ratings, taken longi-tudinally over the course of therapy. We hypothesized that youth would

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 149

demonstrate significant improvement in both symptoms and behavior,and that TA would be a significant predictor of that improvement.

METHOD

Youth Participants

The initial sample included 85 youth from a large Midwestern out-of-home family-style residential treatment facility for delinquent youth whowere referred to an onsite psychotherapy clinic from the fall of 2001 tothe spring of 2002. Those youth that only had one session or who werealready receiving onsite clinical services prior to initiating data collectionin the fall of 2001 were removed from the sample, resulting in 71 eligibleparticipants. Ages ranged from 11.9 to 18.6 (M = 15.9, SD = 1.78). Par-ticipants were 52% male, 48% female. Length of stay at the facility priorto therapy referral ranged from 9 to 911 days (M = 254, SD = 244).Youth were referred for therapy by direct-care staff or clinical supervisorsfor a range of problems including externalizing behavior (e.g., aggres-sion), affective problems (e.g., depression), socialization concerns, schoolproblems, sexual issues, and clinical exotica (e.g., tics, etc.). Most youthpresented with problems in multiple areas. The number of therapysessions ranged from 2 to 32 (M = 12.6, SD = 7.7). Over the course of thetherapeutic episode, 28 youth (39%) on average had weekly sessions, 35youth (49%) averaged one session every two weeks, and 8 youth (12%)averaged one session every three to five weeks. Most of the youth had afixed schedule, but for many there was a trend to reduce the frequency oftherapy over time.

Setting

Described in detail elsewhere (Handwerk, Field, & Friman, 2000), theresidential program utilizes the Teaching-Family Model (TFM). TheTFM entails treatment given by Family-Teachers (trained, married cou-ples who live in a family-style home), a highly developed social skillscurriculum, elaborate motivation system, self-government, emphasis onnormalization and de-emphasis of mental illness, and a behaviorallyoriented model emphasizing critical elements common to most parenttraining programs. Most youth in the program do not receive “therapy”per se, as the model of care espouses that those closest to the youth (e.g.,Family-Teachers) are the most potent “therapy agents.” Nevertheless,

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


approximately 15%–20% of youth in the facility are referred for adjunctivetherapeutic services. Typically, these youth demonstrate intense behav-ioral and emotional problems that cannot be adequately addressed in theresidential-home environment, present with mental health issues that arebeyond the expertise of the Family-Teachers, and may be at risk of beingterminated from the program and possibly sent to a higher level of care.

Clinicians were seven predoctoral psychology interns from AmericanPsychological Association accredited programs completing their intern-ships at the facility. All held licenses as masters level therapists in thestate. Clinicians received weekly supervision with one of two PhD psy-chologists, weekly supervision with a licensed masters level practitioner,and twice weekly group supervision including didactic training on variousinterventions for psychological problems encountered by adolescents(e.g., depression). In the overwhelming majority of cases, the supervisorsdid not stipulate the specific form of therapy, though elements of therapywere specifically mandated (e.g., the use of exposure techniques for youthwith anxiety problems; certain elements of behavioral activation for youthwho were depressed; habit reversal for habit disorders; etc). No attemptwas made to quantify or to measure the therapy process. Most youth wereon a weekly therapy schedule, however, some were moved to a biweeklyor a more than once a week schedule, depending on individual circum-stances.

Measures

Therapeutic Alliance Measure

The therapeutic alliance scale used was Doucette and Bickman’s WorkingRelationship Scale (WRS; Bickman et al., 2004; Doucette, 2004), whichwas completed independently after each session by the youth and theirtherapist. The WRS contains 32 statements that are rated on a 4-pointLikert scale (1 = Strongly Agree to 4 = Strongly Disagree; range = 32–128)and has been found to demonstrate strong reliability (alpha = .83 to .90;Bickman et al.). The reliability for the WRS using the data in this studywas strong, with α = .95 and an average weekly test-retest reliabilityacross the first ten sessions of .76. The WRS consists of items spanningthree domains: resistance to therapy (e.g., “Being in therapy is usually awaste of time”), bond/relationship (e.g., “It matters to me what my therapist/clinician thinks about me”), and collaboration on goals/tasks (e.g.,“My therapist/clinician and I work on my problems as a team”). A higherscore on the WRS indicates a better working relationship.

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 151

Symptom Screener

The Symptom Screener (SS) is a 35-item measure (Doucette, 2004).There was also strong reliability for the SS using the data in this study,with α = .94 and an average weekly test-retest reliability across the first10 sessions of .65 (the lower correlation here is expected for a scaledesigned to measure change over time). The SS contains statements thatrespondents rate on a 4-point Likert scale (1 = Never to 4 = Always;range = 35–140). The symptom screener consists of items spanning inter-nalizing problems (e.g., “I worried about most things”) and externalizingproblems (e.g., “I argued with adults”). For this measure, lower scoreindicates fewer symptoms. The SS was used as one measure of therapyoutcome and was completed by the youth prior to each therapy session.

Daily Incident Report

Direct observations of behaviors were gathered from the Daily IncidentReport (DIR). This report logs all significant events that occur at the resi-dential facility each day (e.g., extreme noncompliance, aggressive behav-ior, psychiatric visits). Each recorded incident includes a descriptivenarrative of the behavior or event and at least one code (some incidentsmay include more than one code). Each code has been operationalized viaa brief description of the code along with a prototypical example (e.g.,“Physical assault (on staff): youth assaults a staff member. Injury may ormay not have resulted, but aggressive physical contact occurred. Examplesinclude biting, choking, kicking, punching, pushing.”).

The reliability of the DIR has been established in several studies. First,Wright (2001) investigated how likely Family-Teachers were to reportyouth’s problem behaviors to clinical supervisors. Using a questionnairedistributed to 54 Family-Teachers containing 43 scenarios, reporting reli-ability for all events was 83.5%, indicating a moderate but acceptablelevel of agreement between Family-Teachers and clinical supervisors.Additionally, Larzelere (1996) conducted analyses of inter-coder reliabilityof the narratives by administrative staff. Kappa coefficients ranged from.66 to .97 (M = .91) for codes entered for the same narratives by differentcoders. Therefore, at the level of coding the narrative descriptions, theDIR possesses good to excellent reliability. Taken together, both atthe level of reporting and coding, the DIR appears to possess adequatereliability.

We used a weekly summed total of unique occurrences for 40 incidentcodes that measured a variety of behavioral problems (e.g., substance use,

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


property damage, running away, physical aggression). These 40 codesrepresent oppositional defiant and conduct problems, self-destructivebehavior, and substance abuse that are entered into the database andreflect each youth’s progress in the program. Thus, we had a weekly“incident score,” with no real upper limit, for each youth receiving thera-peutic services, even if they did not have a therapy session every week.

Datasets

The mean number of sessions per youth was about 13. However, manyyouth had considerably more sessions, with a one youth having 32 therapy ses-sions. As most youth (87%) had 20 or fewer therapy sessions, we limited theanalyses to the first 20 sessions to avoid the situation where those few caseswith more therapy sessions might bias the results. For the DIR, we used datafrom the first 20 weeks in which the youth had received therapeutic services.

Hierarchical Linear Modeling

Hierarchical Linear Modeling (HLM) was chosen to examine the impactof youth and clinician alliance on symptom and antisocial behavior overtime (Raudenbush & Bryk, 2002). Especially useful in this study, HLMallows for random effects to be estimated for incomplete data as individualgrowth curves can be extrapolated from their existing data points (Silverstein &Long, 1998). While every effort was made to administer each measure everysession, 47% of the youth had at least one missing measure on one or moreoccasion. While this would be a problem for most multivariate analyses,HLM allowed us to utilize the data from all our youth. A final advantage ofHLM is that it does not require that the time intervals (e.g., time betweensessions) be equal (Silverstein & Long, 1998). Specifically, in this dataset,some youth were on a two-week therapy interval whereas others receivedservices every week or more.

RESULTS

Descriptive Statistics

Working Relationship Scale

Scores on the Youth WRS ranged from 48 to 128 (M = 108.3, SD =15.05). For the Clinician WRS, scores ranged from 65 to 110 (M = 91.59,SD = 6.62). For both measures, higher scores indicated higher levels of

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 153

therapeutic alliance. Correlations between the youth and clinician thera-peutic alliance were low at the beginning of therapy (r = .34) and declinedsomewhat at the end of therapy (r = .24).

Symptom Screener

During the study, scores on the Youth SS ranged from 35 to 137 (M = 48.02,SD = 13.57), with higher scores indicating more problem behaviors.

Daily Incident Report

The number of incidents per youth per week ranged from 0 to 24(M = 1.11; SD = 2.76). The mode and median were 0, indicating thatmost youth on most weeks did not have any incidents.

Analyses of TA and Outcome

The first step when developing a HLM model is to create an uncondi-tional base model that does not include any predictors. The base modeldetermines if there is significant variation in the outcome variable in ques-tion and serves as a baseline against which subsequent models are com-pared. In this study, all unconditional base models tested were significantand are not discussed in detail. For all models, we used indicators of time asLevel 1 variables. For the WRS and SS scales, session number was used asthe Level 1 variable and week in therapy was the Level 1 time variable forthe DIR. This distinction was made because the WRS and SS scales werecollected at the time of treatment and youth did not always follow a weeklytherapy schedule. DIR data, on the other hand, was invariably collecteddaily and so was modeled weekly (because of low daily behavioral frequen-cies). In both cases, the time variable was log-transformed to provide a lessbiased and more reliable path across repeated measures (Howard, Kopta,Krause, & Orlinsky, 1986). Finally, individual youth level characteristics(e.g., gender, number of days in the current residential placement prior toentering therapy) were introduced into the model as Level 2 variables.

Therapeutic Alliance Measure

Figure 1 shows both the Youth and Clinician WRS scores for the first20 sessions, with zero indicating the first session. Thus, all analyses willbe interpreted using the first session as the intercept. Average youth WRSscores increased slightly from the first to the second session and thenremained relatively invariant over the remainder of the first 20 therapy

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


sessions. HLM confirmed this by indicating that the intercept was signifi-cant, β = 107.16, t(70) = 65.15, p < .001, whereas change over sessionswas not, β = .15, t(70) = .23, p = .81. This indicates that the meanslope does not vary over the course of therapy. However, to check if therewas significant between subjects variation, the Level 2 variables ofdays in care before therapy, race and gender were entered into the model(e.g., Do males have a different slope than females?). Results indicate noLevel 2 between subject variability. Due to the relatively stable natureof youth TA over the course of therapy, we used mean youth WRSover the course of therapy as a Level 2 predictor variable in subsequentanalyses.

Clinician WRS fluctuated somewhat during the first 20 therapy ses-sions, with a significant intercept, indicating that it is statistically differentfrom zero, β = 90.92, t(70) = 125.9, p < .001, and slope, indicating anincrease in alliance over time that was a significant, β = .82, t(70) = 2.33,p = .023. However, despite the statistical results, we decided to use themean clinician WRS as a Level 2 predictor variable in subsequent mod-els for three reasons: (a) the increase in WRS over 20 sessions is small(.07 per session over 20 sessions = a 1.4 increase in overall TA score on a32–128 scale); (b) much like the youth WRS, the Level 2 variables werenot significant indicators of between subject variability; and (c) to maintain

FIGURE 1. Mean youth and clinician therapeutic alliance score over thecourse of 20 therapy sessions.

Youth TA

Clinician TA

Session Number

Mea

n TA

80

75

70

65

60

55

501 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 155

analytical consistency with the youth TA scores, and greater parsimonywhen developing predictive models.

Symptom Screener: Level 1 Model

Figure 2 indicates that youth entered therapy with a high number ofsymptoms that decreased in a curvilinear fashion over time. Specifically,symptoms rapidly decreased in first few sessions and then leveled offduring subsequent therapy sessions. Given the curvilinear nature of thedata, we used the log-linear transformation of Session number +1 to cap-ture this curvilinear trend. For this model, the intercept, β = 61.65, t(70) =34.55, p < .001, and the log-transformed session number, β = −7.30,t(70) = −9.80, p = .001, were statistically significant, which reflects theelevated number of symptoms at the onset of therapy and significantsymptom reduction thereafter. Overall, this model accounted for about a57% reduction from the unconditional model in youth-reported symptoms.

Symptom Screener: Combined Model (Levels 1 and 2)

A key feature of HLM is that it allows the intercept and slope parametersfrom the Level 1 (the within subjects) model to be used as dependent vari-ables in the Level 2 (the between subjects) model (Bray, Adams, Getz, &Baer, 2001). This allows for the determination of the amount of varianceaccounted for by the between-subjects variables. This “combined” modelexamined five time-invariant between subject variables: mean youth TAover the course of therapy, mean clinician TA over the course of therapy,youth gender, the number of prior days in the current residential setting

FIGURE 2. Youth symptom screen over the course of 20 therapy sessions.

Time (sessions)

Sym

ptom

s

35

30

25

20

15

10

5

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


before entering therapy, and race. In addition to the intercept and log-transformed session number that were significant in the previous model,mean youth therapeutic alliance of the course of therapy (YTH TA) wasfound to be a significant Level 2 predictor, β = .133, t(70) = 2.21, p < .05, butonly accounted for about a 3% of the variability in the reduction of SymptomScreener scores over the course of therapy (see Table 1). Those youth whohad higher mean TA scores reported a quicker reduction in symptoms.

Daily Incident Report: Level 1 Model

Problem behavior, as measured by the DIR, decreased during the first20 weeks in therapy, eventually approaching the program average ofabout 0.5 incidents per week per youth (See Figure 3). Because of the rel-atively linear nature of the data, the log-transformation of week in therapywas used as the Level 1 predictor variable. Further, week in therapy was

TABLE 1. Combined Model for Symptom Screener

β SE t df p

InterceptAverage symptoms 61.69 1.77 34.77 69 .000Mean Youth TA −.19 .14 −1.30 69 .199

SlopeSession # (log transformed) −7.35 .73 −10.05 69 .000Mean Youth TA .13 .06 2.21 69 .03

Note. Deviance = 6914.48; df = 4.

FIGURE 3. Mean daily incident report over the course of the first 20 weeks intherapy.

Time (Weeks)

Program Average

Num

ber

per

Wee

k

2.01.81.61.41.21.00.80.60.40.20.0

10 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 157

used instead of session number because we were able to calculate the DIR“score” for week in therapy whereas session number did not necessarilyfollow a weekly schedule. The intercept, β = 1.48, t(70) = 5.39, p < .001, wassignificant and the slope approached significance, β = −.097, t(70) = −1.64,p = .104. This indicates that youth were involved in a significant numberof disruptive incidents at the beginning therapy and there was a trend forthis frequency to reduce during the first 20 weeks. Overall, when com-pared to the base model, number of weeks in therapy accounted for lessthan 1% of the variance. Further, there was not a significant amount ofunaccounted for error left on the slope, suggesting no need to furtherinvestigate Level 2 variables on this time variable.

Daily Incident Report: Combined Model (Levels 1 and 2)

Much like the model used for the Symptom Screener outcome variable,we examined the five time invariant between-subjects variables: meanyouth TA over the course of therapy, mean clinician TA over the course oftherapy, youth gender, race, and the number of prior days in the current res-idential setting before entering therapy (see Table 2). The number of priordays in the current residential setting was the only significant variable onthe intercept, β = 1.48, t(69) = 5.51, p < .001, where those who had moredays in the program prior to entering therapy had fewer behavioral inci-dents upon entering therapy, β = −.002, t(69) = −2.34, p < .05.

DISCUSSION

Consistent with previous findings of therapy conducted in residential caresettings, youth improved during the course of therapy. On a self-report

TABLE 2. Combined Model for Daily Incident Report With Error on Time as Fixed

Variable b SE t df p

InterceptAverage number of incidents 1.48 .268 5.51 69 .000Prior days in programa −.002 .0008 −2.34 69 .02

SlopeWeek in therapy (log transformed) −.095 .059 −1.61 70 .11

Note. Deviance = 4896.08; df = 2.aLevel 2 predictor, grand mean centered

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


measure, youth ratings on a symptom checklist showed substantialimprovements, and much of the improvement occurred early in the ther-apy. On a more objective measure of in-program behavior, decreases inproblem behaviors during the course of therapy were not statisticallysignificant. However, on a pragmatic level, youth entered therapy demon-strating approximately twice the rate of problem behaviors on the DIRrelative to other youth at the facility. By the end of therapy, behaviorproblems approached levels equivalent to facility norms. One of the rea-sons for these results may have been attributable to the low base rate andhigh variability of problem behavior as measured by the DIR.

Similar to other findings, there was only modest to poor agreement onTA between clinicians and youths. Agreement between clinicians andyouth declined over the course of therapy. On average, TA was relativelystable and did not change substantially throughout the course of therapy,though individual alliances did show variability over the course of therapy.Consistent with other reports (Kendall & Sugarman, 1997; Shirk &Karver, 2003), average youth ratings of TA were relatively high.

Although TA has been found to be a modest, but consistent, predictorof outcome for children and adolescents (Shirk & Karver, 2003), ratingsof alliance were only marginally related to either symptom reduction ordaily behavior in this study. The only significant relationship was foundbetween youth-rated TA over the course of therapy and self-reportedsymptom reduction. However, TA only accounted for about 3% of thevariance on symptom improvement, which is less than has been reportedin previous meta-analyses (r2 = .06; Shirk & Karver, 2003). One primarydifference between this study and previous studies is the multiplemeasures of TA and outcome over the course of therapy. We used aver-age ratings of TA that may be a more accurate representation of TA overthe course of therapy than measures of TA collected on one occasion (cf.,de Roten et al., 2004). Many previous studies investigating the relationshipbetween alliance and outcome for youth have assessed TA toward the endof therapy, potentially confounding the relationship between TA and out-come. Further, our measure of TA was a traditional measure of TA, incor-porating aspects of the relationship, collaboration, and resistance. Thus,the incongruity of the current results with prior reports may be partiallyattributed to differences in the measurement of salient constructs.

It is important to note that therapy is not the principal mode of treat-ment for youth at this residential care facility. Primary intervention occurswithin a family-style environment, where trained paraprofessionals imple-ment a well-articulated, monitored, and supervised model of treatment

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 159

(Daly & Dowd, 1992). In this setting, youth have many potentially significantadults in their lives, including Family-Teachers, Assistant Family-Teachers,coaches, school teachers, clinical supervisors, mentors, and clergy, all ofwhom are trained and integrated within the treatment program. Thus, theTA that occurs between youth in this program and the adults that spendthe most time with them may be a more important contributor to interven-tion outcomes than the TA between the youth and their therapist. Further,given that youth were in the program, on average, for 8 months prior tobeginning therapy, it is possible that youth experienced a “TA haloeffect,” meaning that the youth had learned that adults in this environmentwere generally caring and could be trusted.

Many therapists working with children and adolescents understandablyreport that they place primary emphases on the relationship betweenthemselves and their clients (Digiuseppe, Linscott, & Jilton, 1996;Johnson et al., 2002; Oetzel & Scherer, 2003). Relationship is at the heartof most models of therapy for both adults and children (Bachelor &Horvath, 1999). Nevertheless, the results of this study suggest that thespecific relationship between therapists and youths at this treatment facil-ity was not primarily responsible for the improvements seen for youthduring the course of therapy. While not trivializing the role of TA in atherapeutic setting, these results suggest that other factors may be as, oreven more, relevant and salient in producing change in adolescent ther-apy. Unlike adult therapy outcomes where a strong case for therapyequivalence exists, behavior therapy tends to produce more positiveresults with children and adolescents (Weisz et al., 1995). Perhaps whattherapists do with children (i.e., the strategies, tactics, advice) is moreimportant than whether youth clients like their therapist (or vice versa).While youth with the lowest TA scores experienced many of the sameoutcomes as youth with the highest TA scores, it should be kept in mindthat the range of TA scores was fairly truncated and fairly high overall.Perhaps a relationship between TA and outcome would have been foundhad TA scores covered the full range of possible scores. While a strongpositive therapeutic relationship may be helpful to establish the contextfor positive change in youth, it appears to be neither entirely necessarynor sufficient in producing positive outcomes for children in this setting.

Although we analyzed several client variables (i.e., time in programprior to therapy, gender), missing from this study is evidence to suggestwhat else might account for the variance in therapy outcomes, such asspecific techniques. We made no attempt to measure the therapy process.This prohibits definitive statements regarding the relative contribution of

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


alliance as opposed to specific techniques. In future research, it will beimportant to measure specific therapist behavior that contributes to alli-ance (cf., Creed & Kendall, 2005) in order to better understand variablesthat affect alliance and whether these behaviors ultimately affect therapyoutcome. Unlike the adult literature, very few child therapy studies havedirectly compared the relative influence of specific technique and alliance(Kaufman et al., 2005; Shirk & Karver, 2003).

There are also limitations of this study that restrict the generaliz-ability of these results. Therapy was conducted by predoctoral internscompleting their internship. Therapists were highly monitored andsupervised, and supervision entailed direct feedback regarding processvariables, albeit based on clinicians’ reports of what went on in thetherapy room. Whether these results would generalize to more skilledor experienced therapists, or those who were not receiving extensivesupervision is a question for future research. Additionally, our studywas conducted in a residential program utilizing the TFM, which maylimit the generalizability of our findings to youth in other residentialsettings.

Also, though our objective outcome measure (i.e., DIR) is animprovement over relying solely on participants’ ratings, it was highlyloaded on externalizing behaviors. This limits the conclusions that canbe drawn regarding the relationship between TA and therapy outcomefor internalizing problems (cf., Kendall et al., 1997). Also, the DIRwas a more objective measure of outcome than clinician or youth rat-ings, but not totally independent of the process. Family-Teachers werenot blind to a youth’s participation in therapy, though the context ofDIR reporting (i.e., in the morning, every day, for all 500+ youth inthe program) minimizes this concern. Further, the DIR emphasizesmore serious problems than those the youth report on in the SS. Thismay have contributed to the lack of statistical congruence betweenanalyses of change over time on the DIR and SS, although the lack ofstatistically significant change on the DIR over time belies the sub-stantive and clinically meaningful change youth demonstrated as theirbehaviors essentially reached local norms. Although a more indepen-dent source of behavior than other self-reporting measures, these limi-tations make the DIR less sensitive to more subtle behavioral andattitudinal change.

As others have reported, average ratings of TA were mostly positiveand relatively stable over time. Although highly stable and positiveratings could be interpreted as rating bias, it might also represent a “true”

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 161

reflection of the adolescents regard for their therapist (Creed & Kendall,2005). Nevertheless, statistically speaking, truncation of range will diminishthe magnitude of correlations. One potential approach to overcome thisproblem is using observational coding schemes for TA. Although obser-vational coding systems have provided some promise in reducing theseproblems (McLeod & Weisz, 2005), the feasibility of using such systemsin everyday practice seems limited.

Finally, although collecting multiple measures of TA and outcome isboth conceptually and statistically advantageous to singular measure-ments, reports from the adolescents indicated extreme frustration withprocess of filling out both SS and TA ratings every session. We sur-veyed a subsample of adolescents (n = 35) during the course of theproject to elicit their impressions. Fifty percent of the adolescentssurveyed indicated they believed it took too long to fill out the forms,66% believed filling out the forms “wasn’t helpful,” and 68% indicatedthey thought filling out the forms “was a waste of time.” We havesubsequently thinned our schedule for adolescents to complete TAforms to the first, second, and every fourth session thereafter to eliminatefrustration while continuing to assess TA longitudinally. We are hopingthat developing a more sensitive measure of TA embedded in a lessstringent collection schedule for youth will provide an accurate pictureof the role of TA and therapy outcome.

In conclusion, although this study examined youth in a relativelyunique treatment setting (in-house clinical services within a residentialcare program), our results were not consistent with the notion that thequality of the alliance between therapist and youth is of the utmostimportance in accounting for symptom change. There is a paucity ofresearch in the area of child-therapist alliance, and the majority of stud-ies in this area have significant methodological problems that prohibitdefinitive statements about the importance of alliance in child therapy.As residential care represents a unique environment where there aremultiple treatment providers, research in this setting should expand thescope of potential important alliances to include other care providers.As the role of TA in residential care has been hypothesized to be criticalin outcomes, we suggest that researchers in this area use multiplemeasurements of alliance and outcomes in future studies so that moredefinitive evidence can be obtained on the relationship between allianceand child therapy outcome, contextual variables that affect this relation-ship, and the relative influence of specific techniques and alliance onoutcome.

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


REFERENCES

Bachelor, A., & Horvath, A. (1999). The therapeutic relationship. In M. A. Hubble, B. L.Duncan, & S. D. Miller (Eds.), The heart and soul of change: What works in therapy(pp. 133–178). Washington, DC: American Psychological Association.

Bickman, L., de Andrade, A. R. V., Lambert, E. W., Doucette, A., Sapyta, J., Boyd, A. S.et al. (2004). Youth therapeutic alliance in intensive treatment settings. Journal ofBehavioral Health Services & Research, 31, 134–148.

Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the workingalliance. Psychotherapy: Theory, Research & Practice, 16, 252–260.

Bray, J. H., Adams, G. J., Getz, J. G., & Baer, P. E. (2001). Developmental, family, andethnic in influences on adolescent alcohol usage: A growth curve approach. Journal ofFamily Psychology, 15, 301–314.

Casey, R. J., & Berman, J. S. (1985). The outcome of psychotherapy with children.Psychological Bulletin, 98, 388–400.

Cloitre, M., Chase Stovall-McClough, K., Miranda, R., & Chemtob, C. M. (2004).Therapeutic alliance, negative mood regulation, and treatment outcome in child abuse-related posttraumatic stress disorder. Journal of Consulting & Clinical Psychology, 72,411–416.

Creed, T. A., & Kendall, P. C. (2005). Therapist alliance-building behavior within a cog-nitive-behavioral treatment for anxiety in youth. Journal of Consulting and ClinicalPsychology, 73, 498–505.

Daly, D. L., & Dowd, T. P. (1992). Characteristics of effective, harm-free environmentsfor children in out-of-home care. Child Welfare, 71, 487–496.

de Roten, Y., Fischer, M., Drapeau, M., Beretta, V., Kramer, U., Favre, N. et al. (2004). Isone assessment enough? Patterns of helping alliance development and outcome. Clini-cal Psychology & Psychotherapy, 11, 324–331.

DeVet, K. A., Kim, Y. J., & Charlot-Swilley, D. (2003). The therapeutic relationship inchild therapy: Perspectives of children and mothers. Journal of Clinical Child andAdolescent Psychology, 32, 277–283.

Digiuseppe, R., Linscott, J., & Jilton, R. (1996). Developing the therapeutic alliance inchild-adolescent psychotherapy. Applied & Preventive Psychology, 5, 85–100.

Doucette, A. (2004). Child/Adolescent Measurement System (CAMS): User manualNashville, TN: Author.

Florsheim, P., Shotorbani, S., Guest-Warnick, G., Barratt, T., & Hwang, W.-C. (2000).Role of the working alliance in the treatment of delinquent boys in community-basedprograms. Journal of Clinical Child Psychology, 29, 94–107.

Garcia, J. A., & Weisz, J. R. (2002). When youth mental health care stops: Therapeuticrelationship problems and other reasons for ending youth outpatient treatment. Journalof Consulting & Clinical Psychology, 70, 439–443.

Handwerk, M. L., Field, C., & Friman, P. C. (2000). The iatrogenic effects of group inter-vention for antisocial youth: Premature extrapolations? Journal of Behavioral Educa-tion, 10, 223–238.

Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the WorkingAlliance Inventory. Journal of Counseling Psychology, 36, 223–233.

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 163

Horvath, A. O., & Symonds, B. D. (1991). Relation between working alliance andoutcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology,38, 139–149.

Howard, K. I., Kopta, S. M., Krause, M. S., & Orlinsky, D. E. (1986). The dose-effectrelationship in psychotherapy. American Psychologist, 41, 159–164.

Jensen, P. S., Weersing, R., Hoagwood, K. E., & Goldman, E. (2005). What is the evi-dence for evidence-based treatments? A hard look at our soft underbelly. MentalHealth Services Research, 7, 53–74.

Johnson, L. N., Wright, D. W., & Ketring, S. A. (2002). The therapeutic alliance in home-based family therapy: Is it predictive of outcome? Journal of Marital & FamilyTherapy, 28, 93–102.

Kaufman, N. K., Rohde, P., Seeley, J. R., Clarke, G. N., & Stice, E. (2005). Potentialmediators of cognitive-behavioral therapy for adolescents with comorbid majordepression and conduct disorder. Journal of Consulting & Clinical Psychology, 73,38–46.

Kazdin, A. E. (1982). The token economy: A decade later. Journal of Applied BehaviorAnalysis, 15, 431–445.

Kazdin, A. E., Bass, D., Ayers, W. A., & Rodgers, A. (1990). Empirical and clinical focusof child and adolescent psychotherapy research. Journal of Consulting & ClinicalPsychology, 58, 729–740.

Kazdin, A. E., Marciano, P. L., & Whitley, M. K. (2005). The therapeutic alliance incognitive-behavioral treatment of children referred for oppositional, aggressive, andantisocial behavior. Journal of Consulting and Clinical Psychology, 73, 726–730.

Kendall, P. C., Flannery-Schroeder, E., Panichelli-Mindel, S. M., Southam-Gerow,M., Henin, A., & Warman, M. (1997). Therapy for youths with anxiety disorders:A second randomized clinical trial. Journal of Consulting & Clinical Psychology,65, 366–380.

Kendall, P. C., & Sugarman, A. (1997). Attrition in the treatment of childhood anxietydisorders. Journal of Consulting & Clinical Psychology, 65, 883–888.

Lambert, M. J., & Barley, D. E. (2001). Research summary on the therapeutic relationshipand psychotherapy outcome. Psychotherapy: Theory, Research, Practice, Training,38, 357–361.

Larzelere, R. E. (1996). Inter-coder reliabilities and construct groupings for some importantcodes on the Daily Incident Report. (NRI 001–96). Boys Town, NE: Father Flanagan’sBoys’ Home.

Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the therapeutic alliancewith outcome and other variables: A meta-analytic review. Journal of Consulting &Clinical Psychology, 68, 438–450.

McLeod, B. D., & Weisz, J. R. (2005). The therapy process observational coding system-alliance scale: Measure characteristics and prediction of outcome in usual clinicalpractice. Journal of Consulting and Clinical Psychology, 73, 323–333.

Oetzel, K. B., & Scherer, D. G. (2003). Therapeutic engagement with adolescents inpsychotherapy. Psychotherapy: Theory, Research, Practice, Training, 40, 215–225.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications andData Analysis Methods (2nd ed.) Thousand Oaks, CA: Sage.

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014


Robbins, M. S., Turner, C. W., Alexander, J. F., & Perez, G. A. (2005). Alliance and drop-out in family therapy for adolescents with behavior problems: Individual and systemiceffects. Journal of Family Psychology, 17, 534–544.

Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship vari-ables in child and adolescent therapy: A meta-analytic review. Journal of Consulting &Clinical Psychology, 71, 452–464.

Silverstein, M., & Long, J. D. (1998). Trajectories of grandparents’ perceived solidaritywith adult grandchildren: A growth curve analysis over 23 years. Journal of Marriage& the Family, 60, 912–923.

Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H. (1997). Ameta-analysis of outcome studies comparing bona fide psychotherapies: Empiricially,“all must have prizes.” Psychological Bulletin, 122, 203–215.

Weiss, B., & Weisz, J. R. (1995). Relative effectiveness of behavioral versus nonbehavioralchild psychotherapy. Journal of Consulting and Clinical Psychology, 63, 317–320.

Weisz, J. R., Weiss, B., Alicke, M. D., & Klotz, M. L. (1987). Effectiveness of psycho-therapy with children and adolescents: A meta-analysis for clinicians. Journal ofConsulting & Clinical Psychology, 55, 542–549.

Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects ofpsychotherapy with children and adolescents revisited: A meta-analysis of treatmentoutcome studies. Psychological Bulletin, 117, 450–468.

Wright, D. J. (2001). The relationship of anxiety and externalizing disorders. DissertationAbstracts International: Section B: The Sciences and Engineering, 62(4), 2001,2084B.

BIOGRAPHICAL NOTES

Michael L. Handwerk, PhD, was Director of Clinical Services, Research, andInternship Training at Boys Town at the time of this study. Currently, he is a ClinicalPsychologist for Harrisburg Medical Center in Harrisburg, IL. He has publishedover 30 articles and chapters on assessment and intervention for childhood problems.His research interests include outcomes of out-of-home care and assessment of andinterventions for disruptive behavior disorders.

Jonathan C. Huefner, PhD, is a Research Scientist, National Research Institute forChild and Family Studies, Father Flanagan’s Boys’ Home, Boys Town, Nebraska, USA.He has 23 years experience conducting research, and has over 30 published articles.Dr. Huefner’s recent work has focused primarily on outcomes studies, gender differencesin response to treatment, and examining negative peer contagion in residential care.

Jay L. Ringle, MA, is a Research Analyst, National Research Institute for Child andFamily Studies, Father Flanagan’s Boys’ Home, Boys Town, Nebraska, USA. Jay hasworked at Boys Town for 8 years and his work has focused on conducting long-termoutcome studies of youth in residential care, developing a suicide screening measure forat-risk children and examining negative peer contagion among youth in residential care.

Brigid K. Howard, MA, is a Research Analyst, National Research Institute for Childand Family Studies, Father Flanagan’s Boys’ Home, Boys Town, Nebraska, USA. Brigid

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Handwerk et al. 165

has worked at Boys Town for 5 years and has been involved in the administration of theComputerized Diagnostic Interview Schedule for Children (DISC) and, more recentlywith follow-up research.

Stephen H. Soper, MS, was a Research Analyst in the Department of Clinical Services &Research. Steve has passed on since the time of the study. With Masters Degrees in bothSocial Welfare and Computer Science, Steve was fond of saying he loved his job becauseit combined his passion for computers with his passion for studying human behavior. Hewas a dedicated researcher who was always willing to lend a hand others. He has been andwill continue to be missed.

Julie K. Almquist, MA, is Senior Clinician for the Girls and Boys Town BehavioralPediatric Outpatient Clinic, and has been at Boys Town for 14 years. Julie providestherapy services to children, adolescents, and their families, and consultation to schoolsand other childcare providers.

M. Beth Chmelka, BS, is a Research Analyst, National Research Institute for Child andFamily Studies, Father Flanagan’s Boys’ Home, Boys Town, Nebraska, USA. She has20 years of research experience in the residential child care field. Her most recent focus isoutcome follow-up studies for out-of-home and in-home family services.

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

Dow

nloa

ded

by [

Uni

vers

iteit

Tw

ente

] at

15:

16 2

9 N

ovem

ber

2014

the role of therapeutic alliance in therapy outcomes for youth in residential care

Documents