psychotherapists' assessments of their development at different career levels

Psychotherapy Volume 36/Fall 1999/Number 3

PSYCHOTHERAPISTS' ASSESSMENTS OF THEIRDEVELOPMENT AT DIFFERENT CAREER LEVELS

DAVID ORLINSKYUniversity of Chicago

HANSRUEDI AMBUHLUniversity of Bern

JEAN-FRANgOIS BOTERMANSCatholic University ofLouvain

JOHN DAVISUniversity of Warwick

M. HELGE R0NNESTADUniversity of Oslo

ULRIKE WILLUTZKIRuhr-University Bochum

MANFRED CffiRPKAUniversity of Heidelberg

MARCIA DAVISNorth Warwickshire Health Service

and the SPR Collaborative Research Network

Experiences of current and careerprofessional development were surveyedfor about 3,900 psychotherapists at allcareer stages in several westerncountries. Participants includedpsychologists, psychiatrists, and othermental health practitioners representinga broad range of theoretical orientations.Measures of experienced developmentwere consistent across professional anddemographic categories. As expected,perceived therapeutic mastery increased

The authors gratefully acknowledge major contributions tothis project by Drs. Paul Gerin and Alice Dazord (INSERM,France) and other colleagues including (alphabetically): LarryBeutler (Santa Barbara); Peter Buchheim (Munich); ManfredCierpka (Heidelberg); Eric Friis-Jorgensen (Copenhagen);Horst Kachele (Ulm); Ekaterina Kalmykova (Moscow); JanMeyerberg (Karlsruhe); John Norcross (Scranton); TerryNorthcut (Chicago); Barbara Parks (Chicago); Elena Scherb(Buenos Aires); Thomas Schroder (Derbyshire); Gaby Shefler(Jerusalem); Dan Stiwne (Linkoping); Margarita Tarragona(Mexico City); Antonio Vasco (Lisbon); Anna Von der Lippe(Oslo) and Hadas Wiseman (Haifa).

Correspondence regarding this article should be addressedto David Orlinsky, Committee on Human Development, Uni-versity of Chicago, Chicago, IL 60637. E-mail: <[email protected]>.

with increasing years in practice, butcurrently experienced growth remainedat a high and unexpectedly constant levelacross career cohorts, including the mostsenior practitioners. Overall findingssuggest that therapists experiencedevelopment simultaneously as agradually increasing sense ofprofessional expertise and as an ongoingsense of continual improvement,interpreted as essential to maintainingtheir motivation and morale.Contributions of these aspects oftherapist development to process andoutcome are discussed.

This article aims to make two contributions toknowledge about the professional development ofpsychotherapists. The first concerns the thera-pist's experience of professional developmentacross the entire career span. The second concernsthe relation between therapists' assessments oftheir development and an objective measure oftime in therapeutic practice.

Of what importance is the development of psy-chotherapists beyond the initial years of trainingand education? If development is understood to

203

D. Orlinsky et al.

mean "becoming a therapist," then what meaningcan the term have for those who are alreadytrained and certified as psychotherapists? To an-swer properly, a distinction must be drawn be-tween the external criteria set for graduation andlicensing as a therapist and the internal qualitiesrequired for being therapeutic. In the formersense, development is essentially complete whenthe external criteria have been met. In the lattersense, development may be understood as attain-ment of increasing expertise in a task that thera-pists perceive as highly challenging in its com-plexity (Skovholt & R0nnestad, 1995).

It is clear that development is a matter of greatimportance to therapists themselves. For exam-ple, in a previous study (Orlinsky et al., 1999)nearly 3,800 therapists were asked, "How im-portant to you is your further development as atherapist?" The mean rating for this large groupwas 4.5 on a 0-5 scale (SD = 0.9), and thecorrelation of these ratings with length of practice(range 3 months to 50 years) was - .04. Thissuggests that therapists at all career levels arestrongly motivated to continue their development,although it is not clear that they necessarily dodevelop. Dawes (1994) questioned whether psy-chotherapists increase their professional skillsafter they attain a basic competence level. Whilecompetence and development are not synonymousconcepts, it seems reasonable to assume that formany, development presupposes increased mas-tery. It is not clear, however, that a developmentalparadigm best captures changes that therapists ex-perience during their professional lives (Hol-loway, 1987).

Unfortunately, most models of therapist andcounselor development (e.g., Fleming, 1953;Hess, 1987; Hogan, 1964; Loganbill, Hardy, &Delworth, 1982; Stoltenberg & Delworth, 1987)have focused primarily on the early phases ofprofessional growth, and empirical research hasmainly examined supervisees in their student orimmediate postgraduate years (Skovholt, R0nnes-tad, & Jennings, 1997). This situation has notchanged much since Stein and Lambert (1984)found that the average experience level in studiesof therapist expertise was only 2.9 years. With thepossible exception of the life-span developmentalmodel of Skovholt and R0nnestad (1995), theexisting literature gives only a sketchy accountof professional development in the later years.

Thus an interesting and potentially importantquestion may be asked about whether, and to what

extent, psychotherapists continue to experienceprofessional development after they become inde-pendent practitioners. A further question may alsobe raised about the meaning that continued profes-sional development has for psychotherapists, in-cluding those who have achieved seniority in theirfield. These questions will be addressed throughthe analysis of an extensive body of data collectedin an international survey of psychotherapists,which included a number of direct and indirectassessments of professional development (Orlin-sky et al., 1999). Specifically, we ask the follow-ing questions.

1. What is the relation between perceived thera-peutic mastery and level of professional experi-ence? It may be that mastery increases duringthe early training years but not much beyond,as suggested by Dawes (1994), or that masterycontinues to increase over the course of the thera-peutic career either at a constant or deceleratingrate. Some therapists may also experience a de-cline in perceived therapeutic mastery as a func-tion of "burnout" (Maslach, 1982), but it is un-clear whether that phenomenon is related tocareer phase.

2. What is the relation between currently expe-rienced growth and level of professional experi-ence? Skovholt and R0nnestad (1995) reportedthat veteran therapists, reflecting on their pastand current experiences, described themselves aspresently developing at a gradual pace. This con-trasted with the urgent sense of change describedby less experienced practitioners. If their inter-views with 100 American psychologists prove tobe typical of therapists in general, we would ex-pect to find that currently experienced growth de-clines as the number of years in practice in-creases. It is less plausible (even though logicallypossible) to expect that therapists will continue toexperience constant or increasing levels of currentgrowth throughout their professional careers.

3. What is the relation between a therapist'sperceived therapeutic mastery and currently expe-rienced growth as a therapist? Does that relationvary at different phases of the professional career?Although it would require a longitudinal study toinfer a causal relation between these two aspectsof development, exploration of the question withcross-sectional data can lead to the formation ofpotentially interesting hypotheses. For example,novice therapists might be expected to show asubstantial overlap in ratings of their currentgrowth and their therapeutic competence. By con-

204

Therapists' Assessments of Development

trast, an attenuated relationship between thosetwo variables would be expected if senior thera-pists experience higher levels of therapeutic mas-tery but only slight or moderate levels of cur-rent growth.

These questions will be examined using datacollected in a large survey of psychotherapistsof different professional backgrounds, theoreticalorientations, and nationalities.

Method

ParticipantsA total of 3,958 psychotherapists1 from West-

ern countries were available in the 1998 databaseof the Collaborative Research Network (CRN) ofthe Society for Psychotherapy Research (SPR)(Oriinsky et al., 1999). Of these, 2,443 (61.9%)identified themselves as psychologists, 955 (24.2%)were psychiatrists (or, in Germany, physicians inthe specialty of psychosomatics and psychother-apy), 239 (6.1%) were social workers, and the re-mainder (7.8%) were drawn from other professionalbackgrounds. In terms of theoretical orientation,62% of the therapists reported being strongly influ-enced in practice by analytic-psychodynamic ap-proaches, 32% by humanistic approaches, 24%by cognitive approaches, 23% by systemic ap-proaches, and 14% by behavioral approaches(multiple ratings were allowed from each respon-dent). The mean number of years in practice was11.2 (SD = 8.9), with a median of 10 and arange from less than 1 to as many as 52 years.Nearly all (94%) practiced individual psychother-apy. In addition, 35% practiced couples therapy,34% group psychotherapy, and 28% family ther-apy. About half of the therapists (48%) were inpart-time or full-time private practice. There were2,201 women in the sample (56.1%) and 1,723men (43.9%). They ranged in age from 22 to 90with a median of 42.6 years (M = 43.25, SD =10.3). The main nationalities represented in thesample were Germany (N = 1,059), Norway(N = 804), the United States (N = 680), Switzer-land (N = 255), and Portugal (N = 188). Smallernumbers were from other European countries(Austria, Belgium, Denmark, France, Italy, Swe-den, and the United Kingdom) as well as fromArgentina, Mexico, Russia, and Israel.

MeasuresThe measures used in this study were derived

from the Development of Psychotherapist Com-mon Core Questionnaires (DPCCQ), a lengthymultipurpose set of instruments designed to sur-vey varied aspects of the professional and per-sonal experiences of psychotherapists, counsel-ors, and other mental health workers.2 Thegeneral content and rationale of the DPCCQ havebeen presented in a report by Oriinsky et al.(1999), which also provides descriptive data andfactor analyses on several subsets of items rele-vant to therapists' assessments of their perceivedcareer development and currently experienced de-velopment. The dependent and independent vari-ables of the present study are based on thoseprior analyses.

Dependent variables. A factor analytically de-rived scale of "Perceived Therapeutic Mastery"was based on the following items (each rated by3,172 therapists on a scale from 0 [Not at all] to5 [Very]):

Overall, at the present time:

1. How much mastery do you have of the tech-niques and strategies involved in practicingtherapy?

2. How well do you understand what happensmoment-by-moment during therapy sessions?

3. How much precision, subtlety, and finessehave you attained in your therapeutic work?

4. How capable do you feel to guide the develop-ment of other psychotherapists?

A second factor analytically derived scale of"Currently Experienced Growth" was based onthe following items (each rated by 3,135 thera-pists on a scale from 0 [Not at all] to 5 [Very]):

In your recent psychotherapeutic work:

1. How much do you feel you are changing asa therapist?

2. How much does this change feel like progressor improvement?

3. How much do you feel you are overcomingpast limitations as a therapist?

4. How much do you feel you are becoming moreskillful in practicing therapy?

1 Subsequent Ns and percentages vary slightly due to smallamounts of missing data for specific variables.

2 A copy of the DPCCQ may be obtained by writing to thefirst author.

205

D. Orlinsky et al.

5. How much do you feel you are deepening yourunderstanding of therapy?

6. How much do you feel a growing sense ofenthusiasm about doing therapy?

Composite scale scores were derived by addingitem scores with unit weight and dividing by thenumber of items to retain the original scale an-chors. The internal consistency of each scale wasassessed by computing Cronbach's alpha. Thealpha for perceived therapeutic mastery was .84and for currently experienced growth was .85.

Independent variable. Professional therapeuticexperience was assessed by asking the followingquestion: "Overall, how long is it since you firstbegan to practice psychotherapy? (Include prac-tice during training, but exclude any periods dur-ing which you did not practice.)" Therapists re-sponded by indicating the number of years andmonths since they actually started to do therapy.

Descriptive variables. Therapists completedthe DPCCQ anonymously but were requested toidentify the country where they lived, their sex,the month and year of their birth, and the currentmonth and year (from which the therapist's agewas computed), in addition to reporting other per-sonal and demographic information not relevantto the present study. Concerning their back-ground, therapists were asked, "What is your pro-fessional identity? That is, how do you refer toyourself in professional contexts?" The followinglist of alternatives was supplied, with the instruc-tion to "check as many as apply to you: Psychia-trist; Psychologist; Social Worker; Psychoanalyst;Psychotherapist; Counselor; Physician; Nurse;Other (specified)." To assess theoretical orienta-tions, therapists were asked, "How much is yourcurrent therapeutic practice guided by each of thefollowing theoretical frameworks?" Ratings weremade (using a scale from 0 [Not at all] to 5 [Verygreatly]) for each of the following: "Analytic/Psychodynamic; Behavioral; Cognitive; Human-istic; Systemic; Other (specified)." Multiple rat-ings were allowed, and endorsements of 4 or 5were taken to indicate a "salient" theoreticalinfluence.

ProcedureThe DPCCQ inventory was constructed in 1990

by members of the international Society for Psy-chotherapy Research (SPR) who shared an inter-est in the training and development of psychother-apists (Orlinsky et al., 1991). This team, which

has become known as the SPR Collaborative Re-search Network, consisted initially of 12 psychol-ogists, psychiatrists, and social workers ofdiverse theoretical orientations working in Bel-gium, France, Germany, the Netherlands, Swit-zerland, the United Kingdom, and the UnitedStates. This has subsequently expanded to includeadditional colleagues in Argentina, Australia,Austria, Brazil, China, Denmark, Finland,Greece, Israel, Italy, Japan, Korea, Mexico, NewZealand, Norway, Poland, Portugal, Russia,Spain, and Sweden. The initial version of theDPCCQ contained 370 items, composed in En-glish but concurrently translated into French andGerman. Additional translations have been madeby native-speaking research colleagues into Chi-nese, Danish, Finnish, Greek, Hebrew, Italian,Japanese, Korean, Norwegian, Polish, Portu-guese, Russian, Spanish, and Swedish. Each ofthe items in each translation was rated by twoindependent bilingual judges on scales of accu-racy and idiomatic correctness and were acceptedonly when found to be good or excellent in eachrespect. Item translations judged to be fair or poorwere submitted to a third expert bilingual judgeand revised until all three agreed on the result.Sections of the DPCCQ that used single wordsin checklists were also assessed by independent"back translation" into English.

Since 1991, these translations of the DPCCQhave been used by CRN colleagues to collect datafrom mental health workers of all professions andtheoretical orientations in countries where sig-nificant populations of psychotherapists are to befound. Because the professional groups that prac-tice psychotherapy vary from country to country,however, and because the practice of psychother-apy is not yet legally controlled in numerouscountries, representative sampling of the interna-tional population of professional therapists wasvirtually impossible. Consequently, different datacollection strategies were employed in differentcountries. In some cases, this has involved ran-dom sampling of members of professional organi-zations or using lists of therapists and counselorspublished in local telephone directories; in othercases, solicitations of attendees at professionalconferences have been made with the cooperationand support of conference organizers. Faculty andstudents of academic departments and centers in-volved in training mental health professionalshave also contributed data, as have independentpractitioners approached by colleagues who are

206


CRN members. In all cases, the participation oftherapists has been voluntary, anonymous, andwithout remuneration. As it usually takes about1.5 to 2 hours to complete the DPCCQ, the seri-ous motivation of therapists who have contributeddata is virtually assured.

Since the overall study is primarily discovery-oriented rather than aimed at hypothesis-testing,the strategy selected for initial data collection fo-cused on the accumulation of a large data-basewith the internal diversity to permit disaggrega-tion into meaningful subgroups. Thus, while thesample might not be fully generalizable with re-spect to the inherently ill-defined populationsfrom which they were drawn, the generality offindings across diverse subgroups can be as-sessed. The collection of sufficient descriptivedata to ensure the detailed delineation of therapist

characteristics further allows for tentative gener-alization or "transferability" of findings to thera-pists with similar professional and demographiccharacteristics (Lincoln & Guba, 1985).

ResultsDescriptive data on the three basic variables in

this study are shown in Table 1, which demon-strates a broad range of years in practice (M =11.20.SD = 8.88) and relatively high levels bothin perceived therapeutic mastery and currentlyexperienced growth (means of 3.4 and 3.55, re-spectively, on a 0-5 scale). Table 1 also indicatesthe degree of consistency or generality acrosstherapist subgroups by calculating the effect size(ES) of the difference between subgroup meansand the total sample mean, as recommended byElliott, Stiles, and Shapiro (1993). Only three

TABLE 1. Means and Effect Sizes for Practice Duration, Perceived Therapeutic Mastery, andCurrently Experienced Growth

PerceivedTherapeutic

Total Sample

Therapist Subgroup

ProfessionMedicinePsychologyOther

OrientationAnalytic-dynamicCognitive-behavioralHumanisticSystemicBroad-spectrumOther

SexFemaleMale

NationGermanyNorwayPortugalSwitzerlandUSA

Years in

M =SD =

M

9.6312.697.88

11.3911.5911.5510.1113.1111.38

9.5713.19

8.7611.2110.1012.9215.47

Practice

11.208.88

ESt

.18

.17

.37

.02

.04

.04

.12

.22

.02

.18

.22

.28

.00

.12

.19

.48

Mastery

M = 3SD = .

M

3.203.533.08

3.383.413.473.313.823.53

3.323.53

3.243.343.533.553.55

.4183

ES

.03

.14

.40

.04

.00

.08

.12

.50

.15

.11

.14

.20

.08

.15

.18

.17

CurrentlyExperienced

Growth

MSD

M

3.383.563.69

3.533.443.603.653.793.66

3.613.48

3.383.573.573.493.66

= 3.55= .79

ES

.21

.01

.17

.03

.13

.07

.13

.30

.13

.07

.09

.22

.03

.02

.08

.14

Note, t Effect size of difference between subgroup mean and total sample mean; moderateES defind as s= .4 and is underlined.

207

D. Orlinsky et al.

moderate differences (ES > .4 but < .7) wereobserved. (ES rather than traditional p valueswere calculated because extremely small effectswould be statistically significant with the large Nof this study.)

One difference was that therapists affiliatedwith professions other than medicine or psychol-ogy (i.e., the relatively few social workers,nurses, and "lay" analysts in our sample) weremoderately lower (ES = .40) in level of per-ceived therapeutic mastery. A second moderatedifference in level of perceived therapeutic mas-tery (ES - .50) was observed in the relativelysmall group of broad-spectrum therapists whorated 4 or 5 (on a 0-5 scale) for four or moretheoretical orientations. A third difference wasthat American therapists in this sample were mod-erately more (ES = .48) experienced than theaverage, although they did not differ from thera-pists from four other countries referred to in Table

1 in levels of perceived therapeutic mastery andcurrently experienced development.

Otherwise, 45 of the 48 effect sizes computedshowed only small (ES > .2 but < .4) or negligi-ble (ES < .2) differences from the general samplemean based on profession, theoretical orientation,sex, and nationality. This implies that the inde-pendent and dependent variables in this studyshow a substantial degree of generality across alarge and diverse international sample of psycho-therapists, and that any relationships observedamong them probably are not attributable to dif-ferences in profession, theoretical orientation,sex, or nationality.

Table 2 shows product-moment correlationsamong years in practice, perceived therapeuticmastery, and currently experienced growth, bothfor the total sample and for each of the subgroupsseparately. Overall and within every subgroup,perceived therapeutic mastery was positively and

TABLE 2. Correlations of Practice Duration, Perceived Therapeutic Mastery, and Currently Experienced Growth

Perceived TherapeuticMastery by Years in

Currently ExperiencedGrowth by Years in

Perceived TherapeuticMastery by Current

Sample

Total

ProfessionMedicinePsychologyOther

OrientationAnalytic-dynamicCognitive-behavioralHumanisticSystemicBroad-spectrumOther

SexFemaleMale

NationGermanyNorwayPortugalSwitzerlandUSA

N

3036-3112

477-4832079-2155473-475

905-933333-357441-446264-271147-153732-745

1734-17641290-1335

550-562704-755176-181221-227580-643

Practice*

.53

.49

.50

.59

.54

.57

.53

.55

.42

.44

.53

.51

.52

.41

.27

.30

.67

Practice§

-.02

.02-.03

.02

-.04-.03-.10-.13-.06-.00

-.02.01

-.04-.03-.03

.05-.03

Growtht

.27

.33

.27

.27

.18

.26

.21

.11

.31

.31

.27

.30

.26

.34

.38

.34

.20

Notes. * p < .0001 for all correlations.§ All correlations nonsignificant except Humanistic (p = .03) and Systemic (p = .04).t p < .0001 for all correlations except r = .11.

208


significantly related to therapists' years in practice(p < .0001). The total sample correlation of .53indicates that 28% of the variance in perceivedtherapeutic mastery is predicted by practiceduration.

By contrast, the correlations between years inpractice and currently experienced growth wereessentially nil, both for the total sample (r =— .02) and for the separate subgroups. The latterwere mainly negative, ranging in magnitude from+ .05 to — . 13. Only two of 15 were statisticallysignificant despite the large sample numbers in-volved, years of practice and currently experi-enced growth were slightly negatively correlatedfor systemic therapists (r = -. 13, p = .04) andfor humanistic therapists (r = — . 10, p = .03).Thus, currently experienced growth was reportedby most therapists at the same levels regardlessof the length of time they had practiced, for exam-ple, whether they were novices or veteranpractitioners.

The different relationships of perceived thera-peutic mastery and of currently experiencedgrowth to therapists' years in practice are illus-trated in Figure 1. The distribution of years inpractice was divided into deciles for conve-nience,3 resulting in the following cohort defini-tions: (a) 0 to < 1.33 years of, n = 370; (b) 1.33to <3.15 years, n = 402; (c) 3.15 to <5 years,n = 315; (d) 5 to <7.25 years, n = 456;(e) 7.25 to <10 years, n = 387; (f) 10 to <12years, n = 338; (g) 12 to <15 years, n = 400;(h) 15 to <18 years, n = 377; (i) 18 to <23years, n = 404; (j) 23 to 52 years, n = 414.

For perceived therapeutic mastery, Figure 1shows a negatively accelerated increase in meanvalues (rated on a 0-5 scale) over time, progress-ing from a low of 2.28 for beginners to 2.76,3.09, 3.30, 3.38, 3.65, 3.74, 3.72, 3.81, and4.04 for the most senior group. Another way ofshowing the progression is to note the percentageof therapists in each cohort reporting high and

3 Conceptually based divisions of the career span into phaseor stage categories typically depend on institutional param-eters such as the training period within a profession (e.g.,masters-level, doctoral-level, or postdoctoral level in psychol-ogy) and/or legal parameters (e.g., licensure). Since theseparameters vary greatly among professions within and amongcountries, a descriptive statistical approach was taken to theconstruction of career cohorts for initial use with this multipro-fessional international database.

low levels of mastery (arbitrarily defined as ^4.0 on a 0-5 scale for high, and ̂ 3 for low).By this criterion the proportion claiming a highmastery in the total sample was 31.6%, but thisfigure is relatively meaningless because the levelschanged across the 10 experience level cohortsfrom 2.6% to 6.6%, 11.3%, 19.9%, 21.1%,37.8%, 44.3%, 45.1%, 49.0%, and finally to63.5%. Correspondingly, the proportion of thera-pists reporting low mastery across the 10 experi-ence level cohorts declined from 83.2% to 69.2%,52.3%, 39.8%, 35.2%, 17.5%, 14.8%, 16.6%,15.8%, and to only 7.6% for the most seniortherapists.

For currently experienced growth, on the otherhand, Figure 1 shows that therapists in all careercohorts reported similarly high mean levels thatremained close to the total sample mean of 3.5on a 0-5 scale, as follows: 3.47, 3.62, 3.59,3.56, 3.52, 3.59, 3.61, 3.54, 3.53, 3.48. Usingthe definitions of "high" and "low" given above,the proportion in the total sample reporting a highlevel of current professional growth was 33.6%,and the deviation from this among cohorts hadno consistent direction and was never more than4%. Even for extreme groups, mean levels ofcurrently experienced growth were 3.47 for nov-ices and 3.48 for the most senior therapists. Theproportion in the total sample reporting a lowlevel of current professional growth was 23.5%,fluctuating among cohorts by no more than 5%.

Despite the distinctly different relationships toyears in practice, Table 2 also shows that perceivedtherapeutic mastery and currently experiencedgrowth were moderately but positively and consis-tently correlated. The correlations ranged betweentherapist subgroups from r = . 11 (the only nonsig-nificant correlation) to r = .38 with a total samplecorrelation of r = .27 (p < .0001), indicating 7%of shared variance. This may indicate that therapistswho feel competent also tend to feel as if they arecurrently improving, that therapists who feel theyare currently improving also tend to feel high levelsof mastery, or that a third factor is exerting a smallbut significant influence in common on the twovariables (7% of shared variance).

Discussion

General Limitations

One general limitation of this study is that thesubjects, although numerous and diverse in pro-fessional and demographic characteristics, are not

209

D. Orlinsky et al.

4.5 -

23-52

Years in Practice (deciles)

Figure 1. Perceived therapeutic mastery and currently experienced growth by therapist career cohort.

a random sample of the international populationof psychotherapists at large. Thus there is no as-surance that our findings will prove representativeof therapists of all sorts in all places, althoughthis is probably true of all studies in the field.Given the geographical distribution and profes-sional diversity of professional psychotherapistsin the world today, generalizability can be assuredonly for arbitrarily restricted subgroups. For ex-ample, a random sample of American psycholo-gists who belong to one or more of the AmericanPsychological Association (APA) divisions thattherapists join (e.g., 12, 17, 29, 39) would notnecessarily be representative of American psy-chologists who do not belong to the APA or tothose particular divisions. It is likely that such arandom sample would be even less representativeof American therapists in other professions or ofpsychologists in other countries. As a practicalalternative, we have used our large and internallydiversified data set to assess the generality of ob-served effects across professional and demo-graphic characteristics, and we have shown thatthe measures and findings were consistent (for

these therapists at least) across differences in pro-fessional background, theoretical orientation,sex, and nationality. Readers may generalize ten-tatively to therapist groups of interest to them onthe basis of similarity to our sample.

Another limitation of this study is the fact thatthe measures of development were grounded inthe therapists' experiences and do not necessarilycorrespond to ratings of mastery or growth thatmight be made from other observational perspec-tives, for example, that of supervisors. Past re-search on therapeutic process and outcome (e.g.,Orlinsky, Grawe, & Parks, 1994; Strupp, Hadley,& Gomes-Schwartz, 1977) indicates that differ-ences may also be expected between assessmentsmade by supervisors or peers and therapists' ownratings of their professional development. Clearlythose other perspectives are important to under-standing therapist development and should be in-vestigated, but it was the intent of the presentstudy to focus on the therapists' experience,which has an intrinsic interest both for therapiststhemselves (as witnessed by their participation inour lengthy survey) and for researchers. Differ-

210


ences between therapists and others in perspec-tives on development would constitute an im-portant topic for research.4

Relation Between Dependent Variables

There was a moderate but consistent and sig-nificantly positive correlation (r = .27) betweenperceived therapeutic mastery and currently expe-rienced growth. Since causality cannot be inferredfrom a correlation, this finding may reflect eithera) a tendency among therapists who feel compe-tent (in all experience level cohorts) also to feelthat they are currently improving, or b) a trendamong therapists who feel they are currently im-proving (in all experience cohorts) also to feelhigh levels of mastery. Alternatively, some thirdelement common to both may be exerting a smallbut significant influence (7% of shared variance),which may reflect method variance attributable toboth variables being paper-and-pencil measurestaken on the same occasion (Fiske, 1971). In con-junction with their differential relationship toyears in practice, however, the modest level ofthe observed correlation at least makes clear thatthere is little redundancy between the two scales.Previous analyses indicated that they reflect dif-ferent dimensions of what professional develop-ment means to psychotherapists (Orlinsky etal., 1999).

Perceived Therapeutic Mastery

The scale of perceived therapeutic mastery wasclearly and consistently related to years in prac-tice, as anticipated. These scores reflect each ther-apist's assessment of his or her "mastery . . . ofthe techniques and strategies involved in practic-ing therapy," understanding of "what happensmoment-by-moment during therapy sessions,""precision, subtlety, and finesse" in therapeutic

4 Najavits and Strupp (1994) reported a study of 16 moder-ately experienced, psychodynamically-oriented Americantherapists, in which they compared the four most and fourleast "effective" therapists (based on researchers' ratings ofclient outcome and treatment retention) and found that themore effective therapists made significantly more self-criticalcomments concerning their performance in the third sessionof short-term time-limited therapy. This limited finding initself implies little with respect to therapist-based measuresof therapist development but suggests the importance of ex-ploring relations between diverse observational perspec-tives.

work, and ability "to guide the development ofother psychotherapists." Thus, therapists ratedmastery in the traditional senses of craft profi-ciency and of teaching ability (Orlinsky, 1999).

Because the observed relation between yearsin practice and perceived therapeutic mastery wasbased on cross-sectional rather than longitudinaldata, however, one cannot be sure that the resultwholly reflects an increased sense of masteryamong therapists as they gain experience. Someor all of the effect might be due to either selectiveattrition or historical cohort effects. Selective re-tirement from the field by practitioners who expe-rience the lowest levels of therapeutic masterycould explain the effect, but this attrition wouldhave had to continue over the entire career spanin order to produce the continual increments inlevel of mastery observed in this study. An alter-native explanation is that therapists in the oldestcohort felt high levels of perceived therapeuticmastery, that the youngest feel little mastery evenas they gain experience; thus, the progressive in-crease in felt mastery observed among succes-sively more senior cohorts reflects a historicaltrend toward lower professional self-confidencerather than a developmental trend toward greaterself-confidence. Logically, these alternativehypotheses can only be excluded by a longitudinalstudy of therapists' development encompassingmuch of the therapist's career. Until such a long-term study is done, researchers have no recourseother than to rely on relevant theory and personalobservations in judging the plausibility of alterna-tive interpretations.5 Accepting the uncertainty in-herent in this situation, it seems plausible to inter-pret the observed relationship between years inpractice and perceived therapeutic mastery as areflection of a genuine developmental trend. De-velopment here is understood in the traditionalsense as a long-term process by which prac-titioners gradually refine their natural talents, ac-quire technical mastery of their craft, and over-come personal limitations in professional work.This view is supported by significant positive cor-relations of perceived therapeutic mastery with

5 The most practical alternative would be to combine cross-sectional analyses of relatively large samples, as was donehere, with a series of short-term, longitudinal analyses usingsmaller samples, on the model of Lowenthal, Thurnher, andChiriboga (1975); these should cover a number of points dis-tributed across the career span, not just the student years.

211

D. Orlinsky et al.

both direct and derived retrospective estimates oftherapists' career development (Orlinsky et al.,1999). It also seems plausible to us that a portion(although probably not more than a small portion)of the observed relationship may be due to contin-uous selective attrition of the least self-confidenttherapists from the field of the effect. Of the inter-pretations considered, the hypothesis that cohortdifferences are due to a historical artifact seemsmost implausible.

Currently Experienced Growth

Contrary to initial expectation, the therapists'sense of currently experienced growth did notdecline as a function of years in practice but re-mained at a generally high level, even amongthose who had been in practice for more thantwo decades. Survey responses indicated that thissense of growth reflected experiences of "chang-ing," "improving," "becoming more skillful," a"deepening . . . understanding of therapy,""overcoming past limitations as a therapist," andfeeling "a growing sense of enthusiasm about do-ing therapy." These experiences were reportedto considerable extent by therapists at all careerlevels, as evidenced by a total sample mean of3.5 (one point above the middle of the 0-5 scale).

The lack of covariation between practice dura-tion and current professional growth cannot beattributed to lack of variability or to unreliabilityin either variable. On the other hand, it seemsquite implausible to suppose that very experi-enced practitioners, who (as we have seen) typi-cally feel a strong sense of therapeutic mastery,were actually growing to the same extent as thosewho are really just learning their craft. The ques-tion is how to interpret the result, which wasobserved in each of the therapist subgroups aswell as the sample at large.

Is this persistent, fairly strong sense of positivegrowth at all career stages just a collective illu-sion, possibly fostered in the psychotherapeuticprofession by a cultural commitment to the 19thcentury idea of progress reborn as an individualis-tic ethic of development or "personal growth"(see, e.g., Collingwood, 1946; Cushman, 1995;White, 1983)? Such an interpretation would seemplausible if therapists were uniform in reportinghigh levels of current growth, but in fact onlyabout a third of the therapists in each cohort metthe criterion we set to define high current growth,while a fifth to a fourth in each cohort met the

criterion set for low growth. There was about asmuch variation in currently experienced growthas in perceived therapeutic mastery (SD = .79and .83, respectively; see Table 1).

The most plausible hypothesis we can offer atpresent is that the therapists' sense of currentlyexperienced growth reflects a renewal of the mo-rale and motivation needed to practice therapy, areplenishment of the energy and refreshing of theacumen demanded by therapeutic work. Thisinterpretation recognizes that currently experi-enced growth is a measure grounded in the thera-pist's subjective experience, and that its level var-ies significantly among therapists but not in termsof their career cohort, profession, orientation, andso on. If currently experienced growth is viewedas the "regrowth" or restoration of recurrentlydepleted resources, it is understandable that it isas essential to highly experienced practitioners asto trainees. On this view, currently experiencedgrowth and the restoration it signals are neededas a consequence of the continuous investmentthat therapists make in their work rather than asa result of their inexperience.

Support for the plausibility of this hypothesiscomes from extensive research demonstrating thatpsychotherapeutic work often is stressful and im-poses a psychological cost on the providers ofcare (see, e.g., Deutsch, 1984; Dryden, 1997;Farber, 1983, 1985; Farber & Heifetz, 1981,1982; Grosch & Olsen, 1994; Hellman, Mor-rison, & Abramowitz, 1987; Maslach, 1982;Pines & Maslach, 1978; Raquepaw & Miller,1989). Findings show that therapists and othercare-givers tend to become depleted by the stress-ful aspect of their work and imply the need forexperiences that will renew their motivation andmorale. If therapeutic practice is to be tolerablesession after session, year after year, as it clearlyis for most therapists, there must be a processby which the therapist's interest and energy arecontinually renewed.

Fortunately, if the statements of many prac-titioners are a reliable guide, practice seems toprovide a great deal of stimulation and satisfactionfor therapists, not just stress or frustration (e.g.,Burton, 1972; Dryden & Spurting, 1989; Guy,1987). These personal accounts of the intrinsicrewards of therapeutic work are generally con-firmed by available studies (e.g., Farber &Heifetz, 1982; Henry, Sims, & Spray, 1971; Nor-cross & Guy, 1989; Orlinsky & Howard, 1977;

212


Parks & Orlinsky, 1998). Csikszentmihalyi's(1990) well-known work on "flow" shows howthe stimulation and intrinsic satisfaction experi-enced in work by therapists (and by others in thework or leisure activities they find absorbing) canbe a source of growth. This practice-based satis-faction can be further enhanced by effective su-pervision, continuing-education workshops, dis-cussions with colleagues, and other experiencessupporting the "continuous professional reflec-tion" that prior qualitative research has linked todevelopment in therapists (R0nnestad & Skov-holt, 1991; Skovholt & R0nnestad, 1995).

This interpretation of currently experiencedgrowth can also be placed in the broader contextof process-outcome research, which has demon-strated that outcome is related to therapists' abil-ity to provide their clients with the experienceof an empathic, responsively caring relationship(Orlinsky, Grawe, & Parks, 1994). To elicit suchperceptions, therapists need to approach each cli-ent and each session with energy, interest, andpersuasive optimism about the possibility ofmeaningful change. Therapy done in a routine orhalfhearted manner by bored, tired, or frustratedtherapists is unlikely to be helpful, and may bedamaging if patients blame themselves for thenegativity or lack of empathy they perceive(Henry, Schacht, & Strupp, 1990; Henry &Strupp, 1994). The view advanced here suggeststhat current professional growth is more than anoptional bonus that practitioners may or may notget from their therapeutic work, but is an essentialpositive outcome for therapists that is necessaryfor continued effective practice.

The idea of outcome for patients is taken forgranted because it is the raison d'etre of therapy;the idea of outcome for therapists is less familiar.Yet therapy has positive and negative effects fortherapists too, though the spheres of life differ inwhich these outcomes are felt. Therapy's primaryimpact for patients is in the sphere of personallife, affecting self-concept and self-managementskills, emotional balance and psychosomatic well-being, and personal and family relations. By con-trast, the impact of therapy for therapists is (orshould be) primarily in the sphere of professionallife in the forms of short-term and long-term pro-fessional development. The short-term impact oftherapy for therapists is felt in the presence orabsence of currently experienced growth. Thelong-term impact of therapy for therapists is found

in the initially rapid and then relatively gradualincrement of perceived therapeutic mastery.6

In our view, both aspects of therapist develop-ment contribute in turn to the influence that thera-pists can have in producing positive outcomes forpatients. The therapist's positive morale, energy,and openness (continually replenished by cur-rently experienced growth) contribute to the ther-apeutic process by reviving and supporting thepatient's morale (Frank, 1974) and by helpingtherapists maintain a nondefensive attitude whenresolving alliance ruptures (Safran, Muran, &Samstag, 1994) or healing empathic failures (Ko-hut, 1984). The therapist's sense of professionalcompetence, understanding, and resourcefulness(reflecting perceived therapeutic mastery) pro-vides self-confidence in dealing with challengingcases and may be the reason why, for experiencedbut not for inexperienced practitioners, the thera-pist's affective state is able to influence and mod-ify that of the client (Howard, Krause, & Orlin-sky, 1969).

Therapist Development and Clinical EffectivenessThese considerations lead directly to the ques-

tion of whether therapist development is relatedto clinical effectiveness. Several recent studieshave investigated the differential clinical effec-tiveness of therapists (Blatt, Sanislow, Zuroff,& Pilkonis, 1996; Crits-Christoph et al., 1991;Lafferty, Beutler, & Crago, 1989; Luborsky,McLellan, Diguer, Woody, & Seligman, 1997;Najavits & Strupp, 1994), but so far none havesought to relate clinical effectiveness directly tomeasures of therapist development. However,Beutler, Machado, and Neufeldt (1994) reviewedstudies that used duration of practice as a pre-

6 Conceiving the two aspects of development experiencedby psychotherapists as short-term and long-term effects oftherapy for therapists suggests a parallel with the phase modelof patient outcome proposed by Howard, Lueger, Mating,and Martinovich (1993), which involved a sequential focuson "remoralization," "remediation," and "rehabilitation." Pro-fessional development in the sense of an ongoing renewalof therapist motivation and work morale corresponds to theremoralization phase of client outcome, which is based onFrank's (1974) concept of outcome as a restoration of morale.Professional development viewed as the long-term refinementof talent, acquisition of technique, and transcendence of per-sonal limitations as a therapist corresponds to the later remedi-ation and rehabilitation phases of client outcome.

213

D. Orlinsky et al.

dictor of outcome and concluded, with some res-ervations, "that length of therapist experience byitself [italics added] is neither a strong nor a sig-nificant predictor of amount of improvement" inclients (p. 249).

The study presented here has shown that lengthof therapist experience (measured by years inpractice) is a reasonable predictor of therapists'perceived therapeutic mastery but not of their cur-rently experienced growth. In studying the rela-tion between these aspects of therapist develop-ment and clinical effectiveness, researchersshould bear in mind that process-outcome re-search has generally shown outcome to be moreconsistently associated with the quality of pa-tient's participation in therapy than with the thera-pist's (Orlinsky, Grawe, & Parks, 1994). Thequality of therapists' participation has some in-fluence on their clients' involvement, but profes-sional development very probably is just oneamong several factors that determine the qualityof the therapist's participation.7 Thus the relation-ship between therapist's professional developmentand patient outcome is a complex one mediatedand moderated by a number of intervening patient,therapist, and interactive process variables. Accord-ingly, the size (ES) of association between therapistdevelopment and clinical effectiveness is likely tobe modest at best, and in seeking to test the connec-tion, researchers should be prepared to use largesamples, as in the present study, to make the testfair (Kazdin, 1994, p. 45ff).

References

BEUTLER, L. E., MACHADO, P. P. P., & NEUFELDT, S. A.(1994). Therapist variables. In A. E. Bergin & S. L. Gar-field (Eds.), Handbook of psychotherapy and behaviorchange (4th ed., pp. 229-269). New York: John Wiley.

BLATT, S. J., SANISLOW, C. A., ZUROFF, D. C., &PILKONIS,P. A. (1996). Characteristics of effective therapists: Furtheranalyses of data from the National Institute of MentalHealth Treatment of Depression Collaborative ResearchProgram. Journal of Consulting and Clinical Psychology,64, 1276-1284.

BURTON, A. (Ed.). (1972). Twelve therapists. San Francisco:Jossey-Bass.

COUJNOWOOD, R. G. (1946). The idea of history. London:Oxford University Press.

CRTTS-CHRISTOPH, P., BARANACKIE, K., KURCIAS, J. S.,BECK, A. T., CARROLL, K., PERRY, K., LUBORSKY, L.,McLELLAN, A. T., WOODY, G. E., THOMPSON, L., GAL-

7 Another obvious factor is talent (e.g., Orlinsky, Boter-mans, & R0nnestad, 1998).

LAGHER, D., & ZTTRIN, C. (1991). Meta-analysis of thera-pist effects in psychotherapy outcome studies. Psychother-apy Research, I, 81-91.

CSIKSZENTMIHALYI, M. (1990). Flow: The psychology of opti-mal experience. New York: Harper & Row.

CUSHMAN, P. (1995). Constructing the self, constructingAmerica: A cultural history of psychotherapy. Reading,MA: Addison-Wesley.

DAWES, R. M. (1994). House of cards: Psychology and psy-chotherapy built on myth. Toronto: Maxwell Macmillan.

DEUTSCH, C. J. (1984). Self-reported sources of stress amongpsychotherapists. Professional Psychology: Research &Practice, 15, 833-845.

DRYDEN, W. (1997). Therapists' dilemmas. London: Sage.DRYDEN, W., & SPURLING, L. (Eds.). (1989). On becoming

a psychotherapist. New York: Tavistock/Routledge.ELLIOTT, R., STILES, W. B., & SHAPIRO, D. A. (1993). "Are

some psychotherapies more equivalent than others?" InT. R. Giles (Ed.), Handbook of effective psychotherapy(pp. 455-479). New York: Plenum Press.

PARSER, B. A. (Ed.). (1983). Stress and burnout in the humanservice profession. New York: Pergamon.

PARSER, B. A. (1985). Clinical psychologists' perceptions ofpsychotherapeutic work. Clinical Psychologist, 38,10-13.

PARSER, B. A., & HEIFETZ, L. J. (1981). The satisfactionsand stresses of psychotherapeutic work: A factor analyticstudy. Professional Psychology, 12, 621-630.

PARSER, B. A., & HEIFETZ, L. J. (1982). The process anddimensions of burnout in psychotherapists. ProfessionalPsychology, 13, 293-301.

FEKE, D. W. (1971). Measuring the concepts of personality.Chicago: Aldine.

FLEMING, J. (1953). The role of supervision in psychiatrictraining. Bulletin of the Menninger Clinic, 17, 157-159.

FRANK, J. D. (1974). Psychotherapy: The restoration of mo-rale. American Journal of Psychiatry, 131, 271-274.

GROSCH, W. N., & OLSEN, D. C. (1994). When helping startsto hurt: A new look at burnout among psychotherapists.New York: W. W. Norton.

GUY, J. D. (1987). The personal life of the psychotherapist:The impact of clinical practice on the therapist's intimaterelationships and emotional well-being. New York:Wiley-Interscience.

HELLMANN, I. D., MORRISON, T. L., & ABRAMOWTTZ, S. I.(1987). Therapist experience and the stresses of psychother-apeutic work. Psychotherapy, 24, 171-177.

HENRY, W. E., SIMS, J. H., & SPRAY, S. L. (1971). The fifthprofession: Becoming a psychotherapist. San Francisco:Jossey-Bass.

HENRY, W. P., SCHACHT, T. E., & STRUPP, H. H. (1990).Patient and therapist introject, interpersonal process, anddifferential psychotherapy outcome. Journal of Consultingand Clinical Psychology, 58, 768-774.

HENRY, W. P., & STRUPP, H. H. (1994). The therapeuticalliance as interpersonal process. In A. O. Horvath & L. S.Greenberg (Eds.), The working alliance: Theory, research,and practice (pp. 51-88). New York: John Wiley.

HESS, A. K. (Ed.). (1987). Psychotherapy supervision: The-ory, research, and practice. New York: John Wiley.

HOG AN, R. A. (1964). Issues and approaches in supervision.Psychotherapy: Theory, Research, and Practice, 1, 139-141.

HOLLOWAY, E. L. (1987). Developmental models of supervi-sion: Is it development? Professional Psychology: Researchand Practice, 18, 209-216.

214


HOWARD, K. I., KRAUSE, M. S., & ORLINSKY, D. E. (1969).Direction of affective influence in psychotherapy. Journalof Consulting and Clinical Psychology, 33, 614-620.

HOWARD, K. I., LUBGER, R. J., MALINO, M. S., & MARTINO-VICH, Z. (1993). A phase model of psychotherapy outcome:Causal mediation of change. Journal of Consulting andClinical Psychology, 61(4), 678-685.

KAZDIN, A. E. (1994). Methodology, design, and evaluationin psychotherapy research. In A. E. Bergin & S. L. Garfield(Eds.), Handbook of psychotherapy and behavior change(4th ed., pp. 19-71). New York: John Wiley.

KOHUT, H. (1984). How does analysis cure? Chicago: Univer-sity of Chicago Press.

LAFFERTY, P., BEUTLER, L. E., & CRAGO, M. (1989). Differ-ences between more and less effective psychotherapists: Astudy of select therapist variables. Journal of Consultingand Clinical Psychology, 57, 76-80.

LINCOLN, Y. S., & CUBA, E. G. (1985). Naturalistic inquiry.Newbury Park, CA: Sage.

LOGANBILL, C., HARDY, E., & DELWORTH, U. (1982). Super-vision: A conceptual model. Counseling Psychologist,10, 342.

LOWENTHAL, M. F., THURNHER, M., & CHDUBOGA, D.

(1975). Four stages of life. San Francisco: Jossey-Bass.LUBORSKY, L., McLELLAN, A. T., DlGUER, L., WOODY, G.,

& SELIOMAN, D. A. (1997). The psychotherapist matters:Comparison of outcomes across twenty-two therapists andseven patient samples. Clinical Psychology: Science andPractice, 4, 53-65.

MASIACH, C. (1982). Burnout: The cost of caring. New York:Prentice Hall.

NAJAVTTS, L. M., & STRUPP, H. H. (1994). Differences inthe effectiveness of psychodynamic therapists: A process-outcome study. Psychotherapy, 31, 114-123.

NORCROSS, J. C., & GUY, J. D. (1989). Ten therapists: Theprocess of becoming and being. In W. Dryden & L. Spur-ling (Eds.), On becoming a psychotherapist (pp. 215-239).New York: Tavistock/Routledge.

ORLINSKY, D. E. (1999). The master therapist: Ideal characteror clinical fiction? Journal of Counseling Psychology,46, 12-15.

ORLINSKY, D. E., AMBOHL, H., RCNNESTAD, M. H., DAVIS,J. D., GERIN, P., DAVIS, M., WILLUTZKI, U., BOTERMANS,J.-F., DAZORD, A., QERPKA, M., AAPRO, N., BUCHHEIM,P., BAE, S., DAVIDSON, C., FRUS-JORGENSEN, E., Joo, E.,KALMYKOVA, E., MEYERBERG, J., NORTHCUT, T., PARKS,B., SCHERB, E., SCHRODER, T., SHEFLER, G., STTWNE, D.,STUART, S., TARRAGONA, M., VASCO, A. B., & WISEMAN,H. (1999). The development of psychotherapists: Concepts,questions, and methods of a collaborative internationalstudy. Psychotherapy Research, 9(2), 127-153.

ORLINSKY, D. E., BOTERMANS, J.-F., & RCNNESTAD, M. H.(1998). Psychotherapeutic talent is the skill that therapistshave already when they start training: An empirical analy-sis. Abstracts of the 29th Annual Meeting of the Societyfor Psychotherapy Research (p. 224).

ORUNSKY, D. E., GERIN, P., DAVIS, J., DAVIS, M., AMBUHL,H., WILLUTZKI, U., DAZORD, A., DAVIDSON, C., BOTER-MANS, J.-F., AAPRO, N., BACKX, W., & IAHNS, J.-F.(1991). The international study of the development of psy-chotherapists: Conceptual organizational of the CommonCore Questionnaires. Unpublished manuscript, Universityof Chicago, Committee on Human Development.

ORLINSKY, D. E., GRAWE, K., & PARKS, B. K. (1994). Pro-cess and outcome in psychotherapy—noch einitial. InA. E. Bergin & S. L. Garfield (Eds.), Handbook of psycho-therapy and behavior change (4th ed., pp. 270-376). NewYork: John Wiley.

ORUNSKY, D. E., & HOWARD, K. I. (1977). The therapist'sexperience of psychotherapy. In A. Gurman & A. Razin(Eds.), Effective psychotherapy: A research handbook (pp.566-589). New York: Pergamon Press.

PARKS, B. K., & ORUNSKY, D. E. (1998). Intrinsic motivationfor psychotherapeutic work. Unpublished manuscript, Uni-versity of Chicago, Committee on Human Development.

PINES, A., & MASLACH, C. (1978). Characteristics of staffburnout in mental health settings. Hospital & CommunityPsychiatry, 29, 233-237.

RAQUEPAW, J. M., & MILLER, R. S. (1989). Psychotherapistburnout: A componential analysis. Professional Psychol-ogy: Research & Practice, 20, 32-36.

R0NNESTAD, M. H., & SKOVHOLT, T. M. (1991). En modellfor profesjonell utvikling og stagnasjon hos terapeuter ogradgivere. Tidsskrift for Norsk psykologforening. (A modelfor the professional development and stagnation of thera-pists and counselors). Journal of the Norwegian Psycholog-ical Association, 28, 555-567.

SAFRAN, J. D., MURAN, J. C., & SAMSTAG, L. W. (1994).Resolving therapeutic alliance ruptures: A task analytic in-vestigation. In A. O. Horvath & L. S. Greenberg (Eds.),The working alliance: Theory, research, and practice (pp.225-255). New York: John Wiley.

SKOVHOLT, T. M., & RCNNESTAD, M. H. (1995). The evolvingprofessional self: Stages and themes in therapists and coun-selor development. Chichester, UK: John wiley.

SKOVHOLT, T. M., RCNNESTAD, M. H., & JENNINGS, L.(1997). In search of expertise in counseling, psychotherapy,and professional psychology. Educational Psychology Re-view, 9, 361-369.

STEIN, D. M., & LAMBERT, M. J. (1984). On the relationshipbetween therapist experience and psychotherapy outcome.Clinical Psychology Review, 4, 1-16.

STOLTENBERG, C., & DELWORTH, U. (1987). Supervisingcounselors and therapists: A developmental approach. SanFrancisco: Jossey-Bass.

STRUPP, H. H., HADLEY, S. W, & GOMES-SCHWARTZ, B.(1977). Psychotherapy for better or worse: The problemof negative effects. New York: Jason Aronson.

WHITE, S. H. (1983). The idea of development in develop-mental psychology. In R. M. Lerner (Ed.), Developmentalpsychology: Historical and philosophical perspectives (pp.55-77). Hillsdale, NJ: Lawrene Erlbaum Associates.

215

psychotherapists' assessments of their development at different career levels

Documents