Initiating International Collaborative Research

Projects in Mental Health:

A Progress Report on the International Project on the

Effectiveness of Psychotherapy and Psychotherapy Training (IPEPPT)

International Project on the Effectiveness of Psychotherapy and Psychotherapy Training (IPEPPT) Formally initiated, June 2004, by:

Italian Coordinamento Nazionale Scuole di Psicoterapia (CNSP; >5,000 therapists)

Italian Federation of Psychotherapy Associations (FIAP; 21 psychotherapy associations: >10,000 therapists)

General Goal: To improve psychotherapy and psychotherapy training in a broad range of theoretical approaches by encouraging systematic research in therapy training institutes and university-based training clinics.

Political purpose: Increase status of involved organizations

IPEPPT General Scientific Steering Committee Robert Elliott, Scientific Director (University of Toledo-

USA) Alberto Zucconi, Coordinator (University of Siena-Italy) David Orlinsky (University of Chicago-USA) Franz Caspar (University of Freiburg) Louis Castonguay (Pennsylvania State University-USA) Glenys Parry (University of Sheffield-UK) Bernhard Strauss (Friedrich Schiller University Jena-

Germany)

IPEPPT: Current Status Still in formation stage Not a single study The “Project” = Promoting practice-based

research in Europe, North American and elsewhere

Finding partners/applying for EU funding Creating/finding tools

E.g., conceptual/organizing concepts Finding/translating instruments

KU Leuven piloting Dutch-language systematic case study protocol

IPEPPT Immediate Objectives 1. To construct a list of agreed-upon general

pantheoretical recommendations for evaluating: Key aspects of therapy, especially in training centers Key aspects of therapy training outcome Not a “Core Battery”

2. To facilitate the development of specific treatment and training outcome protocols for particular: Therapy approaches (e.g., Systemic therapy) Client populations (e.g., people living with schizophrenia) Linguistic/national groups (e.g., Italy)

3. To facilitate national/international collaborations

IPEPPT Draft Research Framework Such a project requires a guiding

conceptual framework for determining what to measure and how to measure it

Work-in-progress 8 measurement domains:

4 Research themes 2 Levels (Star design)

Research Theme A. General/ Pantheoretical

B. Treatment- Specific

I. Therapy Outcome

e.g., general problem severity

e.g., theory-based dysfunctional processes

II. Therapy Process

e.g., therapeutic alliance

e.g., therapist techniques

III. Client/Ther-apist Background

e.g., demographics e.g., preference for type of therapy

IV. Training Outcome

e.g., productive vs. unproductive practice pattern

e.g., therapist skill development

Framework: Eight Therapy Measurement Domains, with examples of key concepts

Structure: (1) “Star” Design Main body of the star = General outcome/

process protocol Shared by all orientations (General/

Pantheoretical) Provides common metric

Star rays = Specialized protocols for different therapy approaches and different countries (Treatment/Population/Language Specific)

“Star” Design for Sample Concepts within Therapy Outcome Domain for Studies of Four

Different Therapies

Qualitative perceptions of

change

General problem severity

Interpersonal/ relational issues

Dysfunctional Attitudes

Target Problems

Implicit Cognitive Biases

Self-Ideal Discrepancy

Experiential Access

Self-Esteem

Level of Object Relations

Maturity of Defenses

CCRT Change

Relational Satisfaction

Family Environment

Interpersonal Empathy

CBTCBT Experi-Experi-entialential

Psycho-Psycho-dynamicdynamic

Family/Family/CouplesCouples

Structure: (2) Nested Priority Lists

Not a single “core battery” Allow flexibility while encouraging

consistency within & across approaches Three Levels of Priorities:

1) Measurement domains are prioritized2) Within each measurement domain, key concepts

are ranked by approximate importance3) For each concept, available instruments are also

described (researchers prioritize)

Research Theme A. General/ Pantheoretical

B. Treatment- Specific

I. Therapy Outcome

e.g., general problem severity

e.g., theory-based dysfunctional processes

II. Therapy Process

e.g., therapeutic alliance

e.g., therapist techniques

III. Client/Ther-apist Background

e.g., demographics e.g., preference for type of therapy

IV. Training Outcome

e.g., productive vs. unproductive practice pattern

e.g., therapist skill development

Framework: Eight Therapy Measurement Domains, with examples of key concepts

Example: General Therapy Outcome Domain

Key concepts in a possible recommended priority order: (“Star”) (1) General problem severity (quantitative)

Give every 2 sessions to reduce data loss from drop-out (2) Interpersonal/relational functioning (3) Qualitative perceptions of change (4) Individualized problems/goals (5) Health care utilization/costs (6) Quality of life/life satisfaction/well-being

Instrument (length) Reference ShorterForms(length)

Scale Basis,Points &Time Frame

Cost Non-EnglishTrans-lations

Symptom Checklist-90-Revised (SCL-90-R) (90 items)

Derogatiset al., 1976

BriefSymptomInventory (53items)

Distress5 pointsPast week

$2 each(scoringextra)

10 (incl.Norwe-gian)

Clinical Outcomes inRoutine EvaluationOutcome Measure(CORE-OM; 34items)

Evans etal., 2002

CORE-SF(18 items)

Frequency5 pointsPast week

Free Italian,Slovak,Norwe-gian

OutcomeQuestionnaire (OQ-45) (45 items)

Lambert etal., 1996

OQ-30 (30items); OQ-10 (10 items)

Frequency5 pointsPast week

One-timelicensingfee (e.g.,$30 for astudent)

German,Dutch,Spanish,Italian

Treatment OutcomePackage (TOP) AdultClinical Scales v4.0(58 items)

Kraus etal., 2005

TOP 37 (37items)

Frequency6 pointsPast 2 weeks

Free Spanish

Common Client Problem Severity Instruments

Research Theme A. General/ Pantheoretical

B. Treatment- Specific

I. Therapy Outcome

e.g., general problem severity

e.g., theory-based dysfunctional processes

II. Therapy Process

e.g., therapeutic alliance

e.g., therapist techniques

III. Client/Ther-apist Background

e.g., demographics e.g., preference for type of therapy

IV. Training Outcome

e.g., productive vs. unproductive practice pattern

e.g., therapist skill development

Framework: Eight Therapy Measurement Domains, with examples of key concepts

Example: General Therapy Process Domain

Key concepts in possible recommended priority order: (1) Therapeutic alliance (2) Perceived helpful aspects of therapy (3) Therapist and client response modes (4) Perceived session effectiveness

Common Therapeutic Alliance Instruments

Instrument(items)

Reference Scale Basis,Points &Time Frame

Subscales Informant Non-EnglishTranslations

WorkingAllianceInventory(WAI) (36items; 12 itemshort form)

Horvath &Greenberg,1989;Hatcher,2005a

Frequency7 points(revised shortform: 5points)

BondTaskagreementGoalagreement

ClientTherapistObserver

DutchDanishFrenchItalianNorwegian

CaliforniaPsychotherapyAlliance Scale(CALPAS; 24items)

Gaston &Marmar,1994

Agreement-Disagreement6 points

PatientWorkingCapacityPatientCommitmentTherapistUnderstandingandInvolvementWorkingStrategyConsensus

ClientTherapist

FrenchPortugueseItalian

Penn HelpingAllianceQuestionnaire-II(Haq-II; 19items)

Luborskyet al., 1996

Agreement7 points

-- ClientTherapist

GermanFrenchNorwegianDutch

Different Levels of Research Protocol are Possible

I. Minimum Protocol II. Systematic Case Study Protocol III. Maximum Protocol

Other Protocols: IV. General Training Protocols V. Specific Treatment Protocols

I. A Recommended Minimum Protocol: Applications

Easy to use: Limited to one measure from each of the first three research domains

Can use with own clients Provides basic treatment monitoring for

individuals & agencies Other versions are possible (e.g., different

outcome or process measures)

I. A Recommended Minimum Protocol: Elements

(1) General therapy outcome instrument Client problem severity Give at odd-numbered sessions (short form)

(2) General therapy process Therapeutic alliance (use short from)

(3) Client/therapist background measure Standard practice: Client/ therapist demographics Client diagnosis, presenting problems Type of therapy

II. Systematic Case Study Protocol: Applications

Use for student case study requirements Meets emerging standards for systematic single

case research New online journal: Pragmatic Case Studies in

Psychotherapy (Rutgers University, Editor: Fishman)

II. Systematic Case Study Protocol: Elements

A. Therapy Outcome: (1) Weekly/biweekly outcome measure+ (2) At least one other quantitative outcome measure+ (3) Qualitative outcome assessment (e.g., post-therapy interview)

B. Therapy Process (1) Therapeutic alliance+ (2) Detailed record of therapy (process notes and/or recordings)+ (3) Qualitative perception of helpful aspects (post-session and/or post-

therapy) C. Client/therapist background

Client/ therapist demographics; client diagnosis, presenting problem; type of therapy

II. Systematic Case Study Protocol: Research Questions

(1) Did the client change substantially over the course of therapy?

(2) If the client changed, did therapy make a substantial contribution?

(3) What brought about the client’s changes?

II. Systematic Case Study Protocol: Emerging Evidence Standards (1) Rich case record, including both quantitative &

qualitative data (2) Replication/convergence across methods (3) Critical examination of alternative views (e.g.,

Hermeneutic Single Case Efficacy Design, Elliott, 2002): Non-change explanations (e.g., measurement error) Non-therapy explanations (e.g., extra-therapy events)

(4) Narrative coherence Narrative model of predisposing and process factors Use for generalizing to other cases

Include measures of at least one concept in each of the eight domains

Appropriate for research centers (e.g., Center for the Study of Experiential Psychotherapy)

Also consortia of cooperating centers: Each center measure some variables

III. Maximum Protocol

IV. General Training Research Protocols: Issues Outcomes of therapy training not well understood Difficulties:

Lack of agreed-upon measures of therapist functioning and skill

Must measure therapist change longitudinally over several years of training

Possible applications: Use research to improve training Meet requirements of accrediting and funding agencies

IV. General Training Research Protocols: Promising Concepts

Quality of therapist professional involvement and growth (e.g., Orlinsky &Rønnestad; Collaborative Research Network [CRN])

Qualitative perceptions of effects and important aspects of training (e.g., qualitative interviews)

General therapist facilitative interpersonal skills (e.g., coping with common difficulties)

Change in therapist self concept (e.g., Scilligo, SASB Introject scales)

V. Specific Treatment Protocols

= Star rays Applications: For specific theoretical approaches,

client populations, or language groups Requires working committee for each group

Identify relevant therapy outcomes, processes, background variables (or training outcomes)

Do protocol and measure development research Establish virtual communities for exchanging ideas

V. Specific Protocol Example:

Person-Centered and Experiential Psychotherapy International Research Group (PCEP-IRG)

Current core members: University of Toledo (Elliott & team) Ohio University (Anderson & team) Katholieke Universiteit Leuven (Leijssen & team) Universities of Strathclyde & Abertay, Scotland

(McLeod, Cooper)

V. PCEP-IRG Outcome Protocol: Promising Developments

Center for the Study of Experiential Therapy Research Protocol (CSEP- 2):

Self-determined problems/goals: Personal Questionnaire (PQ-10)

Self-concept (content & coherence) Qualitative Self-Description interview Tennessee Self-Concept Scale 2 (long, short forms)

Experiential processing: Toronto Alexithymia Scale (TAS-20)

Need positive mental health measures, self-coherence, etc.

V. Specific Training Research Protocols

Some Possible Types of Specific Training Outcomes: Mastery of therapy theory/knowledge

Case formulation skill (e.g., use clinical vignettes) Treatment-specific intervention skill Therapist personal development (e.g., maturity,

identification with orientation, values)

Promising New Therapy Research Methods Make this Work Possible

Systematic qualitative research methods Interpretive single case designs (Fishman,

Elliott) Using early outcome to identify & repair

problems (Lambert: Signal alarm methods) New, powerful psychometric methods

(Rasch analysis/Item Response Theory) Virtual communities (Community Zero)

Lessons Learned So Far Importance of vision and leadership Leading by example is important Both immediate and longterm payoffs offs are

important These things eat up a lot of time Grant funding is not necessary and can

complicate broad collaborations Clinically interesting examples are useful to

persuading skeptical possible participants Progress is possible in creating a research

climate in training institutes and schools

Invitation to Dialogue - 1

1) Provide comments and suggestions on the framework & concepts presented here: Robert.Elliott@utoledo.edu

2) Form or join online discussion groups or virtual communities

Closed sites; must apply for membership General info: www.communityzero.com/ipepp Example: www.communityzero.com/pcepirp

Open site under development: www.ipeppt.net

3) Begin implementing the minimum protocol design with your own clients and in your own training setting.

Invitation to Dialogue - 2

4) Convert traditional case presentation training requirements into systematic case study exercises

5) Help with translations of key research instruments

6) Contribute to psychometric research: Improve existing instruments Equate different instruments for same concepts

7) Collaborate with groups with similar interests to generate data for pooling.