Session by Session Monitoring of Outcome and Alliance

David C. Low

Family & Systemic Psychotherapist

Feedback Informed Treatment (ORS & SRS) Trainer

david.low1964@gmail.com

Tel. 01473-280416

Aim of the Presentation

• Provide introduction to the research and clinical rational for Session by Session monitoring of outcome and alliance.

• Consider how session by session measuring fits with systemic thinking and practice.

• Introduce the Outcome Rating Scale (ORS) and Session Rating Scale (SRS) and how they can be used with families.

• Introduce how the measures can be used to bring the clients’ voice into clinical supervision.

“So What?”5 Minute Exercise

• Get in small groups of 3.

• Introduce yourselves.

• Choose someone to feedback themes to the rest of us.

• Very briefly discuss the relevance of session by session measuring has to each of you, and why any part of it would be important to you.

The Turn to “Outcome”

• “Outcome,” “Evidence,” “Effectiveness” &“Accountability” are the watchwords of the day.

• Part of a world wide discourse and trend and not specific to mental health.

What Works in Therapy: the Tale of Two Paradigms

• The Medical Model Evidence Based Practice Paradigm:

– Diagnosis + RCT Tx = Good Outcome?

– NICE and PBR “What Works for Whom.”

To

• The Contextual Model Practice Based Evidence Paradigm:

– Is this relationship between this client and this therapist working for this young person/family at this time and place?

– i.e. Session by Session Measuring

Session by Session Monitoringthe Evidence

• Currently, 13 RCT’s involving 12,374 clinically, culturally, and economically diverse clients provides evidence that routine monitoring and feedback

– As much as doubles treatment “effect size”;

– Decreases drop-out rates by as much as half;

– Decreases deterioration by 33%;

– Reduces hospitalizations and shortened length of stay by 66%;

– Significantly reduced cost of care (non-feedback groups increased).

Miller, S.D. (2010). Psychometrics of the ORS and SRS. Results from RCT’s and meta-analyses of routine outcome monitoring and feedback: The available evidence. http://www.scottdmiller.com/?q=blog/1&page=2.

How Does Session by Session Measures Fit with Systemic Thinking?

• Cybernetics from Greek kubernētēs steersman, from kubernan to steer, control

• Cybernetics concerned with self-regulating systems and how their stability and functioning were maintained through mechanisms of information, control and feedback.

• A key notion of cybernetic is the principle of feedback– how information about the results or consequences of a previous action return or loop back into the system in order regulate subsequent sessions.

• Session by session, ultra brief, reliable and valid measures provide feedback into the therapeutic system to inform when we are on track and when we are off track and need to adjust or do something different in order to have a positive outcome.

Why do I need Formal Measures as I Already ask for Feedback?

• Routine tracking of client progress using standardized measures is especially important given clinicians’ tendencies to:

– be overly optimistic about the meaning of clients’ lack of progress

– their over optimism of eventual improvement in the face of slow progress,

– and their failure to judge when clients are headed toward a negative outcome.

Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring,

and feedback in clinical practice. Washington, DC: American Psychological Association.

Self-Assessment Bias:Clinicians Have an Overly Positive Assessment of

Personal Performance

• 25% of mental health professionals viewed their skill to be at the 90th percentile when compared to their peers

• Most thought they were above average.

• None viewed themselves as below average.

• Clinicians tended to overestimate their rates of client improvement, and underestimate their rates of client deterioration.

• Using session by session measures keeps us grounded, and improving our performance and outcomes.

Walfish, S, McAlister, B., O'Donnell, P., & Lambert, M.J. (2012)An Investigation of Self-assessment Bias in Mental Health Providers. Psychological Reports: Volume 110, Issue, pp639-644.

Benefits of Session by Session Measuring

• Puts the client’s voice at center stage

• Allows services to be client and family driven

• Brings consumers into the inner circle of decisions

• Partners in monitoring the benefit and fit of services

• Collaboration, the Next Generation: Going boldly where we have not gone before.

Ok, But Every Session?

• The evidence demonstrates that frequent measurement, leads to improved outcomes, decreases deterioration rates, and reduces no shows.

• Would you advise a parent that it is ok to be consistent some of the time with parenting strategy and still expect their preferred outcome.

• Can you imagine a GP who is treating you for high blood pressure saying,  “Do I need to measure the blood pressure EVERY TIME?”

Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring,

and feedback in clinical practice. Washington, DC: American Psychological Association.

The purpose of the ORS & SRS:• To give therapists ongoing, formalized

feedback about the progress and process of therapy, allowing them to maximize the effect of services.

• The ultra brief design and formats for different age groups including small children, allows all members of the family an important feedback voice into the therapeutic system, and enhances everyone’s engagement and participation in family work.

• The ORS and SRS have good validity and reliability, and high feasibility.

Reluctant Using the ORS & SRS as It Might Infringe on Therapy Time

• Using the ORS and SRS to get feedback about the benefit & fit of treatment need not be cumbersome or intrusive on therapy time.

• Clients appreciate your dedication to getting it right,

• It is painless and can fit your

natural way of working.

Using the ORS and SRS to Help Level Power Imbalances

• Invites clients into inner circle of decision making

• Available in multiple languages

• Gives a voice to diversity and the disenfranchised

• Local culture and context over privileged knowledge

ORS & CORS Integrating Outcome Into Care

• The ORS, a brief measure of global subjective distress, is administered at the beginning of each session to get real time feedback from young people and carers regarding their perspective of treatment progress.

• There is a version for those 13 to adult, and a younger version with smiley faces called CORS for 6-12 year olds.

Postmodernism and ORS/CORS• Other measures focus on symptoms and

diagnostic constructs and tends to measure dysfunction (or rather peoples perception on constructed notions of dysfunction), and increase possibilities of inviting a person to adopt a 'mental illness' identity.

• In contrast, the ORS is a simple generic measure that takes at face value what people construct as meaningful for themselves.

• It is easy and feasible to integrate into a variety of practice modalities including post modern models that help clients re-story and develop new narratives.

Introducing the ORS/CORS to Young Person and Carer

“The ORS is an outcome measure that allowsus to track where you’re at, how you’re doing,how things are changing or if they are not. Itallows us to determine whether the therapy isbeing helpful so we can do something differentif it’s not helping. It also is way to make surethat your perspective stays central here, thatwe are addressing what you think is mostimportant. It only takes a minute to fill out andmost clients find it to be very helpful. Wouldyou like to give it a try? Are you ok with that?”

“Ok, so let me go over the instructions with you”.

Introducing the ORS/SRS to other Members of the Family

• At most first interviews you won’t know where the distress is, so you can ask all family members present to complete the ORS on themselves. This allows you to "see" who is distressed.

• To carer and other family members present: “I would also be grateful if the rest of you would also fill the form out on yourselves to help me understand how things are going for you, and to make sure what we are doing here is benefitting you too.

Even if things are going ok for you, I would be grateful if you could do this anyway, so we can monitor and ensure that whatever we do together doesn't impact you negatively.”

Outcome Rating Scale (ORS)(Ages 13 to Adult)

Individually(Personal well-being)

I----------------------------------------------------------------------I

Interpersonally(Family, close relationships)

I----------------------------------------------------------------------I

Socially (Work, school, friendships)

I----------------------------------------------------------------------I

Overall(General sense of well-being)

I----------------------------------------------------------------------I

Child Outcome Rating Scale (CORS) Ages 6-12

Me(How am I doing?)

I------------------------------------------------------------------------------------I

Family(How are things in my family?)

I------------------------------------------------------------------------------------I

School(How am I doing at school?)

I------------------------------------------------------------------------------------I

Everything(How is everything going?)

I------------------------------------------------------------------------------------I

The ORS: The Bare Bones

• The ORS has no specific content other than the 4 domains—a skeleton to which clients add the flesh & blood of their experiences.

• At the moment clients connect the marks with what they find distressing, the ORS is a meaningful measure of progress & potent clinical tool. Get a good rating

First Session: Connect ORS to theClient’s Described Experience

Principles:• Where is the distress?

• Constructing meaning of the scores i.e. Connect the client’s described experience of their lives to the marks on the scales

• Systemic questioning

Graphing: A Helpful Visual

• Adds a visual component — another way to focus tx

• At a glance tells the story of the client’s progress.

• Indispensable with couples & families, helps discuss multiple viewpoints

CORS Cutoff (age 6-12)

child = 32

carer = 28

ORS Cutoff

Age 13-17= 28

Age 18 + = 25

40

35

30

25

20

15

10

5

0

Date

ORS/CORS Graph

Making the Measures Work with Families

• Invite all members of the family to Complete the ORS or CORS.

• Invite all members of the family to interpret.

• Summarize progress as entry point into the meeting.

15 Year Old ORS

• What stands out from this teens scores in addition to being above the clinical cut off which is 28?

• Suppose the mother’s collateral scoring of the teen is much lower than the teens self report scoring, how could we use this information to begin or focus the session?

7.6

5.7

8.0

7.5

28.8

Linking Outcome to Treatment

• See clients more frequently when the slope of change is steep.

• Begin to space the visits as the rate of change lessens.

• See clients as long as there is meaningful change & they desire to continue.

ORS/CORSReliable Change Index

• Reliable Change Index = 5

Reliable

Deterioration5 point or more

worsening from pre-treatment

score

Reliable

Change5 point or more

improvement from pre-

treatment score

Clinically Significant Change!5 point or more improvement from pre-

treatment score and crossed the cutoff score for age group

Alliance comprised of four empirically established components which the SRS Measures:

1. agreement on goals, meaning or purpose of treatment;

2. agreement on the means and methods used;

3. agreement on the therapist’s role (including being perceived as warm, empathic and genuine; and

4. accommodating the client’s preferences.

Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of change 2nd ed.: Delivering what works in therapy (pp.113-141).Washington, D.C.: American Psychological Association

Why is Monitoring the Alliance So Crucial?

• Researchers have repeatedly found that the therapeutic alliance as rated by the client, is one of the best predictors of outcome across different types of therapy including psychopharmocology.

• The feedback from the SRS helps clinicians make the necessary adjustments to align simultaneously with all family members in the pursuit of common agreement on goals and tasks in therapy.

Martin, D.J., Garske, J.P., & Davis, M.K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68, 438-450.

Wampold, B.E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Erlbaum.;

Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of change 2nd ed.: Delivering what works in therapy (pp.113-141).Washington, D.C.: American Psychological Association.

But I Already Ask My Clients for Feedback, Why do I Need use an Alliance Measure?

• Research has shown that clinicians are poor at gauging their client’s experience of the alliance and they need to request more formal real time alliance feedback.

• The key here is that people THINK they ask more often than they do.

• Plus, and more importantly, since the data says it’s the client's point of view that is most predictive of outcome, you don't want to leave it up to the therapist to decide WHEN to ask. 

Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of change 2nd ed.: Delivering what works in therapy (pp.113-141).Washington, D.C.: American Psychological Association.

Anker, M.G., Duncan, B.L., Owen, J., & Sparks, J.A. (2010). The alliance in couple therapy: Partner influence, early change, ad alliance patterns in a naturalistic sample. Journal Consulting and Clinical Psychology, 78(5), 635-645.

Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice. Washington, DC: American Psychological Association.

Quickest Way to Prevent Drop Out

• Clients drop out for 2 reasons: therapy is not helping (monitor outcome) & alliance problems—not engaged or turned on.

• Direct way to improve effectiveness is to keep people engaged in therapy.

• Gotta measure the alliance to improve your successes!

SRS & CSRSUsing the Client’s Experience of the Alliance to Guide

Practice

• The SRS is administered at the end of each session to get feedback on the alliance and therapy process from young people and carers perspective.

• Like the ORS there is a version for those 13 on up, and a younger version called CORS for 6-12 year olds.

• There is also a Group Version called GSRS.

Introducing the SRS/CSRS“Ok, we need to end in a couple minutes, but before we do I would be grateful if you would take a minute to fill out this other form which asks your opinion about our work together today?

I am not hoping for a perfect score: life is not perfect and I know I am not either. I rely on this feedback to keep me on track, and let me know when I am off track and need to make some changes to ensure that what we are doing is benefitting you.

The ultimate purpose of using these forms is to make every possible effort to make our work together beneficial. There might be some small things that you think you might be able to live with or are not important, but your feedback on even the small things are critical to me as it can be the small things that make the difference. If anything feels missing, or I said something that did not sit right or left you feeling blamed, you would be doing me the best favor if you let me know. Can you help me out?

Ok. Here is one for each of you. Remember please feel free to give your honest opinion when filling this out. Ok?”

Session Rating Scale (SRS)(Ages 13 to Adult)

Relationship

I-----------------------------------------------------------------------I

Goals and Topics

I------------------------------------------------------------------------I

Approach or Method

I-----------------------------------------------------------------------I

OverallI------------------------------------------------------------------------I

I did not feel heard,

understood, and respected.

I did feel heard

understood, and

respected.

We did not work on or talk about what I wanted to work on

and talk about.

We did work on or talk about what I

wanted to work on and talk about.

The therapist’s approach is not a good fit for me.

The therapist’s approach is a good fit

for me.

There was something missing in

the session today.

Overall, today’s session was right

for me.

Child Session Rating Scale (CSRS)Ages 6-12

Listening

I-----------------------------------------------I

How Important

I-----------------------------------------------I

What We Did

I-----------------------------------------------I

Overall

I-----------------------------------------------I

did not always listen to me.

What we did and talked about was

not really that important to me.

I did not like what we did today.

I wish we could do

something different.

listened to me.

What we did and talked about

were important to

me.

I liked what we did today.

I hope we do the same kind of things next

time.

SRS/CSRS GraphSRS Cutoff =36

Discuss

40

35

30

25

20

15

10

5

0

Date

SRS Scores Above the Cutoff Score: Keep Convey Negative Feedback is a Gift

• Keep encouraging client to let you know…

• No bad news on the SRS! Not a measure of competence or anything negative about you or the client.

• Negative Feedback is a gift from the client that helps you to be better.

• Unless you really want it, you are unlikely to get it.

• You won’t get it from everyone.

What About Below 36 or 9cm

• Scores < 36 or 9cms should be discussed.

• Don’t expect specifics or revelations—any feedback is a godsend

• “Is there anything else I could have done, something I should have done more of or less of, some question or topic I should have asked, was there anything I said that did not sit right with you?”

Low SRS Scores Can be OK

Caution:

• Don’t assume a low SRS score means that there are alliance issues.

• The most effective therapists tend to have lower SRS scores and seem to be more skilled at generating honest feedback and using negative feedback to improve performance and outcomes

• Key is to consider if the ORS/CORS scores are improving. If not, there may be a rupture or problem with the alliance.

Does Not Seem to be Improving“Okay, so things haven’t changed since the

last time we talked. How do you make sense of that?

Should we be doing something different here,

or should we continue on course steady as we go?

If we are going to stay on the same track, how long should we go before getting worried?

When will we know when to say ‘when’ and do something different?”

Using Session by Session Measures in Supervision

• The measures and the graphs bring the feedback and voice of the young person and carer directly into the supervisory session.

• Supervisors and supervisees can utilize the measures and graphs from a variety of cases to reflect on potential areas for professional growth and development.

How to Get the Measures Free

• The ORS/CORS and SRS/CSRS measures are licensed for member groups of CORC and CYP IAPT by Scott D. Miller.

• Individual clinicians may also obtain a licence and the measures at no cost in English, and languages other than English, at www.scottdmiller.com.

• Note: When down loading the measures for the first time, you may need to adjust the analogue scales to make sure they are exactly 10cm. Beyond that adaptation the license agreement involves no alterations to the ORS/CORS and SRS/CSRS.

Further Help and Training in using the ORS/CORS and SRS/CSRS

• Information on Training in how to integrate real time outcome and alliance feedback using the ORS & SRS to improve clinical effectiveness with young people and families contact:

– David C. Low, Family and Systemic Psychotherapist, david.low1964@gmail.com, 01473-280416

– Scott D. Miller, Ph.D., Director of the International Center for Clinical Excellence -- info@scottdmiller.com

“So What, Now?”5 Minute Exercise

• Get back into your small groups of 3.

• Choose someone to feedback themes to the rest of us.

• Very briefly discuss what you each might try doing as a result of the material covered this morning.