Aspects of the CORE system

Chris Evans (chris@psyctc.org) 5th Conference on Psychiatric Research in the North,

Stokmarknes, Norway, 3.ix.03

Outline of session

IntroductionAims driving the CORE systemInstruments in the CORE system

CORE-OMPsychometrics of the CORE-OMTranslations

CORE-AThe researcher/practitioner gap and …Methods for CORE and outcome measurement

AcknowledgementsCST

Self

Michael Barkham

Frank Margison

PTRCJanice Connell

Kerry Audin

Chris Marshall

Liz Benson …and others

CIMSJohn Mellor-Clark

Alex Curtis-Jenkins

Richard Evans

InstitutionalArtemis Foundation

Mental Health Foundation

Counselling in Primary Care Trust

SPR(UK) and SPR(UK) Northern Group

My NHS backing: Nottinghamshire Healthcare NHS Trust & Tavistock & Portman NHS Trust

Relationship between measures

CORE system design:the central CORE

CO R E-O MCO R E-A

Re fe re ntia l se lf-re p o rt

m e a sure s, e .g . BDI-I o r II

Re fe re ntial ra

te d

m ea sure s, e

.g . HRSD

Re fe re ntia l e xte rna l

m e a sure s, e .g . we ig ht

Lo c a l m e a sure s Q ua lita tive d e sc rip to rs

Aims driving the CORE system

To help bridge the research/practice gapBy providing instruments which should:

be pragmatic and user-friendlyhave acceptable psychometric propertiesbe used on a broad basis: “copyleft”be both easy to score by hand and also

computer-scannablebe supported by at least one co-ordinating centreprovide bases for extensions to other domains

CORE-OM(page 1)

CORE-OM(page 2)

CORE-OM: internal reliability

AlphaNon-clinical(n = 1,009)

Clinical(n = 713)

Subjective Well Being .77 .75

Problems .90 .88

Functioning .86 .87

Risk .79 .79

All non-risk items .94 .94

All items .94 .94

CORE-OM: test-retest stability

rho W S F R NR All

Well-being .88 .76 .82 .50 .85 .85

Symptoms .80 .87 .84 .54 .88 .88

Functioning .75 .68 .87 .44 .81 .80

Risk .39 .48 .48 .64 .50 .51

All non-risk items .85 .83 .91 .52 .91 .90

All items .85 .83 .91 .52 .90 .90

CORE-OM: factor structure

Component Number

33312927252321191715131197531

Eig

envalu

e

14

12

10

8

6

4

2

0

CORE-OM: validityclinical cf. non-clinical groups

8631084N =

Sample

ClinicalNon-clinical

All

item

s

4

3

2

1

0

CORE-OM: validitygender, age & ethnicity

Small gender differencesd is effect size = |difference in means|/s.d.

d: 1.7 for clinical/non-clinical differenced: .2 for gender in non-clinical sampled: .04 for gender in clinical sample

Very small age effectsIn students:

non-white vs. white & English 1st language vs. notinternal reliabilities equalno significant difference in % unusable

CORE-OM: validityconvergent & discriminant

W S F R NR All n

BDI I .77 .78 .78 .59 .84 .85 251

BDI II .79 .74 .78 .32 .83 .81 29

BAI .56 .68 .55 .39 .65 .65 218

BSI .63 .76 .71 .62 .79 .81 97

SCL .68 .87 .79 .83 .85 .88 34

GHQ-A .43 .60 .44 .30 .56 .55 69

GHQ-B .55 .61 .57 .30 .64 .64 69

GHQ .67 .66 .65 .56 .72 .75 69

GHQ-C .60 .52 .60 .44 .62 .63 69

IIP .48 .58 .65 .45 .64 .65 246

GHQ-D .63 .47 .55 .69 .58 .63 69

CORE-OM: validitysensitive to change

404040404040404040404040N =

A2A-R2-RR2RF2FS2SW2W

95

% C

I

3.0

2.0

1.0

0.0

Short and acceptableReliableValid:

huge clinical/non-clinical effectsmall effects of age, gender and ethnicitysensitive to change

CORE-OM: summary

TranslationsCriteria & partnership

Strict criteria to ensure good translations and useful evidence of psychometrics and referential data

Partnership with a local lead, Chris Evans is CST partner for all translations

Copyright to be held by CORE System TrustMeasures to be copyleft: no cost copying, no profit

Currently:Norwegian translation underway led by Professor

Vidje HansenGujarati translation essentially completeItalian, German, French Canadian and British Sign

Language under way

CORE-ANot a therapist rating scaleTwo parts, each two sides of A4:

Therapy Assessment Form (TAF)Demographics

Gender, age, referral & contact dates, living situation

Clinical historyPrevious treatments & current medication

Clinical stateProblem & risk ratings, ICD-10 diagnoses

Outcome of assessment

CORE-A:TAF p.1

CORE-A:TAF p.2

CORE-A continuedEnd of Therapy form (EoT)

Logistics of the therapySession numbers & frequency, dates, follow-up

Description of the therapyTheory, modality, mode of ending

Clinical state at endingProblem & risk ratings (as in TAF), any benefits

of therapy, change in medicationTherapy experience

Motivation, working alliance, psychological mindedness

CORE-A:EoT p.1

CORE-A:EoT p.2

The researcher/practitioner gapPatient

Therapist

Researcher

The new scenario

Patient

Therapist

Politician/manager

Researcher

Methods to bridge the gap

Measure individual change: RCSC Confidence intervals for all grouped data

Graphical presentation of all dataSpatial metaphor and levels:“drill”

&“zoom”

Methods 1: RCSCReliable change:

change more than expected 95% of time on basis of unreliability of measurement

Clinically significant change:Change to score more characteristic of non-

clinical than clinical population:A: 2 s.d. improvementB: within 2 s.d. of non-clinical meanC: beyond the crossing point of clinical & non-

clinical Gaussian distributions

RCSC: CSC methods

0 1 2 3 4

0.0

0.2

0.4

0.6

0.8

1.0

Non-clin. mean Clinical mean

CSC criterion C

CSC criterion A

CSC criterion B

RCSC: graphical presentation

Confidence intervals & graphics

Services

%

02

04

06

08

01

00

% with the second CORE-A (EOT) form

14

96

2

18

23

13

85

30

08 13

89

13

62

10

40

7 44

8 10

87

15

64

1 38

7 18

05

4

5

92

7 81

2

10

86

13

79

91

5 32

9

13

08

96

8

18

19

12

32

16

57

95

0 17

44

20

21

53

2 20

3

CIs & graphics: add reference

Services

%

02

04

06

08

01

00

% with the second CORE-A (EOT) form

14

96

2

18

23

13

85

30

08 13

89

13

62

10

40

7 44

8 10

87

15

64

1 38

7 18

05

4

5

92

7 81

2

10

86

13

79

91

5 32

9

13

08

96

8

18

19

12

32

16

57

95

0 17

44

20

21

53

2 20

3

CIs & graphics: add CIs

Services

%

02

04

06

08

01

00

% with the second CORE-A (EOT) form1

49

6

315

2

932

18

23

40

13

85

69

30

08

135

13

89

128

13

62

49

10

40

153

7

430

44

8

55

10

87

51

15

64

60

1

2863

87

691

80

5113

4

168

5

164

92

7

75

81

2

102

10

86

62

13

79

165

91

5

102

32

9

323

13

08

142

96

8

639

18

19

141

12

32

690

16

57

98

95

0

203

17

44

101

20

21

77

53

2

300

20

3

173

CIs & graphics: add summary

Services

%

02

04

06

08

01

00

% with the second CORE-A (EOT) form1

49

6

315

2

932

18

23

40

13

85

69

30

08

135

13

89

128

13

62

49

10

40

153

7

430

44

8

55

10

87

51

15

64

60

1

2863

87

691

80

5113

4

168

5

164

92

7

75

81

2

102

10

86

62

13

79

165

91

5

102

32

9

323

13

08

142

96

8

639

18

19

141

12

32

690

16

57

98

95

0

203

17

44

101

20

21

77

53

2

300

20

3

173

Overall proportion = 72.1%Maximum = 98.8%Minimum = 33%Ratio, max:min = 2.99Chi square = 1181.75 d.f. = 32 p = 0

Number "significantly" high 13Number "significantly" low 10Number "significantly" different 23

Cis & graphics: notched boxplots

Referral Assessment Pre-Therapy Discharge 6 Month Follow-Up

Occasion

0

10

20

30

40

50

60

Graphics: traffic lights

0

10

20

30

40

50

60

70

80

90

1001

% clients achieving clinical & reliable

change

% achieved by 9 similar services

0

10

20

30

40

50

60

70

80

90

1001

% clients reaching planned ending for therapy

% achieved by 9 similar services

0

1 0

2 0

3 0

4 0

5 0

6 0

7 0

8 0

9 0

1 0 01

% of clients accepted for therapy

% achieved by 9 similar services

Traffic lights on CI plots

Services

%

20

40

60

80

10

0

% with CORE-OM at assessment1

08

7

5

30

08

1

4 13

08

7

18

23

53

2 13

89

96

8

10

40

18

05

32

9 20

21

13

85

2

95

0

12

32

15

64

18

19

14

96

81

2

38

7

92

7

16

57

20

3

10

86

91

5

13

79

13

62

44

8

17

44

Quantile boundaries at: 39.2, 83, 92.5, 94.7, 100

Drill & zoom: multilevel awareness1. Items within scores2. Measures 3. Occasions (sessions) within episodes4. Episodes within sequence of episodes5. Episodes/occasions within individuals6. Therapists7. Services8. “Settings”9. Locality10. Country

Spatial metaphorDrill in To a different variable or measure

Drill down To a person in a group of data

Drill backard or forward

Keep “back/forward” chronological, don’t forget the referral processes

Zoom out Take in context from other clients, therapists or services

Find nearest neighbours

Give context to observations by finding other similar units

Helicopter up Take wider view in two dimensions, e.g. more clients and profile of change within clients over time

Stand back More context, e.g. referral policies, procedures, funding

Providing support: data input

“It’s really simple and easy to use. I’m not very computer literate, but I’d got to grips with it in less than an hour”

Providing support: RCSC & drilling

“The clinical and reliable change graph is invaluable. As a service manager it gives me instant access to where we can look to improve our service provision”

Providing support: reports

“I never realised that writing a report could be so simple, all I need to do is copy the tables I need from CORE-PC, paste them in Word, and write my interpretations.”

Websites

http://coreims.co.uk/

http://www.psyctc.org/stats/COREtools/