consumer-operated service programs results
DESCRIPTION
Consumer-Operated Service Programs Results. Jean Campbell, Ph.D. Greg Teague, Ph.D. E. Sally Rogers, Sc.D. Asya Lyass, M.S. Ph.D. (Cand.) FROM INNOVATIONS TO PRACTICE: THE PROMISE AND CHALLENGE OF ACHIEVING RECOVERY FOR ALL Hyatt Regency Hotel in Cambridge, MA., on April 14, 2008. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/1.jpg)
Consumer-Operated Service Programs Results
Jean Campbell, Ph.D.Greg Teague, Ph.D.
E. Sally Rogers, Sc.D.Asya Lyass, M.S. Ph.D. (Cand.)
FROM INNOVATIONS TO PRACTICE: THE PROMISE AND CHALLENGE OF
ACHIEVING RECOVERY FOR ALLHyatt Regency Hotel in Cambridge, MA., on April 14, 2008
![Page 2: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/2.jpg)
2
Background
Over the past three decades peer-run support services have matured, diversified, increased in numbers across the United
States
![Page 3: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/3.jpg)
3
What is a COSP? A Consumer–Operated Services Program is
peer-run service program that is administratively controlled and operated by mental health consumers and emphasizes self-help as its operational approach.
Today, most COSPs operate as an adjunct to traditional mental health services within the continuum of community care.
![Page 4: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/4.jpg)
4
COSP Service Models Mutual Support Groups Drop-in Centers Education and Advocacy Programs Multi-Service Agencies Specialized Supportive Programs Warm Lines
![Page 5: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/5.jpg)
5
Number of COSP Services SAMHSA estimates there are 7,467 groups,
organizations, and services run by and for mental health consumers and/or families in the United States (2002) 44.4% are mental health mutual support groups 40.4% are mental health self-help organizations 15.2% are consumer-operated service programs
![Page 6: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/6.jpg)
6
COSP Effectiveness Studies using non-randomized control
groups or pretest scores as comparisons found that participation in peer support Reduces psychiatric symptoms Decreases hospitalization Enlarges social networks Enhances self-esteem and social functioning
![Page 7: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/7.jpg)
7
COSP Effectiveness Recent studies using randomized control
groups and pretest scores as comparisons have found that participation in peer support promotes wellness (a sense of well-being). Hope Empowerment Social connectedness Meaning in life Self-esteem/personhood
![Page 8: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/8.jpg)
8
COSP Study Overview Eight program sites
CA, CT, FL, IL, ME, MO, PA, TN Three general program models
Drop-In (4 sites) Mutual Support (2 sites) Education/Advocacy (2 sites)
![Page 9: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/9.jpg)
9
Participating Study Sites
![Page 10: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/10.jpg)
10
COSP Study Overview One-year longitudinal follow-up
4 measurement points: 0, 4, 8, 12 months Participants
Persons 18+ with diagnosable mental / behavioral / emotional disorder and functional impairment
N = 1827 enrolled in study; 1600+ in analysis Common interview protocol Logic Model Conventional RCT approach
Intent-to-treat analysis Optimized, common a priori hypothesis
![Page 11: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/11.jpg)
11
Primary Hypothesis
Participants offered both traditional and consumer-operated services would show greater improvement in well-being over time than participants offered only traditional mental health services.
![Page 12: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/12.jpg)
12
Well-Being Outcome Rationale
To develop a measure that was supported by theory and peer literature
Hypothesized to be most sensitive to primary peer support program effect
Short term outcome Realization of participants that “We are not alone.”
![Page 13: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/13.jpg)
13
Composite Well-being Measure: Constituent Scales Scales incorporated in Well-being measure (WB2)
Total Herth Hope Index Meaning of Life Framework Subscale Empowerment / Making Decisions (EMD) Self-
esteem/ Self-efficacy Subscale Recovery Assessment Scale (RAS) Goal Subscale Recovery Assessment Scale (RAS) Hope Subscale
Internal consistency Cronbach’s alpha = .92 at all four measurement
points
![Page 14: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/14.jpg)
14
Results: Well-being Over Time by Random Assignment Group
WB2 by Assignment Group - Overall
-.2
-.1
.0
.1
.2
.3
Base(N=1600)
4mo(N=1441)
8mo(N=1357)
12mo(N=1272)
TMHS, 797COS+, 803
![Page 15: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/15.jpg)
15
Intent-To-Treat Results for Well-being – All Sites Increase in well-being over time for
participants overall Possible differences in change in well-being
over time across sites Significantly greater increase in well-being for
persons offered use of COSP (p < .05) overall
Possibly important differences in this effect across site
![Page 16: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/16.jpg)
16
Well-being Effect Sizes By SiteWB2 Effect Sizes by Site
-.4
-.2
.0
.2
.4
.6
1 2 3 4 5 6 7 8
ES
![Page 17: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/17.jpg)
17
Well-being Effect Sizes By Site: Between Conditions & Within COSP+
WB2 Effect Sizes: Time, Condition by Site
-.4
-.2
.0
.2
.4
.6
1 2 3 4 5 6 7 8
ES-CES
![Page 18: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/18.jpg)
18
Well-being Effect Sizes By Site: Between and Within Conditions
WB2 Effect Sizes: Time, Condition by Site
-.4
-.2
.0
.2
.4
.6
1 2 3 4 5 6 7 8
ES-TES-CES
![Page 19: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/19.jpg)
19
Intent-To-Treat Results for Well-being – Seven Sites Increase in Well-being over time for participants
overall No significant site difference in change in Well-
being over time – very consistent positive ES for COSP across all 7 sites, along with positive trend for both groups
Significantly greater increase in Well-being for persons offered use of COSP (p < .01)
Negligible differences in this effect across site Site variations in experimental effect for this
outcome are driven by variations in TMHS programs
![Page 20: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/20.jpg)
20
As-Treated Analysis Engagement rates and adherence to assigned condition
low 57% of participants assigned to COS+ used COS 15% of participants assigned to TMHS only used COS
Consequently, ITT results may be too conservative
Scores calculated for propensity to use COS
Analysis limited to middle third of propensity Removed those who especially seek/avoid COS Effects can be attributed to intervention, not selection Smaller N means positive findings are conservative
![Page 21: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/21.jpg)
21
As-Treated Analysis: Two Measures of Engagement in COSP
Binary measure of simple engagement – visited at least once vs. didn’t use at all
Three-point measure of intensity of use No use Low use = less than study median (8.5 visits to
COSP over 12-month period) High use = more than study median
![Page 22: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/22.jpg)
22
Well-being Over Time By Engagement (Middle 1/3 Propensity)
WB2 over Time by Engagement Group
-.2
-.1
.0
.1
.2
.3
Base(N=516)
4mo(N=479)
8mo(N=449)
12mo(N=404)
TMHS, 331COS+, 185
![Page 23: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/23.jpg)
23
Well-being By Intensity of COSP Use (Middle 1/3 Propensity)
WB2 by COS Use (Middle Propensity Group)
-.2
-.1
.0
.1
.2
.3
Base(N=516)
4mo(N=479)
8mo(N=449)
12mo(N=404)
High, 77Low, 79None, 360
![Page 24: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/24.jpg)
24
COSP-Findings for Well-being Significant gains in well-being were found for
the group that was offered use of COSP (the formal experiment – intent-to-treat analysis)
Greater gains in well-being were found for the group of participants who actually used COSP services (as-treated analysis)
Greatest gains in well-being were found for the group of participants who used COSP the most, those in the upper half of frequency of use
Variations in effects across sites were unrelated to formal COSP type
![Page 25: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/25.jpg)
25
Sub-Study: How is empowerment effected by involvement in consumer run programs
Why study empowerment?Many programs espouse empowerment as both
an aspect of the program structure (“empowering practices”) and as an outcome to be achieved
However, few rigorous studies of the effects of consumer run programs on empowerment exist
Most studies not randomized, small, descriptiveComplicating matters: there is no consensus of
the definition of empowerment
![Page 26: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/26.jpg)
26
Sub-Study: Empowerment Making Decisions Empowerment (MDE), 28 item
scale to measure subjective feelings of empowerment: self-efficacy, perceived power, optimism about and control over the future, and community activism
Personal Empowerment (PE), 20 item tool with 2 subscales: Choice and Reduction in Chance
Organizational Empowerment (OME), 17 item scale about involvement in an community, organization or club
![Page 27: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/27.jpg)
27
Purpose
Determine whether COSP sites varied with respect to differences between E & C or change over time
Describe pattern in change over time
Examine whether “engagement intensity” (binary or no, low, high use) resulted in greater changes in empowerment
![Page 28: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/28.jpg)
28
Intent to Treat Results Results using strict (ITT) approach yielded marginal
results--perhaps due to modest engagement in the COSP
Results of ITT analyses with both measures of personal empowerment had overall results below threshold for significance and very small effect sizes
As with well-being analyses, one site obscured the more positive results from remaining sites.
Without this site, both measures showed a significant, small and positive effects
There was still significant cross-site variation remaining for PE Choice
![Page 29: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/29.jpg)
29
Analysis of differences in slopes over time
No difference between groups over time
0
2
4
6
8
10
0 1 2 3 4 5 6
timeou
tcom
e C
E
Difference between groups over time
0
5
10
15
20
25
30
0 1 2 3 4 5 6
time
outc
ome C
E
![Page 30: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/30.jpg)
30
Effect sizes for the difference in slopes between the experimental and the control group over time - MD
Site 1Site 2 Site 3 Site 4 Site 5 Site 6 Site 7
Site 8
All
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
Effe
ct S
ize
N=109
N=561
N=118
N=224
N=222N=143
N=133
N=93
N=1603
![Page 31: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/31.jpg)
31
Differences between groups over time-personal empowerment Sample items: “I can pretty much determine what
will happen in my life” “People working together can have an effect on their community”
Half of the sites showed a positive difference between E and C groups in personal empowerment (MD) over time with an effect size of .2 or more
One site had a significant negative difference in change over time between E and C
One site had slight negative difference; remaining had no difference
Conclusion: positive effect on personal empowerment from participation in COSP
![Page 32: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/32.jpg)
32
Effect sizes for the difference in slopes between the experimental and the control group over time - PE (Choice)
Site 1 Site 2 Site 3
Site 4
Site 5
Site 6
Site 7
Site 8
All
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Effe
ct S
ize
N=113
N=115
N=559 N=225
N=223
N=143
N=133
N=93
N=1604
![Page 33: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/33.jpg)
33
Differences between groups over time- PE-Choice
Sample item: “How much choice do you have about how you spend your free time”
Three sites had positive difference in change over time between E and C groups with effect sizes above .2
Five sites had a negative difference in change over time, or no change
Conclusion: Some COSP affect perception of choice
![Page 34: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/34.jpg)
34
Effect sizes for the difference in slopes between the experimental and the control group over time - PE (Reduction in chance)
Site 1
Site 2
Site 3
Site 4
Site 5
Site 6
Site 7
Site 8
All
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Effe
ct S
ize
N=113
N=555 N=117 N=222 N=219 N=143
N=135
N=90 N=1594
![Page 35: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/35.jpg)
35
Differences between groups over time-PE-Reduction in Chance
Sample item: “How likely is it that you will get enough to eat in the next month”
Two sites had positive differences in change over time between E and C
Six sites had no positive differences in change over time between E and C, or negative differences
Conclusion: Perception of life being left to chance or control over life not affected by COSP
![Page 36: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/36.jpg)
36
Organizational Empowerment There was a general downward trend in
organizational empowerment but significant variation across sites
There was less decline among COSP programs than traditional programs
Data raises questions about the fit of construct “organizational empowerment” with COSP programs
Conclusion: Relatively few participants (in either condition) experienced an increase in the number of organizational roles or activities tapped in this measure.
![Page 37: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/37.jpg)
37
As treated analyses Needed to go beyond ITT analyses because of
modest engagement in the COSP for experimental folks
Using a procedure described earlier and propensity scores, created two groups of balanced individuals
Individuals in both groups showed equal tendency to attend COSP
Yielded a better comparison of E and C participants
![Page 38: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/38.jpg)
38
Results
Results of the as-treated analyses confirmed the inferences that emerged from the ITT analyses:
Use of consumer-operated services was positively associated with increases in personal empowerment as measured by both the MD and the PE Choice scales, results that held without significant variation across the eight sites. Gains from using COSPs become apparent only with higher levels of use.
![Page 39: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/39.jpg)
39
Conclusions
Results support the conclusion that COSPs in general have a positive impact on aspects of empowerment (MD and PE Choice scales)
Some COSPs have the effect of improving empowerment, while others are less effective
Effects significant but small in magnitude and consistent with other studies
![Page 40: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/40.jpg)
40
Conclusions
Individuals with greater engagement in and attendance at COSPs fared better in their personal empowerment outcomes
Additional analyses suggested positive changes in self-efficacy items: “I believe I am a person of worth”; “I am usually confident about the decisions I make”.
COSP may have more of an effect on self-efficacy than other aspects of personal empowerment
![Page 41: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/41.jpg)
41
What is Fidelity?
Fidelity measures provide an objective rating system to assess the extent that components of a program are faithfully implemented according to intended program model, theory, or philosophy.
![Page 42: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/42.jpg)
42
Why Measure Fidelity?
The use of fidelity measures has become a widely accepted methodological tool in mental health services research and serves a number of important purposes.
![Page 43: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/43.jpg)
43
Why Measure Fidelity?
In addition to establishing that a set of well-defined services leads to predicted outcomes, it is critical for researchers to establish the integrity of the service delivery.
Providers need to adhere to critical elements of an evidence-based practice in order to achieve the positive outcomes identified in the original research.
![Page 44: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/44.jpg)
44
COSP Fidelity Measurement
Common measurement, diverse programs Analytic challenges
Some theoretically important program aspects were not being measured
Comparison conditions were highly variable Initial program measurement goal
Develop cross-site program-level implementation measure
Assess interventions within traditional pooled data framework
![Page 45: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/45.jpg)
45
Fidelity Assessment Common Ingredients Tool: FACIT
Process of Developing FACIT Identification of common ingredients Definition of common ingredients Feasible performance indicators (48 items) Performance anchors (typically 4-5) Involvement of COSP directors and staff as well as
researchers/evaluators at all stages of FACIT development process
Involvement of the CAP in the definition of common ingredients
![Page 46: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/46.jpg)
46
FACIT Operationalization
Data collection During site visits, questions devised to elicit
information about common ingredients from program directors and program staff.
COSP recipient focus group Program observation
While FACIT was developed to measure characteristics thought to be common to COSPs, it was also used to measure the extent to which TMHS were “consumer-friendly.”
![Page 47: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/47.jpg)
47
FACIT Operationalization
Independent rating After site visits, site visitors independently rated each
program on each dimension. Conciliation
Site visitors came to agreement on any dimensions on which there was disagreement.
![Page 48: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/48.jpg)
48
FACIT Operationalization
Pilot testing of FACIT (Round 1 site visits) Average interrater reliability of items .8; use of
FACIT feasible with both COSP and Traditional Mental Health Services
FACIT Psychometrics CIs present within COSPs Differences between COSPs and TMHS detected Differences among COSP models
![Page 49: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/49.jpg)
49
FACIT Operationalization COSP Fidelity Measurement (Round 2 site
visits) Analysis & further psychometrics
Factor analysis and internal consistency within major scales
Identification of subscales for use in fidelity-outcome analyses
![Page 50: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/50.jpg)
50
FACIT Scales and SubscalesUsed in COSP Study STRUCTURE
Consumer Ownership Responsiveness
ENVIRONMENT Inclusion Accessibility
BELIEF SYSTEMS Peer Ideology Choice & Respect Spirituality & Accountability
PEER SUPPORT Encouragement Self-Expression
EDUCATION ADVOCACY
![Page 51: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/51.jpg)
51
FACIT Scale Scores: COSP and Traditional Services
0 20 40 60 80 100
ADVOCACY
EDUCATION
PEER SUPPORT
BELIEFSYSTEMS
ENVIRONMENT
STRUCTURE
% of possible subscale total
COSPTMHS
![Page 52: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/52.jpg)
52
All overall FACIT scores for COSP were significantly higher than the score for any TMHS
Mean COSP scores were higher than TMHS on all main subscales
COSP scored higher than their respective TMHS on most subscales
FACIT Scores: COSP vs. Traditional Services
![Page 53: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/53.jpg)
53
Comparing Fidelity & Outcome (COS+ vs. TMHS – 8 sites)
Sample Standardized Outcome and Fidelity By Site
-2
-1
0
1
2
1 2 3 4 5 6 7 8
s.d.
Empowerment (MD) Mean of FACIT Process Scales
![Page 54: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/54.jpg)
54
Mean Correlations: FACIT Scales and Selected Outcomes (8 Sites)
Mean Correlations of FACIT Scales With Outcomes(WB2, MD, Social Inclus & QOL, Life Sat)
-.4 -.2 .0 .2 .4 .6 .8
ADVOCACY
EDUCATION
PEER SUPPORT
BELIEF SYSTEMS
ENVIRONMENT
STRUCTURE
Pearson r
![Page 55: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/55.jpg)
55
FACIT Process Scales & Selected Outcomes: Between & Within Programs
Correlation Coefficients: Mean FACIT Process Scores & Selected Outcomes - Experimental & Observational
.0
.1
.2
.3
.4
.5
.6
.7
.8
Emp (MD) Wellbeing Socializ Life Sat
Pear
son
r
COS+ vs T-only All 16 programs over time
![Page 56: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/56.jpg)
56
FACIT Process Scales and Specific Outcomes
FACIT Process Scales & Selected Outcomes (N=8)
.0
.2
.4
.6
.8
1.0
ENVIRONMENT BELIEF SYSTEMS PEER SUPPORT
Pear
son
r
Empowerment Wellbeing Socializ Life Satisf
![Page 57: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/57.jpg)
57
Key Ingredients Related To COSP Outcomes: Scales & Subscales Environment
Inclusion subscale: Low/no cost; protective program rules; positive social environment; sense of community; lack of coerciveness
Belief Systems Choice & Respect subscale: Choices about
participation; acceptance and respect for diversity Peer Support
Self-Expression subscale: Artistic expression; participants telling own stories
![Page 58: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/58.jpg)
58
Summary: Fidelity and Outcome Analysis COSP are effective in producing gains in
recovery-related domains over a 12-month period
Gains in these areas are strongly related to specific program (fidelity) ingredients, measured at the program level
Critical ingredients are stronger in COSP but are present in traditional programs as well
Critical ingredients related to other outcome domains, independent of program type (COSP TMHS) or experimental effect
![Page 59: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/59.jpg)
59
COSP-Substantive Findings Evidence base for COSP as discrete
programs Adding COSP to traditional services adds to
well-being Effect is both incremental and compensatory:
impact is strongest where TMHS programs are weakest
Evidence base for recovery theory Program features specified for and found in
COSP are related to increases in well-being independent of setting
![Page 60: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/60.jpg)
60
COSP-Policy Implications Results provide evidence of COSP
effectiveness Results reinforce commitment to recovery–
oriented services Particular consumer-supported program features
contribute to recovery and are effective across diverse settings
Program features central to COSP should also be fostered within TMHS
![Page 61: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/61.jpg)
61
COSP Next Steps
COSP Recognized by SAMHSA as an EBP COSP Evidence-Based Practices Kit
developed KIT materials organized around FACIT domains and
common ingredients FACIT protocol available
KIT in field review
![Page 62: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/62.jpg)
62
COSP Next Steps
“Raising All Boats” Initiative part of Missouri DMH Transformation Efforts FACIT implemented at 5 Drop-in Centers E-FACIT and Users’ Guide developed COSP ToolKIT tested in program training FACIT revised for piloting at 5 Warmlines FACIT results used to guide CQI efforts
![Page 63: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/63.jpg)
6363
Using the e-FACIT As a Self-Assessment CQI Tool Easy Administration (2 persons)
Interviews with scheduled questions from protocol Focus groups Observation
Easy Scoring 48 items Anchored scoring
Easy Data Entry Excel 2003 program
Easy Charting The E-FACIT Workbook developed to automatically
produce charts
![Page 64: Consumer-Operated Service Programs Results](https://reader035.vdocuments.mx/reader035/viewer/2022062411/56816686550346895dda32fe/html5/thumbnails/64.jpg)
6464
Using the e-FACIT As a Self-Assessment CQI Tool Conduct annual assessments
CQI Team Outside evaluator
Share results with staff & membership Target areas to improve Develop Quality Improvement Plan
Implement COSP Toolkit (targeted areas) Revise or enhance programmatic approaches Augment staff supervision & training