applying implementation science to improve care in

Applying Implementation Science to

Improve Care in Community Settings

Gregory A. Aarons, Ph.D.

University of California, San Diego

Department of Psychiatry

Child & Adolescent Services Research Center

Presented at: Be There San Diego: University of Best Practices 03 April, 2017

Implementation Research

Implementation research:– Scientific study of the use of strategies to adopt and

integrate evidence-based health interventions into clinical and community settings in order to improve patient outcomes and benefit population health.

Dissemination research:– Scientific study of targeted distribution of information

and intervention materials to a specific public health or clinical practice audience. The intent is to understand how best to spread and sustain knowledge and the associated evidence-based interventions.

R01: https://grants.nih.gov/grants/guide/pa-files/PAR-16-238.html



https://grants.nih.gov/grants/guide/pa-files/PAR-16-238.html



Implementation

Outcomes

Feasibility

Fidelity

Penetration

Acceptability

Sustainability

Uptake

Costs

*IOM Standards of Care

What?

QIs

ESTs

How?

Implementation

Strategies

Implementation Research Methods

Service

Outcomes*

Efficiency

Safety

Effectiveness

Equity

Patient-

centeredness

Timeliness

Patient

Outcomes

Clinical/health

status

Symptoms

Function

Satisfaction

CONTEXT

CONTEXT

CO

NT

EX

TC

ON

TE

XT

Conceptual Model of Implementation Research

Proctor et al. (2009)

Implementation

Outcomes

Feasibility

Fidelity

Penetration

Acceptability

Sustainability

Uptake

Costs

*IOM Standards of Care

What?

QIs

ESTs

How?

Implementation

Strategies

Implementation Research Methods

Service

Outcomes*

Efficiency

Safety

Effectiveness

Equity

Patient-

centeredness

Timeliness

Patient

Outcomes

Clinical/health

status

Symptoms

Function

Satisfaction

CONTEXT

CONTEXT

CO

NT

EX

TC

ON

TE

XT

The UsualThe Core of

Implementation

Science

Conceptual Model of Implementation Research

Proctor et al. (2009)

Review of Models (Tabak, et al., 2012)

Reviewed 61 models – Models (aka “theories” or “frameworks”)

– Frameworks evaluated on:

Construct flexibility– Broad highly operationalized

Focus on dissemination vs. implementation– D-only D=I I-only

Socioecologic framework level– Individual Community System

Source: Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and

practice: models for dissemination and implementation research. American journal of preventive medicine,

43(3), 337-350.

Most frameworks also are adapted or modified in practice

Source: Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice: models

for dissemination and implementation research. American journal of preventive medicine, 43(3), 337-350.

Common Elements of Frameworks

Multiple Levels

– Implementation occurs in complex systems

– Need to identify concerns at different levels

Process of Implementation– Implementation occurs over time

– There may be relatively discrete phases or stages

CFIR Matrix

Source: Damschroder et al., 2009

Consolidated Framework for Implementation

Research

Damschroder, L.J., et al., Fostering implementation of health services research findings into practice: a consolidated

framework for advancing implementation science. Implement Sci, 2009. 4: p. 50.

Damschroder, L. Consolidated Framework for Implementation Research (CFIR) Wiki. 2010 November 8, 2010 [cited 2011

October 15]; Available from: http://wiki.cfirwiki.net.

http://wiki.cfirwiki.net/

Consolidated Framework for Implementation Research (CFIR)

Intervention characteristics– Intervention source

– Evidence strength/quality

– Relative advantage

– Adaptability

– Trialability

– Complexity

– Design/packaging

– Cost

Inner setting– Structural Characteristics

– Implementation climateTension for change

Compatibility

Relative priority

Organizational incentives/rewards

Goals/feedback

Learning climate

– Readiness for implementationLeadership engagement

Available resources

Access to information and knowledge

Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of

health services research findings into practice: a consolidated framework for advancing implementation science. Implement Science

4(1), 50.

Outer setting- Patient needs and resources- Cosmopolitanism (org networks)- Implementation climate- Peer pressure- External Policies/incentives

Characteristics of the individuals involved– Knowledge/beliefs/ (attitudes?)

– Self-efficacy

– Individual stage of change

– Identification with organization

Process of implementation– Planning

– Engaging

– Executing

– Reflecting and evaluating

Exploration, Preparation, Implementation,

Sustainment (EPIS) Model

Key phases of the implementation process

Multilevel

Frames implementation factors across levels within each phase

Enumerates common and unique factors across levels and across phases

Source: Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in

public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4-23.

Service Environment

Leadership

Legislation

Policies

Resources

Service Contracts

Inter-organizational Environment

Relationship of service

system with CBOs

Relationships between

provider organizations

Collaboration

Competition

Co-opetition

Outer Context

EBP Characteristics

EBP Developers

Purveyors / Implementation Support

EBP

System

Fit

EBP Organization

Fit

Individual Adopter Characteristics

Attitudes to EBP

Commitment to EBP

Citizenship

Org. Commitment

Job Satisfaction

Turnover Intentions

Turnover

Intra-Organizational Characteristics

Leadership

Implementation Leadership

Policies

Structure

MIS/EHR

Data monitoring/feedback

Inner Context

EBP Provider

Fit

Patients/Consumers

Need

Advocacy

Interconnections

Interactions-Linkages-Relationships

Procurement-Contracting

EBP

Patient

/ Client

Fit

Aarons et al. (2014)

Sustainment Outcomes

QUALITATIVE

Level of Institutionalization

Continued use of EBP

Adaptations

QUANTITATIVE

Provider Reach/Penetration

Client Reach/Penetration

Org. Sustainment Climate

Fidelity

Determinant & Process Framework Example: EPIS

Implementation & Sustainment Model

Service Environment

Leadership

Legislation

Policies

Resources

Service Contracts

Inter-organizational Environment

Relationship of service

system with CBOs

Relationships between

provider organizations

Collaboration

Competition

Co-opetition

Outer Context

Patients/Consumers

Need

Advocacy


Outer Context


Individual Adopter Characteristics

Attitudes to EBP

Commitment to EBP

Citizenship

Org. Commitment

Job Satisfaction

Turnover Intentions

Turnover

Intra-Organizational Characteristics

Leadership


Policies

Structure

MIS/EHR

Data monitoring/feedback

Inner Context



Inner Context

EBP Characteristics

EBP Developers


EBP

System

Fit

EBP Organization

Fit

EBP Provider

Fit

EBP

Patient

/ Client

Fit



EBP Fit – Values-Innovation Fit

Outer Context

EBP Characteristics

EBP Developers


Inner Context

Interconnections

Interactions-Linkages-Relationships

Procurement-Contracting



Networks – Linkages - Collaborations

Why Consider Multiple Phases?

Characterizes process of implementation

Develops a way to think about what supports are needed during the implementation process

Helps in providing a “long-term view”

Helps in planning and conducting implementation

Aarons, G.A., Hurlburt, M. & Horwitz, S.M. (2011). Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Administration and Policy in Mental Health and Mental Health Services Research.38, 4-23.

EXPLORATION

OUTER CONTEXT

Sociopolitical Context

Legislation

Policies

Monitoring and review

Funding

Service grants

Research grants

Foundation grants

Continuity of funding

Client Advocacy

Consumer organizations

Interorganizational networks

Direct networking

Indirect networking

Professional organizations

Clearinghouses

Technical assistance centers

INNER CONTEXT

Organizational characteristics

Absorptive capacity

Knowledge/skills

Readiness for change

Receptive context

Culture

Climate

Leadership

Individual adopter characteristics

Values

Goals

Social Networks

Perceived need for change

PREPARATION

OUTER CONTEXT

Sociopolitical

Federal legislation

Local enactment

Definitions of “evidence”

Funding

Support tied to federal and

state policies

Client advocacy

National advocacy

Class action lawsuits


Organizational linkages

Leadership ties

Information transmission

Formal

Informal

INNER CONTEXT


Size

Role specialization

Knowledge/skills/expertise

Values

Leadership

Culture embedding

Championing adoption

IMPLEMENTATION

OUTER CONTEXT

Sociopolitical

Legislative priorities

Administrative costs

Funding

Training

Sustained fiscal support

Contracting arrangements

Community based organizations.


Professional associations

Cross-sector

Contractor associations

Information sharing

Cross discipline translation

Intervention developers

Engagement in implementation

Leadership

Cross level congruence

Effective leadership practices

INNER CONTEXT

Organizational Characteristics

Structure

Priorities/goals

Readiness for change

Receptive context

Culture/climate

Innovation-values fit

EBP structural fit

EBP ideological fit

Individual adopter characteristics

Demographics

Adaptability

Attitudes toward EBP

SUSTAINMENT

OUTER CONTEXT

Sociopolitical

Leadership

Policies

Federal initiatives

State initiatives

Local service system

Consent decrees

Funding

Fit with existing service funds

Cost absorptive capacity

Workforce stability impacts

Public-academic collaboration

Ongoing positive relationships

Valuing multiple perspectives

INNER CONTEXT


Leadership

Embedded EBP culture

Critical mass of EBP provision

Social network support

Fidelity monitoring/support

EBP Role clarity

Fidelity support system

Supportive coaching

Staffing

Staff selection criteria

Validated selection procedures

Aarons, G.A., Hurlburt, M. & Horwitz, S.M. (2011). Advancing a Conceptual Model of Evidence-Based Practice Implementation in

Public Service Sectors. Administration and Policy in Mental Health and Mental Health Services Research.38, 4-23.

EPIS Model Adapted for Specific Context

(Child/Adolescent Mental Health) Based on Systematic Review

Novins, D.K., Green, A.E., Legha, R.K., & Aarons, G.A. (2013). Dissemination and Implementation of Evidence-Based Practices for Child and Adolescent Mental Health: A Systematic Review. Journal of the American Academy of Child and Adolescent Psychiatry. 52(10), 1009-1025

Implementation Strategies

Address specific factors identified in implementation frameworks

Discrete implementation strategies – Clinical reminders, training only

Multifaceted implementation strategies– Training + reminders

– Training + fidelity monitoring + coaching

Blended implementation strategies (comprehensive)– Community Development Team strategy (CDT)

– Interagency Collaborative Team strategy (ICT)

– Dynamic Adaptation Process strategy (DAP)

– Leadership and Organizational Change for

Implementation (LOCI)

Source: Powell , McMillen, Proctor et al (2011). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2) 123-157.

Domains of StrategiesType of Strategy Description Context Level N

Planning Info gathering, leadership, relationships Outer/Inner n=17

Education Training, materials, influence

stakeholders

Inner/Outer n=16

Financing Incentives, financial support Inner/Outer n=9

Restructuring Change roles, create teams, alter record

systems, create relationships

Inner/Outer n=7

Quality

Management

MIS + feedback, clinical reminders,

decision support, PDSA cycles

Inner/Outer n=16

Policy Change Licensure, accreditation, certification,

mandates

Outer/Inner n=3

Source: Powell , McMillen, Proctor et al (2011). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2) 123-157.

Aligning leadership

22

Leadership for implementation

– Full range leadership

– Implementation Leadership

Climate embedding mechanisms

– Primary

– Secondary

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255-274.

Change Oriented

Leadership

Transformational Leadership

23

Change Oriented

Leadership

Transactional Leadership

24

Non-Leadership

Passive-Avoidant Leadership

25

Full Range Leadership Model

Act i

ve

Pass

I

ve

Less Effective

More Effective

Passive-Avoidant

Transformational

Transactional

26

Implementation Focused


27

Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The Implementation Leadership Scale (ILS): Development of a Brief Measure of Unit Level Implementation Leadership. Implementation Science, 9(1) 45.


The Implementation Leadership Scale– Identifies specific behaviors that leaders may enact to support and engage

followers in the larger goal of successful EBP implementation

– Focused on leader behaviors related to climate embedding mechanisms to promote strategic climates

– 1. Proactive Leadership“Has a plan to address implementation of EBP”

– 2. Knowledgeable Leadership“Is able to answer staff questions about EBP”

– 3. Supportive Leadership“Recognizes and appreciates employee efforts toward successful implementation of EBP

– 4. Perseverant Leadership“Carries on through the challenges of implementing EBP”

– Implementation Leadership Total Scale Total α = .97, 12 total items

Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The Implementation Leadership Scale (ILS): Development of a Brief Measure of Unit Level Implementation Leadership. Implementation Science, 9(1) 45.

28

Mean ILS Discrepancy

3.01

2.82

1.751.9

2.82.87

0

1

2

3

4

33% 35% 38%

Me

an

Im

ple

me

nta

tio

n L

ea

de

rsh

ip S

ca

le S

co

re

Percent of Sample in Each Category

Supervisor Clinican

Leader N=80; Clinician N=513; Standardized scores that differ > .5 SD considered discrepant values

Humble Leader

Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017). The Humble Leader: Association of Discrepancies in Leader and Follower Ratings of Implementation Leadership With Organizational Climate in Mental Health. Psychiatric Services, 68(2) 115-122.

30

ILS Discrepancy and Organizational Climate - Involvement

The curve along the line of incongruence (dotted line) is significant. Discrepancy between staff and supervisor ratings of ILS does influence Involvement. As scores become more discrepant, Involvement increases (a4 = 0.42, t = 3.468, p = 0.001).

Involvement climate is impacted by discrepancy differently depending on who is rating ILS more favorably (direction of discrepancy matters). Involvement is highest at the left corner, where staff rate ILS high and supervisors rate themselves low (a3 = -0.30, t = -3.154, p = 0.002)

Humble leadership predicts better climate for involvement-

Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017). The Humble Leader: Association of Discrepancies in Leader and Follower Ratings of Implementation Leadership With Organizational Climate in Mental Health. Psychiatric Services, 68(2) 115-122. 31

ILS Discrepancy and Organizational Climate - Performance

Feedback

The curve along the line of incongruence (dotted line) is significant, meaning discrepancy between staff and supervisor ratings of ILS does influence Performance Feedback. As scores become more discrepant, Performance Feedback increases (a4 = 0.47, t = 6.289, p = 0.000)

Performance Feedback is impacted by discrepancy differently depending on who is rating ILS more favorably (direction of discrepancy matters). Performance Feedback is highest when staff rate ILS high and supervisors rate themselves low (a3 = -0.37, t = -5.147, p = 0.000)

Humble leadership predicts better climate for performance feedback-

Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017). The Humble Leader: Association of Discrepancies in Leader and Follower Ratings of Implementation Leadership With Organizational Climate in Mental Health. Psychiatric Services, 68(2) 115-122. 32

O2 SUSTAINMENT PROJECT(NIMH R01MH072961)

Mixed-methods examination of EBP sustainment factors in two states representing 87 counties:

– OK Children’s Services System

– 10 California Counties

ICT Project (NIMH)

SKCP/ADAPTS project– (ACF, CDC, HRSA) 33

Outer Context System Leadership Predicts EBI

Sustainment in 11 Service Systems

3.15 3.2

1.37

0

0.5

1

1.5

2

2.5

3

3.5

4

Leadership Competence

Outer Context

Full Sustainment Partial Sustainment Non-Sustainment

*

Aarons, G. A., Green, A. E., Trott, E., Willging, C. E., Torres, E. M., Ehrhart, M. G., & Roesch, S. C. (2016). The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 991-1008.

34

Inner Context Team Leadership Predicts EBI

Sustainment in 11 Service Systems

2.73

2

0.57

2.7

2.28

0.58

2.242.11 2.18

0

0.5

1

1.5

2

2.5

3

3.5

4

Transformational Leadership Transactional Leadership Passive-Avoidant Leadership

Inner Context

Full Sustainment Partial Sustainment Non-Sustainment

**

Aarons, G. A., Green, A. E., Trott, E., Willging, C. E., Torres, E. M., Ehrhart, M. G., & Roesch, S. C. (2016). The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 991-1008.

35

Leadership and Organizational Change for

Implementation (LOCI)

Pilot Study National Institute of Mental Health

NIMH R21MH082731

– Developed and tested in mental health programs

– Small scale (12 programs – 6 LOCI vs. 6 Webinar)

National Institute on Drug Abuse

NIDA R01DA038466

– Large scale

– 60 programs (30 LOCI – 30 Webinar)

36

LOCI Pilot Study OutcomesLOCI (n=5) Compared to Control Condition (n=6)

Feasibility– Engagement in leadership training (t=2.39, p<.05)

– Increased leadership knowledge (t=6.43, p<.001)

Acceptability– Applied what was learned (t=4.80, p<.001)

– Improved leadership behaviors (t=4.88, p<.001)

– Ability to manage change (t=4.54, p<.01)

– Change in behavioral routines (t=4.94, p<.01)

– Changed leadership behaviors (t=4.97, p<.01)

– Increased emphasis on EBP with subordinates (t=4.11, p<.01)

Perceived Utility– Greater overall utility (t=7.20,p<.001)

– Higher utility in managing organizational change (t=5.72, p<.001)

– Greater utility implementing/using EBPs in team (t=4.84, p<.001)

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015). Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 11. 37

0

1

2

3

4

Baseline 3 Month 6 Month

LOCI (n=41)

Webinar (n=59)

Subord

inate

Rate

d L

eader

Support

for

Score

Assessment Time

LOCI Pilot Study

Improved Supportive Leadership

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015). Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 11.

38

Leadership and Organizational Change for Implementation

R01DA038466

Developing leaders for EBP implementation and sustainment– http://implementationleadership.com/index.html

First level alone not sufficient– Strategically involve other levels in a way that minimizes

organizational stress and increases implementation climate development

Intervention needs consideration of timing of:

– Leadership development activitiesDidactic TrainingOngoing very brief coachingCollaboration

– Implementation climate development activities

– Linking organizational and multilevel leader buy in strategy with leader development

Aarons, G. A., Ehrhart, M. G., Moullin, J. C., Torres, E. M., & Green, A. E. (2017). Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol. Implementation Science, 12(1), 29. 39

http://implementationleadership.com/index.html

Overview of LOCI

Developing leaders for EBP implementation takes time and ongoing support– Two conditions

LOCI leader training + coaching + org strategy

Webinar leader training

Intervention needs consideration of timing of:– Leadership development activities

– Implementation climate development activities

– Linking organizational and multilevel leader buy in

strategy with leader development

Strategic alignment

across all levels

On Target =

Implementation Success

ExecutivesCounselors Supervisors

40

Organizational Context for Implementation

Enhance general leadership and “implementation

leadership”

Align leadership to develop strategic climate for

implementation

Support congruence of leadership across

organizational levels

Schein’s (2010) culture/climate “embedding

mechanisms”Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The Implementation Leadership Scale (ILS): Development of a Brief Measure

of Unit Level Implementation Leadership. Implementation Science, 9:45.

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning Leadership Across Systems and Organizations to

Develop a Strategic Climate for Evidence-Based Practice Implementation. Annual review of public health, 35, 255-274.

Ehrhart, M.G., Aarons, G.A., & Farahnak, L.R. (2014). Assessing the Organizational Context for EBP Implementation: The

Development and Validity Testing of the Implementation Climate Scale (ICS). 9:157

Jacobs, S. R., Weiner, B. J., & Bunger, A. C. (2014). Context matters: measuring implementation climate among individuals and

groups. Implementation Science, 9(1), 46.

Schein, E. H. (2010). Organizational culture and leadership (4th Ed). John Wiley & Sons. 41

Primary Climate Embedding Mechanisms

What leaders can do that can be applied at the system,

organization, team or workgroup level

– What leaders pay attention to, measure, control

– How leaders react to critical incidents, crises

– How leaders allocate resources

– Role modeling, teaching, coaching

– How leaders allocate rewards and status

– How leaders recruit, select, promote, excommunicate

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning Leadership Across Systems and Organizations to Develop a Strategic Climate for Evidence-Based Practice Implementation. Annual review of public health, 35, 255-274.

Schein, E. H. (2010). Organizational culture and leadership (4th Ed). John Wiley & Sons. 42

LOCI Leadership Program

LOCI Condition

(12-months)

Didactic Leadership

Training

Leadership Coaching

Data Driven 360-degree

Assessments

Organizational Strategy

4 Org. Strategy Meetings (OSMs)

Monthly OSM Check-in calls

Agency wrap-up to conclude project @

16-months

Strategies in place for sustained EBP

2-day in-person LOCI leadership training

Two 1-day in-person leadership booster trainings

Graduation

Weekly one-on-one coaching calls with LOCI trainer

Monthly group collaborative call with other LOCI leaders

5 web-based surveys assessing climate & leadership

Completed by providers, supervisors, & executives)

Occurs at baseline, 4-, 8-, 12-, & 16-months

Aarons, G. A., Ehrhart, M. G., Moullin, J. C., Torres, E. M., & Green, A. E. (2017). Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol. Implementation Science, 12(1), 29.

43

Where to from Here?

Develop implementation strategies tailored to specific

contexts

Consider at what levels (system, organization, provider,

client) strategies should be aimed and coordinated

Consider phases (e.g., readiness) to tailor D or I

strategies

Requires empathy and understanding of challenges and

motivations of systems, organizations, providers,

patients

Identify metrics to determine success (or not) of

dissemination/implementation efforts

Contact

Gregory Aarons, Ph.D.

Department of Psychiatry

University of California, San Diego

9500 Gilman Drive (0812)

La Jolla, CA 92093-0812

e-mail:

[email protected]

Web:

http://psychiatry.ucsd.edu/faculty/gaarons.html

http://ImplementationLeadership.com

mailto:[email protected]

http://psychiatry.ucsd.edu/faculty/gaarons.html

http://implementationleadership.com/

applying implementation science to improve care in

Documents