Implementing Evidence- Based Practice on a Large

Scale

Marion S. ForgatchZintars G. Beldavs

Oregon Social Learning CenterPresented at the Intervening Early: Progress and Opportunities in Child Service Settings meeting sponsored by the National Institute on Drug Abuse, National Institutes of Health (NIH), September 18-19, 2007.

Support for this project was provided by Grant No. R01 DA16097 from the Prevention Research Branch, NIDA, U.S. PHS, and support from the Behavior Center in Oslo, Norway.

Nationwide Implementation Study in Norway

• International Collaboration OSLC (Forgatch, Patterson, DeGarmo) Norwegian Center for the Study of Behavioral Problems and

Innovative Practices (Atferdsenterret) (Ogden) University of Michigan (Price)

• Nationwide Implementation of PMTO • Study factors contributing to

Implementation Adoption Adaptation Fidelity through successive generations

• Successive Generations of Norwegian Professionals Generation 1 (G1): certified by OSLC professionals (N=34) Generation 2 (G2): certified by G1 (N=79) Generation 3 (G3): certified by G1 & G2 (N=69) Generation 4 (G4): in process (N=74 )

System-wide Implementation in Norway (Ogden, Forgatch et al., 2005)

Establish National Implementation and Research Center

Develop Plans @ County & Municipal Levels

Conduct Therapist Recruitment, Training, & Maintenance Program

Conduct Clinical Outcome Research

Conduct Implementation Research

G1 Specialists

N = 34

(85%)

G2 SpecialistsN = 79(95%)

G3 SpecialistsN=69(95%)

Expected G4 Specialists

N=73

Implementation Feedback Process

Develop a Collaborative Partnership

Adapt for Contexts & Culture

Train Professionals

Evaluate Effectiveness

Evaluate FidelityEvaluate Fidelity

Make SustainableMake Sustainable

Fidelity of Implementation Rating System (FIMP)

Knutson, Forgatch, & Rains (2003)

Rating system: evaluates competent application of PMTO principles

Ratings based on direct observation of intervention sessions (DVD recordings)

Sessions scored: Encouragement & Discipline

Session segments rated: 10-35 minute samples

Principal Component Analysis: Alpha = .947; 1 component extracted; 82.85% of variance explained

9-Point Likert Scale:

Good work = 7-9; Acceptable = 4-6; Needs Work = 1-3

FIMP Content

5 categories:

Knowledge: Proficiency in understanding and application of core principles

Structure: Session management, pacing, leads without dominating

Teaching: Interactive approach to promote parents’ mastery and independent use of PMTO skills & tools

Process: Proficiency in use of clinical and strategic skills, provides safe context in which to learn

Overall (Integration): Growth during session, family satisfaction, likelihood to continue & use, sensitive to context  

PMTO Intervention Model

S

T

PMTO INTERVENTION MODEL

S

PARENTINGPRACTICES

Does Training Increase Fidelity?

G1

Mid Training

Training Cases

81 (2.4)

Early Training

Training Cases

88 (2.6)

Certification

New Cases

70 (2.3)

Estimated Marginal Means of AVERAGE

TIME

321

Est

imat

ed M

argi

nal M

eans

7.2

7.0

6.8

6.6

6.4

6.2

6.0

5.8

5.6

Early Mid Certification

Norway G1 FIMP Across Training

Norway G1 FIMP Across Training

Early Mid Certification

Fidelity Drift Across Generations

G1Fidelity

Covariates:Agency CharacteristicsTherapist CharacteristicsFamily Characteristics

G2Fidelity

Training

G3Fidelity

Training

Fidelity Across Generations Certification

G1 G2 G3

34

56

78

9

131 729 321N=

fimpmn1 fimpmn2 fimpmn3

Fidelity Across Generations All Available FIMPs

G1 G2 G3

Fidelity Model

FidelityChange

Parenting

ChangeChild

Behavior

BaselineMaternalParenting

.53

12 mo.ChangeMaternalParenting

e3

BaselineStepfatherParenting

.66

12 mo.Change

StepfatherParenting

e4

.70

*

chi-square =9.113, df = 12, P =.693, CFI = 1.000, cmindf =.759, rmsea =.000

tp < .10; *p < .05; **p < .01; ***p < .001

DisciplineSessions

EncouragementSessions

.84

**

ChildAge

-.46

.20

ChildGender

-.71**

.26

-.4

2t

InterventionFidelity

.51*

.49*.52***

.83***

R =2

R =2

Fidelity Effects on Change in Observed Effective Parenting: OSLC Stepfamily Sample

(Forgatch, Patterson, & DeGarmo, 2005)

BaselineMother

Parenting

.31

ChangeMother

Parentinge3

BaselineFather

Parenting

.39

ChangeFather

Parentinge4

chi-square =10.882, df = 8, P =.208, CFI = .990, cmindf =1.360, rmsea =.053tp < .10; *p < .05; **p < .01; ***p < .001

Fidelity Effects on Pre/Post Treatment Change in Effective Parenting: Norwegian Sample

.56

DisciplineSessions

e2

.53

EncouragementSessions

e1

InterventionFidelity

R =2

R =2

Generation

-.19*

-.17*

56

78

13 50 51N=

fimpmn1 fimpmn2 fimpmn3

FIMP scores by Generation for Norwegian Model

G1 G2 G3

ReferencesForgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005).

Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training (PMTO). Behavior Therapy, 36, 3-13.

Knutson, N. M., Forgatch, M. S., & Rains, L. A. (2003). Fidelity of Implementation Rating System (FIMP): The training manual for PMTO. Eugene: Oregon Social Learning Center.

Ogden, T., Forgatch, M. S., Askeland, E., Patterson, G. R., & Bullock, B. M. (2005). Implementation of parent management training at the national level: The case of Norway. Journal of Social Work Practice, 19(3), 317-329.