outcomes and methodological considerations across...
TRANSCRIPT
Outcomesandmethodologicalconsiderationsacrossthreecommunity-basedeffectivenesstrialsofASD-focused
interventions
CRIEI2018
SarahRieth,KelseyDickson,LaurenBrookman-Frazee,&Aubyn StahmerCRIEI2018
InterventionwithToddlerswithASD?
Whatworks?(circa2006)
Whatfitsfamiliesneeds?
Whatcan/will
communityprovide?
Training $$$$
CommunityProviders
ResearchTeam
FundingAgency
OccupationalTherapy
InfantMentalHealth
Speech&Language
Medical
Families
Education Behavior
Autism
CBPR
Public(PartC)
PrivateInsurance
ASDexperience
Multi-Cultural
Intervention with Toddlers with ASD?
CommunityProviders
ResearchTeam
FundingAgency Families
Bond * Regulate * Interact * Develop * Guide * Engage
InitialInterventionSelectionProcess
EMT P.L.A.Y. ImPACT
• Parent focused, coaching• Blend of developmental and behavioral strategies (NDBI)
• Local training potential• Range of disciplines• Easy to use materials• Flexibility for agencies and families• Evidence-based
Intervention Selection
Responding to Community Needs
+ =
Responding to Community Needs
EnhancementstoProviderTraining:• Earlysocial-emotionaldevelopment(context)
• Opportunitiesforhands-on-practicethroughout
• Parentengagementfocus(supportcoaching)
• Reflectivepractice• MAlevelpractitioners
AdaptationstoIntervention:• Toddler-ize examples• Emphasisondailyactivities• Rethinking“homework”• Remove“autism”• Smootherintegrationbetweendevelopmentalandbehavioral
Two Recent Studies
TrainTheTrainer–
CommunityCapacity
PilotStudy–Child
Outcomes
Trainers:ModelDeveloper+BRIDGEMentors
Trainers:BRIDGEMentors
PriorStudy– 4agencies,8providers
Train the Trainer Design
CurrentStudy– 14agencies,14leaders,43providers
• Multiple baseline across groups– 3 groups, 4-5 agencies per group
• Part 1: Agency Training – by BRIDGE Mentors– 3 months training, 3 months on-going support– Group A: Agencies with prior involvement– Group B: Community agencies – San Diego– Group C: Community agencies – Sacramento (shortest BL)
• Part 2: Provider Training – by Agency Trainers– Materials provided by research team– Training conducted independently– 3 months training, 3 months on-going support … sort of
Train the Trainer Design
• Measures:– Demographics– Intervention fidelity (10 min sample)– Coaching strategies fidelity– Implementation Survey– Organizational Readiness to Change Survey– Self-rated training fidelity (for Agency Trainers)
Train the Trainer Design
9 10
Training - 3 months
Initial ½ Day OverviewEarly Development
(4 hours)
Coaching – Modeling, practice, feedback
Building Engaging Interactions & Environments
Coaching – Focus on new strategies
Coaching
12 Weeks
Creating Opportunities
Prompting & Rewards
Coaching Coaching
Generalization to Daily Activities
Engaging Parents
Week 1 2 3 4 5 6 7 8 11
Coaching
12
Ongoing Support – 3 months
Initial Intro to RP
RP Group
RP Group
Session1 2 3 Monthly-ish Meetings
Reflective Practice
Twice per month feedback (live or based on videos, group or individual)
Ongoing Feedback
Practice with your own clients
Participants
MexicanAmericanFilipino/aAmericanCaucasianMixed
AsianAmerican
Caucasian
Mixed
NotStated
MexicanAmerican
AgencyTrainers
Race/Ethnicity
Therapists
Discipline
21%Hispanic/Latino
35%Hispanic/Latino
Experience
Area AgencyTrainers(n=14)
Therapists(n=43)
SPEDCredential 29% 35%
EIExperience 15.7years (11.35) 10.1years(9.4)
ASDExperience 15.1years(10.22) 8.88years(8.5)
ParentEdExperience 14years (12.56) 7.63years(4.1)
Range1-35yearsforallcategories
Fidelity Data
0
1
2
3
4
5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
0
1
2
3
4
5
0
1
2
3
4
5
AverageNum
bero
fAreas‘Passed’
GroupC
GroupA
GroupB
Fidelity Data
0
0.5
0
0.5
0
0.5
Percen
tageofO
bservatio
nsatF
idelity
Baseline Training
GroupC
GroupA
GroupB
Two Recent Studies
TrainTheTrainer–
CommunityCapacity
PilotStudy–Child&Family
Outcomes
• Study Design– Community trial of Project ImPACT for Toddlers, N=31– Assignment to agency by Regional Center
• Trained therapists• Usual care therapists
– Pre and post assessment by research team
Pilot Study Design
• Child Measures– Characterization:
• ADOS-T• Mullen Scales of Early Leaning
– Pre – Post:• CSBS IT Checklist• Vineland (Parent)• MB-CDI (Parent)• Social Communication Checklist (Therapist)
Pilot Study Design
UsualCare(n=16) ImPACT (n=15)
Age 22.94mo(SD=6.21) 21.47mo(SD=4.73)
Gender 7males(43.8%) 13males(86.7%)
Ethnicity 7Hispanic/Latino (43.8%) 7Hispanic/Latino (46.7%)
Languages
Spanish 5(31.4%) 6(40%)
ADOS-T TotalScore* 9.94 15.80
Pilot Study Participants
CSBS– ByGroup
0
2
4
6
8
10
12
14
16
Intake EndofTreatment Follow-up
SymbolicComposite
UsualCare ImPACT
*
CSBS– ByGroup
0
5
10
15
20
25
Intake EndofTreatment Follow-up
SocialComposite
UsualCare ImPACT
+
ParentCoachingApproach
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
1 2 3 4 5 6
Percen
tageofS
essio
nswith
Paren
tPresent
TypeofParentInvolvementinSession
UsualCare
PIforT
0123456
Score
Code
TrainedTherapists
UsualCareTherapists
ParentEngagementStrateiges
Sustainment
0
10
20
30
40
50
60
70
2016 2017
012345678910
2016 2017
Agencies Children
Thank [email protected]
ResearchsupportedbyInstituteofEducationSciences,AutismSpeaks,andtheNationalInstituteofMentalHealth
ASAP Intervention: Outcomes and Methodological Challenges
Brian Boyd1 and Linda Watson2, Michael Alessandri3, Grace Baranek4, Betsy Crais2, Amy Donaldson4, LeAnne Johnson6, Anibal Gutierrez3, Stephanie Reszka2
1University of Kansas, 2UNC-Chapel Hill, 3University of Miami, 4University of Southern California, 5Portland State University, 6University of Minnesota
Overview of ASAP Intervention• Advancing Social-communication And Play (ASAP) is a
manualized, classroom-based intervention for preschoolers with ASD
• ASAP was designed as a supplemental intervention to improve children’s social-communication and play skills
• Developed based on the work on Kasari et al. (2006) & through a prior IES Goal 2 grant (PI: L. Watson)
Kasari, C., Freeman, S., & Paparella, T. (2006). Joint attention and symbolic play in young children with autism: A randomized controlled intervention study. JCPP, 47, 611-620.
ASAP Intervention Components
ASAP Intervention
Team-based
Coaching
Group instruction
Focus on targeted goals
1:1 instruction
Assessment & Monitoring
ASAP Intervention ContentSocial-communication Goals
• Social interaction• Requesting• Joint attention• 20 sequenced objectives
Play Goals
• Exploratory• Relational• Functional• Symbolic• 21 sequenced objectives
Goal MasteryASAP defines mastery as:
“the child showing at least 3 spontaneous occurrences of the targeted skill”
Criteria:• All 3 spontaneous occurrences on same day • Occurs across different situations and varied activities
• During and/or outside of planned, instructional activities
CURRENT ASAP STUDY
Study Design• Multisite, cluster randomized trial with randomization at classroom level• Equal allocation to ASAP or “business-as-usual” control group• Recruited 78 (of 80) classrooms, averaging 2 preschoolers with ASD per
classroom• Classroom teams participated for 1 school year
CONSORT DiagramClassrooms
randomized (n = 78)
Children enrolled (n=161)Intervention classrooms
(n = 40)
Classrooms remained in the study (n = 39)
Children in intervention classrooms (n=85)
Children completing the intervention (n = 82)
Control classrooms (n = 38)
Classrooms remained in the study (n = 36)
Children in control classrooms (n=76)
Children remaining in the control group (n = 73)
ASAP OUTCOMES
Proximal Child OutcomesSocial-Communication (S-C)
• Developed a direct observational coding system
• S-C is coded based on examiner-child interactions during ADOS administration (3x/year)
• Generates scores for social interaction, requesting, joint attention & total S-C
Play
• Adapted a direct observational coding system
• Play is coded based on examiner-child interactions during SPA administration (3x/year)
• Generates scores for exploratory, relational, functional, symbolic & total play
Ungerer, J.A. & Sigman, M. (1981). Symbolic play and language comprehension in autistic children. AACP, 20, 318-337.
Proximal Outcomes• No significant group x time interactions
• No evidence of change over time for either group
Secondary Child Outcomes
• Coding scheme based on Adamson, Bakeman & colleagues
• Data collected at beginning and end of school year in child’s classroom
• 3 five min. observations collected per child (total of 15 min. at each time point)
Adamson, L., Bakeman, R., Russell, C., Deckner, D., (2000). Coding symbol-infused engagement states technical report 9. Atlanta, GA: Developmental Laboratory Dept of Psychology, GSU
Engagement
UnengagementSignificant group x time interactions in favor of ASAP group
d = -.56
Some EngagementSignificant group x time interactions in favor of ASAP group
d = -.50
Methodological Considerations
• Engagement data were collected in children’s classrooms during their normal routines by blind observers
• Social-communication and play data were collected outside of child’s classrooms and semi-structured assessments were administered by naïve examiners
Context-bound
Highly Generalized
ASAP FIDELITY
Teacher Interview
0.00
2.00
4.00
6.00
8.00
10.00
12.00
T1 T2 T3
Adherence
Control Intervention
Methodological Considerations• Measuring a decision-making process
• Sampling across multiple providers for team-implemented interventions
• Determining level of fidelity needed to achieve effects on proximal outcomes
Summary• Found significant differences for secondary but not primary outcomes
• Raises questions about generalization of findings beyond classroom context
• Captured self-reported teacher adherence and differentiated ASAP from BAU classrooms• Measuring how well teachers and teams make data-informed decisions is still a work in
progress• 39 intervention classrooms reflect at least 117 ASAP team members who implemented the
intervention
AcknowledgementsCo-PIs and Investigators
• Linda Watson • Michael Alessandri• Grace Baranek• Betsy Crais• Amy Donaldson• LeAnne Johnson• Anibal Gutierrez
Coaches & Project Coordinators
• Project Coordinators• Stephanie Reszka, Meg Parlade,
LeAnne Hidde, Hannah Sanford
• Coaches• Jen Neitzel, Jessica Dykstra-
Steinbrenner, Kaitlyn Wilson, Emily Monn, Donna Barrow, Cecilia Alvarez-Tabio
AcknowledgmentsProject Staff
• UNC-CH: Jessica Amsbary, Cory Clark, Jenni Reiff, Chetna Sethi, Katie Belardi, Caroline McCarty, Sarah Griffin, Jessica Pulliam, Sallie Nowell, Amanda Plummer, Maggie Fitch, Diana Zellner, Kristin Hodgson, Doug DeMeyer
• UM: Christine Ghilain, Hoa Lam Schneider, Jessica Weber
• UMN: Kelsey Young, Julianne Bazyk, Hannah Vukelich
• PSU: Heather Demosthenes, Haylee Jenkins
Schools & Families
• Thanks to all of the school personnel and families of children with ASD who participated
Funding Source
The research reported here was supported by the Institute of Education Sciences, U.S.DOE, through grant R324A110256.
ExaminingtheEfficacyofClassroomPivotalResponseTeaching
JessicaSuhrheinrich,SarahR.Rieth,&Aubyn C.Stahmer
U.S.DepartmentofEducationGrants:R324A130349,R324B07002
HowcanwemakePRTcomponentsworkforteachers?
Takingturnsandmanagingmaterialswithagroupof
children
Usingpreferredmaterialswithagroup
ofchildren
MakingPRTworkwithothermandatedcurricula/IEPgoals
Providingdirectreinforcementduring
academictasks
Trainingparaprofessionals
Collectingdata
SystematicAdaptationofPRTfortheClassroom
FocusGroups
DevelopManualizedProgram
TestPossibleChanges
Adaptations• Usingcomponentsingroups• Facilitationofturntaking• Usingpreferredmaterialsinacademictasks• UsingmultipleexemplarsResources• Lessonplanning• ExamplesacrossIEPgoaldomains
• Communication;Social;Academic;Behavior
• Datacollection• Paraprofessionaltraining
SystematicAdaptationofPRTfortheClassroom
AdvisoryBoardFeedbackateachStage
FocusGroups
PilotTestAdaptedModel
TestPossibleChanges
(Stahmer,Suhrheinrich,&Rieth,2016)
RandomizedControlledEfficacyTrial
35SchoolDistricts
21EligibleDistricts
19 DistrictsAgreedto
Participate(90%)
17Districtswithparticipatingteachers
2Districtswithnoenrolledteachers
2DistrictsNoResponse
14Districts<15ASD
17schooldistricts
109teachers
76paraprofessionals
257students
94%Female9.2(r=1-34)yearsofexperience
83%Male5.8(r=3-11)yearsold• Autism(92%)• AutismSpectrum(7%)• Non-Spectrum(1%)
SchoolYear
2012/2013
2013/2014
2014/2015
2015/2016
RandomizedWaitlistControlDesign
SchoolYearClassof2013(35teachers)
2012/2013 TrainingYear
2013/2014 Follow-upYear
2014/2015 -----
2015/2016 -----
RandomizedWaitlistControlDesign
SchoolYearClassof2013(35teachers)
Classof2014(30teachers)
2012/2013 TrainingYear ObservationYear
2013/2014 Follow-upYear Training Year
2014/2015 ----- Follow-upYear
2015/2016 ----- -----
RandomizedWaitlistControlDesign
SchoolYearClassof2013(35teachers)
Classof2014(30teachers)
Classof2015(29teachers)
2012/2013 TrainingYear ObservationYear -----
2013/2014 Follow-upYear Training Year ObservationYear
2014/2015 ----- Follow-upYear Training Year
2015/2016 ----- ----- Follow-upYear
RandomizedWaitlistControlDesign
AutismOnly
Cross-Cat
Inclusion
Mild/Mod
Mod/Severe
Resource
ClassroomType
TrainingProtocolTrainingOthers
IntegratingCPRTintoyourclassroom
GroupInstructionwithCPRT
UsingCPRTwithIndividualStudents
CPRTPrinciplesandComponents
AppliedBehaviorAnalysis
• 12hrs grouptraining• Withindistrict• Integratedactivities
• Targetstudents• Focalactivities
CoachingProtocolProcedures:1)Observe;AssessFidelityofImplementation2)Provideverbalandwrittenfeedback,discuss,answerquestions
• Teachers 7.6(1.2)• Paraprofessionals 5.5(1.1)
AverageNumberofSessions
TeacherOutcomes
• HigherfidelityofCPRTforteacherswhocompletedtraining(B=0.27,p=.001).
• TeachersreportusingCPRTfor50min/day,4days/week
TeacherFidelity
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%TrainingYear
Percen
tofp
articipantsre
aching
masterycriteria
TeacherSatisfaction
• 100%ofteachersweresatisfiedorverysatisfiedwiththequalityoftrainingreceived.
• 89%ofteachersreportedCPRTassuccessfulorverysuccessfulforstudentswithASD.
TeacherFidelityatFollow-up
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
TrainingYear
Follow-upYear
Percen
tofp
articipantsre
aching
masterycriteria
TeacherReportedSustainment
• Sustainmentsurveycompleted18moposttraining
Survey Questions Agree orStronglyAgree
Overall, IcontinuetouseCPRT 93%
IammotivatedtouseCPRT. 86%
IusethefullCPRTprotocol withnewstudentswhodidn’tparticipateintheoriginaltraining.
53%
IusepartsoftheCPRTprotocolorcertainCPRTstrategieswithnewstudentswhodidn’tparticipateintheoriginaltraining.
90%
StudentOutcomes
• GoalAttainmentScaling• Socialgoals(t(64)=3.60,p<.001)
• ThePervasiveDevelopmentalDisordersBehaviorInventory(PDDBI)• Sensory/perceptualapproachbehaviors(p=.004)• Repetitive/pragmaticproblems(trend,p=.07)
Stilltoanalyze…
• Childengagementdata• Potentialmoderators
• Teacherfidelity• Studentgains
Discussion
• TeachersupportforCPRT
• Continuedexplorationoffactorsrelatedtoteacherfidelityanduse
• Howcanwemoreeffectivelycapturestudentoutcomes?OR,whyarestudentoutcomessolimited?
• Whatadditionalsupportsareneededforsustainmentofpractice?