Embedding Progress Monitoring in Interventions to Support Social-Emotional and Communication

Development I

Kathleen Baggett, Ph.D.Kathryn Bigelow, Ph.D.

Prepared for the Head Start Center for InclusionNovember 3, 2010

Session GoalsExplore the relevance and importance of

progress monitoring for improving child outcomes in EHS

Provide a brief overview of progress monitoring

Consider some examples of progress monitoring tools and their application to interventions

Stimulate discussion about how progress monitoring can be applied within your work

What is Progress Monitoring... and why is it important?Scientifically-based practice used for

assessing child progress and thus, intervention effectiveness

Allows for documentation of progress and gives us the ability to adjust the intervention to meet intervention goals

Provides data on intervention effectiveness that can be provided to a wide audience (administrators, parents, researchers, funders)

Adapted from National Center on Student Progress Monitoring

Progress MonitoringMeasures should occur on a regular, frequent

basisShould be brief and easy to administer Should be sensitive to the intervention after

relatively short periods of timeShould provide information that is readily

understandable to a range of individuals (administrators, parents, etc)

Should allow for monitoring progress of individual children, and groups of children

Adapted from National Center on Student Progress Monitoring

What is Fidelity Monitoring...and why is it important?

Documentation of the degree to which an intervention is implemented as originally planned

Without fidelity, we do not know if our results, good or bad, are a result of the intervention

Tells us where efforts to increase adherence to the intervention protocol are needed

Crucial to determining if an intervention is effective

Fidelity MonitoringMeasures must be closely tied to intervention protocol Measures should be based on a clear and objective

description of the steps involved in the intervention After observations, calculate the percentage of accurate

implementation, provide feedback, and take steps to increase accuracy, if necessary

Measurement should take place on an on-going basis, as implementation can change across providers or over time

For checking growth in caregivers’ responsiveness to their child in ways that promote positive social-emotional behavior.

IPCI

Home Visiting Nurses

Mental HealthTherapists

Part CEI

Home Visitors

EHSHome Visitors

Parent-Child Interaction

Help interventionists know when intervention is neededHelp interventionists see when they are making a differenceHelp interventionists know when an intervention change is neededHelp supervisors facilitate intervention-planningHelp program directors understand when programs need improvement

While there are many measures of parent-child interaction, tools designed specifically for practitioners to guide intervention decision-making have been lacking.

Practitioners need practical tools that can:

•Monitor progress•Identify risky interactions•Guide intervention decision-making

IPCI Purpose

Progress Monitoring for all children

If cautions are identified, increase monitoring

If concerns are identified, Begin intervention

Administered in family homes or other caregiving settings (biological home, foster home, center-based care) Children 3-42 months of age and a familiar caregiver

Four semi-structured activities are observed for a total of 10 minutes

Free Play Looking at Books Distraction Task Dressing

12 items are rated on a 4-point scale following observation

Videotaping is not required (but useful for intervention purposes)

IPCI Administration

IPCI ActivitiesLooking at

Books

Distraction

Dressing

Free play

IPCI ActivitiesFree play (4

minutes)

“Whatever it is that you and your child like to do together- something your child loves”

IPCI ActivitiesLooking at Books (2

minutes)

“Here are some books for you and your child to look at together- however you and your child would like to spend time with these books is fine.”

IPCI ActivitiesDistraction Task (2

minutes)

“There are times when parents need to keep children away from things- either because they may be dangerous or just inappropriate”. Let’s see how your child responds when there are materials that are off limits. Please keep child on the blanket and away from the recorder and keys”.

IPCI ActivitiesDressing (2 minutes)

“Whatever it’s like to get dressed in the morning” (shirt, socks, shoes)

Caregiver Facilitators Conveys acceptance and

warmth Makes descriptive

comments Follows child’s lead Maintains and extends

Caregiver Interrupters Uses criticism, harsh

tone Uses intrusions and

restrictions

Child Engagement Positive feedback Sustained engagement Follow-through

Child Distress Fuss, cry Tantrum Frozen, watchful, withdrawn

IPCI Domains and Behaviors

Caregiver and Child Behaviors are rated on a 4-point scale O = Not at all 1 = Rarely 2 = Sometimes but inconsistently 3 = Often and consistently

Behavior Ratings

The IPCI’s Home:

With other early childhood general outcomes measures

Early Communication IndicatorEarly Social IndicatorEarly Problem Solving IndicatorEarly Motor Indicator

Types of IPCI Reports

Home Visitors and SupervisorsProgram AdministratorAgency Administrator

Kansas Model Demonstration Center Goal of project is to support early intervention

providers in using evidence-based practices to help build the capacity of parents and educators in promoting communication of young children

Fidelity and progress monitoring include documentation of: Parent and teacher use of communication

strategies to monitor intervention progress and fidelity and provide feedback , and child communication

Parent and teacher self-report of communication strategies to monitor progress, provide feedback, and prompt use of strategies

Fidelity of implementation of communication strategies by providers, who are delivering intervention to parents and teachers

Monitoring Parent/Teacher Strategy Use and Child CommunicationParent and teacher use of communication promoting strategies

measured using a frequency count of strategy use during 30-min observations

Also measure child communication (gestures, vocalizations, words, multiple words)

Providers share graphs with parents and teachers monthly to provide feedback on use of strategies, illustrate connection between adult and child communication, and show growth in child communication over time

Provider uses this as an opportunity to provide positive feedback, address challenges, and prompt further use of strategies throughout routines

Dotted line shows frequency of use of communication promoting strategies by a parent, and bars show child communication

Parent and Teacher Self-Report Use of Communication StrategiesParent and teacher complete self-checks of their use of

communication promoting strategies during a variety of routines

Rate frequency with “not today”, “rarely”, “sometimes”, and “often”

Providers collect forms and use data to prompt discussion with parent/teacher about use of the strategies, provide positive feedback, and share additional ideas for embedding strategies in new routines or activities

Functions as a prompt to remember to use strategies, but also allows parents or teachers to self-monitor their use of strategies

Please fill out the form based on how of ten you used each strategy during daily routines over the course of one week:

Child Name: _ _____________________

Date: _ _ _______________

Person Completing: _ _ ________________ Group/ Meal/ Other:

Provider: _ _ _______________________ Free Play Reading Outdoor Circle Time Snacktime ________

Comment and Wait

Described actions, toys, or activities N R S O N R S O N R S O N R S O N R S O N R S O

Ask Questions and Wait

Asked "Yes/ No" and Open-Ended Questions N R S O N R S O N R S O N R S O N R S O N R S O

Respond by Adding a Little More

I mitated child talk and expanded by adding new inf o N R S O N R S O N R S O N R S O N R S O N R S O

Comments:

© 2004 University of Kansas. Promoting Communication Strategies Checklist: Home (Reproduce with Permission Only) Walkerd@ku.edu

Neve

r

Rar

ely

Som

eti

mes

Oft

en

Communication Strategies Self -Check: TeacherModel Demonstration Center on Promoting Communication

Neve

r

Rar

ely

Som

eti

mes

Oft

en

Neve

r

Rar

ely

Som

eti

mes

Oft

en

Date:_________________________

Parent :________________________ Dressing &

Child: _________________________ Play Toileting Outings Reading Mealtime OtherPerson Completing: _______________

Commented and Labeled Not Used Not Used Not Used Not Used Not Used Not Used

Described actions, toys or activitiesRarely Used Rarely Used Rarely Used Rarely Used Rarely Used Rarely Used

Sometimes Sometimes Sometimes Sometimes Sometimes Sometimes

Of ten Used Of ten Used Of ten Used Of ten Used Of ten Used Of ten Used

Environmental Arrangement Strategies

Books, activities provided? yes no Daily schedule and activity changes? yes no

Books available throughout the day; Flexible schedule; activity changes smooth; have a book area minimal waiting

Materials available and arranged? yes no Family I nteractions? yes no

Toys out, arranged in view to Surroundings encourage f amily interactionsencourage interaction

NOTES:

Measure of Fidelity of Implementation: ProvidersSimple checklist used to document the strategies shared with

parents/teachers, strategies observed during visits, and routines in which strategy use was addressed

Providers fill this out themselves after each visit and regular reliability checks are completed by intervention coach

Data are summarized and a report is provided monthly, summarizing data for each family

Intervention coach distributes reports to providers in meetings or in person, and posts on password-protected web site

Coaches use this as an opportunity to discuss provider’s strategies for sharing intervention with parents/teachers, provide positive feedback, address challenges, and plan next steps in intervention

Juniper Gardens MDC Communication Strategies□ Environ. Arrangement □ Following child’s lead□ Commenting/labeling □ Imitating/expanding□ Open-ended questions □ Positive attention/praise□ Providing choices □ Fill in blank/time delay Strategies discussed or taught in:

□ Play □ Meal/snack time □ Toilet/personal care□ Household routines □ Reading/ books □ Circle/Group □ Outings □ Outdoor □ Other (desc): ________Observed parent using strategies in:

□ Play □ Meal/snack time □ Toilet/personal care□ Household routines □ Reading/ books □ Circle/Group □ Outings □ Outdoor □ Other (desc): ________Discussed Manual/DVD? □ Yes □ No

Reviewed Self-Checklist? □ Yes □ No □ N/A

Discussed Graph? □ Yes □ No □ N/A

Juniper Gardens MDC Provider _____ Child ________ Date ___ / ___ / ___Worked with (check all that apply):

□ Child (directly) □ Parent□ Teacher

Language is the Key Strategies used, taught or discussed:

□ Comment and Wait□ Ask Questions and Wait□ Respond by Adding More (expansion)□ Following Child’s Lead □ Other (desc): __________________Strategies discussed or taught During:

□ Play □ Reading/ books □ Daily Activities □ Other ____________

Observed parent/teacher using strategies during:

□ Play □ Reading/ books □ Daily Activities □ Other ____________

Introduced Lang is the Key Manual/DVD for first time? □ Yes □ No □ n/a Reviewed Self-Checklist? □ Yes □ No Reviewed Self-Checklist? □ Yes □ No Discussed Graph? □ Yes □ No □ n/a

MDC Parent Summary for Providers Site:

Child(ren): Parent: Part C Provider:

Data Summary from the Provider Observation Checklist

Total visits with checklist to 18 Most recent checklist entered: 4/15/2010 (Date of Visit)

Strategies Addressed with This Parent So Far Activities in Which Strategies Were Addressed # Visits # Visits Environmental arrangement 8 Play 16 Following child's lead / 7 Meal / snack time 4 Commenting / labeling 10 Toileting / dressing / personal 6

Imitating / expanding 13 Household routines 8 Open-ended questions 3 Reading / looking at books 9 Positive attention and praise 7 Circle time 0

Providing choices 9 Outings 0 Fill in blank / time delay 9 Outdoors 0 Other 1 Other 2

Strategies Observed During Visits Report Date: 5/26/2010 # Visits Group: No Graphs Environmental arrangement 12 Following child's lead / 16 Intervention Commenting / labeling 17 Start:Date: 4/25/2009

Imitating / expanding 17 Notes: Open-ended questions 13 Positive attention and praise 17 __________________________________________

Providing choices 5 __________________________________________ Fill in blank / time delay 4 Other 0 __________________________________________

Provider Summary Report showing strategies addressed with a given family so far, activities in which strategies were addressed, and strategies observed throughout the visits.

Fidelity and Progress Monitoring Working TogetherFidelity monitoring through observations of parents and

teachers, and of early intervention providers offers a wealth of information on whether intervention is being implemented as planned

Fidelity monitoring through observations of parents, teachers, and providers tells us the degree to which intervention is being implemented as originally planned (and initially evaluated)

Progress monitoring shows us how parents and teachers are using strategies and how child communication is growing and how social-emotional behavior is changing

Provides greater confidence in effectiveness of the intervention

References and ResourcesCarta, J., Greenwood, C., Walker, D., & Buzhardt,

J. (Eds.). Individual Growth and Developmental Indicators: Tools for Monitoring Progress and Measuring Growth in Very Young Children. Baltimore, MD: Brookes.

IGDI website: http://www.igdi.ku.edu/

National Center on Student Progress Monitoring – www.studentprogress.org