Components Associated With Home Visiting ProgramOutcomes: A Meta-analysis

abstractBACKGROUND: Although several systematic reviews have concludedthat home visiting has strong evidence of effectiveness, individual eval-uations have produced inconsistent results. We used a component-based, domain-specific approach to determine which characteristicsmost strongly predict outcomes.

METHODS: Medline and PsycINFO searches were used to identify eval-uations of universal and selective home visiting programs implementedin the United States. Coders trained to the study criterion coded char-acteristics of research design, program content, and service delivery.We conducted random-effects, inverse-variance–weighted linear regres-sions by using program characteristics to predict effect sizes on 6outcome domains (birth outcomes, parenting behavior and skills, ma-ternal life course, child cognitive outcomes, child physical health, andchild maltreatment).

RESULTS: Aggregated to a single effect size per study (k = 51), the meaneffect size was 0.20 (95% confidence interval: 0.14 to 0.27), with a rangeof –0.68 to 3.95. Mean effect sizes were significant and positive for 3 ofthe 6 outcome domains (maternal life course outcomes, child cogni-tive outcomes, and parent behaviors and skills), with heterogeneity ofeffect sizes in all 6 outcome domains. Research design characteristicsgenerally did not predict effect sizes. No consistent pattern of effectivecomponents emerged across all outcome domains.

CONCLUSIONS: Home visiting programs demonstrated small but sig-nificant overall effects, with wide variability in the size of domain-specific effects and in the components that significantly predicteddomain-specific effects. Communities may need complementary or al-ternative strategies to home visiting programs to ensure widespreadimpact on these 6 important public health outcomes. Pediatrics2013;132:S100–S109

AUTHORS: Jill H. Filene, MPH,a Jennifer W. Kaminski, PhD,b

Linda Anne Valle, PhD,b and Patrice Cachat, MSWa

aJames Bell Associates, Arlington, Virginia; and bCenters forDisease Control and Prevention, Atlanta, Georgia

KEY WORDSbirth outcomes, child cognitive development, child maltreatment,child physical health, early childhood, effectiveness, homevisiting, infancy, maternal life course, meta-analysis, parentingbehavior

ABBREVIATIONSCI—confidence intervalHomVEE—Home Visiting Evidence of Effectiveness

Ms Cachat collected data and critically reviewed the manuscript;Ms Filene conceptualized and designed the study, designed thedata collection instruments, and drafted the initial manuscript;Dr Kaminski conceptualized and designed the study, designedthe data collection instruments, conducted the analyses, anddrafted the initial manuscript; and Dr Valle conceptualized anddesigned the study, designed the data collection instruments,and reviewed and revised the manuscript. All authors approvedthe final manuscript as submitted.

The findings and conclusions in this report are those of theauthors and do not necessarily represent the official position ofthe Centers for Disease Control and Prevention.

www.pediatrics.org/cgi/doi/10.1542/peds.2013-1021H

doi:10.1542/peds.2013-1021H

Accepted for publication Aug 26, 2013

Address correspondence to Jill H. Filene, MPH, James BellAssociates, 3033 Wilson Blvd, Suite 650, Arlington, VA 22201.E-mail: filene@jbassoc.com

PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).

Copyright © 2013 by the American Academy of Pediatrics

FINANCIAL DISCLOSURE: The authors have indicated they haveno financial relationships relevant to this article to disclose.

FUNDING: This study was supported by The Pew Center on theStates. The views expressed are those of the authors and do notnecessarily reflect the views of the Pew Center on the States orThe Pew Charitable Trusts.

POTENTIAL CONFLICT OF INTEREST: The authors have indicatedthey have no potential conflicts of interest to disclose.

S100 FILENE et al by guest on November 22, 2020www.aappublications.org/newsDownloaded from

Early childhood marks a period of rapidgrowth and development that lays thefoundation for future health and successin school and life.1 Because parents playa critical role in shaping children’s earlydevelopment, interventions that reachfamilies in these early years have greatpotential for producing long-term bene-fits.2 Prenatal and early-childhood homevisiting is a widely endorsed method fordelivering a vast array of preventive andearly intervention services to families inneed of support. By engaging familiesin home visiting programs during theprenatal or early-childhood period, pro-viders seek to improve children’s long-term developmental trajectories byfostering improved parenting knowl-edge and skills, social support, copingand problem-solving skills, and accessto community and health services.3

Despite national and international en-dorsement of home visiting as a strategyto prevent child maltreatment andpromote enhanced functioning andwell-being for children and families,4–8

previous meta-analyses and literaturereviews of home visiting programsacross awide range of outcomes suggestmixed, modest findings depending on theprograms and outcomes examined.6,9–12

A recent review funded by the US De-partment of Health and Human Services,the Home Visiting Evidence of Effective-ness (HomVEE) review, identified 13models that met the department’s crite-ria for effectiveness.13 Across and evenwithin these “evidence-based” models,the findings have been inconsistent,leaving gaps in knowledge about theeffectiveness of home visiting acrossvarious outcome domains. The mixedfindings may be due to program design,thematch betweenprogramcomponentsand expected outcomes, or the quality ofimplementation of the program or theevaluation. Alternatively, the differencesin effects might simply be explained bythe variation in the way home visitingprograms are comprised and delivered.

Best-practice recommendations con-cerning home visiting have generallyeither taken the form of suggestingwholesale adoption of models that havebeen shown to be effective (eg, HomVEE[homevee.acf.hhs.gov], Promising Prac-tices Network [promisingpractices.net])orhavebeenbasedonclinical impressionabout particular approaches (eg, rec-ommendations for a particular scheduleof home visits). Although model ratingsare important forguidingpractitioners inadopting a program model, any partic-ular program may not include the mosteffective combination of components toproduce maximum results for a givenpopulation or community. In addition, asthe Maternal, Infant, and Early ChildhoodHome Visiting Program14 impels in-creased focus on outcomes, a pressingquestion is how to best build the effec-tiveness of a programmodel or enhancemodels that may already be in operation;that is, what elements (eg, content, ser-vice delivery methods) in home visitingprograms seem most important forprogram success?

Although 2 systematic reviews con-ducted before 2002 examined the re-lationship between parent and childoutcomesandasmall subsetofprogramcomponents,12,15 no reviews have fullydisassembled home visiting programsinto individual components or includedstudies conducted during the last de-cade. Therefore, a component analysisapplying meta-analytic techniques wasused to synthesize the results of pub-lished evaluations of home visitingprograms to determine which individ-ual home visiting program componentshave the most power to predict keyparent and child outcomes.

METHODS

Search Strategy

In September 2010, the PsycINFO andMedline databases were searched forliterature published between 1979 and

2010 with evaluations of home visitingprograms.Studieswere limited to thosepublished inEnglishasa journal article,book, or book chapter, although pro-grams could be implemented in anylanguage. Details of the search strategyare outlined in the Appendix. The initialsearchwas designed to be very broadlyinclusive of home visiting programs.

The original literature search resultedin 3252 unduplicated studies. Of these,49 were literature reviews and meta-analyses, from which we identifiedadditional relevant publications. A sec-ondary search was conducted on au-thornames that appearedat least twicein the original search results. In addi-tion,unduplicatedstudies fromHomVEEwere examined. These follow-up strat-egies yielded an additional 1875 re-cords, providing 5127 total abstractsfor possible inclusion.

Study Selection

Inclusion criteria were selected to definethe scope of the meta-analysis as evalu-ationsofuniversalandselective(ie, forat-risk families) programs that used homevisiting as a primary delivery strategy forpregnant women and families with chil-drenfrombirththroughage3years intheUnited States. Programs that conductedonly 1 or 2 home visits were excluded asdissimilar to the rest of the field. Homevisiting programs targeting families forexisting identified problems (eg, familypreservationprogramsorprogramsthatprovided services to families with a sub-stantiatedchildmaltreatmentcase)wereexcluded. Similarly, criteria were se-lected to ensure that evaluation resultscould be generalized to a broad pop-ulation of typically developing childrenand parents. Thus, we excluded pro-grams that targeted parents or childrenbecause of developmental disabilities,chronic illness, feeding disorders, orbereavement because the programsprovide specialized components notfound in thegeneralfieldofhomevisiting.

SUPPLEMENT ARTICLE

PEDIATRICS Volume 132, Supplement 2, November 2013 S101 by guest on November 22, 2020www.aappublications.org/newsDownloaded from

Figure 1 presents the PRISMA flow di-agram for study inclusion. Abstractsidentified in the literature search werescreened by 2 project staff members todetermine eligibility. A study was ex-cluded at this point only if both staffmembers agreed that it met none ofthe inclusion criteria; 525 documentswere retrieved and reviewed in fulltext. To allow for calculation of com-parable effect sizes, studies that used asingle-case evaluation method, lackeda control or comparison group, or didnot contain enough statistical infor-mation to calculate effect sizes wereexcluded. The resulting 126 studieswere coded for meta-analysis; a sub-sample of the 51 articles including the6 outcome measures (maternal lifecourse, birth outcomes, parent behav-iors and skills, child cognitive out-comes, child physical health, and childmaltreatment) selected for this studywere analyzed.

Data Abstraction

Coding forms adapted from Kaminskiet al16 captured information about thedocument, author(s), home visiting pro-gram, participants, evaluation design,

outcome measures, and statisticalresults. Table 1 lists and describes thevariables coded for these analyses. Fullcoding forms are available from the firstauthor. When an article referred toa secondary study or article providingadditional program information, thatsecondary document was obtained, andthe information was coded. Before cod-ing independently, data abstractorswere trained to criteria of coding 3consecutive articles with .90% accu-racy.

Summary Measures

Effect sizes analogous to Cohen’s d sta-tistic17 were calculated from means andstandard deviations whenever possibleor from other reporting methods, in-cluding categorical data, correlations,and odds ratios, by using Comprehen-sive Meta-Analysis 2 software (Biostat,Inc, Englewood, NJ).18 Effect sizes werecalculated based on unadjusted data ifavailable or adjusted data if not. Onceeffect sizes were calculated, they wereexported into SPSS version 20 (IBMSPSSStatistics, IBM Corporation, Armonk, NY)for analyses by using macros for multi-variate analyses of effect sizes.19,20 We

applied Hedges’small sample correctionto all effect sizes before analysis andweighted each by the inverse of thevariance.21

Within and across articles, some sam-ples were represented multiple times(eg, the same sample assessed at dif-ferent time points, assessed with dif-ferentmeasures,orreported indifferentarticles). Including all published reportsof those samples would have alloweda small number of frequently publishedprograms to bias the results. Thus, foreach analysis, we selected or aggre-gated effect sizes such that each sample(eg, a program implemented in a par-ticular location) only provided a singleeffect size for that analysis. Dataonbirthoutcomes at any time point in a studywere included. For all other outcomes,immediate posttest assessments werepreferred. If immediate posttest datawere not available for a particularsample, we included assessments thatoccurred during the intervention butafter two-thirds of the intervention wasdelivered. Follow-up data were excludeddue to a lack of comparability in thelength of follow-up periods. When “total”scores and “subscale” scores from

FIGURE 1PRISMA flow diagram for study inclusion.

S102 FILENE et al by guest on November 22, 2020www.aappublications.org/newsDownloaded from

TABLE 1 Variables Coded for the Analyses and Definitions

Variable Description or Definition

Home visiting content/delivery componentsDevelopmental norms and expectations Information on typical child development, developmental milestones, and child behaviorDevelopmentally appropriate care and routines Using developmentally appropriate behaviors related to satisfying a child’s primary needs

(eg, diapering, dressing, bathing)Safe or clean home environment Information or activities focused on home cleanliness, safety, accident prevention, and first aidStimulating home environment Organizing environment to promote development (eg, books)Responsiveness, sensitivity to cues, and nurturing Providing developmentally appropriate responses to emotional needs, such as physical contact and

affectionDiscipline and behavior management Using age-appropriate discipline or management, including discipline-related communication skillsPromotion of child’s socioemotional development Fostering children’s positive adjustment and well-being, such as positive self-esteem, adaptability,

creativity, and interpersonal comfortPromotion of child’s cognitive development Includes using naturally occurring opportunities to promote child language or knowledge by

describing aspects of the child’s activity or environment and asking questionsPublic assistance Information on obtaining or being directly taught to obtain housing or food assistance (eg, SNAP, WIC, TANF,

AFDC, welfare)Concrete or instrumental assistance Direct provision of resources to address basic needs, including transportation services, respite or child

care, grocery certificatesSelecting appropriate alternative caregivers Information or activities related to finding capable child or respite caregiversParental relationships Enhancing parental relationship (eg, communication between parents)Parental substance use Providing education or direct services related to substance useParental mental health Addressing mental health issues or directly providing mental health servicesPrenatal health Information or activities to promote prenatal health and behavior (eg, diet, nutrition, prenatal care, fetal

development)Family planning or birth spacing Information or activities to promote family planning or birth spacing (eg, optimal intervals, contraception)Self-, stress-, or anger-management Providing services for stress-, anger-, or self-management (eg, self-sufficiency skills, such as time

management)Support group Directly providing a support groupSocial support or social network (need for) Informationandactivitieson the importanceofandhowtoaccesssocial support (eg, teachingparentshow

to identify and access support groups or develop a support network)Adult literacy or academic achievement Information on obtaining GEDs, literacy, or other training or educationProblem solving Teaching the use of problem-solving strategiesGoal setting Teaching parents to engage in goal settingCase management Identifying and linking families to other services and resources (ie, hands-on assistance with

contacting, making appointments, helping with forms or eligibility criteria, advocacy)Rehearsal or role-playing Using rehearsal, practice, or role-playing of techniques or behaviorsHome visitor is professional Using professional home visitors, (eg, nurse, psychologist, social worker)Match between home visitor and client: race/ethnicity Purposive matching of home visitor and client on race and/or ethnicityStandardized curriculum Using an established curriculum or curriculum adapted to family needsProgram delivered in language other than English Program delivered in language other than English

Research design characteristicsRandom assignment Investigators randomly assigned individuals to treatment conditions before the intervention and

maintained group assignment in analyses (eg, studies in which intervention-assigned “nonattenders”were analyzed because comparison participants were coded as not using random assignment)

Assessment of initial equivalence Investigators reported assessment of group equivalence at baseline on either demographic or outcomemeasures

No-treatment comparison group Comparison group for a given effect size received no alternate treatment or servicesHome visiting tested as a stand-alone intervention Intervention group for a given effect size received only the home visiting program (versus receiving the

home visiting program as part of a broader package of interventions)Timing of outcome assessment Outcome was measured at 67% to 90% of treatment implementation versus immediately posttest

Outcome measure categoriesMaternal life course Indicators of maternal health, economic self-sufficiency, educational attainment, and other life outcomes,

such as criminal behavior or subsequent pregnancies and birthsBirth outcomes Indicators of the absence of negative birth outcomes, such as prematurity, low birth weight, or childbirth

complicationsParent behaviors and skills Indicators of parenting behaviors and practices, such as promoting a safe and stimulating home

environment, positive parenting behaviors, well-child visits, and immunizationsChild cognitive outcomes Indicators of cognitive and language developmentChild physical health Indicators of positive health outcomes, including the absence of child injury/ingestion, mortality, and

illnessesChild Maltreatment Indicators of childmaltreatment, including child protective services data and self-report of abusive/harsh

parenting practices

AFDC, Aid to Families With Dependent Children; GEDs, General Educational Development Tests; SNAP, Supplemental Nutrition Assistance Program; TANF, Temporary Assistance for Needy Families;WIC, Supplemental Nutrition Program for Women, Infants, and Children.

SUPPLEMENT ARTICLE

PEDIATRICS Volume 132, Supplement 2, November 2013 S103 by guest on November 22, 2020www.aappublications.org/newsDownloaded from

particular measures were reported,preference was given to the total scoreif it fell within a single outcome cate-gory. When a single study included$3study arms, the effect size most closelyattributable to the effect of only thehome visiting program (eg, treatmentversus no-treatment comparison, ortreatment plus enhancement versusenhancement only) was selected.

Analytic Plan

We first examined overall programeffects on the 6 outcome categories byaggregating to a single effect size perstudy sample. We calculated overallweighted mean effect size, 95% confi-dence interval (CI), and Q and I2 sta-tistics.22 Adhering to the analyticstrategies set forth by Kaminski et al,16

we next investigated outcome-specificmean effect sizes by aggregating toa single effect size per study sample foreach outcome category, as well as CIsand Q and I2 statistics. We used inverse-variance–weighted analyses of vari-ance to examine the impact of 4 indi-cators of methodologic rigor (randomassignment, assessment of initialequivalence, using a pure no-treatmentcomparison group, and testing the ef-fect of the home visiting program asa stand-alone intervention versus aspart of a larger package of inter-ventions) and timing of the outcomemeasure (before versus at the end oftreatment) on effect sizes for eachoutcome category. Finally, we usedinverse-variance–weighted linear re-gression to test the impact of programcomponents on effect sizes, with thegoal of determining the predictors ofstrongest program effects. Only com-ponents theoretically expected to con-tribute to particular outcomes weretested for those outcomes. As the in-tent of the analyses was to model var-iability among studies, all reportedresults were obtained via random-effects models.

RESULTS

Theoverallweightedeffectsizeof thefinalset of 51 studies was 0.20 (95% CI: 0.14 to0.27). The 251 effect sizes ranged from –

0.68 to 3.95. The Q test of homogeneity ofeffect sizes was significant (P , .001),with an I2 value of 65%. Table 2 shows thenumber of studies and summary statis-tics according to outcome category.Three outcome categories (maternal lifecourse, child cognitive outcomes, andparent behaviors and skills) resulted insignificant, positive average effect sizes.Average effects sizes were not signifi-cantly different from zero for birth out-comes, child physical health, and childmaltreatment. Between 52% and 86% ofthe heterogeneity observed for eachoutcome was attributable to true vari-ance rather than to chance, suggestingthe need to further examine the natureof the heterogeneity.

In the inverse-variance–weighted anal-ysis of variances, only 1 research designvariable was a significant predictor ofany outcome: effect sizes of maternallife outcomes were higher among stud-ies reporting outcomes during treat-ment (mean effect size: 0.23 [95% CI: 0.13to 0.33]) than studies reporting out-comes immediately posttest (mean ef-fect size: 0.02 [95% CI: –0.11 to 0.15]).Measurement timing was therefore in-cluded as a covariate in the regressionanalysis of maternal life outcomes.

Resultsoftheinverse-variance–weightedlinear regressions assessing relation-ships between program componentsand effect sizes are presented in Table 3.Controlling for timing of assessment, nocomponents significantly predicted ma-ternal life outcomes. Effect sizes basedon birth outcomes were significantlylarger for programs using nonpro-fessional home visitors, programs thatmatched clients and home visitors onrace and/or ethnicity, and programs thatincluded problem solving. Effect sizesfor the parent behaviors and skills out-come were significantly larger for

programs that taught parents devel-opmental norms and appropriate ex-pectations, discipline and behaviormanagement techniques, responsiveand sensitive parenting practices, andprograms that addressed parentalsubstance use. Children’s cognitiveoutcomes were better in programsthat taught parents responsive andsensitive parenting practices and pro-grams reporting that they requiredparents to role-play or practice skillsduring home visits. Using professionalhome visitors was a significant pre-dictor of better child physical healthoutcomes, as was teaching disciplineand behavior management techniques.However, providing parents with a sup-port group was associated with smallereffect sizes on child physical health.Better child maltreatment outcomeswere associated with teaching parentshow to select alternative caregivers forchildren and problem solving.

To ensure that these results were notunduly influenced by effect sizes basedon results reported in studies as ad-justed statistics, we removed thoseeffect sizesandre-examinedregressionanalyses with significant components.Of the 14 components reported earlieras significant, 3 could not be analyzedwithout the adjusted effect sizes due tolow frequency (the 2 components sig-nificant for child maltreatment out-comes and the relationship betweenchild physical health outcomes andteaching discipline and behavior man-agement techniques). Ten of the other11 components maintained statisticalsignificance in these sensitivity analy-ses. The effect of teaching parentsproblem-solving strategies on birthoutcomeswasno longer significant andthus may be a less robust finding thanother component effects.

DISCUSSION

The overall effect size of home visitingprograms (aggregated across the 6

S104 FILENE et al by guest on November 22, 2020www.aappublications.org/newsDownloaded from

selected outcome domains) was signif-icant and equivalent to approximatelyone-fifthof a standarddeviation favoringthe intervention group. Translated to anodds ratio, suchan effect is equivalent tothe comparison group being∼1.5 timesmore likely to have poorer outcomes.Consistent with results of previousmeta-analyses of home visiting pro-grams,6,9,12,15 parents and children par-ticipating in home visiting programsachieved more positive outcomes over-all than parents and children in control/comparison groups. However, outcome-specific mean effect sizes revealedsignificant but small effects only onmaternal life course, child cognitiveoutcomes, and parent behaviors andskills. In contrast, home visiting pro-grams did not produce significant av-erage effects on 3 frequent programtargets (birth outcomes, child physicalhealth, and child maltreatment), sug-gesting that programs were, on aver-age, not effective in addressing theseoutcomes. The nonsignificant effectsizes, combined with the relativelysmall significant effect sizes, suggestthat communities may need comple-mentary or alternative strategies tohome visiting programs to have agreater impact on these importantpublic health outcomes.

Although surveillance bias (ie, programinvolvement increases the likelihood ofdetecting maltreatment) may partiallyexplain the lackofasignificanteffect sizeon child maltreatment outcomes mea-sured through child protective servicesdata, previous studies have found sur-veillance bias effects to attenuate butnot eliminate group differences where

they exist.23,24 In addition, the presentanalyses included self-reports of abu-sive parenting practices in addition tochild protective services reports. Thus,the presence of a surveillance biaswould likely not fully explain the lack ofstatistical significance.

Research design variables were gen-erally not significantly predictive of ef-fect sizes, whereas many programcomponents were. Similar to othersystematic reviews, no clear and con-sistentpatternofeffectivehomevisitingprogram components emerged acrossoutcome domains.12 Only 3 componentswere predictors of larger effects on.1outcome; 1 of those components wasonly robust for 1 outcome in the sen-sitivity analyses. All other significantcomponents were only predictive ofeffect sizes for a single outcome do-main. These results suggest that the“home visiting” label represents a di-versity of approaches with differingeffectiveness, and that attention tospecific program content and deliverycharacteristics is critical to the effec-tiveness of these programs.

The components that emerged as sig-nificant for .1 outcome (teaching sen-sitive and responsive parenting,teaching discipline and behavior man-agement techniques, and teachingproblem-solving) make intuitive sense;teaching new parenting skills andbehaviors was associated with greatereffects on parenting behaviors, whichmay also translate into more positiveimpacts on other, sometimes more dis-tal, outcomes, such as child cognitivedevelopment, child physical health, andchild maltreatment. Using professional

home visitors was unexpectedly associ-ated with smaller program effects onbirth outcomes but larger effects onchild physical health outcomes. The in-consistency in these results may be dueto the professional background or typeof professional providing the services,as different professionals may be moreor less effective with different healthoutcomes. Alternatively, the inconsistentresultsmight be due to other differencesnot analyzed here between programsusing professional and nonprofessionalhome visitors. Programs that enrollparticipants prenatally and use pro-fessional home visitorsmaywant to lookfor ways to boost their effectiveness,specifically on birth outcomes.

It is important to note that not all com-ponents were tested for each outcome,eitherbecausethecomponentswerenottheoretically linked to the outcome ordue to limited variability of the compo-nent among studies reporting a partic-ular outcome. In addition, nonsignificantcomponents may be contributing toprogramoutcomes(eg,asprecursors toor in combination with other compo-nents) in interactiveways that cannot betested by using these analytic methods.The presence of a significant componentthus indicates a robust effect, but theabsence of significance fora componentdoes not necessarily imply a lack ofimpact. We can only conclude that thenonsignificant components did not bythemselves distinguishmore successfulprograms from less successful pro-grams on that outcome and are thuscomponents that are unlikely to be suf-ficient to produce outcomes they did notsignificantly predict.

TABLE 2 Number of Studies, Mean Effect Sizes, and Results of Heterogeneity Analyses According to Outcome Category

Variable Maternal LifeCourse Outcomes

Birth Outcomes Parent Behaviorsand Skills

Child CognitiveOutcomes

Child PhysicalHealth

Child Maltreatment

No. of studies in analysis 12 14 32 24 15 9Mean effect size (95% CI) 0.20 (0.07 to 0.32)* 0.061 (–0.08 to 0.20) 0.23 (0.13 to 0.33)* 0.25 (0.11 to 0.38)* 0.11 (0.00 to 0.22) 20.08 (–0.24 to 0.07)Heterogeneity analysis P , .02 P , .0001 P , .0001 P , .0001 P , .0001 P , .01I2 52% 86% 75% 78% 80% 65%

*P , .05.

SUPPLEMENT ARTICLE

PEDIATRICS Volume 132, Supplement 2, November 2013 S105 by guest on November 22, 2020www.aappublications.org/newsDownloaded from

TABLE3

Unstandardized

Regression

Coefficients

and95%

CIsFrom

Individual

Inverse-Variance–WeightedLinear

RegressionsPredictingOutcom

eCategory

From

Program

Components

Variable

MaternalLife

Course

Outcom

esBirthOutcom

esParent

BehaviorsandSkills

Child

Cognitive

Outcom

esChild

PhysicalHealth

Child

Maltreatm

ent

Developm

entalnormsandexpectations

0.34

(0.09to0.60)*

0.24

(–0.05

to0.54)

0.08

(–0.24

to0.39)

–

Developm

entally

appropriatecare

androutines

0.04

(–0.23

to0.31)

0.00

(–0.30

to0.30)

20.01

(–0.26

to0.23)

–

Safeor

cleanhomeenvironm

ent

20.12

(–0.39

to–0.15)

20.25

(–0.62

to0.12)

0.04

(–0.21

to0.28)

0.09

(–0.20

to0.38)

Stimulatinghomeenvironm

ent

0.20

(–0.16

to0.56)

0.44

(–0.08

to0.97)

––

Responsiveness,sensitivity

tocues,and

nurturing

0.47

(0.18to0.76)*

0.38

(0.02to0.74)*

–

Disciplineandbehavior

managem

ent

0.29

(0.06to0.51)*

0.41

(0.13to0.68)*

–

Prom

otionofchild’ssocioemotionaldevelopment

20.18

(–0.51

to0.15)

0.21

(–0.09

to0.50)

–

Prom

otionofchild’scognitive

developm

ent

0.01

(–0.26

to0.29)

0.27

(–0.01

to0.55)

–

Homemanagem

ent

––

Publicassistance

–0.15

(–0.22

to0.53)

20.24

(–0.62

to0.14)

–20.06

(–0.36

to0.25)

0.05

(–0.26

to0.36)

Concreteor

instrumentalassistance

–20.06

(–0.44

to0.33)

20.21

(–0.53

to0.12)

0.15

(–0.32

to0.61)

20.07

(–0.35

to0.21)

20.19

(–0.49

to0.11)

Selectingappropriatealternativecaregivers

20.06

(–0.23

to0.11)

0.12

(–0.28

to0.55)

20.15

(–0.48

to0.19)

20.33

(–0.70

to0.05)

0.09

(–0.17

to0.35)

0.26

(0.01to

0.51)*

Parentalrelationships

–0.12

(–0.36

to0.59)

20.09

(–0.52

to0.35)

––

0.02

(–0.30

to0.35)

Parentalsubstanceuse

––

0.32

(0.06to0.60)*

0.07

(–0.35

to0.49)

––

Parentalmentalhealth

0.17

(–0.01

to0.36)

–0.17

(–0.13

to0.47)

20.01

(–0.38

to0.36)

0.08

(–0.23

to0.39)

–

Prenatalhealth

0.22

(–0.14

to0.57)

20.31

(–0.69

to0.07)

0.17

(–0.09

to0.43)

Family

planning

orbirthspacing

0.01

(–0.17

to0.18)

20.08

(–0.42

to0.27)

20.05

(–0.37

to0.26)

Self-,stress-,oranger-managem

ent

––

20.24

(–0.60

to0.13)

––

Supportgroup

––

20.17

(–0.43

to0.09)

0.01

(–0.30

to0.31)

–0.28

(–0.47

to–0.10)*

20.22

(–0.51

to0.06)

Socialsupportorsocialnetwork(needfor)

20.01

(–0.18

to0.16)

20.05

(–0.43

to0.33)

20.20

(–0.46

to0.06)

20.07

(–0.38

to0.24)

0.06

(–0.18

to0.30)

0.01

(–0.29

to0.31)

Adultliteracy

oracadem

icachievem

ent

20.03

(–0.13

to0.20)

20.29

(–0.68

to0.09)

0.05

(–0.26

to0.36)

Problemsolving

0.15

(–0.25

to–0.56)

0.38

(0.03to0.73)*

20.19

(–0.47

to0.10)

0.01

(–0.34

to0.36)

20.04

(–0.29

to0.21)

0.27

(0.03to

0.51)*

Goalsetting

20.14

(–0.30

to0.03)

20.01

(–0.46

to0.44)

20.14

(–0.44

to0.17)

–0.05

(–0.23

to0.32)

0.00

(–0.33

to0.33)

Case

managem

ent

––

20.15

(–0.49

to0.13)

20.17

(–0.48

to0.13)

––

Rehearsalorrole-playing

––

0.09

(–0.22

to0.40)

0.52

(0.13to0.91)*

––

Homevisitorisprofessional

20.04

(–0.24

to0.15)

–0.38

(–0.70

to–0.06)*

0.01

(–0.25

to0.28)

0.02

(–0.29

to0.32)

0.25

(0.04to0.45)*

20.02

(–0.33

to0.28)

Match

betweenhomevisitorandclient:race/ethnicity

0.20

(–0.07

to0.46)

0.39

(0.04to0.75)*

0.09

(–0.20

to0.37)

0.09

( –0.33

to0.50)

–

Standardized

curriculum

0.20

(–0.03

to0.44)

–0.14

(–0.13

to0.41)

0.05

(–0.29

to0.38)

20.01

(–0.27

to0.26)

–

Programdelivered

inlanguage

otherthan

English

0.19

(–0.01

to0.39)

–0.03

(–0.24

to0.29)

20.21

(–0.50

to0.09)

20.20

(–0.42

to0.03)

–

Blankcells

representcom

ponent-outcomerelationships

thatwerenottestedbecausetheparticular

componentwas

notexpectedtocontributetothatoutcom

e.Cells

containing

“–”wereunabletobe

tested

duetoinsufficientvariabilityon

thecomponent

inprogramsmeasuring

thatoutcom

e.*indicatefindings

significantatP,

.05.Formaternallife

course

outcom

esonly,the

timingoftheoutcom

eassessment(ie,duringthefinalone-thirdoftreatm

entversusattheendoftreatm

ent)was

enteredas

acovariatebasedon

theresults

oftheinverse-variance–weightedanalysisofvariancespredictingeffectsizesfrom

studydesign

variables.

S106 FILENE et al by guest on November 22, 2020www.aappublications.org/newsDownloaded from

Our results for the impact of differentcomponents must be taken as correla-tional and not as an experimental ma-nipulation. Our results are also basedonpublished studies and are dependent onthe completeness of reporting of com-ponents within each study. Many theo-retically interesting and relevantprogram characteristics (eg, programdosage, sample demographic charac-teristics,fidelity of implementation, stafftraining, home visitor caseload, study orprogram attrition) could not be testeddue to insufficient numbers of studiesreporting those characteristics. For ex-ample, the timing of enrollment in homevisiting programs during pregnancymight be associated with a program’sability to promote positive birth out-comes; variability in gestation at en-rollment could explain the lack of

significance with birth outcomes.However, this relationship could not betested due to insufficient reporting oninitiation of services.

CONCLUSIONS

The present meta-analysis marksa distinct departure from the commonpractice of recommending the whole-sale adoption of evidence-based pro-grams. Although model ratings areimportant for guiding practitioners inadopting a packaged program model,anyparticularprogrammaynot includethe most effective combination ofcomponents to produce maximumresults. Instead of considering eachprogram as a black box, the codingscheme used in the present studyallowed the authors to disassemblehome visiting programs and examine

the impact of specific components. Theresults suggest that certain existingcomponents are more likely to be as-sociated with positive effects on spe-cific outcomes. Although carefulevaluation of modifications or adapta-tions to existing programs would becritical, changes to include more of thesignificant components identified arelikely to produce programs that aremore potent with respect to theseparent and child outcomes. For otheroutcomes, components that signifi-cantly predict positive outcomes re-main to be identified. Our findings pointto new program and research oppor-tunities within the home visiting field,whether through the development orselection of a home visiting program, orfor improving programs already la-beled efficacious or effective.

REFERENCES

1. Shonkoff J, Phillips D. From Neurons toNeighborhoods: The Science of EarlyChildhood Development. Washington, DC:National Academies Press; 2000

2. Brooks-Gunn J, Markman LB. The contri-bution of parenting to ethnic and racialgaps in school readiness. Future Child.2005;15(1):139–168

3. Guterman N. Stopping Child MaltreatmentBefore it Starts: Emerging Horizons inEarly Home Visitation Services. ThousandOaks, CA: Sage; 2001

4. Krugman RD. Universal home visiting:a recommendation from the US AdvisoryBoard on Child Abuse and Neglect. FutureChild. 1993;3(3):184–191

5. American Academy of Pediatrics. Councilon Child and Adolescent Health. The role ofhome-visitation programs in improvinghealth outcomes for children and families.Pediatrics. 1998;101(3 pt 1):486–489

6. Bilukha O, Hahn RA, Crosby A, et al; TaskForce on Community Preventive Services.The effectiveness of early childhood homevisitation in preventing violence: a system-atic review. Am J Prev Med. 2005;28(2 suppl1):11–39

7. National Governors Association, Center forBest Practices. The benefits and financing

of home visiting programs. Available at:www.nga.org/Files/pdf/BENEFITSFINANCING-HOME.pdf. Accessed August 23, 2008

8. Butchart A, Harvey AP, Mian M, Furniss T.Preventing Child Maltreatment: A Guide toTaking Action and Generating Evidence.Geneva, Switzerland: World Health Organi-zation; 2006

9. Gomby DS. Home Visitation in 2005: Out-comes for Children and Parents. Invest inKids Working Paper No. 7. Washington, DC:Committee for Economic Development; 2005

10. Guterman NB. Enrollment strategies inearly home visitation to prevent physicalchild abuse and neglect and the “universalversus targeted” debate: a meta-analysis ofpopulation-based and screening-basedprograms. Child Abuse Negl. 1999;23(9):863–890

11. Layzer JI, Goodson BD, Bernstein L, Price C.National Evaluation of Family Support Pro-grams. Final Report. Volume A: The Meta-Analysis. Cambridge, MA: Abt Associates,Inc; 2001

12. Sweet MA, Appelbaum MI. Is home visitingan effective strategy? A meta-analytic re-view of home visiting programs for familieswith young children. Child Dev. 2004;75(5):1435–1456

13. Avellar S, Paulsell D, Sama-Miller E, DelGrosso P. Home visiting evidence of effec-tiveness review: executive summary. Officeof Planning, Research and Evaluation,Administration for Children and Families,US Department of Health and HumanServices. Washington, DC. Available at:http://homvee.acf.hhs.gov/HomVEE_Executive_Summary_2012.pdf. Accessed November 22,2012

14. Health Resources Service Administration.Maternal, Infant, and Early Childhood HomeVisiting Program. Available at: http://mchb.hrsa.gov/programs/homevisiting. AccessedDecember 5, 2012

15. Nievar M, Van Egeren LA, Pollard S. A meta-analysis of home visiting programs: mod-erators of improvements in maternal be-havior. Inf Mental Hlth J. 2010;31(5):499–520

16. Kaminski JW, Valle LA, Filene JH, Boyle CL. Ameta-analytic review of components asso-ciated with parent training program effec-tiveness. J Abnorm Child Psychol. 2008;36(4):567–589

17. Hedges LV, Olkin I. Statistical Methods forMeta-Analysis. Boston, MA: Academic Press;1985

SUPPLEMENT ARTICLE

PEDIATRICS Volume 132, Supplement 2, November 2013 S107 by guest on November 22, 2020www.aappublications.org/newsDownloaded from

18. Bornstein M, Hedges L, Higgins J, Roth-stein H. Comprehensive Meta-AnalysisVersion 2. Englewood, NJ: Biostat, Inc;2005

19. Lipsey MW, Wilson D. Practical Meta-Analysis.Thousand Oaks, CA: Sage; 2001

20. Wilson DB. Meta-analysis macros for SAS,SPSS, and Stata. Available at: http://mason.

gmu.edu/∼dwilsonb/ma.html. Accessed January23, 2012

21. Hedges LV. A random effects model foreffect sizes. Psychol Bull. 1983;93(2):388–395

22. Higgins JP, Thompson SG, Deeks JJ, Altman DG.Measuring inconsistency in meta-analyses.BMJ. 2003;327(7414):557–560

23. Olds D, Henderson CR Jr, Kitzman H, Cole R.Effects of prenatal and infancy nurse homevisitation on surveillance of child mal-treatment. Pediatrics. 1995;95(3):365–372

24. Chaffin M, Bard D. Impact of interventionsurveillance bias on analyses of child wel-fare report outcomes. Child Maltreat. 2006;11(4):301–312

S108 FILENE et al by guest on November 22, 2020www.aappublications.org/newsDownloaded from

APPENDIX 1 Search Strategy and Search Terms

Search Step Search Entries

S1. Program/Evaluation Terms (parent and (training or education or program)) or ((support or treatment or interventionprevention) and ((parent or family) and results)

S2. General Program Target Terms (parenting skills or home environment or family relations or parent child relations ormotherchild relations or father child relations or childhood development or at risk protective or(resilient or resilience or resiliency) or child management or competence))

S3. Specific Child Outcome Terms ((youth violenceor juvenile delinquency ordelinquentorconductdisorderorconductproblemor behavior problem or noncompliant or noncompliance or aggression or aggressive or(bully or bullying) or adhd or attention deficit disorder or academic problems or schooladjustment or school problems or school dropout or impulsivity or impulse control orexternalizing or prosocial or problem solving or communication skills or social skills ordiscipline or assertiveness or self esteem or drug abuse or substance abuse or alcohol orsmoking or cigarette or sexual acting out or abuse or neglect ormaltreatment or anxiety ordepression or mental illness or suicide or eating disorder or internalizing or emotionaladjustment or (Child* and (abuse or neglect ormaltreatment or health or injury or violenceor ingestion or poison* or attachment or immuniz* or “emergency department”)) or“infant mortality” or ((juvenile or adolescent) AND delinquen*) or (child and (cognit* orlanguage or “social-emotional” or “socioemotional” or “socio-emotional” or physical orhealth) and development)) or “school readiness” or “school achievement” or “childdevelopment” or “developmental delay” or (child AND behavior*) or (child AND disab*) or((Preterm or “pre-term” or premature) AND birth) or “low birth weight” or “lowbirthweight”

S4. Specific Maternal/Family Outcome Terms ((parent* or family or matern* or mother* or father* or patern*) and (employment or careeror stress or depress* or efficacy or “mental health” or health)) or ((subsequent or teen)AND (birth or pregnan*)) or “home environment” or “self sufficiency” or “self-sufficiency”or (parent* AND (skill* or ability*)) or (reduc* AND (crime or “domestic violence” or “familyviolence” or “intimate partner violence”) or ((community AND coordinate*) or referral*) or(smoking or tobacco)) and (parent and (training or education or program)))

S5. Compiling results from “General Program Target Terms,”“Specific Child Outcome Terms,” and “Specific Maternal/FamilyOutcome Terms” searches

S2 or S3 or S4

S6. Restricting Program Targets/Outcomes to Parenting ProgramEvaluations

S1 and S5

S7. Restricting Relevant Parenting Program Evaluations to thosedelivered in the home

S6 and “home”

S8. Restricting Relevant Home Visiting Program Evaluations to thosepublished in English

Limit S7 to English language

Articles for this meta-analysis were identified from a literature search for a wider set of parenting interventions and thus returned a larger set of article citations than might have beenreturned by a more focused search only for home visiting programs. The overall search strategy built on the search for Kaminski et al16 that was conducted in September 2002 and includedarticles published between 1990 and 2002. On September 16, 2010, we conducted a complementary, updated search to include studies published before 1990 and since 2002. The articlesreturned from the new search were added to the previous database. For simplicity of presentation, the search strategy shown here lists the terms and actions that would have returned the fullset of publication years if the entire search had been conducted on September 16, 2010, instead of in 2 sections.The search discussed here was conducted via OvidSP by using PsycINFO as the database. Terms in quotation marks were searched only as those explicit terms. Terms not in quotation markswere searched as a multipurpose term (ie, .mp) appearing in any relevant field. The option to conduct an “exploded” search was engaged whenever available for an entered search term. An *indicates a “wildcard” search wherein any possible endings of that term were included (eg, behavior* searched for behavior, behaviors, behavioral). The search entries were repeated inMedline with necessary adjustments for that search engine. The returned articles from the 2 databases were then combined. Unpublished dissertations and duplicates were deleted from thefinal set of results.

SUPPLEMENT ARTICLE

PEDIATRICS Volume 132, Supplement 2, November 2013 S109 by guest on November 22, 2020www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2013-1021H2013;132;S100Pediatrics 

Jill H. Filene, Jennifer W. Kaminski, Linda Anne Valle and Patrice CachatMeta-analysis

Components Associated With Home Visiting Program Outcomes: A

ServicesUpdated Information &

http://pediatrics.aappublications.org/content/132/Supplement_2/S100including high resolution figures, can be found at:

References

#BIBLhttp://pediatrics.aappublications.org/content/132/Supplement_2/S100This article cites 12 articles, 3 of which you can access for free at:

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtmlin its entirety can be found online at: Information about reproducing this article in parts (figures, tables) or

Reprintshttp://www.aappublications.org/site/misc/reprints.xhtmlInformation about ordering reprints can be found online:

by guest on November 22, 2020www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2013-1021H2013;132;S100Pediatrics 

Jill H. Filene, Jennifer W. Kaminski, Linda Anne Valle and Patrice CachatMeta-analysis

Components Associated With Home Visiting Program Outcomes: A

http://pediatrics.aappublications.org/content/132/Supplement_2/S100located on the World Wide Web at:

The online version of this article, along with updated information and services, is

by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397. the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, 60143. Copyright © 2013has been published continuously since 1948. Pediatrics is owned, published, and trademarked by Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it

by guest on November 22, 2020www.aappublications.org/newsDownloaded from