assessing the impact of parent involvement programs ... · assessing the impact of parent...

Assessing the Impact of Parent InvolvementPrograms: Research from Outside the World of

Program Evaluation

Eric DearingDepartment of Psychology

My Collaborators

Holly KreiderEllen Mayer

Heather WeissHarvard Family Research Project

Harvard Graduate School of Education

Kathleen McCartneyHarvard Graduate School of Education

Sandra SimpkinsDepartment of Family and Human Development

Arizona State University

Research from Outside the World ofProgram Evaluation

Four Empirical Themes with Implications for Evaluation Work:

1. Involvement is co-constructed.

Research from Outside the World ofProgram Evaluation

Four Empirical Themes with Implications for Evaluation Work:

1. Involvement is co-constructed.

2. Involvement is a dynamic process that can vary withinfamilies.

Research from Outside the World ofProgram Evaluation

Four Empirical Themes with Implications for Evaluation Work:

1. Involvement is co-constructed.

2. Involvement is a dynamic process that can vary withinfamilies.

3. Involvement often has indirect effects on children’sachievement.

Research from Outside the World ofProgram Evaluation

Four Empirical Themes with Implications for Evaluation Work:

1. Involvement is co-constructed.

2. Involvement is a dynamic process that can vary withinfamilies.

3. Involvement often has indirect effects on children’sachievement.

4. Involvement effects can vary across children.

The School Transitions Study

A follow-up investigation of 390 low-income childrenand their families who participated in theComprehensive Child Development Program

Families were 37% African American, 36%European American, and 24% Latino American

Children were followed from kindergarten throughthe fifth grade

Our Operational Definition of Involvement

Home (e.g., reading to/with the child)

School (e.g., attending PTA/PTO meetings)

Home-school communication (e.g., parent-teacher conferences)

Unconventional (e.g., parent-to-parentcommunication)

Involvement is Co-constructed

ContextsOutside the

HomeThe Family

ContextsOutside the

HomeThe Family

FamilyInvolvementin Education

Involvement is Co-constructed

Involvement is Co-constructed

SchoolContext

The Family

FamilyInvolvementin Education

Involvement is Co-constructed

SchoolContext

Supports andServices

Staff and CommunityInvestment

Child and FamilyStrengths

FamilyInvolvement

LiteracyAchievement

The Statistical Importance of Including Context

FamilyProgram

A

FamilyInvolvement

A

B

SchoolContext

FamilyProgram

FamilyInvolvement

The Statistical Importance of Including Context

A

B

C

Involvement is a Dynamic Process

3

3.5

4

4.5

5

5.5

6

1 2 3 4 5 6

School Grade

Fam

ily I

nvo

lvem

ent

K 1 2 3 4 5

Changes in Family Involvement in the School Transitions Studyby Maternal Age

Teenager

20 or Older

Involvement is a Dynamic Process

3

3.5

4

4.5

5

5.5

6

1 2 3 4 5 6

School Grade

Fam

ily I

nvo

lvem

ent

K 1 2 3 4 5

Changes in Family Involvement in the School Transitions Studyby Maternal Age

Teenager

20 or Older

Involvement is a Dynamic Process

3

3.5

4

4.5

5

5.5

6

1 2 3 4 5 6

School Grade

Fam

ily I

nvo

lvem

ent

K 1 2 3 4 5

Changes in Family Involvement in the School Transitions Studyby Maternal Age

Teenager

20 or Older

Involvement as a Dynamic Process

Between-family differences in average level of involvementacross the study were positively associated with between-child differences in average level of literacy performanceacross the study.

In the School Transitions Study:

Involvement as a Dynamic Process

Between-family differences in average level of involvementacross the study were positively associated with between-child differences in average level of literacy performanceacross the study.

Above and beyond these between-family differences,however, increased involvement within families wasassociated with increased literacy performance for children.

In the School Transitions Study:

Involvement as a Dynamic Process

Between-family differences in average level of involvementacross the study were positively associated with between-child differences in average level of literacy performanceacross the study.

Above and beyond these between-family differences,however, increased involvement within families wasassociated with increased literacy performance for children.

In fact, the within-family effect size for involvement wasnearly twice as large as the between-family effect size.

In the School Transitions Study:

The Indirect Effects of Involvement on Children’s Achievement

The Indirect Effects of Involvement on Children’s Achievement

FamilyInvolvement

Feelings aboutLiteracy

LiteracyAchievement

The Indirect Effects of Involvement on Children’s Achievement

FamilyInvolvement

LiteracyAchievement

The Indirect Effects of Involvement on Children’s Achievement

FamilyInvolvement

ChildSelf-efficacy

ChildAchievement

e.g., Hoover-Dempsey & Sadler, 1995; Eccles & Harold, 1993; Bandura et al., 1996

ParentEfficacy

The Indirect Effects of Involvement on Children’s Achievement

http://www.unc.edu/~preacher/sobel/sobel.htm

http://www.public.asu.edu/~davidpm/ripl/mediate.htm

http://users.rcn.com/dakenny/mediate.htm

Involvement Effects Vary Across Children

The Moderating Effect of Maternal Education in the School Transitions Study

0.35

0.4

0.45

0.5

0.55

Low High

Family Educational Involvement

Ave

rage

Pro

port

ion

Cor

rect

on

Woo

dcoc

k-Jo

hnso

n

Lite

racy

(L

ette

r-W

ord)

< High School High School

Moderate

Involvement Effects Vary Across Children

Some moderators that have been demonstrated empirically in thefamily involvement literature:

1. family education (e.g., Dearing et al., 2004).

2. parent-child relationship (e.g., Simpkins et al., 2004).

3. ethnicity (e.g., Hill & Craft, 2003).

4. child age (e.g., Fan & Chen, 2001).

The search for moderators should begin with child, family, orcommunity characteristics that may:

1. strengthen or weaken program effects.

2. modify the meaning of constructs of interest.

Research from Outside the World ofProgram Evaluation

The Implications for Evaluation Work:

1. Assessment of contexts that help determine familyinvolvement can improve the precision of estimated programeffects.

Research from Outside the World ofProgram Evaluation

The Implications for Evaluation Work:

1. Assessment of contexts that help determine familyinvolvement can improve the precision of estimated programeffects.

2. Longitudinal assessments of within-family variations in familyinvolvement can improve the ecologically validity ofinvolvement indicators.

Research from Outside the World ofProgram Evaluation

The Implications for Evaluation Work:

1. Assessment of contexts that help determine familyinvolvement can improve the precision of estimated programeffects.

2. Longitudinal assessments of within-family variations in familyinvolvement can improve the ecologically validity ofinvolvement indicators.

3. Assessing intermediate mechanisms relaying program effectsto children can help capture program effectiveness.

Research from Outside the World ofProgram Evaluation

The Implications for Evaluation Work:

1. Assessment of contexts that help determine familyinvolvement can improve the precision of estimated programeffects.

2. Longitudinal assessments of within-family variations in familyinvolvement can improve the ecologically validity ofinvolvement indicators.

3. Assessing intermediate mechanisms relaying program effectsto children can help capture program effectiveness.

4. Estimating variations in involvement effects can help clarify forwhom involvement matters most.