Educ Res Policy PracDOI 10.1007/s10671-014-9163-y

ORIGINAL PAPER

Effects of preschool intervention strategies on schoolreadiness in kindergarten

Xin Ma · Regena F. Nelson · Jianping Shen ·Huilan Y. Krenn

Received: 2 July 2013 / Accepted: 5 March 2014© Springer Science+Business Media Dordrecht 2014

Abstract Using hierarchical linear modeling, the present study aimed to examine whethertargeted intervention strategies implemented individually during a preschool program exhib-ited any short-term and long-term effects on children’s school readiness in kindergarten,utilizing data gathered through the Supporting Partnerships to Assure Ready Kids (SPARK)initiative. Outcome measures included scales from the Preschool and Kindergarten BehaviorScales and Bracken Basic Concept Scales. The short-term effects highlighted the interventionstrategy of age-appropriate well-designed learning materials that effectively excelled chil-dren in overall conceptual development. We found no long-term effects of any interventionstrategy on school readiness (SPARK children did not exhibit any statistically significantgains in school readiness measures in kindergarten).

Keywords Preschool intervention · School readiness · Short-term and long-term effects

Current research trends in early childhood education (ECE) focus on increasing school readi-ness of children, particularly children with socioeconomic disadvantage. As more resourcesare devoted to early childhood programs, there is a need to evaluate the impact of earlychildhood interventions and services to determine how to best distribute limited resources topopulations that need them the most. Program evaluation provides information on effectiveinterventions and services. It is always important to know how duration and frequency ofinterventions and services influence children’s school readiness. The present study aims toexamine the impact of several intervention strategies on the short-term and long-term schoolreadiness of vulnerable children.

X. Ma (B)University of Kentucky, Lexington, KY, USAe-mail: xin.ma@uky.edu

R. F. Nelson · J. ShenWestern Michigan University, Kalamazoo, MI, USA

H. Y. KrennThe W. K. Kellogg Foundation, Battle Creek, MI, USA

123

X. Ma et al.

1 Review of literature

1.1 School readiness

Currently, kindergarten teachers report that at least 1/3 of the children in kindergarten are notready for school (Ackerman and Barnett 2005). Some children start school up to 1.5 yearsbehind their peers in reading, math, and vocabulary skills (Sadowski 2006). School readinessis a multi-dimensional measure of the learning opportunities in a child’s home, school, andcommunity environments (Carlton and Winsler 1999). The assumption is that all childrenare capable of learning and need the assistance of parents and teachers to guide their learningin cognitively and socially stimulating environments. Therefore, the children who have haddevelopmentally appropriate learning experiences on a consistent basis prior to kindergartenare ready for school (Rodriguez and Tamis-LeMonda 2011).

This implies that school readiness is a modifiable attribute, which is a change from earlierviews of school readiness. Previously, school readiness was seen as a developmental stagethat was biologically determined rather than influenced by one’s environment. Children werescreened to determine if they were developmentally ready for school. If they were not, thenthey were given the gift of time by being held out of school until they were developmentallyready (Bear and Modlin 1987; Ellwein et al. 1991; Meisels 1987).

The current view of school readiness is based on brain research that has shown howchildren’s brain development is influenced by the learning experiences they have startedfrom birth (Bruer 1999; Shore 1997). Children in homes where they are spoken to and readto on a consistent basis have more developed brain structures than children who are not.The children with more developed brain structures perform better on school readiness tests(LaParo and Pianta 2000). To increase the number of children who come to kindergarten readyfor schools, the pre-kindergarten learning environments must be improved for all children(Burchinal et al. 1997; Hart and Risley 1995; Shonkoff and Phillips 2000).

One method for improving pre-kindergarten learning environments is offering interventionprograms for children. Intervention programs can happen at homes and in schools. In homeintervention programs, an educator or a nurse visits the family regularly in their home andprovides information about child development, learning activities, and social services in thecommunity (Powell 1990; Ramey and Ramey 1998). Some home programs also providemental health and career counseling to parents to help remove emotional and economicbarriers that prevent parents from providing a stimulating home environment and interactingwith their children in positive ways. Generally, these programs are for families with childrenthat are birth through age 3.

After age 3, the intervention services are provided in government funded, free preschoolprograms such as Head Start or if needed through early intervention programs that providetreatment and services for children with developmental delays. Research has shown thatchildren who attend high quality preschool programs and/or receive early intervention ser-vices to treat developmental delays are more likely to be ready for school (e.g., Rosenberget al. 2008). Therefore, the research on school readiness has shifted to determining how toimplement high quality intervention programs (Ramey and Ramey 2004).

1.2 Intervention strategies

Overall, intervention strategies have proven to be effective if they meet certain criteria regard-ing length of time, frequency, and comprehensiveness of treatment (Ramey and Ramey 2000).Research has shown that programs that focus on prevention of learning difficulties are less

123

Effects of preschool intervention strategies

expensive than intervention programs that treat an existing learning problem (Ramey et al.1984). Therefore, the timing of program services is crucial. Intervention programs that startprenatally or at birth can focus on prevention by tracking children’s development early. Theresults from early developmental screenings give parents information on how to supporttheir children’s normal development or how to seek out further evaluation if a developmentaldelay is detected. If needed, intervention services are provided until age 5 and supplementpreschool attendance.

Intervention services are effective when they are provided consistently. Children receivingservices to treat a developmental delay show more progress when the treatment occurs dailyrather than once a week. Parents can provide daily treatment or teachers can provide it ina program for children with special needs. Full day programs have more impact than half-day programs. In addition, specialists should evaluate children regularly to determine if thetreatment is working and to create new developmental goals (Farran 1990).

Although the goal of early intervention programs is to increase school readiness, theinterventions are broader than education services. The most successful intervention programsprovide health care, nutrition, social services, and parent education. Taking a comprehensiveapproach to treat the whole child and their family system has the greatest impact on schoolreadiness, future school achievement, and the trajectory of children’s lives into adulthood(Casto and Lewis 1984).

Ideally, all families would have access to intervention services at the time of birth. Duringthe hospital stay, a specialist would visit the family and invite them to participate in a parent-ing program. The parent program options would range from drop-in parent support groupsand workshops in the community to weekly home visits from a nurse. The most effectiveprograms are home visiting programs. One such program is the Nurse Family Partnership(NFP) program (Olds et al. 1999; Weiss 1993). In NFP, a nurse starts visiting the motherduring the 2nd trimester of the pregnancy and continues to visit bi-monthly until the child isage 2. The nurse monitors the health of the mother and the child. The visits focus on com-pleting a curriculum on child development and career planning. Many government fundedpreschool programs have a parent education component that includes home visits (Parkeret al. 1999). Home visits are an effective service model because they remove the economic,childcare, and transportation barriers that prevent families from attending programs in thecommunity (McLoyd 1990). The services are brought to them individually in the comfortof their home. When service providers visit the homes they gain a broader understanding ofthe family system and how well it functions. The consistency of the visitations helps buildtrust between the service provider and the family, which will increase the fidelity of programimplementation by the families. As a result, parents are more likely to change their parentingbehaviors. They tend to engage in more behaviors that will improve school readiness suchas free play, conversations, and reading with their children. In summary, home-based parenteducation programs targeted for families with children from birth to age 3 establish a senseof trust with community programs that can be built on in developing future home–schoolpartnerships (Pianta et al. 1997).

Another component of early intervention is the use of developmental screening to trackchildren’s development. Developmental screening assessments have items that represent chil-dren’s developmental milestone from birth through age 5. The assessment describes how todetermine if the child has met a given developmental milestone. The child’s score is a mea-sure of how the child has performed compared to how the average child performs on theassessment. If the child is performing below average, further evaluations are recommendedto determine if there is a developmental delay. If a developmental delay is not present, thepractitioner will encourage the family to continue to provide opportunities to practice devel-

123

X. Ma et al.

opmental skills through play activities (Parker et al. 1999). The practitioner will explainwhich types of toys and activities will help develop the particular skill. The purpose ofdevelopmental screenings is to provide information to parents that helps them support theirchildren’s normal development. Developmental screenings in isolation should not be usedto make decisions about program placement. Developmental screening assessments are notpsychometrically sensitive enough for making program placement decisions (Bordignon andLam 2004). One criticism is that the norm population for the assessments does not includea representative sample of children from low-income and minority backgrounds (Barona1991; Gredler 1997). Therefore, additional information from diagnostic tests is needed tomake treatment and program decisions (Costenbader et al. 2000).

Diagnostic testing is completed to determine the problem and to create, say, an IndividualFamily Service Plan (IFSP) (Buysse and Bailey 1993). The plan will include developmentalgoals and the treatment plan that will be implemented to meet the goals. The plans may focuson treating a language, speech, physical, hearing, vision, emotional, or cognitive impairment.The treatment will include therapeutic services to address the problem as well as suggestionsfor how parents will be involved in the treatment at home. For example, an emotional impair-ment treatment plan may include a behavior support plan. The plan will provide opportunitiesfor the child to interact with other children to observe positive behavior models and prac-tice social skills. At home, the parents will model and reinforce positive communication andbehavior skills (Blair et al. 1999; Campbell and Ewing 1990; Fox et al. 2002). Treatment plansare designed to be strength-based. The specialist will include activities that highlight the areaswhere the child is performing well and use those skills to improve deficit areas. For instance,a specialist would use songs and puppet play to teach social skills to a child who has stronglanguage skills. Children’s progress toward each goal on the IFSP is monitored throughoutthe treatment cycle. If treatment continues beyond age 3, an Individualized Education Plan(IEP) is developed to replace the IFSP. The IEP is used to provide treatment when the childtransitions into a preschool program. The goal would be to place the child into a preschoolprogram that serves children with special needs in an inclusive environment (Odom 2000).The specialist will work with the preschool teacher to find ways to adapt instruction to meetthe child’s special learning needs in the classroom in ways that support any treatments thechild receives outside of the classroom. Children with severe impairments will need to attenda program for children with special needs and receive more intensive, full-time treatment.Local school districts have preschool programs exclusively for children with special needs.

Preschool classrooms in Head Start are designed to be inclusive environments that canaccommodate children with mild impairments and/or risk factors related to poverty. Whilereceiving intervention services from birth is optimal, research has shown that just 1 year ofhigh-quality preschool can have a significant effect on the school readiness and kindergartenachievement of children in poverty. The goal of Head Start is to provide high-quality preschoolthat focuses on cognitive and social skills and provides health, nutrition, and parent educationservices, thus increasing the value of the preschool experience by expanding the servicesit provides to families (Zigler and Muenchow 1992). As more states cut welfare benefitsfor families, there is an increase in the need for more high quality, affordable child careoptions as parents seek out job training and gain employment (Loeb et al. 2004; Zaslow etal. 1998). Child care programs will be effective in preparing children for school if teachersare trained to implement a developmentally appropriate curriculum that teaches childrenemotional regulation, social skills, and how to carry out their own learning plans based ontheir interest and ability. Teachers must also partner with parents to encourage the use ofplay, books, chores, language, and outings to extend the learning at home (Connell and Prinz2002; Kontos 1991; Peisner-Feinberg and Burchinal 1997).

123

Effects of preschool intervention strategies

Programs that can provide health screenings and services on site are also needed (Oldsand Kitzman 1990). Many young children come to school with chronic illnesses that gountreated because they have not received adequate health care. Children should be screenedfor hearing and vision problems (Currie 2005). Chronic ear infections that go untreated canlead to hearing loss that affects the ability to hear instruction, learn language, and interactwith peers. These children are more likely to have academic and behavior problems. Childrenwho need corrective lenses to see but do not have glasses will struggle with reading and havelow academic achievement. Other infections and illnesses related to tooth decay, asthma, andlead exposure will have similar effects on achievement and behavior. In sum, children whoare in poor health due to inadequate health care and nutrition are less prepared to perform wellin school because they are tired and unable to focus on instruction. Receiving appropriatemedical treatment for their illnesses, including attention deficit disorder, can have a dramaticeffect on school readiness and kindergarten achievement.

Overall, research shows that providing a comprehensive set of intervention services start-ing at birth that address families’ needs with a consistent treatment schedule through age 5 hasa positive impact on children’s school readiness (e.g., Cooper et al. 2009). Nevertheless, givenvarious constraints (e.g., resources), it is often necessary to target a certain group of childrenat a certain time during birth to age 5 with a specific intervention service (instead of a compre-hensive set of intervention services). Is it then possible that (targeted) intervention servicesimplemented individually such as health screening, health care, nutrition programs, par-ent education programs, developmental assessments, early childhood special education, andpreschool programs can also have an important impact? The present study attempts to exam-ine whether (targeted) intervention strategies (implemented individually during a preschoolprogram) exhibit any effects on children’s school readiness in kindergarten, utilizing datagathered through the Supporting Partnerships to Assure Ready Kids (SPARK) initiative.

2 Method

2.1 SPARK and sample

Aimed to create a seamless transition into kindergarten for vulnerable children ages 3–6,SPARK supported efforts to mobilize and strengthen partnerships among families, schools,communities, state agencies, and advocacy groups to work together effectively to promoteearly learning and improve learning outcomes among children. With $43 millions over 5 years(2003–2008), SPARK targeted vulnerable children who were not fully served by other pro-grams and services in selected communities across seven states (Florida, Georgia, Hawaii,Mississippi, New Mexico, North Carolina, and Ohio) and the District of Columbia (seeGreeley and Greeley 2011).

“SPARK’s theory of change can be boiled down to this essence: Strong partnerships amongfamilies, providers, community organizations, and ready schools ensure that children succeedin learning” (Berkley 2003, p. 1; see also Greeley and Greeley 2011). SPARK worked withunderserved parents and communities directly to create plans and solutions to better servevulnerable children. The SPARK approach was to form partnerships of childcare providers,schools, and community stakeholders so as to share promising practices, ideas, and resourceswithin each collaboration (project) across the eight sites. Each collaboration strived to fostercaring environments where all children became active learners with all the supports that theywould need to be ready for school.

Consider as one example a SPARK project implemented in Ohio. This was a parent-focused intervention program seeking collaboration with families, schools, and the

123

X. Ma et al.

community to improve readiness for kindergarten of children (in particular high-need chil-dren). The project provided developmental screenings, referrals and linkages to communityresources, and home- and group-based learning opportunities; trained parents to becomeeffective learning advocates for their children; and provided coaching, professional devel-opment, and technical assistance to teachers and administrators to build social, emotional,physical, and cognitive skills that children need as they transition from home or preschool tokindergarten.

We obtained longitudinal data on a sample of SPARK children and comparison children ina preschool program in which intervention strategies were implemented to SPARK children.Children came from disadvantaged families and neighborhoods. Communities shared a com-mon historical pattern: Children’s school readiness and school failure had constantly been aconcern to the communities. Specifically, two cohorts of vulnerable children took part in thepreschool program with projects across the eight SPARK sites. In each project, children’sspecific needs were identified and corresponding intervention strategies were implementedto target those needs. Although this sample of SPARK children and comparison childrenentered different kindergartens after the preschool program, SPARK managed to follow themthrough kindergarten measuring their school readiness twice (in the fall when they enteredkindergarten and 9 months later in the spring).

Specifically, children in one cohort were measured in the fall of 2008 and again in thespring of 2009, whereas children in the other cohort were measured in the fall of 2009 andagain in the spring of 2010. Because children in both cohorts were similar and went throughan identical preschool program, we combined children from both cohorts together to obtaina much larger sample so as to increase the statistical power of our data analysis. The originalsample of combined cohorts has 732 children.1 In actual data analysis, sample size wasreduced due to missing data on outcome measures at the child level and missing data onteacher characteristics at the teacher level. As a result, the effects of intervention strategieswere examined based on a sample of 477 children with valid outcome measures in the springdata collection.

2.2 Design issues

Children in each cohort attended school (kindergarten) in the fall after a special preschoolprogram in which targeted intervention strategies were implemented. The fall measures wereconsidered pretest measures. Children in each cohort were measured again about 9 monthslater in the spring of the year after. The spring measures were considered posttest measures.Although this design was not a conventional pretest and posttest experiment because therewas neither a treatment condition nor a control group, we did inherit the sample of SPARKchildren (who received targeted intervention strategies) and comparison children from thespecial preschool program. This was a good opportunity to examine the short-term and long-term effects of targeted intervention strategies on promoting school readiness. The short-termeffects were examined by pretest measures because they were given to children once theyentered kindergarten. The long-term effects were examined by gains from pretest to posttestmeasures.

1 The eight sites for SPARK were not randomly selected. However, these sites had disadvantaged communi-ties that were quite typical across the country. What made these sites different was their degree of determi-nation for change and commitment to cooperate with the Foundation. This sample was by no means random.A reasonable description of this sample could be that children came from (nationally) typical, disadvantagedcommunities with strong determination and commitment for change.

123

Effects of preschool intervention strategies

We were methodologically challenged by the design with the attempt to examine thelong-term effects of targeted intervention strategies. First of all, in such a design, gainsobserved from pretest to posttest might come from intervention strategies and kindergarteninstruction. SPARK followed its original sample of SPARK children and comparison childreninto kindergarten, without establishing a control group in kindergarten. Without a controlgroup, our best coping strategy was to control for teacher quality with the assumption thatkindergarten instruction would correlate highly with teacher quality. Statistical control cannever replace a control group. At its best, control over teacher quality could remove some ofthe effects on gains from kindergarten instruction. This implies that the effects of long-termintervention strategies were overestimated in the present analysis, and thus estimates wouldrepresent a sort of upper boundary of the effectiveness of targeted intervention strategies.

Second, different intervention strategies were implemented in different projects across theeight SPARK sites to target various special needs of vulnerable children. This implies thatSPARK children and comparison children were all relative, meaning that SPARK childrenand comparison children were different across projects (i.e., not the same groups of children).Third, not all SPARK children received the same number of intervention strategies (becauseagain these were targeted intervention strategies). Children could receive from one to sevenintervention strategies.2 To cope with these two challenges, we treated intervention strategiesas a whole block so that the effects of a particular strategy were estimated with control overparticipation in other strategies.

2.3 Measures and variables

Outcome measures were obtained from children at the time of entering kindergarten (thefall data collection) and in about 9 months later (the spring data collection). They includedtwo scales from the Preschool and Kindergarten Behavior Scales (PKBS) (Merrell 2003)and two scales from the Bracken Basic Concept Scales—Receptive (BBCS) (Bracken 2006).The PKBS is a behavior checklist administered to parents to identify social skills and prob-lem behaviors of children ages 3–6. The social skills composite measures positive socialskill characteristics of well-adjusted children (e.g., follow instructions from adults, try tounderstand another child’s behavior, separate from parent without extreme distress); a higherscore is desirable and indicates a higher level of social functioning. The problem compositemeasures problem behaviors with young children who are experiencing adjustment problems(e.g., will not share, overly sensitive to criticism or scolding); a lower score is desirable andindicates less problematic behaviors.

The BBCS is a developmentally sensitive measure of children’s basic concept acquisitionand receptive language skills. The school readiness composite collectively represents thereadiness concepts that parents and preschool programs traditionally teach in preparationfor formal education (i.e., colors, letters, numbers/counting, sizes/comparisons, shapes), andthe receptive total composite provides information about a child’s overall conceptual devel-opment (direction/position, self/social awareness, texture/material, quantity, time/sequence).A higher score is desirable in terms of both composites and indicates a higher level of schoolreadiness.

Outcome measures discussed above were dependent variables for data analysis. Key inde-pendent variables were intervention strategies, including learning advocate, consultation toECE setting, initial development screening, SPARK developed learning plan, home visit,

2 Children who received from one to seven intervention strategies were numbered 33, 35, 43, 78, 28, 67, and4, respectively. These counts added up to the sample of 477 children, together with 138 children without anyintervention strategy and 51 children with missing information on participation in intervention strategies.

123

X. Ma et al.

parents as teachers (PAT), health screening, learning materials, accreditation assistance toECE setting, and preschool learning workshops and activities.3 Other independent variableswere used for the purpose of controlling for child characteristics, including gender, age, race,socioeconomic status (measured through receiving financial aid from government), familystructure (measured through the number of adults and the number of children living in a fam-ily), special need status, and insufficient English proficiency (IEP) status. Finally, we derivedkey independent variables for (kindergarten) teachers that captured teacher quality. Variablesthat are powerful indicators of teacher quality are education level and certification status (e.g.,Teddlie and Reynolds 2000). They were used to control for kindergarten instruction based onthe assumption of a high correlation between teacher quality and kindergarten instruction.

These child and teacher variables also provided a good opportunity to further describe oursample (of children and teachers) with descriptive statistics (see Appendix 1). Among bothchild and teacher characteristics, most variables were dichotomous so that means representedpercents of categories coded as 1 (e.g., boys in the sample were 50 %). A few variables werecontinuous (i.e., age, number of adults living in a family, and number of children living in afamily). For example, the average age of children in the sample was 5.41 (approximately 5years and 5 months old).

Intervention strategies were implemented (on a targeted basis) to these children duringthe preschool program. We presented descriptive statistics to illustrate on the participation ofchildren in these intervention strategies (see Appendix 2). Participation status was dichoto-mous coded as 1 if a child participated in a certain intervention strategy. Learning materialsand initial development screening were widely used intervention strategies, implemented to52 and 47 % of children, respectively, in the sample.

2.4 Analytical procedures

After leaving the special preschool program, children attended kindergarten with teacherswho could be identified and in schools that could be identified. We employed hierarchicallinear modeling (HLM) to examine the effects of intervention strategies on gains in outcomemeasures from fall (when children entered kindergarten) to spring (about 9 months later).A three-level HLM model had children nested within teachers nested within schools. Themodel at the child level examined the effects of intervention strategies on gains in outcomemeasures with control over child characteristics. The model at the teacher level controlledfor teacher quality to (at least) partially remove the confounding effects of kindergarteninstruction. The model at the school level was empty mainly to take into account the datahierarchy of teachers coming from different schools. This HLM approach was, therefore, ourprimary means to examine the long-term effects of intervention strategies.

For the short-term effects of intervention strategies, we employed multiple regressionanalysis. Our reasoning was that the data hierarchy of children nested within teachers nestedwithin schools did not yet apply to children as they just entered kindergarten when measuredin the fall (another term to describe kindergarten entrance performance of children wouldbe preschool exit performance of children). Because SPARK children and comparison chil-dren were relative or specific to each intervention strategy (see our discussion earlier), theshort-term effects could be examined within the context of the preschool program with all

3 Intervention strategies were not implemented in any coordinated manner across the sites (precisely acrossthe projects). Again, these were targeted strategies specific to certain needs of children in each project. Evenwhen children received multiple strategies, there was not any coordination to supplement one strategy withanother in a pre-designed manner. In other words, multiple strategies, when implemented together, were largelyadministered independently.

123

Effects of preschool intervention strategies

intervention strategies used as a block (see our discussion earlier). Overall, multiple regres-sion analysis was appropriate to treat SPARK children and comparison children as a singlelevel. Such a regression model examined the short-term effects of intervention strategies on(pretest or fall) outcome measures with control over child characteristics.

3 Results

The goal of our data analysis was twofold. We aimed to examine the short-term effects ofintervention strategies on school readiness measures as a single-level analysis with a multipleregression model (with control over child characteristics). We also intended to examine thelong-term effects of intervention strategies on school readiness measures as a multilevelanalysis with a 3-level HLM model (with control over child characteristics at level 1, teacherquality at level 2, and data hierarchy at level 3). Again, intervention strategies were enteredinto data analysis (both regression and HLM) as a block so that the effects of one strategywere estimated with control over participation in other strategies. To save space, when wediscuss the effects (both short-term and long-term) of one strategy, we omit the expressionof control over participation in other strategies.

3.1 Pretest and posttest gains

Table 1 presents descriptive statistics on pretest and posttest scores in children’s schoolreadiness measures. Results clearly indicated that SPARK children gained little once theyentered kindergarten. Specifically, (positive) gains in social skills were trivial. A drop inthe problem composite indicated a decrease in behavior problems, but the drop was trivial.(Positive) gains in school readiness were also trivial. Gains in overall conceptual development(receptive total composite) were actually negative with a trivial decrease. The examinationof short-term and long-term effects of intervention strategies was conducted within such akindergarten context.

3.2 Short-term effects of intervention strategies

Table 2 presents the multiple regression results estimating the short-term effects of interven-tion strategies on four measures of children’s school readiness. We found no short-term effectsof any strategy on social skills (scores of social skills composite). There were statistically sig-nificant short-term effects of PAT on behavior problems. On the scale of problem composite,

Table 1 Descriptive statistics ofpretest and posttest measures

Measures M SD

Social skills compositePretest 103.34 11.94Posttest 105.13 11.45

Problem compositePretest 100.20 15.37Posttest 99.47 15.67

School readiness compositePretest 93.07 15.47Posttest 94.40 16.14

Receptive total compositePretest 90.71 14.26Posttest 89.30 18.08

123

X. Ma et al.

Table 2 Multiple regression results estimating effects of intervention strategies on school readiness measuresat exit of reschool program

Intervention strategies Social skillscomposite

Problemcomposite

School readinesscomposite

Receptive totalcomposite

Estimate SE Estimate SE Estimate SE Estimate SE

Learning advocate 2.88 2.29 3.21 3.20 3.47 2.95 4.68 2.67

Consultation to ECE setting 2.56 2.37 .10 3.41 2.28 3.03 4.53 2.69

Initial development screening −2.65 2.12 −2.49 2.97 .86 2.73 .01 2.51

SPARK developed learningplan

−.83 2.68 .94 3.79 −1.63 3.46 −6.74* 3.14

Home visit 3.51 2.30 −2.49 3.16 −4.27 3.03 −2.09 2.75

PAT −3.70 3.57 −10.68* 5.09 2.31 4.53 .71 4.16

Health screening −.52 2.06 3.77 2.85 2.25 2.71 3.41 2.41

Learning material −.34 2.01 −1.14 2.75 3.45 2.61 5.68* 2.37

Accreditation assistanceto ECE setting

−4.64 4.68 2.94 6.32 −14.63* 6.95 −5.00 6.11

Learning workshops andactivities

2.15 2.27 −.11 3.09 3.30 2.97 1.95 2.64

SE Standard error* p < .05

higher values indicated more behavior problems. Therefore, children whose parents took partin PAT demonstrated more behavior problems. Caution is needed not to oversimplify thisresult as indicating that PAT had negative effects. We believed that this result in fact indicatedthat for children with more behavior problems their parents were actively involved with PAT.Therefore, behavior problems were actively “tackled” by the strategy of PAT.

We interpreted the negative short-term effects of accreditation assistance to ECE settingon school readiness (scores of school readiness composite) in the same manner (poor schoolreadiness was statistically significantly related to accreditation assistance to ECE setting).When an ECE setting faces challenges for accreditation, it is often because many childrenfail to be ready for school. Accreditation assistance to such an ECE setting almost alwaysincludes effort to better prepare children for school. Therefore, poor school readiness wasactively tackled by the strategy of accreditation assistance to ECE setting.

Similarly, in terms of overall conceptual development (scores of receptive total composite),the adoption of SPARK developed learning plan actively targeted poor overall conceptualdevelopment. Such was the interpretation for the statistically significant, negative short-term effects of SPARK developed learning plan on overall conceptual development. We alsofound a statistically significant, positive short-term effects of learning materials on overallconceptual development. In this case, children who were given more learning materials(to learn) demonstrated better overall conceptual development.

3.3 Long-term effects of intervention strategies

Table 3 presents the HLM results that estimated the long-term effects of intervention strate-gies on children’s gains in four measures of school readiness (social skills composite, problemcomposite, school readiness composite, and receptive total composite) during a period of 9months in kindergarten. Across the four measures, none of intervention strategies demon-strated any statistically significant long-term effects. Therefore, after control over children

123

Effects of preschool intervention strategies

Table 3 HLM results estimating effects of intervention strategies on gains in school readiness measures inkindergarten

Intervention strategies Social skillscomposite

Problemcomposite

School readinesscomposite

Receptive totalcomposite

Estimate SE Estimate SE Estimate SE Estimate SE

Learning advocate 1.91 1.84 .51 2.37 .91 2.69 1.07 2.35

Consultation to ECE setting 1.43 1.95 .41 2.59 2.66 2.84 .89 2.48

Initial development screening −1.64 1.73 −1.79 2.20 −1.19 2.37 −2.95 2.07

SPARK developed learningplan

1.79 2.15 −5.63 2.83 −.30 3.20 1.94 2.87

Home visit −2.52 1.82 2.27 2.33 .74 2.44 −1.42 2.13

PAT −1.82 2.92 −2.93 3.90 −4.60 3.74 −2.71 3.34

Health screening −1.71 1.65 2.29 2.12 1.54 2.25 1.65 1.95

Learning material .38 1.61 3.07 2.05 1.07 2.15 .28 1.91

Accreditation assistance toECE setting

−.35 3.68 −.81 4.64 10.11 6.09 6.50 5.20

Learning workshops andactivities

−.77 1.81 .73 2.29 −3.92 2.46 −2.85 2.13

SE Standard error* p < .05

characteristics at the child level and teacher quality (kindergarten instruction) at the teacherlevel, none of intervention strategies contributed statistically significantly to children’s gainsin school readiness measures during kindergarten.

4 Discussion

It is not uncommon that data from granting agencies (foundations) tend to be complex to workwith. SPARK is certainly such a case. We discussed earlier in detail on the complexity of theSPARK data (and our coping strategies). So, we responded reasonably and adequately to thekey methodological challenges. The challenges that we are facing now are how to make senseof the analytical results. The key issue is how much credit intervention strategies can claim inthe overall effort to make children ready for school. In more technical language, it is how tomake sense of the role or contribution of intervention strategies to measures of school readi-ness in kindergarten. To adequately address this key issue, we use many pieces of statisticalestimates together, including both descriptive and inferential statistics, in order not to over-look details that may hold the key to a reasonable interpretation. We repeat that any discussionon a particular intervention strategy has control over participation in other strategies.

4.1 Making sense of short-term effects

Obviously, the short-term effects of intervention strategies do not need to contend with thecredit issue because as we mentioned earlier the pretest could be considered the exit perfor-mance of children in the preschool program. Nevertheless, the short-term effects scatteredin isolation across school readiness measures. The negative effects of PAT and accreditationassistance to ECE setting can be considered together, because both intervention strategieswere implemented in a very limited manner. PAT was administered to only 4 % of children in

123

X. Ma et al.

the sample, and accreditation assistance to ECE setting only 2 % (see Appendix 2). Becausewe deal with targeted intervention strategies, we have identified a very small portion ofchildren whose behavior problems were disproportional and a very small portion of chil-dren who attended ECE settings historically struggling with preparing children for school.Unfortunately, in their current level of intervention, PAT was unable to correct children withdisproportional behavior problems to the (average) degree of behavior problems of otherchildren and accreditation assistance to ECE setting was unable to improve school readinessof children attending struggling ECE settings to the (average) degree of school readiness ofother children, even though both strategies seem to be rightfully on target to improve schoolreadiness (see Carlton and Winsler 1999). We suggest that PAT and accreditation assistanceto ECE setting need to increase their level of intervention.

The other negative short-term effects were associated with SPARK developed learningplan on overall conceptual development (receptive total composite). This intervention strat-egy was implemented to about one in five children in the sample. If both PAT and accreditationassistance to ECE setting targeted a small portion of “extreme” children, then SPARK devel-oped learning plan targeted a common issue among children. Again, it seems to be wise to useSPARK developed learning plan to create developmentally appropriate learning experiences(Rodriguez and Tamis-LeMonda 2011); unfortunately, in its current level of intervention, itwas unable to close the gap in overall conceptual development between children of concernand children of less concern (children without the need to implement SPARK developedlearning plan). We believe that the level of intervention is the issue again.

SPARK does seem to have the correct intervention strategies for children of concern.However, these negative effects indicated that the correct strategies were not able to closethe gap of concern (i.e., disproportional behavior problems, poor school readiness, and inad-equate overall conceptual development). Whether an intervention strategy targets extremeor common problems of children, we expect that intensity and duration of implementationmake a difference (Ramey and Ramey 2000). To increase the level of intervention, one ofthem or even both of them need to be emphasized. To address duration, these interventionstrategies may need to start earlier than preschool (which is the year before kindergarten).When this is not possible, intensity may need to increase. If we accept the proposition thateducational change takes time in ECE (see Ma et al. 2011), a long-term intervention planneeds to be in order.

There might be some unsung “heroes”—intervention strategies that closed the gap inchildren’s school readiness. Were entrance (into the preschool program) measures of schoolreadiness available, we would be able to identify these heroes. All strategies that showed nostatistically significant effects on the exit measures of school readiness are candidates becausechildren with each strategy performed as well as comparison children (without that strategy).Unfortunately, several intervention strategies advocated in the research literature can only beclassified as candidates such as learning advocate and health screening (e.g., Currie 2005;Odom 2000). Home visits are presented as an effective service model in the literature (e.g.,McLoyd 1990; Parker et al. 1999) but indicated no statistically significant results on anyschool readiness measure in the present analysis. Criticism of developmental screening tests(e.g., Bordignon and Lam 2004; Gredler 1997) may explain our lack of statistically significantresults associated with initial developmental screening.

There is an evident lack of “super heroes”—intervention strategies that make childrenwith them perform better than comparison children. We identified only one super hero—learning materials. In line with the research literature (e.g., Rodriguez and Tamis-LeMonda2011), children to whom this intervention strategy was implemented developed better overallconceptual understanding than comparison children.

123

Effects of preschool intervention strategies

In sum, the short-term effects of SPARK intervention strategies bring marginal news toshare: very few super heroes (strategies) that really increased children’s school readiness,possibly some unsung heroes that closed the gap (only) in school readiness among children,and strategies that were of correct types to implement but failed (due likely to the level ofimplementation) to even close the gap in school readiness among children. All these highlightthe intervention strategy of learning materials (the only super hero) that effectively excelledchildren in overall conceptual development.

4.2 Comments on SPARK’s theory of change

We have found no convincing evidence to support SPARK’s theory of change that emphasizesstrong partnerships among families, providers, communities, and schools (Berkley 2003). Wenote that SPARK’s theory of change was operationalized through the development and imple-mentation of individually targeted intervention strategies. Although this approach indeedrequires strong partnerships, we argue that it is not effective in operationalizing SPARK’stheory of change. With data at hand not equipped to address why, one can only suggest edu-cated speculations. We question whether targeting may function as labeling. Being selected(identified) as targets for a special intervention may to a large extent label children and familiesas abnormal. The resulting negative labeling effects may compromise the effectiveness of anyindividually targeted intervention strategy. Perhaps, any intervention strategy ought to be pro-vided to all children while a varying degree of intervention can be applied to a specific group ofchildren. Another speculation of ours is that the above operationalization does not adequatelyreflect the comprehensive approach of whole child intervention (see Cooper et al. 2009).

4.3 Making sense of long-term effects

It is tempting to notice the lack of any long-term effects of intervention strategies on measuresof school readiness (see Table 3) and then quickly jump to the conclusion that interventionstrategies have no impact on gains in school readiness measures. However, a close look at thedescriptive statistics of pretest and posttest measures shows little gains across all measuresof school readiness (see Table 1). Therefore, neither (9 months of) kindergarten instructionnor intervention strategies implemented in the preschool program had any impact on gainsin school readiness measures. SPARK made all the effort to make children ready for schoolduring preschool. It is, therefore, reasonable to claim that the pretest measures contained allSPARK effort to make children ready for school. However, SPARK children simply stoppedexactly at pretest (or exit of the preschool program). In other words, SPARK children simplydid not gain in school readiness measures in kindergarten. Therefore, it comes to us with nosurprise that there were no long-term effects of any intervention strategy.

Intervention strategies implemented at the preschool level are simply not enough to keepchildren growing in kindergarten. It is our proposition that these intervention strategies canonly assist kindergarten instruction to become more effective. But when kindergarten instruc-tion has no effects, their assisting role also ceases. We believe that, to some extent, SPARKintervention strategies become a “victim” of inadequate kindergarten instruction. We arguethat high-quality kindergarten instruction must go hand in hand with preschool interventionstrategies. After all, one cannot reasonably expect SPARK intervention strategies imple-mented during preschool (ended before kindergarten) to continue to function to promotegrowth during kindergarten. Stated differently, high-quality kindergarten instruction musttake over the center stage and become the main driving force for excellence (e.g., West et al.2000). Kindergarten instruction can and should take advantage of well-designed preschool

123

X. Ma et al.

intervention strategies to promote even better growth. Obviously, kindergarten instructionfailed to do so in the SPARK case.

4.4 Implications for educational practice and research

The research literature emphasizes a comprehensive set of intervention strategies that start atbirth to systematically address families’ needs with a consistent treatment schedule throughage 5 (e.g., Cooper et al. 2009). The present analysis examined whether it was still possible forindividual intervention strategies administered to children in an independent manner to haveany important impact. The results indicated that it was possible for intervention strategiesimplemented in a preschool program to close the gap in school readiness among children andeven excel children in the intervention condition above other children. Therefore, we concludethat it is still possible for a particular intervention strategy (allowing participation in other tar-geted strategies) to impact children’s readiness for school (at the exit of a preschool program).

We particularly suggest that the use of carefully-chosen, well-designed learning materialsholds a great potential to promote children’s school readiness (see also Connell and Prinz2002). The present analysis indicated that this strategy helped children of concern greatlywith their overall conceptual development (their logical reasoning). We suggest that SPARKmakes those learning materials available with careful documentations to promote population-appropriate, well-implemented applications in similar ECE settings (e.g., Rodriguez andTamis-LeMonda 2011).

We identified intervention strategies that were of correct types to implement but failedto close the gap in school readiness among children. We believe that this failure is largelydue to the level of implementation. In line with Ramey and Ramey (2000), we suggest thatall intervention strategies that we have examined in the present analysis (except learningmaterials) need to increase their level of intervention (either duration or intensity or both) tochallenge children enough to be ready for school.

Unfortunately, once children enter kindergarten, it is highly unlikely for a particular inter-vention strategy (implemented in preschool) to continue to create desirable impact on learn-ing. To continue to improve school readiness, we suggest a dual-action approach. Althoughthe present analysis was not equipped to test the comprehensive approach as documented inthe research literature (e.g., Casto and Lewis 1984), we did find that individually targetedintervention strategies had no impact on learning in kindergarten (once allowing participa-tion in other targeted strategies). So, on one hand, a set of well-coordinated interventionstrategies need to work together to overcome the inadequacy of isolated (though targeted)strategies (Casto and Lewis 1984). On the other hand, there must be efforts to improvekindergarten instruction (e.g., West et al. 2000). Kindergartens cannot expect interventionstrategies implemented in preschool to continue to function in kindergarten learning. As wehave argued, kindergarten instruction must take over the center stage of learning and becomethe main driving force for excellence. To some extent, we suggest an extension of well-designed preschool intervention strategies into kindergarten with appropriate adjustment totarget kindergarten children.

The present analysis was a product of a (very beneficial) struggle in research design.We strived to build an experiment-like environment with some ideas from survey studyto function as control. Obviously, an experimental design with randomization if possibleis the only way to unfold the effects of individual intervention strategies or to discern therelative importance of collaborating intervention strategies. We strongly suggest that grantingagencies and future researchers take a true experimental approach to implement interventionstrategies for educational practice and research. It is totally possible to promote community

123

Effects of preschool intervention strategies

wellness and sound research at the same time, with the results of educational practice andresearch far exceeding the investment of valuable resources.

Appendix 1

See Table 4.

Table 4 Descriptive statistics onchildren and teachers

M SD

Child characteristics

Boy .50 .50

Age 5.41 .41

Race-ethnicity

White .25 .43

Black .31 .46

Hispanic .23 .42

Others .21 .41

Socioeconomic status (government financial aid) .59 .49

Family structure

Number of adults living in a family 2.02 .94

Number of children living in a family 2.72 1.27

Special need status .08 .28

IEP status .05 .21

Teacher characteristics

Graduate degree .35 .48

Certified .92 .27

Appendix 2

See Table 5.

Table 5 Descriptive statistics ofparticipation in interventionstrategies

Intervention strategies M SD

Learning advocate .31 .46

Consultation to ECE setting .11 .32

Initial development screening .47 .50

SPARK developed learning plan .22 .42

Home visit .30 .46

PAT .04 .21

Health screening .38 .49

Learning material .52 .50

Accreditation assistance to ECE setting .02 .14

Preschool learning workshops and activities .20 .40

123

X. Ma et al.

References

Ackerman, D. J., & Barnett, W. S. (2005). Prepared for kindergarten: What does readiness mean?. Brunswick,NJ: National Institute for Early Education Research.

Barona, A. (1991). Assessment of multicultural preschool children. In B. A. Bracken (Ed.), The psychoedu-cational assessment of young children (pp. 379–391). Needham, MA: Allyn & Bacon.

Bear, G. G., & Modlin, P. D. (1987). Gesell’s developmental testing: What purpose does it serve? Psychologyin the Schools, 72, 712–7221.

Berkley, A. (2003). The SPARK initiative’s theory of change. Retrieved September 28, 2011 from http://www.smartstartga.org/_downloads/30924_Berkley_Spark_TheoryOfChange.pdf.

Blair, K. C., Umbreit, J., & Bos, C. S. (1999). Using functional assessment and children’s preferences toimprove the behavior of young children with behavioral disorders. Behavioral Disorders, 24, 151–166.

Bordignon, C. M., & Lam, T. C. M. (2004). The early assessment conundrum: Lessons form the past, impli-cations for the future. Psychology in the Schools, 41, 737–749.

Bracken, B. (2006). Bracken Basic Concept Scale (3rd ed.). San Antonio, TX: Pearson Education.Bruer, J. T. (1999). The myth of the first three years: A new understanding of early brain development and

lifelong learning. New York: Free.Burchinal, M. R., Campbell, F. A., Bryant, D. M., Wasik, B. H., & Ramey, C. T. (1997). Early intervention

and mediating processes in cognitive performance of children of low-income African-American families.Child Development, 68, 935–954.

Buysse, V., & Bailey, D. B. (1993). Behavioral and developmental outcomes in young children with disabilitiesin integrated and segregated settings: A review of comparative studies. The Journal of Special Education,26, 434–461.

Campbell, S. B., & Ewing, L. J. (1990). Hard-to-manage preschoolers: Adjustment at age nine and predictorsof continuing symptoms. Journal of Child Psychiatry, 31, 871–889.

Carlton, M. P., & Winsler, A. (1999). School readiness: The need for a paradigm shift. School PsychologyReview, 28, 338–352.

Casto, G., & Lewis, A. (1984). Parent involvement in infant and preschool programs. Division of EarlyChildhood, 9, 49–56.

Connell, C. M., & Prinz, R. J. (2002). The impact of childcare and parent–child interactions on school readinessand social skills development for low-income African-American children. Journal of School Psychology,40, 177–193.

Cooper, J. L., Masi, R., & Vick, J. (2009). Social-emotional development in early childhood. New York:National Center for Children in Poverty, Columbia University.

Costenbader, V., Rohrer, A. M., & DiFonzo, N. (2000). Kindergarten screening: A survey of current practice.Psychology in the Schools, 37, 323–332.

Currie, J. M. (2005). Health disparities and gaps in school readiness. The Future of Children, 15, 117–138.Ellwein, M. C., Walsh, D. J., Eads, G. M., & Miller, A. (1991). Using readiness tests to route kindergarten

students: The snared intersection of psychometrics, policy, and practice. Educational Evaluation and PolicyAnalysis, 13, 159–175.

Farran, D. C. (1990). Effects of intervention with disadvantaged and disabled children: A decade review. In S.J. Meisels & J. P. Shonkoff (Eds.), Handbook of early intervention (pp. 501–539). New York: CambridgeUniversity Press.

Fox, L., Dunlap, G., & Cushing, L. (2002). Early intervention, positive behavior support and transition toschool. Journal of Emotional and Behavioral Disorders, 10, 149–157.

Gredler, G. R. (1997). Issues in early childhood screening and assessment. Psychology in the Schools, 34,99–106.

Greeley, S., & Greeley, B. (2011). Beyond the grant: How the W. K. Kellogg Foundation went beyond grant-making to contribute to a major early childhood initiative. The Foundation Review, 2, 79–93.

Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young American children.Baltimore, MD: Paul H. Brookes.

Kontos, S. J. (1991). Child care quality, family background, and children’s development. Early ChildhoodResearch Quarterly, 6, 249–262.

LaParo, K. M., & Pianta, R. C. (2000). Predicting children’s competence in the early school years: A meta-analytic review. Review of Educational Research, 70, 443–484.

Loeb, S., Fuller, B., Kagan, S. L., & Carrol, B. (2004). Child care in poor communities: Early learning effectsof type, quality, and stability. Child Development, 75, 47–65.

Ma, X., Shen, J., Kavanaugh, A., Lu, X., Brandi, K., Goodman, J., et al. (2011). Effects of quality improvementsystem on child care centers. Journal of Research in Childhood Education, 25, 399–414.

Merrell, K. W. (2003). Preschool and kindergarten behavior scales (2nd ed.). Austin, TX: PRO-ED.

123

Effects of preschool intervention strategies

McLoyd, V. C. (1990). The impact of economic hardship on black families and children: Psychological distress,parenting, and socioeconomic development. Child Development, 61, 311–346.

Meisels, S. J. (1987). Uses and abuses of developmental screening and school readiness testing. Young Children,42, 4–6.

Odom, S. L. (2000). Preschool inclusion: What we know and where we go from here. Topics in Early ChildhoodSpecial Education, 20, 20–27.

Olds, D. L., Henderson, C. R., Kitzman, H. J., Eckenrode, J. J., Cole, R. E., & Tatelbaum, R. C. (1999).Prenatal and infancy home visitation by nurses: Recent findings. The Future of Children, 9, 44–65.

Olds, D., & Kitzman, H. (1990). Can home visitation improve the health of women and children at environ-mental risk? Pediatrics, 86, 108–116.

Parker, F. L., Boak, A. Y., Griffin, K. W., Ripple, C., & Peay, L. (1999). Parent–child relationship, homelearning environment and school readiness. School Psychology Review, 28, 413–425.

Peisner-Feinberg, E. S., & Burchinal, M. R. (1997). Relations between preschool children’s child-care expe-riences and concurrent development: The cost, quality, and outcomes study. Merrill-Palmer Quarterly, 43,451–477.

Pianta, R. C., Nimetz, S. L., & Bennett, E. (1997). Mother–child relationships, teacher–child relationships,and school outcomes in preschool and kindergarten. Early Childhood Research Quarterly, 12, 263–280.

Powell, D. (1990). Home visiting in the early years: Policy and program design decisions. Young Children,45, 65–73.

Ramey, C. T., Yeates, K. O., & Short, E. J. (1984). The plasticity of intellectual development: Insights frompreventative intervention. Child Development, 55, 1913–1925.

Ramey, C. T., & Ramey, S. L. (1998). Early intervention and early experience. American Psychologist, 53,109–120.

Ramey, S. L., & Ramey, C. T. (2000). Early childhood experiences and developmental competence. In J.Waldfogel & S. Danziger (Eds.), Securing the future: Investing in children from birth to college (pp.122–150). New York: Sage Foundation.

Ramey, C. T., & Ramey, S. L. (2004). Early learning and school readiness: Can early intervention make adifference? Merrill-Palmer Quarterly, 50, 471–491.

Rodriguez, E. T., & Tamis-LeMonda, C. S. (2011). Trajectories of the home learning environment acrossthe first 5 years: Associations with children’s vocabulary and literacy skills at pre-kindergarten. ChildDevelopment, 82, 1058–1075.

Rosenberg, S., Zhang, D., & Robinson, C. (2008). Prevalence of developmental delays and participation inearly intervention services for young children. Pediatrics, 121, 1503–1509.

Sadowski, M. (2006). What does effective PreK teaching look like? Retrieved September 28, 2011 from http://www.hepg.org/hel/printarticle/447.

Shonkoff, J. P., & Phillips, D. C. (Eds.). (2000). From neurons to neighborhoods: The science of early childhooddevelopment. Washington, DC: National Academy Press.

Shore, R. (1997). Rethinking the brain: New insights into early development. New York: Families and WorkInstitute.

Teddlie, C., & Reynolds, D. (Eds.). (2000). The international handbook of school effectiveness research.London: Falmer.

Weiss, H. B. (1993). Home visits: Necessary but not sufficient. The Future of Children, 3, 113–128.West, J., Denton, K., & Reaney, L. M. (2000). The kindergarten year: Findings from the Early Childhood

Longitudinal Study Kindergarten Class of 1998–99. Washington, DC: National Center for Education Sta-tistics.

Zaslow, M. J., Oldham, E., Moore, K. A., & Magenheim, E. (1998). Welfare families’ use of early childhoodcare and education programs, and implications for their children’s development. Early Childhood ResearchQuarterly, 13, 535–563.

Zigler, E., & Muenchow, S. (1992). Head Start: The inside story of America’s most successful educationalexperiment. New York: Basic Book.

123