do extra curricular activities in schools improve
TRANSCRIPT
Do extra-curricular activities in schools improveeducational outcomes? A critical reviewand meta-analysis of the literature
Boaz Shulruf
Published online: 2 February 2011
� Springer Science+Business Media B.V. 2011
Abstract Secondary schools tend to sponsor a large number of extra-curricular
activities (ECA) yet little is known about their contribution to students’ educational
outcomes. This meta-analysis aims to determine what it is about ECA participation
that supports positive educational outcomes. Furthermore, this study challenges the
theoretical assumptions about the benefits of participation in ECA. 29 studies (all
except for one based on data collected in the United States) met the search criteria
for inclusion in the analysis. Most effect sizes on academic achievements yielded
from non-specific ECA, academic clubs and journalism were small, as were par-
ticipation in performing arts, sports and leadership activities on a range of educa-
tional outcomes. Although the results show associations between participation in
ECA and educational outcomes, causal effects could not be confirmed. It is con-
cluded that the lack of evidence supporting the causal effects, and thus the common
theoretical assumptions about the effects of ECA on educational outcomes, is due to
methodology limitations in these studies.
Keywords Extra-curricular activities � Systematic review � Secondary schools �Academic achievements
Resume Les activites periscolaires des ecoles ameliorent-elles les resultats des
eleves ? Analyse critique et meta-analyse de la documentation – Les etablissements
secondaires ont tendance a organiser un grand nombre d’activites periscolaires, mais
la contribution de ces dernieres aux resultats scolaires des eleves est faiblement
connue. Cette meta-analyse avait pour but de cerner dans quelle mesure la partic-
ipation a ces activites qui favorise des resultats positifs. D’autre part, cette etude
remet en question les hypotheses theoriques emises sur les benefices de cette
B. Shulruf (&)
Centre of Medical and Health Sciences Education, Faculty of Medical and Health Sciences,
University of Auckland, Auckland, New Zealand
e-mail: [email protected]
123
Int Rev Educ (2010) 56:591–612
DOI 10.1007/s11159-010-9180-x
participation. Vingt-neuf etudes (sauf une fondee sur des donnees collectees aux
Etats-Unis) repondaient au critere de recherche pour etre incluses dans l’analyse. La
plupart des tailles d’effets sur les resultats scolaires retires des activites periscolaires
non specifiques, par les clubs et le journalisme scolaires sont faibles, il en est de
meme pour les arts du spectacle, les activites sportives et celles favorisant l’esprit
d’initiative. Bien que les conclusions de l’analyse etablissent des liens entre par-
ticipation aux activites periscolaires et resultats scolaires, un lien de cause a effet
n’a pu etre demontre. Nous en deduisons que le manque de donnees probantes
etayant ce lien de cause a effet, et donc les hypotheses theoriques courantes sur
l’impact de ces activites sur les resultats scolaires, est du aux limites method-
ologiques de ces etudes.
Zusammenfassung Werden schulische Leistungen durch außercurriculare Akti-
vitaten verbessert? Eine kritische Uberprufung und Metaanalyse der Literatur – Les
activites periscolaires des ecoles ameliorent-elles les resultats des eleves ? Analyse
critique et meta-analyse de la documentation - Viele Sekundarschulen fordern eine
große Anzahl außercurricularer Aktivitaten, uber deren Nutzen fur die schulischen
Leistungen der Schulerinnen und Schuler allerdings wenig bekannt ist. Diese
Metaanalyse soll daruber Aufschluss geben, inwiefern sich die Teilnahme an au-
ßercurricularen Aktivitaten gunstig auf die schulischen Leistungen auswirkt. Zudem
stellt diese Studie die theoretischen Annahmen uber die Vorteile der Beteiligung an
außercurricularen Aktivitaten auf den Prufstand. Die Kriterien fur die Aufnahme in
die Analyse wurden von 29 Studien (alle außer einer auf der Grundlage von Da-
tenerhebungen in den Vereinigten Staaten) erfullt. Unspezifische außercurriculare
Aktivitaten, Wissenschafts-AGs und journalistische Tatigkeiten hatten meist nur
geringe Effekte auf den Schulerfolg. Auch die Beteiligung an Theater-AGs oder
sportlichen Aktivitaten sowie die Ubernahme von Fuhrungsaufgaben wirkten sich
wenig auf bestimmte schulische Leistungen aus. Aus den Ergebnissen lassen sich
zwar gewisse Korrelationen zwischen der Teilnahme an außercurricularen Akti-
vitaten und schulischen Leistungen ablesen, Kausalzusammenhange konnten jedoch
nicht nachgewiesen werden. Die Analyse kommt zu dem Schluss, dass die
mangelnde Nachweisbarkeit kausaler Zusammenhange und somit der gemeinsamen
theoretischen Annahmen uber die Effekte außercurricularer Aktivitaten auf die
schulischen Leistungen der eingeschrankten Methodik dieser Studien geschuldet ist.
Resumen >Mejoran las actividades extracurriculares en las escuelas los resultados
educativos? Revision crıtica y meta-analisis de la literatura – Las escuelas secun-
darias tienden a promover un gran numero de actividades extracurriculares (ECA,
por sus siglas en ingles). Sin embargo, es poco lo que se sabe sobre su aporte a los
resultados educativos de los estudiantes. Este meta-analisis apunta a determinar que
es lo que sostiene los resultados educativos positivos de una participacion en ECA.
Ademas, con este estudio se ponen en tela de juicio las suposiciones teoricas en
cuanto a los beneficios de una participacion en ECA. 29 estudios (todos ellos, con
excepcion de uno, basados en datos recogidos en los Estados Unidos) reunieron los
criterios de busqueda para ser incluidos en el analisis. La mayorıa de los tamanos
del efecto relacionados con logros academicos obtenidos a partir de ECA no
592 B. Shulruf
123
especıficos, clubes academicos y periodismo eran pequenos, igual que la partici-
pacion en artes dramaticas, deportes o actividades de liderazgo sobre una serie de
resultados educativos. Si bien los resultados muestran relaciones entre una partic-
ipacion en ECA y resultados educativos, no se han podido confirmar efectos cau-
sales. Se concluye que la falta de pruebas que sostienen los efectos causales, y con
ello las suposiciones teoricas generales sobre los efectos de ECA sobre resultados
educativos, se debe a las limitaciones metodologicas de estos estudios.
Student performance at secondary school is the main determinant for enrolment in
tertiary education. As part of their school day, many secondary school students take
part in a wide variety of activities, over and above the core curricular teaching and
learning classroom work, which are usually referred to as co-curricular or extra-
curricular activities (ECA). In addressing this demand, whether from students or
other stakeholders, secondary schools allocate considerable resources to the extra
curriculum. Nonetheless, despite a plethora of research studies on the subject, no
conclusive evidence can be drawn from the literature indicating the actual benefit
students gain by participating in ECA. This critical meta-analysis of the literature,
therefore, looks at the benefits and outcomes for students of their participation in
ECA, sponsored by secondary-level schools. It is noted, however, that most of the
literature in this area has been generated from data collected in the United States,
making this paper particularly relevant to the US context.
The aims of this paper are to examine the literature on ECA participation and the
outcomes of such participation, and to attempt to identify the extent to which
Do extra-curricular activities in schools improve educational outcomes? 593
123
participation in ECA affects educational outcomes. This paper not only investigates
the effect sizes of participation in ECA on a range of educational outcomes but also
looks at the validity of the measures used in previous studies, with particular
reference to evidence for causal effects.
Theoretical framework
Extra-curricular activities are an integral component of school life. While
participation rates are known to be very high in the United States (in 1995
approximately 60% of high school sophomores and 70% of seniors participated in at
least one activity; Cooper et al. 1999), there is very limited documentation on the
extent of participation elsewhere.
Investigations into the impact of ECA on students date back as early as the 1930s
with studies documenting the range of activities being offered in schools and
questioning whether participation in certain high school activities could be related
to higher achievement at college (Baxter 1936; Holland 1933). However, despite a
raft of studies on ECA participation being undertaken over nearly eight decades
(mainly in the United States), little is known or understood about the causal effect
between participation in such activities and educational outcomes.
In 1987, Holland and Andre carried out a review of literature relating to extra-
curricular participation and adolescent development with the aim of providing a
critique of methodological approaches and possible directions for future research
(Holland and Andre 1987). However, while this review was relatively comprehen-
sive, it was said to lack the evidence needed to support the conclusions drawn and,
according to Taylor and Chiogioji (1988), it had no theoretical framework or
definitions of its key terms. Notwithstanding, this study did raise the profile of extra-
curricular activity and exposed it as an area which needed more research.
A number of studies, particularly in the last decade, has examined ECA within a
school-sponsored context (e.g. Broh 2002; Davalos et al. 1999; McNeal 1995;
Silliker and Quirk 1997), and many others have taken broader views in terms of
‘‘extra-curricular’’ definition and have included non-school-sponsored activity, such
as community and church-based activities (Eccles and Barber 1999; Gerber 1996;
Lewis 2004); organised and non-organised activities for high school students
(Chambers and Schreiber 2004); as well as social behaviours and high-risk
activities, such as smoking, alcohol consumption and drug taking (Eccles and
Barber 1999; Eccles et al. 2003). For the purposes of this study, ECA have been
defined as those which are school-sponsored and external to the core curriculum. It
is acknowledged that this definition may not be the same as that used by other
authors.
A recent and comprehensive review of literature on ECA for high school students
in the USA (Feldman and Matjasko 2005) found that while ECA are viewed as
being highly important ‘‘developmental settings for adolescents’’, little is under-
stood about the ‘‘contextual influences’’ affecting that development, or the
relationship between participation and outcomes (pp. 160–161). Studies such as
Lewis’ (2004) meta-analysis of extra-curricular participation conclude that the best
594 B. Shulruf
123
academic and social outcomes for students are gained through well-designed,
developmentally appropriate activities but are unable to describe or pinpoint the
particular characteristics contributing to these outcomes. Furthermore, there is very
limited evidence to support the commonly held justification for carrying out ECA.
Extra-curricular studies and previous methodological approaches
Extra-curricular studies examine a multitude of activities. Some authors take a
broad-based approach and examine participation across a range of activities, both
school-sponsored and outside-of-school-time activities; while others nominate and
examine particular extra-curricular areas, such as participation in athletics or in
academically related activities.
Studies which only focus on school-sponsored activities tend to be the exception
rather than the rule. However, in the past decade there has been a move towards
examining ECA within school programmes and their impact on student learning
outcomes. McNeal (1995), for example, examined student involvement in ‘formal’
extra-curricular school activities and their impact on student retention, and Broh
(2002) studied the educational impact of participation in both sport-related and non-
sport-related school-sponsored activities.
Many studies have set out to examine how participation in such activities is
beneficial for students (for example Guest and Schneider 2003). Melnick et al.
(1992) examined the educational effects of interscholastic participation on African-
American and Hispanic boys and girls. They found that while high school athletic
participation was for many a means of being included in social youth groups, it was
not necessarily related to grades and standardised test scores. Another found a
strong link between participation in athletics and the likelihood of students dropping
out of school (Mahoney and Cairns 1997; McNeal 1995); and still more have found
that participation in sports is linked to improved school attendance, academic
outcomes, social relationships and self-esteem (Marsh and Kleitman 2003;
McCarthy 2000; Silliker and Quirk 1997).
The role which participation in ECA plays in educational attainment and cultural
capital gain is also highlighted by the literature (Buoye 2004; Power 1999; Thomas
and Moran 1991) as well as the role which cultural capital plays in attaining higher
academic ranking (Appelman et al. 2003). Group membership through participation
in activities is seen as providing access to the relationships and networks that
influence and support positive outcomes for students, as well as improving the
opportunities to access knowledge and skills which support higher social and/or
cultural status.
Other studies have examined areas such as the association between ECA and
improved chances for college attendance (Kaufman and Gabler 2004; Mahoney
et al. 2003); as well as mentoring and its impact on academic achievement and
dropout rates (Slicker and Palmer 1993). In the latter group, effective mentoring was
found to significantly support improved academic outcomes and retention, while
ineffective mentoring had no effect.
Many studies emphasise the impact of moderating factors such as gender,
ethnicity and socioeconomic status on the level and type of outcome achieved
Do extra-curricular activities in schools improve educational outcomes? 595
123
(Braddock II et al. 1991; Brown and Evans 2002; Diaz 2005; Gerber 1996; Jordan
1999; Lisella and Serwatka 1996). Davalos et al. (1999), for example, found that
both ethnicity and gender were important moderating factors on the impact of ECA
on school retention. On the other hand, in examining the association between ECA
and academic achievement, Valentine et al. (2002) stressed the importance of a
different range of moderating factors to account for why, how and for whom
achievement is supported. These factors included the characteristics of the materials
used in activities, group characteristics or distinctiveness, school size and cultural
issues.
This overview indicates that the research literature on the effects of participation
in ECA is broad. Aiming to identify the extent to which participation in ECA affects
educational outcomes such as attainment, grades, self-concept, aspirations for
higher education and enrolment in higher education institutions, this study focuses
mainly on the validity of the measures used in previous studies and on the strength
of the evidence suggesting causal effects of participation in ECA on these
educational outcomes.
Method
The systematic approach used to carry out this literature review on extra-curricular
activity in the secondary school context is different from the approach taken in
traditional descriptive or narrative reviews. While the purposes of a narrative review
and a systematic review are ostensibly similar – namely to examine the material
pertaining to a particular area – a narrative review does not utilise methods which
allow for the control of potential methodological biases occurring in the literature
(Boaz et al. 2002).
Systematic reviews of literature are carried out to agreed research standards with
the aim of supporting as objective an approach as possible and therefore minimising
bias. The agreed research standards include: using a protocol to guide and plan the
processes to be followed; focusing on a specific question; identifying as much of the
relevant literature as possible through a comprehensive search; making decisions
about the inclusion and exclusion of studies based on methodological criteria;
synthesising research findings and being explicit and transparent (Barr et al. 1999;
Boaz et al. 2002). To help reduce bias further, a collaborative team approach was
used for all decision-making (Kitchenham 2004). The collaborative team included
the author and another two scholars who are familiar with the literature on
systematic reviews and educational intervention in compulsory education systems.
Although systematic reviews, realist reviews, meta-analyses and best evidence
syntheses follow ‘‘highly specified and intentionally inflexible methodology with
the aim of assuring high reliability’’ (Pawson et al. 2004, p. 5), the methods
employed by each approach have attracted significant critique (Davies 2000;
Feinstein 1995; Shapiro 1997; Suri and Clarke 1999; Torgerson 2006). In the
extreme, Feinstein (1995) described these approaches (predominantly meta-
analyses) as ‘‘statistical alchemy of the twenty-first century’’. Although acknowl-
edging the discontent surrounding such strict methodologies, this review adheres to
596 B. Shulruf
123
the standards appropriate to meta-analyses and systematic reviews in terms of its
search and selection strategies and analysis methods in order to minimise possible
biases relating to study selection (Lipsey and Wilson 2001). At the same time,
however, it specifically looks for evidence of causality relating to the effect sizes
identified in the reviewed studies. This is an important contribution to the literature,
particularly since causality is a relatively under-researched topic in meta-analyses of
educational studies.
In summary, this study systematically reviews quantitative studies examining the
causal effects of participation in extra-curricular activity at secondary school level
on students’ educational outcomes in the areas of academic achievement, school
engagement and retention, future aspirations and student self-concept.
Search strategy
This study followed the recommendations of the ESRC UK Centre for Evidence-
Based Policy and Practice for conducting a systematic review search (Boaz et al.
2002). These recommendations include: performing a literature search which
includes both peer-reviewed and non-peer-reviewed literature and encompasses a
wide range of sources including large-scale and small-scale electronic databases,
grey literature such as practitioner journals, books, government reports; taking a
team approach to define keywords and develop exclusion and inclusion criteria and
compiling a list of search tools and other means used to search data bases and other
search sources.
A wide range of academic databases was searched: PsycInfo, ERIC, JSTOR,
Proquest, Proquest Social Science Journals, Expanded Academic, Web of Science,
Sociological Abstracts and Sports Discus. A search of Google Scholar was also
attempted; however, the high number of hits and difficulties in reducing these, even
with keyword definition, led to this source (Google Scholar) being excluded. In
addition, what is known as a ‘‘grey search’’ was carried out which included
evaluation reports and reviews conducted by research institutes (such as the Harvard
Family Research Project), as well as studies previously sourced by team members.
A search of both Campbell Collaboration and Cochrane Collaboration databases
was also conducted and revealed that no reviews have been carried out on the area
of ECA.
Keyword definition
In order to facilitate a comprehensive search of the literature, the review team drew
up a list of possible keywords which described the activity, the research terms used
in examining effectiveness of outcomes, some outcome terms, and some
preliminary exclusion criteria. Thirty-eight keywords were used to describe the
activity and included: ECA, out-of-school activities, non-academic school activities,
learning support and leisure/recreation activities. Research terms included: evalu-
ation, impact, effect, relationship and causal. Outcomes were defined as academic,
engagement and retention. The full list of keywords is shown in Table 1.
Do extra-curricular activities in schools improve educational outcomes? 597
123
Some studies were excluded during the search for the following reasons: (i) if
they reported on data stated to have been collected before 1985; (ii) if the involved
students were less than 13 years of age or over 19 years of age and (iii) if the reports
were not written in English.
The resulting comprehensive search identified 136 studies meeting the
preliminary criteria, although some flexibility was accepted regarding the age-
related exclusion criteria (for example, studies were accepted if they were
longitudinal and tracked students from junior high school through to high school).
Further inclusion and exclusion criteria were set up to assess the quality of studies
and the soundness of their methodology (see Table 2). Studies were then screened
using these additional criteria.
Search results
Out of the 88 studies which met the criteria for population, type of activity and
educational outcomes, 58 included quantitative data and were therefore selected for
further analysis. A final examination of these studies was carried out during the
quantitative data entry process, revealing that only 29 studies described their
methodology and findings in sufficient depth to be accepted for further analysis for
this review (see Table 3).
Each included study is highlighted (*) in the final reference list of this review.
Only one study was undertaken in the United Kingdom; the rest originated in the
United States. Hence, this study is particularly relevant to the US context.
Twenty-three of the studies based their findings on data from secondary sources,
which included national longitudinal studies such as the National Education
Longitudinal Study in 1988 (NELS:88) (NCES 2006) and High School and Beyond
(HSB) (Adelman 1995; Konstantopoulos 2005); and regional longitudinal studies
Table 1 Keyword definitions
Keyword activity definition
Extracurricular activities/engagement; Out-of-school activities; Outside-of-school/outside school;
Out-of-school learning activities; Extra class activities School club membership; School sports
Non-academic student activities; After-school experiences; After-school clubs; School-sponsored
activities; Non-school-sponsored activities; After school; physical activities; Adolescent activities;
Leisure/recreation activities; Adult sponsored structured activities; Extramural activities; Activity
involvement; Art & physical activities
School-based extra-curricular; Non-curricular studies/activities; Youth activities; Homework club;
Student engagement; Careers-related activities; Academic enrichment; Extended learning; Non-class
experience; Extra classroom factors; Leisure-time physical activity; Out-of-school time use; Leisure
time; Structured out-of-school time; SEA/structured extracurricular activities; Learning support;
Academic support; Creative arts; Mentoring/student support
Research terms
Evaluation; Impact; Effect; Outcome; Relationship; Significance; Causal; Indicator; Longitudinal;
Benefits; Comparison; Study; Analysis
Outcome terms
Academic/learning outcome; Future aspirations; Engagement/retention/school holding power;
Retention; Self-worth/Self-concept/self-esteem; (incl. development of leadership & responsibility)
598 B. Shulruf
123
such as the Carolina Longitudinal Study (Robinson and Etheridge 1995) and the
Michigan Study of Adolescent Life Transitions (MSALT) (Barber et al. 2001).
All 29 studies focused, to some extent, on school-sponsored ECA. 19 looked only
at school-sponsored activities; the remaining studies involved both school-
sponsored and non-school-sponsored (but similar) activities. In only four instances
did studies look at the impact of total ECA participation in both school-sponsored
and non-school sponsored activities; the other 25 studies examined the effect of a
range of ECA on a variety of educational outcomes. The ECA included academic,
sporting, leadership, performing and creative arts, vocational and other community
activities. The outcomes considered included academic achievement and aspiration
towards future education or training. School retention or holding power was a
further key outcome measured.
Studies were only included in the meta-analysis if the methodology and data used
enabled fair assessment of their results. Furthermore, only studies providing
statistical data which could be modified to effect sizes were included. It is
acknowledged that adherence to these strict inclusion criteria limits the breadth of
the review, and that it is possible that the omission of studies not meeting any of the
criteria might bias the meta-analysis (Feinstein 1995; Suri and Clarke 1999).
Nonetheless, since the focus of the current study is on the causal effects of ECA, it
was essential to establish, and adhere to, these restrictive selection criteria. It is
Table 2 Inclusion/exclusion criteria
Criteria Description
Study design Quantitative methods included controlled observational studies, studies with
qualitative description supported by other systematic data collection and studies
with qualitative perceptions only (teacher reports only were excluded)
Research measures Data-collection tools were standardised, supporting reliability and validity of data
Sampling Representative sampling and adequacy of sample size were included as criteria
Rigor of
measurement
Studies needed to be sufficiently transparent to allow judgments about the merits
of assessment and the adequacy of data on moderating factors such as ethnicity,
gender, type of activity, peer pressure, number of activities, fit between
participants and activity, or external factors. Transparency also enabled
examination of whether findings were substantiated by data
Data collection and
analysis
Clear, explicit description of data collection and analysis including data on
sampling methods and response rate
Type of activity Structured, school-sponsored/school-based; extra to core curriculum activities;
not employment, work-related
Type of outcome Academic/learning outcome; future aspirations; engagement/retention/school
holding power; self worth/self concept/self esteem (including development of
leadership/responsibility)
Academic/learning
outcome
Future aspirations; engagement/retention/school holding power; self worth/self
concept/self esteem (including development of leadership/responsibility)
Context Adequate description was essential with aims and objectives clearly described
Theoretical
underpinning
Clear and transparent theory was important but not essential
Note: Studies were included if they met the majority of the following criteria for both quality of meth-
odology and focus of the study
Do extra-curricular activities in schools improve educational outcomes? 599
123
suggested, however, that other meta-analyses looking at associations rather than
causal effects could take a much broader approach to their selection criteria.
Limitations
As previously noted, the majority of literature which discusses and investigates
ECA has been undertaken in the USA where such activities are largely externally
contracted rather than run by teachers working in the schools themselves.
Many of the studies are based on secondary data sources involving large national
data sets from longitudinal studies, such as the data from High School and Beyond
Study, and NELS (e.g. Broh 2002; Darling et al. 2005; Marsh 1992; McNeal 1995;
Melnick et al. 1992). The use of secondary data raises the issue of applicability,
since the original data sets were not developed with the purpose of testing the
impact of participation in ECA. As a result, these studies, while valuable, tend to
adopt a similar methodology, highlighting associations between participation and
outcomes with analysis around the pre-defined moderating variables measured in
the original study, such as peer relationships, race and gender. As these studies
rarely have the opportunity to deal with the original data, this presents a potentially
confounding factor in the accuracy of evidence. In addition, the large number of
studies using the same dataset (although utilising somewhat different measures and
sub-samples), has the potential to bias the outcomes of any pooled effect sizes.
A further limitation noted is associated with the common use of secondary data
sources where measures are defined for national or large-scale data collection
purposes. Many of the reviewed studies define educational outcomes in terms of
Table 3 Studies selection
Database Initial
hits
Included
in the first
stage
Included in the
second stage
(met the criteria
for population,
type of activity
and outcomes)
Selected for
quantitative analysis
(included quantitative
data allowing
calculation of effect
sizes)
PsychInfo 3729>>>>>>>>>>>>>>>>>>>>>>>>>=
>>>>>>>>>>>>>>>>>>>>>>>>>;
136 88 29
ERIC 141
JSTOR 7
Snowballing
(grey literature)
58
Expanded academic 390
Proquest Social Science
Journals
23
Proquest 13
Web of Science 24
Sociological abstracts
advanced search
19
Sport Discus 32
600 B. Shulruf
123
academic achievement focusing on Grade Point Averages (GPAs) and scores
(Branch 2003; Chambers and Schreiber 2004; Keith et al. 2004; Lipscomb 2007;
Marsh 1992; Silliker and Quirk 1997); however, Ainley (1994) commented that one
of the issues in studies examining school effectiveness is the emphasis on limited
ranges of achievement measures (i.e. achievement test scores in core curriculum
areas) as indicators of school quality. Ainley (of the Australian Council for
Educational Research), suggested that attention to other indicators, such as student
engagement, might have been more appropriate and might provide a better
understanding of what shapes student outcomes.
Although worthwhile in terms of contributing to the pool of information on the
effect of ECA on student outcomes, few of the studies were positioned within a
coherent theoretical framework. In their recent review of the literature on ECA in
high school, Feldman and Matjasko (2005) commented that ‘‘new methods for
measuring activities and applying an overarching theoretical framework’’ would
help to provide a clearer picture as to the causal mechanisms particular to ECA and
positive outcomes for secondary school students.
Analysis strategy
Statistical data were extracted from the 29 studies. Due to the very wide and diverse
range of activities examined in these studies, clustering was necessary for analysis
purposes (Lipsey and Wilson 2001). In this case, the ECA were clustered into eight
groups, as shown in Table 4.
Outcomes
The outcomes for participation in ECA used in the analysis were: academic
achievement (GPA, Maths, English/Reading and Science); retention or school
attendance; attitudes towards school; tertiary participation and aspirations to tertiary
education; and self-concept.
Table 4 Description of activity clusters
Activity name Description
Sports Any type of individual/team sports activities
Academic Club/
Journalism
Academic clubs that were subject oriented; journalism/yearbook
Performing Arts/
Cheerleading
Any type of performing arts e.g. dance, music, theatre, cheerleading etc.
Student Council Any type of formal student leadership activities
Vocational Club Any types of activities focusing on vocational skills
Mentoring Tutoring, mentoring
Non Sports Activities Activities that did not include sports yet without any specific details on the
type of activity
ECA General ECA activities with no details on the types of activities
Do extra-curricular activities in schools improve educational outcomes? 601
123
Calculating the effect sizes
Effect sizes were calculated from each study using formulae suggested by Lipsey
and Wilson (2001). The total effect sizes (Cohen’s d) for each outcome were
weighted by sample sizes.
Where findings did not meet the significance level of p \ 0.05, an effect size of 0
was imputed as it was assumed that such effects were due to random effects rather
than participation in ECA. This conservative approach was taken (only a few non-
significant effect sizes were negative) for three reasons: (i) most effect sizes
reporting a significance level of p [ 0.05 were small; (ii) non-significant results
(with or without data on the effect size) were reported in studies with a large number
of participants (n [ 300), so the sample size was likely to have little effect on the
significance level; and (iii) some of the reports with non-significant results provided
no data on the effect size. Hence, although this conservative approach may converge
the effect sizes towards zero, it also limits the possible effects of artefacts on the
results, which is important in any study but particularly when there is an attempt to
measure causal effects.
By taking this very conservative approach it was not possible to estimate the
actual effect sizes of ECA, as results using this approach are likely to be biased
downward (given that most non-significant effects were positive) (Lipsey and
Wilson 2001). However, by minimising the chances of false positive or type 2 error
(Abramson 1995) and thus increasing the power of the study, it is believed that this
approach is adequate to address the question ‘‘Do extra-curricular activities affectstudent outcomes?’’ This is particularly important given that critiques of meta-
analyses, which use observational data, suggest that uncontrolled confounders may
bias the results, particularly if the effects are small (Feinstein 1995; Shapiro 1997).
Results
The findings from studies on participation in extra-curricular activity and students’
outcomes suggest a positive relationship between participation in school-sponsored
activities and achievement (see Table 5). However, it is unclear exactly what it is
about these types of activity that causes the learning-related outcomes, or indeed
whether there is a causal relationship at all.
The analysis of the studies undertaken in this review reinforces this point by
demonstrating that while a significant relationship was seen between activities and
outcomes in some instances, no evidence of causality was found. Furthermore, in
many instances effect sizes were low, indicating that there was no meaningful
association at all. Table 5 outlines a summary of the effect sizes, in terms of
learning-related outcomes, and indicates that most effect sizes (Cohen’s d) were
moderate to low. Meaningful effect sizes were found in only two instances: the
effect of general ECA on aspiration to tertiary studies; and the effect of student
council participation on GPA.
Table 5 shows that effect sizes were exhibited mostly where the outcomes related
to grades in general, or specific subjects such as Maths, English/reading and
602 B. Shulruf
123
Science. Interestingly, while the effect size for aspiration to tertiary education
yielded a moderate effect size of 0.48, no significant effect size was found that
related any extra-curricular activity to actual participation in tertiary education.
One of the highest effect sizes found was related to participation in student
council or student leadership bodies. Participation in student leadership bodies
yielded an effect size of 0.47 across three studies on GPA, yet there was no
meaningful effect on other outcomes (see Table 5).
Participation in academic clubs yielded effect sizes of 0.34–0.37 in English/
Reading, Maths and Science (but negligible for GPA; ES = 0.02). This finding
suggests that, although the effect sizes are small, students who participate in
academic clubs have higher grade levels. The number of effect sizes measured for
each outcome related to academic clubs is relatively high (n = 10–16), which gives
higher confidence in the generalisation of this statement (see Table 5). At the same
time, however, the reasons for the relationship between academic clubs and higher
grades are unclear. It may be speculated that more academically-inclined students
choose to participate in academic clubs, but further research is needed to determine
causality in the first instance.
Table 5 Mean effect sizes by type of activity and outcome
Activity cluster Outcome ESa 95% CI N (ES) n (students)
ECA General GPA 0.17 -0.31 to 0.66 3 35,866
ECA General Math 0.38 0.16 to 0.60 4 21,644
ECA General English/Reading 0.30 0.11 to 0.49 4 21,644
ECA General Science 0.27 0.11 to 0.43 4 21,644
ECA General Attitudes toward school 0.20 0.11 to 0.29 5 2,904
ECA General Aspiration to Tertiary 0.48 0.16 to 0.79 3 27,596
Sports GPA 0.15 0.07 to 0.23 15 59,804
Sports Math 0.08 -0.04 to 0.20 14 61,032
Sports English/Reading -0.01 -0.07 to 0.05 13 44,727
Sports Retention 0.31 0.09 to 0.52 5 20,960
Sports Aspiration to Tertiary 0.16 -0.10 to 0.42 4 11,993
Sports Self concept 0.04 -0.04 to 0.13 7 37,034
Academic Club/Journalism GPA 0.02 -0.05 to 0.09 10 32,255
Academic Club/Journalism Math 0.36 0.10 to 0.62 16 35,799
Academic Club/Journalism English/Reading 0.37 0.18 to 0.56 16 35,799
Academic Club/Journalism Science 0.34 0.18 to 0.49 13 10,680
Performing Arts/Cheerleading GPA 0.17 0.01 to 0.34 8 16,925
Performing Arts/Cheerleading Math 0.05 -0.02 to 0.13 11 79,430
Performing Arts/Cheerleading English/Reading 0.05 -0.12 to 0.22 11 79,430
Performing Arts/Cheerleading Retention 0.02 -0.19 to 0.23 3 15,541
Students Council GPA 0.47 -0.02 to 0.95 3 7,283
Students Council Math 0.15 0.04 to 0.26 3 13,344
Students Council English/Reading 0.16 0.04 to 0.28 3 13,344
a Effect sizes are weighted for sample size
Do extra-curricular activities in schools improve educational outcomes? 603
123
Sport was the most commonly investigated extra-curricular activity across the 29
studies. However, the effect of sports activities across the range of learning
outcomes was minimal, and ranged between -0.01 and 0.31. While the results do
suggest that sports activities could be related to school retention, the effect size
(Cohen’s d) of 0.31 is low (Cohen 1977) (Table 5), somewhat limiting the
meaningfulness of this finding; although the results tell us that student participation
in sports is associated with retention, the correlational evidence does not explain
why this might be the case.
There were no meaningful associations found for performing arts/cheerleading
across the range of outcomes, nor for vocational clubs or mentoring. The effect sizes
for performing arts/cheerleading ranged between 0.02 and 0.17 and it is interesting
to note that, unlike sports, performing arts did not have any effect on retention.
Effect size by school grade level
The broader focus of this systematic review is on pathways to higher education; as
such it was clearly important to identify the effect size of ECA by the school grade
level. Results of the full analysis, effect sizes and their level of significance are
shown in Table 6.
The most important finding was the relationship between participation in
academic clubs and results in core curriculum assessments. Effect sizes for
participation in academic clubs at junior and senior high school levels on Maths and
English/Reading outcomes were relatively high (ES = 0.86 and 0.69 respectively);
for Science the effect size was low-moderate (ES = 0.38). None of the academic
clubs that took place in junior schools (J) yielded meaningful effect sizes. These
results clearly indicate that participation in academic clubs in ‘Junior and High’
schools is associated with higher grades in Maths, English and Science (ES = 0.86,
0.69, 0.38 respectively), whereas engagement in academic clubs in junior schools is
not related to students’ achievements. This conclusion is also supported by the
relatively large number of the effect sizes (10) included in this analysis (see
Table 6).
A different picture can be drawn when looking at the relationship between sports
activities and learning-related outcomes. In this case the results show that across
junior and high school levels, all effect sizes but one were very small and for some
the 95% CI indicated inconclusive results. The exception was sports activities
undertaken at a junior school level; this yielded a reasonable (low-moderate) effect
size in terms of retention (ES = 0.30, 95% CI 0.02–0.58).
Analysis of non-specific general ECA at junior through high school level yielded
low to moderate effect sizes for Maths, English and Science outcomes (ES = 0.38,
0.30, 0.27, respectively). At junior school level, non-specific general ECA yielded
high effect size for attitudes towards school (ES = 3.65; 95% CI 0.38–6.92) and
GPA (ES = 1.8; 95% CI -6.88 to 10.48). However, this finding came from two
studies only, so any generalisation should be taken with caution.
It is noteworthy that the data available for the meta-analysis did not allow the
placement of robust control measures for confounding the effects relating to specific
groups’ characteristic or activity types. While this commonly happens when too few
604 B. Shulruf
123
effect sizes are identified, some caution is suggested with regard to the interpretation
of the mean effect sizes reported above.
Effects of data sources
As previously mentioned, the studies included for analysis in this review were
predominantly based on secondary data sources, a methodology which, in terms of
data collection, can lend itself to vulnerability to bias due to limitations around
transparency. To examine the relationship between the results of the studies in this
review and the methods used by the researchers, the association between the data
collection method and the effect sizes was measured. It was found that studies
which used secondary data (data taken from existing databases) yielded effect size
of 0.15 (95% CI 0.14–0.15), whereas primary data (e.g. questionnaires) yielded
higher effect size (ES = 0.27 95% CI 0.26–0.28).
Since these differences are significant, it is suggested that any further systematic
review/meta-analysis in this area should consider the effects of the data sources. It is
particularly important in the use of secondary data as it may mean that the researchers
Table 6 Mean effect sizes by school grade level
Activity cluster Outcome School
levelaESb 95% CI N (ES)
Sports Self concept J & H 0.04 -0.04 to 0.13 7 37,034
Sports Retention J 0.30 0.02 to 0.58 4 20,568
Sports English/Reading J -0.01 -0.35 to 0.32 3 35,197
Sports GPA J 0.17 -0.08 to 0.42 6 22,004
Sports GPA J & H 0.14 0.10 to 0.19 7 37,034
Sports Math J 0.12 -0.66 to 0.90 3 35,197
Sports Math J & H 0.03 0.01 to 0.04 9 25,069
Performing Arts/
Cheerleading
English/Reading J 0.11 -0.13 to 0.34 6 67,857
Performing Arts/
Cheerleading
GPA J 0.18 -0.15 to 0.51 4 15,393
Performing Arts/
Cheerleading
Math J 0.06 -0.07 to 0.19 6 67,857
ECA General Attitudes towards
school
J 3.65 0.38 to 6.92 6 2,904
ECA General Science J & H 0.27 0.11 to 0.43 4 21,644
ECA General English/Reading J & H 0.30 0.11 to 0.49 4 21,644
ECA General GPA J 1.80 -6.88 to 10.48 4 35,866
ECA General Math J & H 0.38 0.16 to 0.60 4 21,644
Academic Club/Journalism Science J & H 0.38 0.20 to 0.55 10 9,530
Academic Club/Journalism English/Reading J & H 0.69 0.38 to 1.00 10 9,530
Academic Club/Journalism GPA J 0.01 -0.07 to 0.09 4 27,839
Academic Club/Journalism Math J & H 0.86 0.41 to 1.31 10 9,530
a J & H Junior and High school; J Junior. b Effect sizes are weighted for sample size
Do extra-curricular activities in schools improve educational outcomes? 605
123
are working blind, rather than being privy to information that has the potential to
elucidate and inform the research. For example, knowing whether a respondent was a
participant or programme operator may change the value of the data.
Discussion
The key issue raised by this critical review and meta-analysis of literature on
participation in ECA and the relationship of participation to school-related
outcomes is linked to the strength of the evidence supporting the effect of ECA
on educational outcomes, and the limitations around the determination of causality.
A further, related issue highlighted is the restriction placed on both the analysis and
the findings due to the lack of empirical data (e.g. attendance and ethnicity data),
and the popular use of data collected from secondary sources.
Causality
The studies investigated in this review set out to illustrate how participation in ECA
is linked to a range of school-related outcomes, including higher academic
achievement, engagement and retention in schooling, development of leadership
and self-responsibility and future tertiary aspirations. While relationships were
determined; there was no evidence of causal effects.
Causal effect
For a relationship to demonstrate causal effect, the following criteria should be
addressed (Abramson 1995, pp. 259–267):
a. Is the association between the suspected cause and the effect shown to be
statistically significant (at the level determined by the researcher)?
b. Is the strength of the association such as correlation between cause and effect
high enough and the actual value context-related?
c. Is there evidence that the greater the exposure to the cause, the greater the
effect? (‘Dose response’ relationship)
d. Has the cause occurred before the effect appears (time response)?
e. Is the effect related to only one cause (specificity)? (If so, the support for causal
effect increases.)
f. Does the association show consistency under different conditions? and;
g. Is there coherence with current knowledge? (Inconsistency with other evidence
weakens the case for causality.)
The evidence provided in some of the studies in this review met some of the
criteria, but no study met them all. Although some associations were statistically
significant (see criterion a), the strength of the association, namely the effect sizes,
were small (criterion b). No evidence could identify dose exposure (criterion c), or
any ‘Time response’ relationship (criterion d), as there was either no measure for
change over time or no indication in these studies that students’ achievement had
606 B. Shulruf
123
increased/decreased over the time. It was impossible to indicate level of specificity
(criterion e), as no study controlled the associations for other known effects on the
outcomes (such as family variables, school experiences etc.). The differences in the
effect sizes across different studies – including negative effect sizes – provide little
support that the evidence presented met the criterion of consistency (criterion f).
Finally, the current knowledge about the impact of ECA on academic achievements
is too scarce to suggest any concrete conclusion (criterion g).
Some caution should be taken, however, in this causality assessment. As Abramson
(1995, p. 261) suggested, ‘‘We can never really ‘prove’ a causal relationship. What we
want, however, is ‘reasonable proof’, strong enough to be used as basis for decision
and action.’’ In practice this means that the more criteria for causality are met, and the
more substantial these criteria are, the more likely it is for a causal effect to be evident.
Thus, ‘‘decisions and action’’ should be made on the best evidence available, and the
task of providing this evidence should be central to further research in this area.
Overall, although some associations could be identified between participation in
ECA and a number of students’ outcomes, there was no robust evidence for causal
effects relating to these associations. Until causality can be shown, how best to
enhance positive school-related outcomes through ECA will remain unclear.
Limitations of study data
The collection of data from the extra-curricular studies presented the review authors
with some difficulties due to the widely varied approaches taken, the multiplicity of
outcomes and the varying definitions of ‘‘outcomes’’. For example, studies used a
wide range of academic outcome measures, including: GPA, Scholastic Assessment
Test (SAT) scores, grades for single core curriculum subjects, course credits, and
self-reports/teacher ratings in the United States; and GCSE in the United Kingdom.
Studies also focused on a range of different core curriculum areas; some examined
English, some Science, some Maths, and others a mixture or all three. There was
also an issue with the range and clustering of ECA. There were no data available on
activity attendance, and limited data available on ethnicity, both of which restricted
the ability to look at trends and patterns.
Furthermore, although the data generally originated from longitudinal studies
with baseline data, there appeared to be no controls in place which might have
enabled a comparison of outcomes by baseline data to have taken place.
Use of secondary data
Most of the studies used secondary data for their analyses but, although these
datasets are large and quite comprehensive, the data stored in these databases were
not originally collected to address specific questions about ECA. For example,
question 82 from the first National Education Longitudinal Study in 1988
(NELS:88) asks: ‘‘Have you or will you have participated’’ in ECA (see Fig. 1).
This question was very likely designed to collect information to answer a specific
research question, and not necessarily the one presented in some of the studies
reviewed. This question asks about the future, and does not distinguish between
Do extra-curricular activities in schools improve educational outcomes? 607
123
reports on actual activities taken, or plans to take these in the future; nor does it ask
for relevant or adequate information related to the outcomes of ECA. In addition,
this question does not distinguish between the level or the type of participation in
ECA, and as such it is impossible to determine, for instance, whether the respondent
is a participant, a leader or an organiser.
Another issue relating to the use of secondary source data, and the NELS:88 in
particular, is the lack of depth to the information collected. General comprehensive
questionnaires, which ask many questions on a broad range of topics, are designed
to capture wide-ranging information rather than collecting in-depth information on
specific topics. As 13 studies included in this review used data from NELS:88, and
all of them used the items from the first wave as presented above, the validity of the
information on participation in ECA is doubtful.
Methodological bias
Further examination of the approaches used in studies based on secondary source
data revealed more methodological flaws. For example, Marsh and Kleitman (2003)
used NELS:88 for their study and sampled 12,084 students from the 1988 cohort.
However, on closer examination, their study included only 4,250 students (35%)
with valid data for all the variables considered in the analysis. Moreover, students
who transferred or dropped out of school were excluded from the main analyses.
Providing no analysis of missing data and excluding the students who left school
raises a critical question about whether or not the effects found in this analysis are
only artefacts. Alternatively, the results could be overturned if all the data were
included in the analysis. For example, one might assume that there were many
students, among the 65% who were excluded from the study, who participated in
ECA but left school for various reasons, including poor outcomes.
None of the other studies which used NELS:88 revealed any analyses of missing
data. In addition, it was found that the data used excluded students who changed
schools and those who did not complete questionnaires (or whose parents did not
complete the questionnaire). The vulnerability for bias is great as students with low
grades are more likely to drop out or change schools and their parents are less likely
to be engaged in school life. Excluding these students from the analysis, while it is
known that the majority of students are active in ECA (Cooper et al. 1999), is likely
to bias the reported results upwards.
These possible biases have a major effect on the analyses carried out in the current
review, since there were quite a few studies which included subsamples from the
NELS:88 project and therefore some participants might have been included in more
than one study. For this reason students’ weightings were not equal across studies and,
82. Have you or will you have participated in any of the following school
activities during the current school year, either as a member, or as an
officer (for example, vice-president, coordinator, team captain)?
Fig. 1 Question 82 from NELS of 1988
608 B. Shulruf
123
as individuals could not be identified, it was impossible to control this bias using
statistical measures. While the conceptual consequences are obvious, this situation
provides further support for the conservative analytical approach undertaken in this
systematic review.
Conclusion
The aim of this systematic review was to ascertain what it is about ECA participation
that supports positive outcomes and why, and to consider how the common
assumptions about the benefits of ECA participation can be validated. The results
show associations rather than causation and raise major concerns regarding the
validity of some of the data and analyses used in the literature. This leads to the
conclusion that the current knowledge on ECA participation does not suggest that
extracurricular activities affect student educational outcomes either positively or
negatively. It is therefore considered essential that further research be carried out to
unravel how participation in ECA contributes to students’ outcomes and why. Such
research should investigate aspects of participation including what motivates
participation, how and why students participate, and how such participation impacts
on their outcomes.
Recommendations for further research
Lack of evidence for causality strongly suggests that the fundamental question
which should be asked is: ‘‘Do extra-curricular activities have any effect on
students’ outcomes?’’ It is therefore recommended that further studies try to identify
the effects of ECA on students’ outcomes by including measures for causality in
their study design. The best measures for causal effects are established by
prospective randomised controlled trials (Abramson 1995) and studies incorporating
this methodology in this area should be promoted. It is noted however, that
undertaking prospective randomised controlled trials is very difficult in social
sciences, particularly due to ethical issues. Thus, the second-best option is to apply
methods of quasi-experiment with an emphasis on causal effects; that is, collecting
data which enable assessment of as many causality criteria as possible.
Implementing such measures in the ECA research is important. Schools in most
countries spend significant resources on ECA but, as this critical review has found,
there is no robust evidence supporting the effectiveness of that expenditure. Thus,
research in this area is crucial, particularly if one wishes to improve school
effectiveness and student outcomes.
References
Evaluation studies reviewed are marked*
Abramson, H. J. (1995). Survey methods in community medicine. New York: Churchill Livingstone.
Do extra-curricular activities in schools improve educational outcomes? 609
123
Adelman, C. (1995). The new college course map and transcript files: Changes in course-taking andachievement, 1972–1993. Based on the postsecondary records from two national longitudinalstudies. Washington, DC: National Institute on Postsecondary Education, Libraries, and Lifelong
Learning.
Ainley, J. (1994). Multiple indicators of high school effectiveness. Paper presented at the Annual meeting
of the American Educational Research Association, New Orleans.
Appelman, M., Gorter, J., Lijesen, M., Onderstal, S., & Venniker, R. (2003). Equal rules or equalopportunities? Demystifying level playing field. Hague: CPB Netherlands Bureau for Economic
Policy Analysis.
*Barber, B. L., Eccles, J. S., & Stone, M. R. (2001). Whatever happened to the jock, the brain, and the
princess? Young adult pathways linked to adolescent activity involvement and social identity.
Journal of Adolescent Research, 16(5), 429–455.
Barr, H., Hammick, M., Koppel, I., & Reeves, S. (1999). Evaluating interprofessional education: Two
systematic reviews for health and social care. British Educational Research Journal, 25(4),
533–544.
Baxter, S. G. (1936). Intelligence and the extra-curriculum activities selected in high school and college.
The School Review, 44(9), 681–688.
Boaz, A., Ashby, D., & Young, K. (2002). Systematic reviews: What have they got to offer evidence basedpolicy and practice? ESRC UK Centre for Evidence Based Policy and Practice, Queen Mary
University of London.
*Braddock II, J. H., Royster, D. A., Winfield, L. F., & Hawkins, R. (1991). Bouncing back: Sports and
academic resilience among African-American males. Education and Urban Society, 24(1), 113–131.
*Branch, J. L. (2003). Extracurricular activities and academic achievement. Hattiesburg: The University
of Southern Mississippi.
*Broh, B. A. (2002). Linking extracurricular programming to academic achievement: Who benefits and
why? Sociology of Education, 75(1), 69–95.
*Brown, R., & Evans, W. P. (2002). Extracurricular activity and ethnicity: Creating greater school
connection among diverse student populations. Urban Education, 37(1), 41–58.
*Buoye, A. J. (2004). Capitalizing in the extra curriculum: Participation, peer influence, and academicachievement. Notre Dame, IN: University of Notre Dame.
*Chambers, E. A., & Schreiber, J. B. (2004). Girls’ academic achievement: Varying associations of
extracurricular activities. Gender and Education, 16(3), 327–346.
Cohen, J. (1977). Statistical power analysis for behavioral sciences. New York: Academic Press.
Cooper, H., Valentine, J. C., Nye, B., & Lindsay, J. J. (1999). Relationships between five after-school
activities and academic achievement. Journal of Educational Psychology, 91(2), 369–378.
*Darling, N., Caldwell, L. L., & Smith, R. (2005). Participation in school-based extracurricular activities
and adolescent adjustment. Journal of Leisure Research, 37(1), 51–77.
*Davalos, D. B., Chavez, E. C., & Guardiola, R. J. (1999). The effects of extracurricular activity, ethnic
identification, and perception of school on student dropout rates. Hispanic Journal of BehavioralSciences, 21(1), 61–77.
Davies, P. (2000). The relevance of systematic reviews to educational policy and practice. Oxford Reviewof Education, 26(3–4), 365–378.
*Diaz, J. D. (2005). School attachment among Latino youth in rural Minnesota. Hispanic Journal ofBehavioral Sciences, 27(3), 300–318.
*Eccles, J. S., & Barber, B. L. (1999). Student council, volunteering, basketball, or marching band: What
kind of extracurricular involvement matters? Journal of Adolescent Research, 14(1), 10–43.
Eccles, J. S., Barber, B. L., Stone, M., & Hunt, J. (2003). Extracurricular activities and adolescent
development. Journal of Social Issues, 59(4), 865–889.
Feinstein, A. R. (1995). Meta-analysis: Statistical alchemy for the 21st century. Journal of ClinicalEpidemiology, 48(1), 71–79.
Feldman, A. F., & Matjasko, J. L. (2005). The role of school-based extracurricular activities in adolescent
development: A comprehensive review and future directions. Review of Educational Research,75(2), 159–210.
*Gerber, S. B. (1996). Extracurricular activities and academic achievement. Journal of Research andDevelopment in Education, 30(1), 42–50.
*Guest, A., & Schneider, B. (2003). Adolescents’ extracurricular participation in context: The mediating
effects of schools, communities, and identity. Sociology of Education, 76(2), 89–109.
610 B. Shulruf
123
Holland, A., & Andre, T. (1987). Participation in extracurricular activities in secondary school: What is
known, what needs to be known? Review of Educational Research, 57(4), 437–466.
Holland, M. N. (1933). Extra-curriculum activities in high schools and intermediate schools in Detroit.
The School Review, 41(10), 759–767.
*Jordan, W. J. (1999). Black high school students’ participation in school-sponsored sports activities:
Effects on school engagement and achievement. Journal of Negro Education, 68(1), 54–71.
*Kaufman, J., & Gabler, J. (2004). Cultural capital and the extracurricular activities of girls and boys in
the college attainment process. Poetics, 32(2), 145–168.
*Keith, T. Z., Diamond-Hallam, C., & Goldenring Fine, J. (2004). Longitudinal effects of in-school and
out-of-school homework on high school grades. School Psychology Quarterly, 19(3), 187–211.
Kitchenham, B. (2004). Procedures for performing systematic reviews (No. 0400011T.1). Keele, UK:
Software Engineering Group, Department of Computer Science, Keele University.
Konstantopoulos, S. (2005). Trends of school effects on student achievement: Evidence from NLS:72,HSB:82, and NELS:92. Bonn: The Institute for the Study of Labor.
Lewis, C. P. (2004). The relation between extracurricular activities with academic and socialcompetencies in school age children: A meta-analysis. College Station: Texas A&M University.
*Lipscomb, S. R. (2007). Secondary school extracurricular involvement and academic achievement: A
fixed effects approach Economics of Education Review, 26(4), 463–472.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: SAGE
Publications.
*Lisella, L. C., & Serwatka, T. S. (1996). Extracurricular participation and academic achievement in
minority students in urban schools. Urban Review, 28(1), 63–80.
*Mahoney, J. L., Cairns, B. D., & Farmer, T. W. (2003). Promoting interpersonal competence and
educational success through extracurricular activity participation. Journal of Educational Psychol-ogy, 95(2), 409–418.
*Mahoney, J. L., & Cairns, R. B. (1997). Do extracurricular activities protect against early school
dropout? Developmental Psychology, 33(2), 241–253.
*Marsh, H. W. (1992). Extracurricular activities: Beneficial extension of the traditional curriculum or
subversion of academic goals? Journal of Educational Psychology, 84(4), 553–562.
*Marsh, H. W., & Kleitman, S. (2003). School athletic participation: Mostly gain with little pain. Journalof Sport & Exercise Psychology, 25(2), 205–228.
*McCarthy, K. J. (2000). The effects of student activity participation, gender, ethnicity, and socio-
economic level on high school student grade point averages and attendance. Boulder: University of
Colorado.
*McNeal, R. B. (1995). Extracurricular activities and high school dropouts. Sociology of Education,68(1), 62–80.
*Melnick, M. J., Sabo, D. F., & Vanfossen, B. (1992). Educational effects of interscholastic athletic
participation on African-American and Hispanic youth. Adolescence, 27(106), 295–308.
NCES. (2006). NELS:88/2000. Fourth follow-up: An overview. Retrieved February 24, 2006, from
http://nces.ed.gov/surveys/nels88/.
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Realist synthesis: An introduction.
Manchester: ESRC Research Methods Programme, University of Manchester.
*Power, A. R. (1999). Getting involved and getting ahead: Extracurricular participation and theeducational attainment process. Notre Dame, IN: University of Notre Dame.
Robinson, J., & Etheridge, B. (1995). The 1994–95 North Carolina State testing results. Multiple-choiceend-of-grade and end-of-course Tests including English II essay. Raleigh: North Carolina State
Department of Public Instruction.
Shapiro, S. (1997). Is meta-analysis a valid approach to the evaluation of small effects in observational
studies? Journal of Clinical Epidemiology, 50(3), 223–229.
*Silliker, S. A., & Quirk, J. T. (1997). The effect of extracurricular activity participation on the academic
performance of male and female high school students. School Counselor, 44(4), 288–293.
*Slicker, E. K., & Palmer, D. J. (1993). Mentoring at-risk high school students: Evaluation of a school-
based program. School Counselor, 40(5), 327–334.
Suri, H., & Clarke, D. (1999). Revisiting methods of literature synthesis. Melbourne: University of
Melbourne.
Taylor, J. L., & Chiogioji, E. N. (1988). The Holland and Andre study on extracurricular activities:
Imbalanced and incomplete. Review of Educational Research, 58(1), 99–105.
Do extra-curricular activities in schools improve educational outcomes? 611
123
Thomas, W. B., & Moran, K. J. (1991). The stratification of school knowledge through extracurricular
activities in an urban high-school. Urban Education, 26(3), 285–300.
Torgerson, C. J. (2006). Publication bias: The Achilles’ heel of systematic reviews? British Journal ofEducational Studies, 54(1), 89–102.
Valentine, J. C., Cooper, B., Bettencourt, A., & DuBois, D. L. (2002). Out-of-school activities and
academic achievement: The mediating role of self-beliefs. Educational Psychologist, 37(4),
245–256.
The author
Dr. Boaz Shulruf is the Deputy Head of the Centre of Medical and Health Sciences Education at the
Faculty of Medical and Health Sciences, University of Auckland. His main research interest is in
methodology of research in education, cross-cultural psychology, social sciences and health. His recent
publications include articles on student pathways in the secondary and tertiary education; methodology of
longitudinal studies; and collectivism and individualism.
612 B. Shulruf
123
Copyright of International Review of Education / Internationale Zeitschrift für Erziehungswissenschaft is the
property of Springer Science & Business Media B.V. and its content may not be copied or emailed to multiple
sites or posted to a listserv without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.