Measuring Impacts of One AfterSchool Literacy Program

HELP ME TO BECOME A BETTER READER

March 2013

Jennifer Jones, Virginia Commonwealth University

In Sunshine County, parental demand for after-school chi ld care, academic accountabi l i ty pol icy changes, and research l inking lack of adult supervis ion after-school with r isk-taking behaviors, dr ives both pr ivate and publ ic funding sources for after-school programs.

In 2002, the ci t izens of Sunshine County, Virginia experienced high crime rates during the hours r ight after school whi le parents were working and many students were left at home unsupervised.

As a result of community complaints, the Sunshine County Pol ice Department partnered with other c ivic organizations to pinpoint areas in need of improvement. The task force ident ified several “high cr ime” neighborhoods where chi ldren were left unsupervised at home during after school hours.

The task force uncovered the need for high qual i ty after school programs to assist the famil ies and teens with l i teracy development as wel l as reduce neighborhood cr imes. Grant funds were secured to develop an after school program which would come to be known as “Brain Bui lders.”

“O

N A

NY

GIV

EN

DA

Y, TH

ER

E A

RE

AP

PR

OX

IMA

TE

LY E

IGH

T M

ILLION

UN

SU

PE

RV

ISE

D

CH

ILDR

EN

BE

TW

EE

N T

HE

AG

ES

OF

5 A

ND

14

DU

RIN

G

AF

TE

R-S

CH

OO

L HO

UR

S.”

1

Executive Summary

The Call to Action

Bra in Bui lders i s a school -year long (August through June) after school program designed to bo lster inte l lectual achievement and provide marginal ized students wi th the tools they need to succeed in l i fe .

The Bra in Bui lders program is current ly funct ional in three publ ic schools in Sunshine County – Every K id Counts Middle School , K ids R Us Elementary School , and Wonderfu l Wor ld Elementary School . In 2012,  85 students in grades K-8 part ic ipated in the after school program. This year, 50 students are scheduled to part ic ipate.

The Bra in Bui lder ’s After School Program operates every week on Tuesdays and Thursdays between the hours of 3 :30 and 5:30 P.M. The program is housed at Every K id Counts Middle School . Act iv i t ies take p lace in a var ie ty of sett ings to inc lude: a c lassroom, a computer lab, a gymnasium and weather permitt ing, outdoor fields .

Executive Summary

Brain Builders Program

At this particular center, 12 students attend regularly. The majority of the students have participated for longer than one school year. The program includes academic skills improvement, character development, and community service opportunities for the youth and the families it serves.

This program has existed since 2002 and has recently seen budget cuts which have prevented program expansion.

The economically disadvantaged students attending Every Kid Counts Middle School are struggling to perform in reading on state standardized assessments. The longitudinal data collected from the state database suggests that there has been a steady decline in performance for this subpopulation of students. Thus, this program was designed to reach this subpopulation and provide these much needed services.

The purpose of this study is to measure whether the Brain Bui lders After-school program in Sunshine County is having any measurable posit ive outcome on the l i teracy level of the students i t services at Every Kid Counts Middle School .

Research Questions1. How are students selected for

participation in this program? Who decides?

2. What does the attendance/participat ion rate look l ike long term?

3. How closely do the volunteers from Brain Bui lders work with academic content teachers to plan?

4. Which strategies do the Brain Bui lders Program employ to bui ld character; develop educational goals; and provide community outreach services?

“THE PU

RPOSE O

F THIS EVALU

ATION

IS TO M

EASURE TH

E LITERACY EFFEC

TS, IF ANY, TH

AT THE BRAIN

BUILD

ERS PRO

GRAM

HAS O

N TH

E YOU

TH IT SERVES.”

JEN

N JO

NE

S

Executive Summary

Measuring Programmatic Outcomes on Literacy

The Evaluator approached the topic via a “construct iv ists lens.” I t was necessary to speak to the program part ic ipants to al low them to te l l the story. Thus, the evaluator p lanned several interv iews, fo l lowed up by informal open-ended surveys and observed the dai ly events in real t ime at the Brain Bui lder’s Center.

The evaluator had to obtain permiss ion to interv iew part ic ipants. A letter was sent home with each of the 12 students and three were returned granting permiss ion for the evaluat ion.

ObservationsThe three students were fo l lowed c losely for approximately three months (0ver 15 vis i ts) . The evaluator was observing sess ions to see how part ic ipants interacted with others, responded to academic l i teracy lessons, and how they incorporated the ski l ls into pract ice.

Executive Summary

Evaluation Methods

InterviewsThe evaluator interviewed the three students, the program director, two academic literacy teachers, and the assistant principal. Questions were formulated in advance but the actual interviews were iterative in nature to allow for free conversation regarding the program.

SurveysFollow-up survey that were open-ended in nature were sent out to all interviewees a few weeks after the initial interview to build validity since the sole evaluator had no other researcher reviewing data for comparison.

ArtifactsThe evaluator used program artifacts such as new articles, pictures and newsletters to strengthen collected data. The pictures were priceless in providing rich details regarding the programs community outreach efforts.

Read ing Compre hen s ion Weakn essesI f o u n d t h a t re a d i n g c o m p re h e n s i o n p o s e d a p ro b l e m f o r a l l t h re e s tu d e n t s t h a t p a r t i c i p a t e d i n t h e e v a l u a t i o n . M o s t o f t h e m w e re re c e i v i n g s o m e t y p e o f re a d i n g c o m p re h e n s i o n s t r a t e g i e s i n t e r v e n t i o n w h e t h e r i t b e t h ro u g h B r a i n B u i l d e r s , i n c l a s s , o r t h ro u g h s o m e o t h e r i n - s c h o o l p ro g r a m . T h e s t u d e n t s re p o r t e d t h a t t h e y j u s t re a d a n d a n s w e re d q u e s t i on s i n t h e i r E n g l i s h / L a n g u a g e A r t s c o u r s e s . T h e s t u d e n t s re p o r t e d t h a t t h e y e n j o y e d m a t h m u c h m o re t h a n re a d i n g b e c a u s e t h e y f e l t t h a t t h e y w e re b e t t e r a t m a t h t h a n re a d i n g . E a c h o f t h e s t u d e n t s k n e w th e i r o w n s t re n g th s a n d w e a k n e s s e s a s t h e y p e r t a i n e d t o l i t e r a c y d e v e l o p m e n t .

Lack o f Programmat i c P lann in gI f o u n d t h a t w h i l e t h e B r a i n B u i l d e r ’ s S t a ff ex p re s s e d t h a t t h e y p l a n n e d w e l l a n d o f t e n w i t h t h e s t a ff a t t h e s c h o o l , o n e o f t h e t e a c h e r s i n t e r v i e w e d a n d th e A s s i s t a n t P r i n c i p a l f e l t o t h e r w i s e . B o t h l a d i e s i n d i c a t e d a “n e e d t o p l a n t o g e t h e r f o r t h e c h i l d re n . ”

Part i c ipant Se lec t ionT h e e v a l u a to r i n q u i re d a b o u t h ow s t u d e n t s a re s e l e c te d f o r p a r t i c i p a t i o n i n t h e B r a i n B u i l d e r s P ro g r a m . I a s ke d t h e p ro g r a m d i re c t o r a n d h e ex p l a i n e d t h a t , “ s t u d e n t s a re s e l e c t e d b a s e d o n th re e c r i t e r i a ; 1 . T h e y h a v e t o q u a l i f y f o r f re e o r re d u c e d l u n c h , 2 . T h e y h a v e t o l i v e i n c o m m u n i t i e s i d e n t i fi e d i n t h e o r i g i n a l g r a n t c o n d u c t e d b y t h e S u n s h i n e C ou n t y Po l i c e D e p a r t m e n t a n d 3 . T h e y h a v e t o w a n t t o b e h e re . ” T h e A s s i s t a n t P r i n c i p a l ex p l a i n e d t h a t t h e re i s a l o n g “w a i t i n g l i s t ” t o g e t i n t o t h e p ro g r a m a n d t h a t “ s tu d e n t s w h o m i s b e h a v e a re n o t p e rm i t t e d t o a t t e n d t h e p ro g r a m . ” Wh e n a s ke d w h o re c o m m e n d s s t u d e n t s f o r p a r t i c i p a t i o n , t h e d i re c t o r re s p o n d e d , “ I t i s a t e a m a p p ro a c h i n v o l v i n g m y s e l f , t h e a s s i s t a n t p r i n c i p a l , p a re n t s a n d t h e t e a c h e r s . ”

Executive Summary

Evaluation Findings

Actively Pursue FundingSources

Funding sources fo r a fter school in i t ia t ives are wan ing. Th is program was born out o f a ca l l fo r act ion f rom community members to reduce cr imes tak ing p lace as a resu l t o f unsuperv ised teens dur ing after work and school hours . I wou ld suggest that the same agenc ies cont inue to work together and communicate to garner suppor t for ra i s ing the necessary funds to susta in th is va luab le program.

I t i s my hope that my evaluat ion w i l l p rovoke fur ther d iscuss ions regard ing l i teracy development and how to meet the needs o f those teens st rugg l ing by hear ing the i r vo ices . Others shou ld fee l we lcomed to use my profi les f rom th is eva luat ion as a veh ic le to provoke conversat ions regard ing how to meet s t rugg l ing teens l i teracy needs and development .

Extended T ime forProfess ional Learning & Planning

Research has suggested that l i teracy development at the middle and high school levels is impeded by structural barr iers such as lack of effective sk i l ls training to meet learner’s needs.

I would l ike to add that in order to expand l i teracy instruction for al l learners, we need to be partnering with teachers, administrators and community members al ike to raise awareness and garner support for l i teracy programs. The staff at Brain Bui lders and at Every Kid Counts Middle need to plan together to strategical ly eradicate i l l i teracy for these students

Executive Summary

Recommendations for Improvement

After extensive review of the findings, the evaluator recommends:

About the Author:

I came to this evaluation project with great interest. Having grown up in the

inner city of Baltimore with the label of “free and reduced lunch” and attending many after school programs in my youth, I knew that reading was my salvation and could elevate me to achieve more; therefore my interest in literacy and its importance has never escaped me.

It became my lifelong endeavor to educate others and share with them my love of learning and reading. I am a certified teacher and administrator with over 15 years of experience in PK-12 education. I believe that my training and experiences in working with the diverse students I have served has provided the strength and fortitude to conduct this study.

1 National Institute on Out-of-School Time. (2003). Fact Sheet on school-age children’s out of school time. Wellesley, MA: Wellesley College, Center for Research on Women.