CRIOP Professional Development: Program Evaluation 2012-2013Evaluatio

Susan Chambers Cantrell, Ed.D.Pamela Correll, M.A.Victor Malo-Juvera, Ed.D.

Professional Development Components

Summer training (1day per teacher)

Grade-level planning (1/2 day per teacher)

School and classroom-based coaching (average of 50.4 hours per teacher)

Day-long follow-up sessions (3 days per teacher)

Research QuestionsWhat was the impact of the project on teachers’ sense of

efficacy for culturally responsive instruction?

What was teachers’ implementation of the CRIOP model in classrooms?

What were teachers’ perceptions of their successes and challenges in implementing the CRIOP model in classrooms?

What were changes in students’ achievement in reading and math during the project?

What was the relationship between implementation of the CRIOP and student achievement?

27 Teachers at Participating Schools

School A School B School C School D02468

10121416

14

3 35

Number of Teachers

School Enrollment and F/R Lunch

School A School B School C School D0

100

200

300

400

500

600

700

588

336 294207

Number Self-pay

Number F/R Lunch

Stu

den

t En

rollm

en

t

Measures and Data Collection

Culturally Responsive Observation Protocol

(CRIOP)Teacher Interviews

Teacher Survey• Culturally Responsive

Teaching Self-Efficacy Scale

• Siwatu, 2007

Student Achievement Measures• Measures of Academic

Progress (MAP)• Think Link

Classroom Implementation of CRIOPTeachers’ CRIOP scores were significantly higher at the second

observation, Wilks’s ^ = .67, F(1, 22) = 13.64, p = .001.

Fall Spring15

15.5

16

16.5

17

17.5

18

18.5

19

16.35

18.52

Results: Teachers’ Perceptions—Success

Improved student learning

Increased student engagement

Enhanced classroom relationships

Heightened teacher knowledge

Improved instructional practices

Improved student learning

Being culturally responsive is just really letting them build up the lesson on their own based on what they know . . . Just seeing their backgrounds helps me decide what I am going to teach. It is different than other years when I had this “a through z“ plan. And it is okay that I have gotten off the beaten path, because I will tell you, their learning has grown ten-fold.

Increased student engagement

Students were “more eager to learn and more excited to learn.”

“The engagement was better.”

“In the past, it has been more teacher driven. But this year I tried to let the students do more on their own. We kind of let them lead the discussions . . . We didn’t want to stop the conversations because they were really into it.”

Enhanced classroom relationships

Students wanted “to be more like a family” and have a “sense of community and trust”.

“The biggest success I have had with being more culturally responsive is my classroom climate is wonderful . . . My ELLs feel like they are not only a part of the learning, but the teaching, and that makes them so proud”.

Heightened teacher knowledge

“I learned to identify different cultural responses that might affect how a student learns”.

Teachers learned “great new ideas” and new perspectives from the model lessons presented by school coaches.

Improved instructional practices

CRIOP professional development coaches “help us plan lessons to make them more culturally responsive and to make them more engaging for students, and that has been super helpful”.

The CRIOP coach “taught me different ways to interact with my students; more engaging ways that I would have never thought of.”

Results: Teachers’ Perceptions—Challenges

Language barriers

Parental involvement

Understanding culturally responsive instruction

Instructional decision making

Language barriers

“One of my ESL students is just very quiet and very soft spoken, so I really haven’t gotten to know a lot about her at home. Even if I sit with her. It is just that language.”

“I wanted to bridge the gap between parents, because it is hard to communicate when you have a language barrier there. So, I wanted to try to figure out, “How am I going to connect with them? How am I going to call them and tell them their child did this?”

Parental involvement

“Probably the biggest thing [challenge] would be that family component, and getting families in here and bringing some of that history into the classroom.”

“The parents have no idea how to help the students, and homework doesn’t always translate right over into English. I want more parents involved, but I don’t how to get them involved and get it translated over into English.”

Understanding culturally responsive instruction

“I think one of my biggest challenges would be just trying to understand that everyone comes from something different . . . I don’t ever want to offend anybody, and that is just a challenge because you don’t want to overstep your boundaries. So it is just a challenge knowing that everyone is different and . . . You need to reach every one of those kids in the classroom”.

Instructional decision making

“We haven’t used it [culturally responsive instruction] like we’d like to. I think if we can truly do the culturally responsive instruction, . . . I think they [students] would do much better. I think to get it truly in place is the biggest challenge and having the flexibility in the classroom to make some decisions . . . about how a student needs to receive . . instruction . . . It’s a huge challenge, feeling like they trust you enough to do that.”

Teachers’ Efficacy Beliefs from Teacher Surveys

Fall 2012 M Spring 2013 M2600

2700

2800

2900

3000

3100

3200

3300

2846.88

3266.08

*Change in mean = 419.20, p < .001

Student Demographics

589 students were enrolled in 27 participating

teachers’ classrooms

80.5% of students in participating

teachers’ classrooms received free or reduced lunch

25% of students in participating teachers’

classrooms were English language learners (ELLs)

Students’ Ethnicity

Caucasian37%

African American

27%

Hispanic30%

Other6%

Results: Student Achievement

Students’ MAP and ThinkLink reading scores were significantly higher at spring administration as compared to fall.

Students’ MAP and ThinkLink mathematics scores were significantly higher at spring administration as compared to fall.

MAP Growth Gains for ELLs

Test results indicate a number of ELLs at every grade level made greater than one year’s growth over the course of the year.

MAP scoring data provided by Northwest Evaluation Association(NWEA), developers of the assessment, was consulted for expected growth in reading and math. Growth targets were calculated by subtracting fall administration expected scores from spring administration expected scores.

ELLs Making More Than One Year’s Growth on MAP

•Reading 76% •Math 87%

Kindergarten

•Reading 36%•Math 74%

First Grade

•Reading 57%•Math 72%

Second Grade

ELL MAP Reading Growth Gains

K 1st 2nd0

5

10

15

20

25

15.216.6

13.7

*20.24

15.19 *14.82

MAP Norm Reading Growth

ELL Reading Growth

*Note. Gains > one year’s growth.

ELL MAP Math Growth Gains

K 1st 2nd0

5

10

15

20

25

30

15.4 16.2

13.1

*26.56

*19.91

*16.21

MAP Norm Math Growth

ELL Math Growth

*Note. Gains > one year’s growth.

Effects of Teachers’ CRIOP Implementation on Student Achievement

Reading* Math168

170

172

174

176

178

180

182

184

Low CRIOP Implementa-tion

High CRIOP Implementa-tionM

AP

Score

s

†

* p <.05, † p < .001

Questions?