How effective is our Elementary Teacher Education Program?

Two Projects: AGILE

Teacher Work Sample:

Georgia A. CobbsTrent Atkins

The University of Montana

Background

“The current political climate with its attention to teacher and student accountability and the shift in schooling from a norm-referenced, textbook driven system to a learner-centered, standards-based system has highlighted the need for a framework like work sampling that offers fodder for potential theoretical and empirical connections between preparation, teaching practices, and P-12 student learning”(Girod, Schalock & Cohen, 2006).

Teacher Education Needs

Teacher education under scrutiny

How to link teacher preparation to practice to K-12 learning?

Complex correlation (Berliner, 2002)

Need for congruence between Higher Education and PK-12

Need to improve literacy instruction preparation

How to measure effectiveness?

Basis for AGILE & TWS

Need to blur the line between general education & special education

Many barriers in PK-12 are same issues in higher education

Ultimately, many of the issues in PK-12 are responsibility of higher education

AGILE Guiding Questions

Does training, engagement and ongoing support in RTI have a positive impact on:

#1 Student-teacher view of self-efficacy?

#2 Student-teacher view of measurement?

#3 Student-teacher knowledge of the principles of Effective Instructional Practices?

AGILEGuiding Question cont

#4 K-3 student reading skills?

#5 K-3 student spelling skills?

#6 K-3 student early numeracy skills?

#7 K-3 student math computation skills?

AGILE Methods

2 professors met w/ (3) student teachers about 3-4/ month

Collected data about every 2 weeks on Elem students

If students were not improving modify instruction

No specific strategies prescribed (RTI)

Math Computation

Oral Counting

K-3 Student Data cont.

Implementation Highlights

School placements have been easier

Schools appreciate the training we are providing to students

More meaningful, transactional relationship with schools

Student-teachers are receiving more focused university supervision from UM faculty (Darrell and Trent)

AGILEEpiphanies

The complete lack of training in intervention strategies

Taking on assessment and evidence-based intervention strategies during student teaching is too much

Assessment and evidence-based intervention strategies must be integrated into the teaching preparation curriculum

Oregon Teacher Work Sample

Developed over the last 20 years

Standardized a method for TWS

Over 1,000 Student teachers

10,000 K-12 students

Why TWS at UM?

To prepare teachers to make a difference in the learning of children

Teach Teacher Candidates to make data driven decisions.

Help to ensure teachers meet ethical obligations

To validate effectiveness of Teacher Ed programs

Purpose of TWS

Effort to move the Teacher Education Program into a new era

Experience of documenting teaching effectiveness

Measure knowledge & skill gains of PK-12 students

Assess using a pre/post action research design.

TWSResearch Questions

Q1 At what grade was the most impact made?

Q2 Was there an impact difference between male and female students?

Q3 Is there an impact difference among the types of students?

Q4 Is there an impact difference among the types of lessons?

Methodology

Senior level Math Methods 2011-2012

Required assignment in methods

Teach a math lesson with a pre-post assessment

Email me the spreadsheet of data

Answer assigned questions

Math Lesson (Researcher coded)

Book

Manipulative

Technology

Book/Manipulative

Book/Technology

Technology/Manipulative

Other

Data Collection

Assignment to my UG/G Students (N=102)

Part of their requirements for my course

Report back to me

Some worked in pairs

Data incomplete or no clear pre-post (narrative no test scores)

Final Teacher Candidate Data N=76

Suggestions for Assessment

Same for both the pre & the post assessment.

Or two assessments should be very similar

These could be chapter or unit test included within a curriculum. In other cases you may be asked to use an assessment that is not part of a specific curriculum. In other situations, you may need to create your own assessment.

Data Reporting

Student Male/Female Ethnicity Learning Needs Post-Assessment Pre-Assessment % Change

Student #1

Student #2

• ANONYMOUS results to me

• Set up a spreadsheet like below

Demograhpics

Gender Male Female Unknown0

100

200

300

400

500

600

700

N=1169 Elementary Students

Gender   Ethnicity   Needs  

Male 603 (51.6%) Native 13 NIN 937 (80.2%)

Female 528 (45.2%) Asian 7 G & T 62 (5.3%)

Unknown 38 (3.3%) Black 3 SpEd 94 (8%)

Total 1169 Latino 4 ELL 2 (.2%)

    Caucasian 311 Unkn 74 (6.3%)

    Unknown 831 Total 1169

    Total 1169    

Demographics

Mean Score Differences by

Grade LevelReport

Change

Gradelevel Mean N Std. Deviation

Kinder11.26 137 28.325

1st35.44 32 38.210

1st/2nd Combo35.74 38 31.395

2nd14.78 187 24.700

3rd21.73 280 27.933

4th13.17 268 24.995

5th23.32 164 37.227

6th10.41 61 37.314

5th/6th Combo45.50 2 17.678

Total17.93 1169 29.957

Findings for GRADE LEVEL

To conduct analyses, grade were grouped into three categories: Early (K,1, and 2), Middle (3 and 4), and Late (5 and 6).

When grouped this way, there were slight differences in mean scores Early (17.25), Middle (17.54), and Late (20.05).

No statistically significant differences found.

Mean Differences by GENDER

Report

Change

Gender Mean N Std. Deviation

Female

18.65 528 31.203

Male

16.63 603 28.799

Total

17.57 1131 29.949

Findings for Gender

Females (18.65) scored slightly higher than male (16.63) students.

Differences were not statistically significant.

Mean Differences by ETHNICITY

Report

Change

ETHNICITY2 Mean N Std. Deviation

Diverse

18.41 27 21.548

Non-Diverse

20.78 311 29.373

Total

20.59 338 28.808

Findings for ETHNICITY

To conduct analyses, the ethnicity variable was dichotomized.

“Non-diverse” students (20.78) scored higher than diverse students (18.41).

Differences were not statistically significant.

LEARNING NEEDS

Needs

  Frequency Percent Valid Percent Cumulative Percent

Valid

NIN 937 80.2 85.6 85.6

G&T 62 5.3 5.7 91.2

SpEd 94 8.0 8.6 99.8

ELL 2 .2 .2 100.0

Total 1095 93.7 100.0  

Missing Unknown 74 6.3    

Total 1169 100.0    

Percent by LEARNING NEEDS

Mean Differences by LEARNING NEEDS

Report

Change

Needs Mean N Std. Deviation

NIN18.19 937 29.880

G&T11.56 62 20.394

SpEd22.57 94 30.791

ELL40.50 2 21.920

Total18.23 1095 29.549

Findings for LEARNING NEEDS

To conduct analyses, due to the small number, ELL (n = 2 ) students were removed.

Students in special education scored highest (22.57), students with no identified needs scored in the middle (18.19), and students who are identified as gifted and talented made the smallest gains (11.56)

Differences were not statistically significant.

LESSON TYPE

LessonType

  Frequency Percent Valid Percent Cumulative Percent

Valid

Book348 29.8 29.8 29.8

Book/Tech90 7.7 7.7 37.5

Tech60 5.1 5.1 42.6

Tech/Manip201 17.2 17.2 59.8

Manipulatives413 35.3 35.3 95.1

Book/Manip57 4.9 4.9 100.0

Total1169 100.0 100.0  

Percent by LESSON TYPE

Mean Differences by LESSON TYPE

Report

Change

LessonType Mean N Std. Deviation

Book19.25 348 32.484

Book/Tech6.47 90 14.342

Tech7.80 60 23.415

Tech/Manip23.93 201 23.868

Manipulatives14.79 413 31.336

Book/Manip 40.26 57 31.227

Total17.93 1169 29.957

Findings for LESSON TYPE

Statistically significant differences among the lesson types!!!!

Book w/manipulatives had the most impact (statistical significance compared to all others).

Followed by Technology combined with manipulatives (statistical significance compared to all other but the book only).

Findings for LESSON TYPE

Statistically significant differences among lesson types

Book w/ manipulatives most impact (statistical significance compared to all others).

Technology w/ manipulatives (statistical significance compared to all other but the book only).

Findings continued

Book by itself was third most impactful (statistical differences when compared to the book and technology, the book and manipulatives)

Least impactful lesson types were manipulatives (14.49)technology (7.80) book w/ with technology (6.47)

Limitations

Not a standard assessment

Teacher Candidates not versed in Research Design

Coding of lesson type

Incomplete data: unknown demographics

Error in recording/analysis by teacher candidate or researcher

Next Steps

Make a Google Form!

Standardize data collection

Teach Research Design to Teacher Candidates

Other ideas?

Conclusions

Book/manipulative lesson may be most effective for teacher candidates Need to research this more

Trend to have teachers make more data driven decisions

UM Teacher Candidates are teaching with data-driven decisions, perhaps more teachers will carry on as well

Questions?

Comments?

Thank you!

References

Berliner, D. (2002). Educational research: The hardest science of all. Educational Researcher, 31(8), 18-20.

Girod, M., Schalock, M. & Cohen, N. (2006). The Teacher Work Sample as a Context for Research. Paper presented at the annual meeting for AACTE, San Diego, CA.