Preliminary Highlights from the Noyce National Program Evaluation

May 30, 2013Ellen BobronnikovCris Price

Abt Associates | pg 2

Study Overview

Implementation study examines the Noyce Program across awards from PI, Faculty, Recipients, and K-12 Administrators’ perspectives

– Data sources: Web surveys and interviews

Impact study assesses program’s impact on teacher certification and employment in high-need districts– Data sources: Teacher certification and employment data

from 5 states and annual “monitoring” data entered by PIs

Abt Associates | pg 3

Selected Implementation Study Research Questions

What activities do teacher preparation programs use to prepare and support Noyce recipients?

What are STEM faculty responsibilities for preparing K–12 mathematics and science teachers?

What activities do Noyce recipients engage in?

Abt Associates | pg 4

Noyce Has Funded Recipients Across the Country

Abt Associates | pg 5

Number of Recipients First Supported Each School Year

2003 2004 2005 2006 2007 2008 2009 2010 20110

200

400

600

800

1000

1200

1400

1600

1800

Master Teaching Fellow

Teaching Fellow

Intern

Stipend Recipient

Scholarship Recipient

Num

ber o

f Rec

ipie

nts

Abt Associates | pg 6

Results: Activities to Prepare and Support Noyce Recipients Noyce recipients in teacher preparation, including

TFs, enrolled in same range of courses available/required of all students in teacher preparation– Additional activities were available to Noyce recipients

Supports for Noyce recipients who were teaching included mentoring, PD, courses, conference support

Supports for MTFs included leadership training and educational resources

Supports for interns included hands-on experience with K–12 students, courses to introduce them to teaching, and internships in various settings

Abt Associates | pg 7

Results: STEM Faculty Involvement with Noyce

After receiving the Noyce award…

– Over half of faculty/PI respondents reported increased STEM faculty member engagement in training STEM K–12 teachers

– About a quarter of STEM faculty respondents reported changes in their teaching due to the Noyce Program (e.g., focus on active learning, adapting course content to needs of teachers)

Abt Associates | pg 8

Results: Recipient Experiences While in Teacher Prep

Most recipients decided to enter K–12 teaching during or after college

Just under half decided to teach in high-need districts at the time of application to Noyce

Almost all completed student teaching in a STEM subject area, and 80% taught in a high need district

Majority of recipients reported that they felt adequately prepared for teaching responsibilities

Abt Associates | pg 9

Results: Recipient Experiences While Teaching

Induction supports were primarily received in the first year of teaching, other than support for conferences, which was received throughout

Common leadership roles held by Noyce teachers included mentoring, committee service, departmental leadership, advising student/school organizations

Almost all Noyce teachers reported that they plan to complete their Noyce teaching obligation and continue teaching science/ math in a high-need school district

Abt Associates | pg 10

Results: TF Experiences TFs must be in a Master’s Program and are required

to teach in a high-need district for 4 years while receiving a salary supplement

Four-fifths of TFs reported that they had decided to teach some time during or after college; one-third were career changers

Almost all TFs indicated that their teacher prep program used a cohort model and included K-12 teachers as science/math mentors; half of TFs participated in a PLC during their first year teaching

Abt Associates | pg 11

Results: MTF Experiences

MTFs are required to teach in a high-need district for 5 years while receiving a salary supplement

MTFs reported involvement in leadership activities prior to Noyce, but generally reported involvement in more leadership activities post-Noyce

The majority of MTFs planned to retain both classroom AND leadership responsibilities; fewer indicated plans to assume primarily leadership roles or to shift to higher education

Abt Associates | pg 12

Results: Intern Experiences

One-third of respondents who had applied for the Noyce internship had not considered teaching prior to learning about Noyce

Common internship settings included math/science camps, research labs, schools, or museums

About half of intern respondents indicated their interest in working as a K–12 teacher increased after participating in the Noyce internship

Now, a transition to impact study …

Abt Associates | pg 13

Impact Study Research Questions

Does an IHE’s receipt of a Noyce grant affect its production of graduates who are certified by their state to teach STEM content?

Does an IHE’s receipt of a Noyce grant affect its production of certified STEM teachers who take teaching jobs in high-need schools?

What is the impact of Noyce on student achievement?

Abt Associates | pg 14

Years to Certification and Teaching

Among all recipients:

– Nearly two-thirds had received their teaching certificate

Among those who had enough time to complete program and earn certifications (2+ years from first receipt of support):

– 83% have been certified to teach

Among recipients who have had at least 2 years to find a teaching position after certification:

– 90% had taught in high-need districts, in fulfillment of their service requirement

Abt Associates | pg 15

Program is being implemented as intended…

Most recipients received STEM certifications and entered teaching in high-need schools (according to monitoring data)

However, this alone does not answer the question: “Would recipients have earned STEM certifications and taught in a high-need schools in the absence of the program?”

Abt Associates | pg 16

Preliminary Findings from Teacher Impact Study Two of the 5 study states had significant positive

impact estimates on STEM certification and employment in high-need schools

– Impact represents an additional 4-5 teachers per IHE per impact year, about what we would expect from monitoring data

Two study states had impact estimates that were not significantly different than zero

One study state had a large significant negative impact estimate

Abt Associates | pg 17

Teacher Impact Study Limitations

All study states except one had small numbers of Noyce IHEs that graduated recipients early enough to be included in state datasets

Year-to-year variation in numbers of recipients who were certified and/or employed within IHEs was often large relative to the expected size of the impact per IHE per year

Impact estimates are NOT from a randomized study

– The quasi-experimental comparative short interrupted time series approach that was used to estimate impacts depends on models and assumptions

Abt Associates | pg 18

Next Steps

Examine state contexts to better understand the mixed findings in the teacher impact study.

Collect certification and employment data from another large state to include in teacher impact study

Collect an additional year of data from some study states

Conduct student impact analyses in 3 districts

Prepare dissemination report