aft 7/12/04 marywood university using data for decision support and planning

AFT 7/12/04 Marywood University

Using Datafor

Decision Supportand

Planning

AFT 7/12/04 Marywood University

Presenters

Dr. Marie Loftus Dr. Barbara R. Sadowski

Marywood Universitywww.marywood.edu

AFT 7/12/04 Marywood University

Session Overview

Group activity followed by discussion

Starting the research process

Break

Group activity – questionnaire analysis and discussion

Research results

Question and answer

AFT 7/12/04 Marywood University

Group Task 1 Develop 2-3 questions of interest

For each question list:

– a) what data could be used – b) factors that might affect the process of

obtaining data

_ c) potential problems with existing data

– d) environmental factors that might bias the data or interpretation of the results

– e) relevant existing data sources

AFT 7/12/04 Marywood University

To determine if a teacher’s Thinking Math training has an effect on student achievement in Mathematics.

Research Purpose

AFT 7/12/04 Marywood University

Questions Considered

Does the mathematics training of teachers have an effect on the math achievement of students they teach?

Does professional development training of teachers translate into higher student achievement?

AFT 7/12/04 Marywood University

Practicality of Research Options

Question 1:

– Difficult to get data on teacher’s background in mathematics

– Measurement difficulty in defining college course work with in-service professional development

Question 2: - Teacher training data available- Non-transient teacher population- Well-documented training program- Student achievement data available

AFT 7/12/04 Marywood University

Other Variables Considered

What constituted teacher training? How many TM workshops?

Did TM trained teachers actually implement the principles? Which ones were implemented? How confident did the teachers feel about their ability to implement?

Was years of teaching experience a factor in how well TM was implemented?

AFT 7/12/04 Marywood University

Other Variables Considered

Was number of years teaching at grade level a factor?

Was the math background of the teacher a factor in how well TM was implemented?

How much of the time did the teacher actually implement TM when teaching math?

AFT 7/12/04 Marywood University

Other Variables Considered

What student variables might affect the score on the achievement test? Reading level? SES? Title 1 support? The number of TM trained teachers they had before reaching fifth grade?

Would school climate affect how the teachers felt about implementing TM math principles? Did the principal support their use of TM?

Ability grouping in the assignment of students

AFT 7/12/04 Marywood University

Research Question 1

Is the number of years students have a TM trained math teacher a predictor of how they will perform on a math achievement test?

AFT 7/12/04 Marywood University

Total enrollment 8,673 students

Economically disadvantaged 54%

Mobility 25.2%

Schools 18–13 elementary (Pre K-5)

Teachers 712–strong collective bargaining history with the Scranton Federation of Teachers

Scranton PA School District

AFT 7/12/04 Marywood University

Data Collection

Survey instrument developed collaboratively by Thinking Math trainers and researchers

Information session held for teachers and administrators

Surveys administered to TM teachers

Student data collected at school sites

AFT 7/12/04 Marywood University

Sample

The study sample was teachers and 5th grade students in:

4 of 13 elementary schools

–3 schools with TM trained teachers at the 5th grade level. –1 school with TM trained teachers, none at the 5th grade level

AFT 7/12/04 Marywood University

Sample Schools

All offer Title 1 reading, two also offer math

Fifth grade enrollments are 70, 55, 44, and 34

AFT 7/12/04 Marywood University

Sample Schools

Percentages of low income students: 28, 36, 46, and 77% (PA Profiles, 1999- 2000)

Class organization

– heterogeneous and homogeneous– departmentalized and self-contained

AFT 7/12/04 Marywood University

Group Task 2

Review/analyze teacher questionnaire results

Consider possible reasons for the results of the scale item means

Are the scales correlated? Which ones?

What other variables are related to the scales?

What changes would you make to improve the instrument?

AFT 7/12/04 Marywood University

Teacher Survey Results

AFT 7/12/04 Marywood University

Teachers Surveyed

33 of 37 are female

Pre-K - 5, Title 1, Learning Support

10% have < ten years teaching experience

70% have twenty or more years of experience

average of 1.5 ER&D courses beyond TM

AFT 7/12/04 Marywood UniversityNCTM 4/ 10/ 03 Scranton SD - Marywood University

Teacher Survey Scales

Confidence Scale ( 7 items)

Rate yourself on your confidence in... e.g., accepting multiple solutions.

1 = Not confident 10 = Very confident

Scale Mean = 8.57 s.d. = 0.80

AFT 7/12/04 Marywood University

Teacher Survey Scales

Implementation Scale (7 items)

To what extent do you... e.g., have your students explain and justify their solutions to problems?

1= Not at all 10 = To a great extent

Scale Mean = 8.1 s.d. = 0.89

AFT 7/12/04 Marywood University

Influence of TM Scale Items

1. How frequently do you incorporate the TM principles in math instruction?

2. To what degree has your delivery of math instruction changed as a result of TM training?

3. How has the depth of the math content you are teaching changed since your TM training?

4. How much has TM research added to your content knowledge of mathematics?

Scale Mean = 8.19 s.d. = 0.97

AFT 7/12/04 Marywood University

Correlations

Confidence Implementation

Influence of TM .395* .429**

Confidence .877**

* p<.05 **p < .01

n = 37, two-tailed

AFT 7/12/04 Marywood University

Regression

Implementation as the dependent variable

Influence of TM as independent variable

Confidence in TM Principles as independent variable

89% of the variance in implementation is accounted for by the two independent variables

AFT 7/12/04 Marywood University

Experience and Implementation

Teacher Experience N Mean s.d.

Less than 20 years 12 7.86 0.81

More than 20 years 25 8.33 0.95

AFT 7/12/04 Marywood University

Teacher Comments on TM

“I now change problems to reflect my class, linking school math to real world experiences.”

“I now use more manipulatives.”

“It has given me new insight for teaching multiple strategies and meeting individual needs.”

AFT 7/12/04 Marywood University

Student Achievement Results

AFT 7/12/04 Marywood University

SAT Math Problem Solving

PSSA Math

SAT Math Procedures

.798** .777**

SAT Math Problem Solving

.819**

**p < .01, (two-tailed); N = 167

Correlations: SAT – PSSA Math

AFT 7/12/04 Marywood University

SAT

Reading PSSA

Reading SAT Math Procedures .626** .620**

SAT Math Problem Solving .694** .648**

PSSA Mathematics .765** .693** **p < .01, (two-tailed); N = 167

Correlations: Reading - Math

AFT 7/12/04 Marywood University

Research Hypotheses

Students with 5th grade TM trained teachers will perform significantly better on PSSA 5th grade Math tests compared to the control group.

Students with 5th grade TM trained teachers will perform significantly better on PSSA 5th grade Math tests than the district and state.

AFT 7/12/04 Marywood University

Math Performance Levels

Level*** Thinking

Math District

Advanced 19.2% 14.8%

Proficient 26.3 25.9

Basic 36.5 31.2

Below Basic 18.0 28.0

***2 = 14.6; df =3, p < .005

AFT 7/12/04 Marywood University

Math Performance Levels

Level Thinking

Math State

Advanced 19.2% 22.5%

Proficient 26.3 30.5

Basic 36.5 24.6

Below Basic 18.0 22.4

Chi-square is not significant.

AFT 7/12/04 Marywood University

Mean Comparisons

Thinking

Math Control

PSSA Math** 1307 1216

SAT Math Procedures*

672 644

SAT Problem Solving***

665 631

*t = 2.40 ; df = 39, p < .05

**t = 2.74 ; df = 195, p < .01

***t = 4.0 ; df = 41, p < .000

AFT 7/12/04 Marywood University

Mean Comparisons

Thinking

Math District

PSSA Math** 1307 1270

SAT Math

Procedures*** 672 594

SAT Problem Solving***

665 609

**t = 2.91 ; df = 166, p < .005

***t > 20 ; df = 167, p < .000

AFT 7/12/04 Marywood University

Mean Comparisons

Thinking

Math State

PSSA Math 1307 1310

Mean difference is not significant.

AFT 7/12/04 Marywood University

Teacher-Student Results

AFT 7/12/04 Marywood University

Mean Comparisons > 2 TM teachers 2 or fewer

SAT Problem Solving** 665 655

SAT Math Procedures* 671 664

Reading Comprehension*** 671 660

*t = -1.76 ; df = 197, p = .081

**t = -0.91 ; df = 197 p = .081

***t = -2.04; df = 197, p < .05

AFT 7/12/04 Marywood University

Research Question

Can math achievement be predicted from teacher experience, professional development and implementation of the TM principles?

AFT 7/12/04 Marywood University

Analysis

A multiple regression was run with SAT math procedures as the criterion variable, and three independent variables: years of experience at grade level, professional development and scale score on implementation of TM principles.

A second multiple regression used SAT problem solving as the dependent variable.

AFT 7/12/04 Marywood University

Math Procedures Regression

Standardized Coefficients

t

Implementation of TM Principles .497 3.52***

Experience -.158 -1.85*

Professional Development -.191 N.S.

*p =.066

*** p < .001

AFT 7/12/04 Marywood University

Math Problem Solving Regression

Standardized Coefficients

t

Implementation of TM Principles

.451 3.18**

Experience -.130 N.S.

Professional Development

-.151 N.S.

** p < .05

AFT 7/12/04 Marywood University

Reading Comprehension Regression

Standardized Coefficients

t

Implementation of TM Principles .549 3.88***

Experience -.077 N.S.

Professional Development -.311 -2.30**

* * p = .023

*** p < .000

AFT 7/12/04 Marywood University

Conclusions

Teacher confidence in TM principles and perception of the importance of TM are predictors of implementation which is in turn a predictor of student mathematics achievement.

The number of years a student has a TM trained teacher is a predictor of student achievement in mathematics.

AFT 7/12/04 Marywood University

Conclusions

Students who have a 5th grade TM trained teacher performed significantly better on the PSSA 5Th grade Math Test than the control group and the district.

AFT 7/12/04 Marywood University

Research Purpose

The research results suggest that TM training does have a significant effect on student achievement.

AFT 7/12/04 Marywood University

Topics for Further Research

Does TM training affect student achievement in Reading Comprehension?

What is the relationship between TM training, student reading comprehension and student mathematics problem-solving achievement?

AFT 7/12/04 Marywood University

Questions?