Using Data to Improve Instruction in Science:Do we teach in light of the data or in spite of the data?

Michael HortonCoordinator

Riverside County Office of Education

Three Questions

• Do we teach in light of the data?

• Do we teach in spite of the data?

• Do we teach in an absence of data?

The Answer to these questions is . . .

Yes!

In Light of the Data

The idea that is sweeping the country:

Professional Learning Communities

is the ultimate example of teaching in light of data.

Professional Learning Communities

PLCs are built upon 3 key questions:

1) What do we want students to learn?

2) How will we know if they learned it?

3) What will we do if they don’t?

Which researchers support PLC?

Professional Learning Communities

Rick DuFour

Rebecca DuFour

Roland Barth

Robert Eaker

Michael Fullan

Larry Lezotte Doug Reeves

Jon Saphier

Mike Schmoker

Dennis Sparks

Rick Stiggins

Robert Marzano

Milbrae McLaughlin Tom Many

“The most promising strategy for sustained, substantive school improvement is building the capacity of school personnel to function as a professional learning community.” - Milbrae McLaughlin

Which groups support PLC?

Professional Learning Communities

Southwest Educational Development Laboratory

Mid-Continent Research for Education and Learning (McREL)

National Staff Development Council

Association for Supervision and Curriculum Development (ASCD)

National Science Teachers Association (NSTA)

“Professional development programs should be based on the needs of science educators—of both individuals and members of collaborative groups.” -NSTA Position Statement

National School Boards Association

Real-World Example

“X” High School in Riverside County-2 Chemistry teachers decided to look at data and measure efforts to

improve that data

Began a program to complete 50+ chemistry labs at home beginning in 2002-2003

In 2002, had an API of 575, a statewide rank of 3, and identified by CDE as an “underperforming school”

66% low SES and rising, 40% of parents never graduated high school, high EL population

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2002 2003 2004 2005 2006

FBB

BB

Basic

Prof

Adv

Chemistry Mean Scale Score in California 2006, 326 at PHS, 331Geometry Mean Scale Score in California 2006, 312 at PHS, 281

Real-World Example: The Data

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Geom

etry

Algebr

a 2

Chem

istry

Geom

etry

Algebr

a 2

Chem

istry

Geom

etry

Algebr

a 2

Chem

istry

Geom

etry

Algebr

a 2

Chem

istry

Geom

etry

Algebr

a 2

Chem

istry

Adv.

Prof.

Basic

BB

FBB

2002 2003 2004 2005 2006

Real-World Example: The Data

Real-World Example: The Data

In Spite of the Data

• Real District “X” was scoring very well on the 8th grade science test.

• 9th graders were required to take Integrated Science and test results were poor.

• 10th graders took Biology and test results were excellent again.

• This continued for at least 4 years.

• Many teachers concluded that Integrated Science was the problem.

In Spite of the Data

2005-2006

In Spite of the Data

The good news is that in 2006-2007, they began to look at their curriculum and did 2 things:

1. Aligned the Integrated courses more closely with the Integrated 1 blueprint.2. Tested to see if Earth Science would be a better option for 9th graders.

The 9th grade data improved.

2006-2007

In Light of the Data

More In Spite of the Data

Based upon recent TIMMS data, “U.S. twelfth graders scored below the international average and among the lowest of the 21 TIMSS nations in both mathematics and science general knowledge in the final year of secondary school.”

NAEP data show that California students score 11 points below national average on the 8 th grade science test. On the 2005 NAEP, California 8th graders scored second-to-last in the country.

California is below average nationally and the nation is below average worldwide in science.

California is among a small minority of states that only require 2 years of science to graduate.

2 years 3 years 4 years

More In Spite of the Data

More In Spite of the Data

Didactic Lecture:

Cognitive science says that lecture and recall doesn’t work.“Research tells us that the human brain can hold about seven items in its short term working memory . . . The number of new items that students are expected to remember in the typical science lecture is vastly greater.” Wieman, 2007

Interactive engagement versus performance, Hake 2002

Research in the classroom says that lecture and recall doesn’t work

More In Spite of the DataYet, we still teach primarily via didactic lecture and recall.

Recently, 1,500 classroom observations were logged and characteristics measured. This is what was found:

- Classrooms in which research-based strategies were being used: 0.2%- Classrooms in which there was evidence of higher-order thinking: 3%- Classrooms in which fewer than half of students were paying attention: 85%- Classrooms in which students were completing worksheets: 52%- Classrooms in which non-instructional activities were occurring: 35%

In my duty of performing Williams Visits this year, I visited 500 classrooms at a variety of grade levels and a variety of subject areas.

- Number of classrooms in which I saw anything other than lecture, bookwork, and

worksheets: 4

In the Absence of Data

In the Absence of Data

In the Absence of Data

About 10 years ago, a reform movement began to switch physics to 9th grade.

There was no data supporting this idea, only anecdote.

Many schools and districts switched and did not collect any data to judge its effectiveness.

To this date, no formal study has been done to measure its effectiveness.

In 2001, Leon Lederman estimated that over 100 schools (60% private) had switched to physics first nationally.

As of today, no data has been published showing its effect.

A website now lists 299 schools teaching physics first, certainly not a complete listing.

“Physics first schools are not quantitatively documenting the degree of their success” (Pasero, 2001).

In the Absence of Data

I have since collected data from California schools showing:

1) Physics first destroys physics test scores2) Physics first hurts chemistry test scores3) Physics first does not affect biology test scores4) Physics first does not increase enrollment in higher physics classes

Analyzing the standards has also showed:

1) Physics standards are highest on Bloom’s Taxonomy2) Very little chemistry and physics are needed to be proficient in the CA biology

standards3) All of the chemistry and physics needed is in the middle school standards4) Calculators are not even allowed on the physics test, this is not a mathematics issue

Still, over 13,000 9th graders took the physics CST last year.

Good News

- 13,000 is down from 16,000 in 2006; 17,000 in 2005; and 15,000 in 2004

- There were physics first schools that had 0% proficiency on the test yet no physics last schools with 0% proficiency on either 9th grade biology nor 11th grade physics. This is an improvement for those schools.

Conflicting Data: Homework

“A typical homework-completing high school student will outperform students who do not do homework by 69% on standardized tests.”

- Harris Cooper

“Homework-completing junior high students outperform homework non-completers by 35%.  There seems to be no difference in scores in the elementary grades.”

-Debbie Reese

“[R]esearch fails to demonstrate homework's effectiveness as an instructional tool and recommended changing the “default state” from an expectation that homework will be assigned to an expectation that homework will not be assigned.”

-Kohn, 2006

“According to Kohn, teachers should only assign homework when they can justify that the assignments are “beneficial” ideally involving students in activities appropriate for the home, such as performing an experiment in the kitchen. . .”

How to Use Data

1) Focus on student achievement2) CST data is valuable, but too infrequent to change daily practice3) Collaborate and share best practices4) Set SMART goals5) Measure progress6) Review progress and make changes as necessary

I taught my dog to whistle. I don’t hear anything.I said I taught him. I

didn’t say he learned it.

How to Use Data

Questions to consider when looking at CST or other data:(Adapted from a Dennis Fox presentation, Los Angeles County Office of Education)

1) How has the distribution of scores changed over the different subject areas over the last 3 years?

2) What is the “net gain” or “net loss” at each performance level for each subject area over the last 3 years?

3) What is the net gain or net loss for the combined performance levels (FBB/BB and Prof/Adv) for each subject area over the last 3 years?

4) Which subject area has made the most growth over the last 3 years?

5) Which subject area has made the least growth or no growth over the last 3 years?

Questions to consider when looking at CST or other data:(Adapted from a Dennis Fox presentation, Los Angeles County Office of Education)

6) During which year did the least growth occur for each subject?7) What are possible contributing factors to the above answers?8) How has the distribution of performance levels changed for

different subgroups?9) How does each change compare to your SMART goal?10) What factor(s) will you change before your next common

assessment?

How to Use Data

Contact Information

Michael Horton

mhorton@rcoe.us

Download PowerPoints and whitepapers and discuss issues, ask questions, or make comments about any of my presentations at:

csta.notlong.com