Educative Instructional Materials for Middle School Science

Susan M. Kowalski, Janet Carlson, Pamela Van Scotter, Paul Beardsley, Brooke N. Bourdelát-Parks,

Stephen R. Getty, Betty Stennett

The Challenge

•  Middle school is a critical time for –  Inspiring student interest in science – Establishing foundational understandings in science (Hanson, 2004; Beghetto, 2007)

•  To inspire interest and promote understanding, we need – Effective teaching practices – Coherent, rigorous, focused instructional materials

The Challenge

•  Preponderance of materials are “fragmented” (AAAS, 2001; Schmidt, 2001)

•  Two-thirds of middle school science teachers have a degree in a field other than science (Fulp, 2002)

•  Teachers teaching out-of-field rely heavily on materials (Ball & Feiman-Nemser, 1988)

•  Problems are most severe in low-income schools

Our Work to Address the Challenge

•  Develop instructional materials that are beneficial to both teachers and students

•  Part of an IES-funded Goal 2 development project (Grant # R305A080422)

•  Materials iteratively developed and tested (pilot study plus two field tests)

•  Materials include eight key features

Key Features of Materials

1.  Rigorous, coherent, and focused curriculum 2.  BSCS 5E Instructional Model 3.  Comprehensive assessment package 4.  Metacognition strategies 5.  Literacy strategies 6.  Collaborative learning 7.  Educative teacher materials 8.  Relevant examples and motivating contexts

Theoretical Framework •  To support teachers, curricular materials

should be educative. •  Educative materials are designed to

promote teacher learning as well as student learning.

•  Nine heuristics articulate roles for materials in three major areas – PCK for Science Topics – PCK for Scientific Inquiry – Subject Matter Knowledge

(Davis & Krajcik, 2005).

Design Heuristics—Area 1 PCK for Science Topics

Materials have a role in supporting teachers in 1.  engaging students with topic-specific

scientific phenomena 2.  using instructional representations that

support student understanding 3.  anticipating, understanding, and dealing with

students’ ideas about science

Design Heuristics—Area 2 PCK for Scientific Inquiry

Materials have a role in supporting teachers in 4.  engaging students in questions 5.  engaging students with collecting and

analyzing data 6.  engaging students in designing

investigations 7.  engaging students in making explanations

based on evidence 8.  promoting scientific communication

Design Heuristics—Area 3 Subject Matter Knowledge

Materials have a role in supporting teachers in 9.  the development of [teachers’] subject

matter knowledge

Excerpt from the Design Heuristic

Instructional materials should support teachers in anticipating, understanding, and dealing with students’ ideas about science

– Curriculum materials should help teachers recognize the importance of students’ ideas and help teachers identify likely student ideas within a topic

– Curriculum materials should help teachers gain insight into how they might be able to deal with the ideas in their teaching, for example, by giving suggestions of thought experiments likely to promote the development of more scientific ideas.

Example Science and Society Unit, Chapter 5

•  Chapter rated “comprehensive and thorough” by an external evaluator

•  Evidence: –  In the Engage phase, the Process and Procedure

section (p. 12) highlights a particular idea that students might have: “Some may say that they don’t want to know anything about diabetes.” The TE provides a suggestion for how to address this idea: “Encourage them to think of questions a person who just found out they had diabetes might want to know.”

Conducting the Research

•  Iterative development of – Student materials and – Teacher materials

•  External reviews of teacher materials – Evaluation rubric aligned with nine heuristics – Each element scored on a scale

•  Little or none •  Some •  Comprehensive and thorough

Process

•  Revised materials as necessary to attend to all elements of all nine heuristics after first field test and external review

•  Tested materials around the US in second field test – 24 field test teachers – Approximately 2000 students – 3-day field test orientation for teachers

Methods

•  Observed Teachers – RTOP (Sawada et al., 2002) – BSCS Fidelity of Implementation Rubric

(BSCS, 2009) •  Administered student tests (Pre and Post)

Results

•  First, a caveat Because the purpose of the data collection was to inform revisions to the materials, the data are not of sufficient scope to make broad generalizations. For example, there was no comparison group in this study; therefore, we cannot make claims about the benefit of these materials over others.

Results – Nature of the Materials

•  Successful incorporation of Davis and Krajcik heuristics: – Attended to 22 out of 24 elements based on

results of first review – Revised materials to attend to remaining

elements

Results – Teacher Practice Evidence of teacher use of reform-based practice: •  Mean RTOP score for this study = 63.3/100 •  Nationally for MS science = 50/100

(Sawada et al., 2002)

•  Finding aligns with that of another BSCS study: –  HS materials –  Random assignment –  External researchers conducting observations –  BSCS teachers had RTOP scores > 2 standard

deviations above control teachers

Results – Teacher Practice/Student Learning

•  Mean Fidelity of Implementation (FoI) score: 88% •  Significant correlation between FoI and RTOP

r = .423; p = .040  

•  Possible association with student learning –  Hierarchical linear modeling (students nested within

teachers) –  FoI neared significance (p = .056) in predicting mean

student posttest (adjusted by pretest and student demographics)

–  RTOP non-significant

Results in short…

Use of the materials is positively associated with reform-based teaching practices and is also associated with improved student achievement.

Implications

•  Instructional materials can be beneficial for both teachers and students

•  Educative materials may enhance teacher practice and student learning

•  This project has created a model of educative materials that can be used by others for additional research

Future Research

•  Efficacy trial to increase our confidence in making causal claims between use of the instructional materials and their effects on teacher practice.

•  Further development work (for grades 6-7)

Dissemination Model

•  Ongoing R&D model for materials through online curriculum dissemination beginning Fall 2012

•  Materials available in exchange for data •  Website: elearn.bscs.org (requires a

registration key, email support@bscs.org )

Thanks to the Team!

•  Our field-test teachers and students •  Our external reviewers •  Pam Van Scotter & Janet Carlson (PIs) •  Susan Kowalski (project lead–research) •  Betty Stennett (project lead–development) •  Paul Beardsley, Brooke N. Bourdelát-

Parks, Stephen R. Getty (developers and researchers)

Thank you!

For questions contact: jcarlson@bscs.org

BSCS 5415 Mark Dabling Blvd.

Colorado Springs, CO 80918 719.531.5550 www.bscs.org

References American Association for the Advancement of Science (AAAS). (2001). Atlas of science literacy. Washington, DC: Project 2061 and the National Science Teachers Association. Ball, D. L., & Feiman-Nemser, S. (1988). Using textbooks and teachers’ guides: A dilemma for beginning teachers and teacher educators. Curriculum Inquiry, 18(4), 401–423. Beghetto, R. A. (2007). Factors associated with middle and secondary students’ perceived science competence. Journal of Research in Science Teaching, 44(6), 800–814. Biological Sciences Curriculum Study (BSCS). (2009). Fidelity of Implementation of the BSCS 5E Instructional Model Assessment Tool—Students in the Classroom. Colorado Springs, CO: Author. Davis, E. A., & Krajcik, J. S. (2005). Designing educative curriculum materials to promote teacher learning. Educational Researcher, 34(3), pp. 3-14. Fulp, S. L. (2002). 2000 national survey of science and mathematics education: Status of middle school science teaching. Chapel Hill, NC: Horizon Research, Inc. Sawada, D., Piburn, M. D., Judson, E., Turley, J., Falconer, K., Benford, R., & Bloom, I. (2002). Measuring reform practices in science and mathematics classrooms: The reformed teaching observation protocol. School Science & Mathematics, 102(6), 245–253. Schmidt, W. H. (2001). Why schools matter: A cross-national comparison of curriculum and learning. San Francisco, CA: Jossey-Bass. Schmidt, W. H., McKnight, C. C., & Raizen, S.A. (1997). Splintered vision: An investigation of U.S. science and mathematics education. Boston: Kluwer Academic. Taylor, J. A., Van Scotter, P., & Coulson, D. (2007). Bridging research on learning and student achievement: The role of instructional materials. Science Educator, 16(1), 44–50.