professional development of science teachers as a reflection of large-scale assessment

AVIVA KLIEGER and NURIT BAR-YOSSEF

PROFESSIONAL DEVELOPMENT OF SCIENCE TEACHERSAS A REFLECTION OF LARGE-SCALE ASSESSMENT

Received: 20 March 2009; Accepted: 26 March 2010

ABSTRACT. Professional development (PD) of teachers comprises a major challenge inmany countries. The empirical relations between teaching–learning processes and studentachievements occupy educators who construct teachers’ in-service training programs.Student achievements serve as a measure for testing improvements in learning. Manyinvestigators view teachers’ content–didactic knowledge as influencing the quality ofteaching and student achievements. Large-scale assessments (national and internationalevaluation frameworks) in the Israeli educational system offer teachers opportunities forimproving their teaching–learning processes and student achievements. We developed aprogram for the PD of science teachers based on the curriculum and large-scaleassessments. The model is long term and integrates theoretical and practical knowledge.The activities should be effective and cooperative and should use diverse teachingmethods and integrate advanced technologies. This article presents considerations for theconstruction of the program, its goals, its performance, teachers’ reports on thecomponents, and the contribution of the program’s components to their PD.

KEY WORDS: large-scale assessment, professional community of science teachers,professional development, student achievements, teacher education, teachers’ knowledge

INTRODUCTION

Teachers play a significant role in preparing students for a changing andglobal world. Teachers’ professional development (PD) is a key toeducational change (Hoban, 2002). Teachers have to change theirviewpoints, attitudes, and teaching methods if they are to implement thechanges (Guskey, 2003, Vannatta & Fordham, 2004). According toFullan (2001), teachers lack effective PD programs for gaining deepknowledge and change.

Conventional teacher education models have been criticized becausethey do not take into account the teachers’ personal practical knowledge,experiences, and needs, do not view the teachers as active professionallearners or reflective practitioners, do not combine theory learning withpractice, and do not support teachers’ implementation (Zhan &Robertson, 2009). The present research reports a model of a PD program

International Journal of Science and Mathematics Education (2011) 9: 771Y791# National Science Council, Taiwan (2010)

that was developed in order to improve the teachers’ teaching method andthe students’ achievements.

Literature Review

Professional Development and the Quality of Teaching. Focusing onteacher empowerment empowers students and affects the quality ofteaching and the students’ learning and achievements (Darling-Hammond,1998a; Glenn, 2000; Hoban, 2002). Kennedy (1998) and Wenglinsky(2002) demonstrated a direct relation between characteristics of PD andstudent’s achievements in the sciences and mathematics. Sanders &Rivers (1996) found significant progress and a reduction in achievementgaps among students tested in comparative tests in the third and fifthgrades as a result of the quality of teaching to which they were exposed.According to Zuzovsky (2005), improvements in student achievements inthe Trends in International Mathematics and Science Study (TIMSS) inIsrael can be attributed to intensive PD activities.

The knowledge required of teachers in various fields should be mappedin order to construct a learning program for PD. Teachers need several typesof knowledge about learning. They should be flexible and reflective,familiar with technology, resourceful, and imaginative (Darling-Hammond,1998b; Hoban, 2005).

Professional Development: Planning, Framework, and Evaluation.Researchers generally agree on the components of high-quality PD whichinclude emphasis on content knowledge, an extended duration, andcompatibility with other learning activities (Kennedy, 1999; Garet, Porter,Desimone, Birman & Youn, 2001; Loucks-Horsley, Love, Stiles, Mundry& Hewson, 2003). Madigan (2001) indicated that teachers’ PD shouldinclude eight essential components: activities should be directed to thestudents’ learning; the schools should use data for making decisionsregarding the content and type of activities that will produce PD; activitiesshould be based on a valid practical research; teachers’ mastery of thecontent; a long-term, effective, and focused program; activities should becompatible with contents that are learned in teacher education and undergoevaluation processes; and PD should be adapted to the evaluation standardsand the school’s curriculum. Teachers who participated in longer-termeducation frameworks had a greater tendency to adopt the change in thecurriculum than teachers who did not participate in these frameworks (USANational Science Board (NSB), 2008). Students of teachers who

AVIVA KLIEGER AND NURIT BAR-YOSSEF772

participated in long-term teacher education achieved higher matriculationscores than students of other teachers (Cohen & Hill, 2001; Viadero, 2005).

A literature review (NSB, 2008) found that a database of thecomponents of effective PD programs began to be developed by theeducation research community. These components include long-termprograms whose contents are integrated into the teachers’ everyday workand which emphasize teamwork and collaboration (Cohen & Hill, 2000;Garet et al., 2001). According to Guskey (1998), five critical hierarchicallevels of information should be considered when evaluating PD: theparticipants’ reactions; their learning; measuring knowledge, skills, andnew attitudes; organizational support and change; the participants’ use ofnew knowledge and skills; and student learning outcomes.

Large-Scale Assessment. Large-scale assessment is necessary for obtaininginformation on the achievements of large numbers of students. Programevaluations supply information on the relationship between achievementsand factors that may explain or contribute to these achievements. Large-scale assessment can also be used for screening or placement purposes(National Research Council, 2003).

Large-Scale Assessment in Israel. In Israel, large-scale assessment iscarried out by state matriculation exams and the Growth and Effective-ness Measures for Schools (GEMS). The Ministry of Education has beendeveloping a system of indicators—GEMS—because of a growingawareness that a final assessment at the end of the school educationalprocess is not enough. The GEMS examines the learning environmentand achievements in mathematics, the sciences, English, and language(fifth and eighth grades). The international assessments frameworks thatare implemented in Israel include: TIMSS, Progress in InternationalReading Literacy Study, Second Information on Technology in EducationStudy (SITES), and Programme for International Student Assessment(PISA). In the 1999 TIMSS, Israel’s average in science was 468, belowthe world average (488). A decrease in science scores occurred between1995 and 1999, whereas an increase was observed between 1999 and 2003.In 2003, the average score in sciences was 488 (world average=474). Israelhas been participating in the PISA since 2000. Israel’s average in scienceswas below the world average. Israel did not participate in the PISA in 2003.

Professional Development and Large-Scale Assessment. Internationalassessment frameworks have implications for science education, includingPD of science teachers (Lin, 2008). Figure 1 presents the existing

PROFESSIONAL DEVELOPMENT AS REFLECTION OF ASSESSMENT 773

evaluation frameworks in the sciences in the junior high schools and theirinfluence on science teachers’ PD. It can be seen that the GEMS in scienceis influenced by the external international evaluation frameworks such asthe TIMSS and PISA. All three external evaluation frameworks influencethe construction of the science teachers’ tests. Low scores in the TIMSS(1999) and GEMS (2000–2002) have acted as a lever for developing amodel of PD for science teachers. One of the uses for large-scaleassessment is to connect to assessment in the classroom (NRC, 2003).

About the Study

The Teacher Education Model. When planning the PD of the scienceteachers, we used existing models and characteristics that have beenidentified as enabling changes in teaching methods, including PD thattakes place for over 3 years, promotes changes in the teachers’instructional behaviors and in classroom environment; teamwork andcollaborative work in professional learning communities, dealing inrelevant contents that are integrated in the teachers’ work in the classroomand emphasis on expansion of the teachers’ scientific knowledge(Clewell, Campbell & Perlman, 2004; Cohen & Hill, 2000; Garet et al.,2001; Kennedy, 1999).

The teacher education reported here was constructed in collaborationwith an academic center, the supervision of science and technology, anddistrict instructors in sciences from the central district. Shulman’s (1997)principles were used to develop a science teachers’ community whichincluded science teachers, teachers’ teachers, and experts in science andtechnology.

The main goal of developing the professional learners’ community inscience was to improve students’ achievements. Some of the goals were

INTERNATIONAL EVALUATION

TIMSS, PISA

NATIONAL EVALUATION

INTERNAL EXAMINATIONS

GEMS

Growth and Effectiveness

Measures for schools

Influence the structure,

skills and topics

Leads to PD

Figure 1. Large-scale assessment in the sciences and teachers’ PD


derived from locating difficulties raised by the external evaluation.Figure 2 describes the general goals of the teacher education program.

We first focused on goals that would afford solutions to difficulties thatarose following the external assessment tests. The short-term goals were theconstruction of test items that include skills and scientific inquiry,improvement of evaluations and measurement in the class, teamwork anddiverse learning strategies. The PD took place in face to face meetingsevery other week and in virtual encounters. The teachers received broadprofessional support from 11 instructors who were experts in the disciplineand in pedagogy and a district supervisor for science and technology.

The physical encounters included workshops and lectures. The virtualencounters were accompanied by a site and the activity in the site includeda forum, a resources bank constructed by the teachers and instructors,learning units, and lectures by experts. Experts gave enrichment lectures oncontents such as transport systems, ecology, electricity and magnetism, andnutrition and on pedagogical issues such as misconceptions, developmentof higher-order cognitive skills, and construction of tests. The resourcesbase included didactic suggestions in the learned fields, a store of test items,

Developing and experimenting test item integration content and

skills

Creating learning sequences, integrating

content and skills according to TIMSS

and GEMS

Determining key concepts of the learning process

Improving measurement and

evaluation processes

Identifying misconceptions and developing teaching

strategies

Promoting professional and interdisciplinary

team work

Identifying knowledge needed as a pre-

requisite for teaching

Developing teaching strategies for a

heterogeneous class

Raising the teacher’s

self-confidence

Developing distance

learning skills

Teacher education program

Figure 2. Goals of the teacher education program


and relevant articles. Implementation was carried out in the schools byteachers who participated in the PD program. The teacher educationbegan in 2001 and is still on-going, with a scope of 84–112 h annuallyper teacher.

The Science Teachers. School teams of science teachers from 57 juniorhigh schools in towns and rural settlements participated in the PD (196teachers in all). The teachers have seniority in teaching sciences, anacademic degree, and a teaching certificate. We examined the attitudes ofscience teachers towards the components of PD.

THE OBJECTIVES OF THE STUDY

This research presents a process that began following the low achieve-ments in sciences of students in Israel in external assessment frameworks(TIMSS, PISA, and GEMS). Development of a PD program for thescience teachers was carried out based on the findings of externalevaluation frameworks. The research questions were: (1) To what extentdo teachers believe that the components of the PD program contribute tothe teaching and evaluation processes which science teachers implement?(2) Did an improvement take place in the students’ achievements in large-scale assessment after the teachers’ in-service education?

METHODS

Participants

The research population included 196 science teachers who participatedin the teacher education program for 3 years in three PD centers in thecentral district (the questionnaire was given to all 196 teachers whoparticipated in the PD, but only 55 teachers returned the questionnaire).The teachers have at least a bachelor’s degree in the sciences and mosthave more than 10 years seniority in teaching. Sixteen teachers from fourschools were also interviewed. These teachers were chosen randomly fromschools in which most of the teachers participated in the PD program.

Research Design

A mixed method (quantitative and qualitative) was used (Erickson, 1998).The integration of findings from both methods has a complementary


value for obtaining a more reasonable picture, with internal and externalvalidity. The study was performed using a case study investigation andwas based on topic-oriented multiple case studies, out of a desire toachieve an in-depth understanding of the researched phenomenon (Stake,2000). The quantitative part of the research is of the experimental type ofresearch design in the category characterized by the one-time event study.In this design, a single group is observed at a particular point of time aftera particular event considered to have created a change. In the presentresearch, the long-term teacher education for the junior high scienceteachers comprises the event to which we refer as generating a change.The event is researched at the end of a 4-year education period.

The Research Instruments

Data organization and collection was carried out using quantitative(questionnaires) and qualitative (interviews) research instruments. Thequalitative research instruments included analysis of documents and asemistructured interview (Patton, 1980). Content analysis of the scienceand technology curriculum was carried out as well as of evaluationreports in science and technology within the national framework (GEMS)and within international frameworks (TIMSS and PISA). Semistructuredinterviews were held with the science teachers in their school and referredto the contribution of the teacher education to teaching science. Thequestions that were asked included: What is your opinion on the teachereducation and what contribution did it make? How is this expressed inyour work? In which aspects was a change generated in the school? Theframework of categories of analysis of the interviews was designed fromthe interviews (emic) and the framework of categories for analysis of thedocuments was designed from the external evaluation frameworks and thescience and technology curriculum (epic). The quantitative research toolincluded an attitudes questionnaire on a four-point Likert scale from 1—not at all to 4—to a very great extent. The 43 questions in thequestionnaire referred to their attitudes to the PD components.

Construction of the questionnaire was based on requirements of thecurriculum in science such as teamwork, a questionnaire that wasadministered by the instruction division of the Ministry of Education inteacher centers (which was validated and found reliable by experts), andfindings out of feedbacks given during the course after each topic.Relevant questions for the PD which the science teachers underwent werechosen from the above questionnaire. The reliability of the adaption wastested by ten teachers’ teachers and instructors who went over the


questionnaire before it was administered to the teachers and were asked tosort the sentences/words into categories.

The mean extent (in percent) of consent in the different categories was:teaching–learning strategies—94; scientific inquiry and technologicalmethod—93; test items—82; teamwork—98; and distance learning—76.The total mean extent was 87. The low mean in distance learning was dueto one sentence: “The division of work between the junior high schoolssaved time” that was also suitable for other categories, such as teamwork.

Data Analysis

Content analysis of the junior high science curriculum and the long-termPD program was carried out. They were analyzed according to acategories design that integrates the knowledge components identifiedduring content analysis of the curriculum and according to the categoriesof the internal referees as derived from the intentions of the programdevelopers. Semistructured interviews with science teachers were held.The analysis unit for the interviews was determined as a statement orsentence of the interviewee which expresses an idea, attitude, or concept.The final stage was designing a system of categories for analysis of theinterviews, i.e., attitudes towards teamwork, contribution of the PD topersonal empowerment, and implementation in teaching–learning strate-gies, scientific inquiry, technological method, teamwork, and the PD ingeneral. Quantitative data analysis was carried out by calculating themean score of the teachers’ answers on a four-point Likert scale.

RESULTS

Teaching–Learning Strategies

The data from the questionnaires were analyzed by calculating the meansand standard deviations (SD) of each item. The findings indicated that theteachers perceived themselves as gaining from most of the teaching–learning–evaluation strategies components. The mean contribution ofusing worksheets and integrating skills in contents was to a great extent.Most teachers felt that the development and use of worksheets made thebiggest contribution to their PD. In the field of evaluation, thecontribution of using quizzes and tests was almost “to a great extent.”The mean contribution of variety in homework, performing experiments,and using new school books was moderate, whereas the contribution ofcomputerized tasks and carrying out projects contributed to a small


extent. Table 1 presents the attitudes on the contribution of the teaching–learning strategies components to their development.

Scientific Inquiry and Technological Method

Table 2 summarizes the findings regarding the teachers’ attitudes towardsthe contribution of working on the component of the scientific inquiry.The findings indicate that the mean extent of all the parameters ofscientific inquiry and technological method was close to “to a greatextent” (3 on a scale of 4). However, parameters related to the experimentwere perceived as contributing to a smaller extent.

Test Items

PD in the field of measurement and evaluation included the constructionof test items. The teachers constructed tests and used them in theirclassrooms. They were asked about the use of the various parameters inconstructing tests after they had finished the teacher education. Table 3presents the findings regarding the various components when constructinga test. The component of constructing tests contributed greatly to thescience teachers. The mean was 93 in five of the components, i.e., theteachers benefited to a great extent. Only a few teachers did not benefit atall, especially in integrating graphs in the questions.

Teamwork

Teamwork was one of the important components of the PD process. Theteachers apparently benefited from all parameters to a great extent.Table 4 presents the contribution of the teacher education to the teachers’teamwork in various aspects, including strengthening teamwork betweendisciplines such as technology and geography. All teachers reported acontribution of teamwork to their work and to their enrichment inknowledge and ideas. The greatest contribution was in cooperation inknowledge and ideas and in strengthening the teamwork between thescience teachers in the school and in the professional community.

Distance Learning

One of the strategies of the teaching processes was distance learningusing a site that accompanied the teacher education. Table 5 presents theteachers’ attitudes regarding the contribution of this strategy to their work.The mean contribution of the PD was close to “a great extent.” However,


TABLE1

Teachingmod

es,teachingaids,and

evaluatio

n:teachers’attitud

eson

thecontribu

tionof

thedifferentcom

ponentsof

theteaching

–learningstrategies

N=55

Distribution(%

)

Not

atall

Toasm

all

extent

Toagreat

extent

Toavery

greatextent

Did

not

answ

erWeigh

ted

mean

SD

Using

worksheets

211

4047

03.33

0.75

Using

differentskills

07

5025

183.22

0.60

Using

quizzes

718

4629

02.96

0.88

Using

tests

913

5325

02.95

0.87

Givingho

mew

ork

727

4620

02.78

0.85

Perform

ingexperiments

920

5016

52.77

0.87

Using

new

text

book

s13

3634

152

2.52

0.91

Onlinetasks

2044

275

42.19

0.83

Perform

ingprojects

2748

182

51.94

0.75


some teachers did not benefit at all, mainly from parameters of using thecomputer in their classrooms after the PD.

In conclusion, most teachers found that the teacher educationcomponents contributed to their PD to a great extent in the fields ofteaching and evaluation strategies as well as in the development ofteamwork and use of distance learning.

Teachers’ Testimonies

The interviewees indicated that teacher education contributed to theteachers’ scientific inquiry, empowerment, teaching–learning strategies,and teamwork.

Teaching–Learning Strategies

The field of teaching–learning strategies benefited in several aspects: (1)The practical implementation of the new learning materials in theclassroom: Teacher 11 said: “The greatness of the in-service educationis that it immediately enters the field and the changes are according to thefield, acquaintance and assimilation of new learning materials.” (2) Thedevelopment of the students’ thinking and discovering students’ mis-conceptions: Teacher 8 said: “The way I stand in front of the class haschanged. I let them think and through the students’ way of thinking Iidentity misconceptions.” (3) The integration of skills in the contents:

TABLE 2

Teachers’ attitudes on the contribution of scientific inquiry and technological method totheir development

N=55

Distribution (%)

Notat all

Smallextent

Greatextent

Very greatextent

Did notanswer

Weightedmean SD

Reading results 5 11 67 15 2 2.93 0.70Reaching conclusions 4 13 70 13 0 2.93 0.63Hypothesis 5 18 57 20 0 2.91 0.78Solving problems 5 22 43 15 15 2.79 0.81Planning an experiment 7 22 56 15 0 2.78 0.79Research question 7 31 44 18 0 2.73 0.85Planning and analyzingan experiment includingcontrol of variables

5 33 51 11 0 2.67 0.75


TABLE3

Teachers’

attitud

eson

thecontribu

tionof

developing

testitems

N=55

Distribution(%

)

Not

atall

Small

extent

Great

extent

Verygreat

extent

Did

not

answ

erWeigh

ted

mean

SD

Iam

moreaw

areof

theim

portantcriteriafora

good

testitem

24

5638

03.31

0.64

Iintegratemoredifferentskillsin

thetestqu

estio

ns0

951

382

3.30

0.63

Iim

prov

edthequ

ality

oftestitemswhich

Iuse

413

3645

23.26

0.83

Iintegratemoredifferentthinking

levelsin

thetest

questio

ns2

758

330

3.22

0.66

Dealin

gin

testitemshelped

mevary

thetypesof

questio

nsin

routinetests

215

4536

23.19

0.75

Iintegratemoretables

inthequ

estio

ns5

2545

250

2.93

0.81

Iintegratemorediagramsin

thequ

estio

ns4

3136

272

2.89

0.86

Iintegratemoregraphs

inthequ

estio

ns4

3338

250

2.85

0.85

Iintegratemorescientific

text

inthequ

estio

ns2

3645

152

2.74

0.73


TABLE4

Teachers’

attitud

eson

thecontribu

tionof

thedifferentcompo

nentsof

team

work

N=55

Distribution(%

)

Not

atall

Small

extent

Great

extent

Verygreat

extent

Did

not

answ

erWeigh

ted

mean

SD

Igained

from

thecoop

erationin

know

ledg

eandideas

04

3856

23.54

0.57

The

team

workam

ongthescienceteachers

streng

thened

020

5129

03.09

0.70

The

team

workbetweenthescienceteachers

andtechno

logy

teachers

streng

thened

045

3113

112.63

0.73

The

team

workbetweenthescienceteachers

andgeog

raph

yteachers

streng

thened

047

2913

112.61

0.73

The

team

workwith

otherteam

sin

thescho

olstreng

thened

062

259

42.45

0.67


TABLE5

Teachers’

attitud

eson

thecontribu

tionof

thedifferentcompo

nentsof

distance

learning

N=55

Not

atall

Small

extent

Great

extent

Verygreat

extent

Did

not

answ

erWeigh

ted

mean

SD

Iam

willingto

trythistype

oflearning

again

09

5136

43.28

0.63

The

possibility

ofworking

onthesite

from

home

during

myfree

timemadeiteasier

forme

512

3640

73.20

0.87

The

division

ofworkbetweenthejunior

high

scho

olssavedtim

e7

1638

354

3.04

0.92

The

subjectsthat

werereferred

toin

thesite

answ

ered

myneeds

424

4722

32.91

0.79

Use

ofthesite

helped

meenrich

mykn

owledg

ein

differentcontentfields

029

5113

72.82

0.67

The

extensiveresource

bank

helped

mecope

with

the

GEMS/TIM

SS

235

3820

52.81

0.79

The

site

facilitated

theteaching

processforme

238

4315

22.72

0.74

The

dialog

uecontinueson

thenet,betweencolleagues

from

otherplaces

enriches

theteaching

–learningprocess

035

3120

92.72

0.88

Use

ofthesite

helped

meenrich

mykn

owledg

ein

differentskills

051

3512

22.61

0.71

Iim

plem

entedactiv

ities

from

thesite

inmyclass

944

2516

62.52

0.90

After

usingthesite

Iusethecompu

termoredu

ring

mywork

2042

2311

42.26

0.92


Teacher 8 said: “This is expressed all the time. When imparting skills inall age groups and enabling them to locate sources of information, askquestions and carry out different surveys.” (4) The change in the scienceteacher’s role in the classroom and proper integration of pedagogy in thescience lessons: Teacher 6 said: “We taught a book, the in-service educationrevolutionized my thinking and perception as a teacher, the nature of thework, and clarified the relation between contents and pedagogy.”

Scientific Inquiry and/or Technological Method

Examples for the contribution to scientific projects that include thescientific inquiry and/or technological method mentioned by the scienceteachers are in the field of research activity and scientific projects andtheir assimilation in the classroom as well as passing the focus to thestudent in these areas: Teacher 3 said: “In constructing projects this gaveme a lot: focused work, personal work on projects is different from frontalwork. More is demanded of the students and they are developed to agreater degree when carrying out projects.”

Teamwork

In the interviews, the teachers indicated the contribution of teachereducation to their teamwork in several aspects, one of which is thedevelopment of a learning professional community in the school: Teacher1 said: “Teamwork was a concept that was not clear, to be a partner in abroad community of teachers, to tear down the walls of the school. Theteamwork opened horizons.” Another aspect is sharing information, work,and support of the staff: Teacher 13 said: “Work in teams contributes alot. We discuss difficulties or prepare tests together. Each one simplycontributes his part, work is very fertile. It seems to me that teamworkwas created in many schools out of commitment. Cooperation contributesto a feeling of belonging.”

The following quote summarizes the most significant contribution inone of the teacher’s own words: Teacher 7 said: “Without the staff I amnothing.”

Attitude Towards the Teacher Education

The interviews also raised the teachers’ attitudes towards the contributionof the PD in general. There were references to the team of lecturers andinstructors, the subjects of the education, and the personal attention. Theattitudes towards the PD were regarding professionalism and the


professional team that led the teacher education: Teacher 14 said: “It isvery good to have good and excellent instructors who work with us andthere is someone who can be asked about any question.” The teachersalso received personal and individual attention according to need: Teacher12 said: “In previous years an instructor accompanied us for the entireyear according to need or feeling. Today it is more according to our needsand requirements.”

In conclusion, the interviews indicate that the teachers benefited on thepersonal level and on the level of implementation in the classroom. Thesefindings, as raised by the interviews, reinforce the findings obtained fromthe questionnaires, especially regarding teamwork, teaching–learningstrategies, and scientific inquiry. The teachers raised subjects in which theywant to develop further and in which they need help and support, such asbuilding lesson plans, analyzing scientific texts, varying teaching methods,and integrating topics learned in the ninth grade, such as energy. Whenasked about additional ideas and suggestions, the teachers asked for moreactivities for the students, prepared suggestions for lesson plans on thesubject of skills: understanding different types of texts, graphs, and tables.

Findings of the External Evaluation

The second research question examined whether an improvement tookplace in the students’ large-scale assessment achievements after theteachers’ education. We used the external evaluation findings for scienceachievements obtained in the TIMSS and the GEMS in order to answerthis question. The GEMS findings were obtained from the Ministry ofEducation and the TIMSS data from the TIMSS site. Individualevaluation for each of the schools between the two GEMS tests (beforeand after 3 years of teacher education) was also carried out. The findingsobtained from the external tests taken after the teachers’ educationprogram indicates improvement in the students’ achievements. Teachersreported (in the interviews) a contribution of the PD on the quality of theirteaching and their students’ achievements. A significant increase occurredin the students’ achievements in the GEMS and in the TIMSS. Theachievements of 99% of the schools from the central district thatparticipated in the GEMS improved (from a mean of 66% to a mean of80%), 1% did not improve, but did not regress (Ministry of Education,2005). An improvement in student achievements in science was alsoobserved in the TIMSS. The mean scores in science increased and are abovethe world average (place 26 to place 23). The results of this research indicatethat the science teachers feel confident in teaching the topics and this sense


of confidence is positively related to learning achievements (Zuzovsky,2005). In conclusion, an increase was observed in the science achievementsof students in the external tests (TIMSS and GEMS).

DISCUSSION

This research presents a process that began following the low achieve-ments in sciences of students in Israel in external assessment frameworks(TIMSS, PISA, and GEMS). These low achievements indicated severalweakness foci: in the fields of content, scientific inquiry, integration ofskills, and even in the structure and formulation of test items developedby the teachers. Development of a professional education system for thescience teachers was carried out based on the findings of externalevaluation frameworks together with the requirements of the curriculum.This is similar to other countries where the education system isundergoing changes following findings of large-scale assessment (Lin,2008). The evaluation findings served as a lever for dealing in topicswhich integrate assessment and teaching–learning processes (Shulman,1997). The goals of the program were to construct a PD programaccording to the principles of a professional learning community withpragmatic achievable aims which meet the teachers’ and the nationaleducation system’s needs.

Teachers’ Attitudes Towards the Contribution of the PD

Many teachers reported a contribution of the PD on the quality of theirteaching and their students’ achievements in all examined components.These findings reinforce our estimate that we located the weak points inthe students’ achievements as reflected in the external assessment. One ofthe main problems that we identified was a gap between the structure andstyle of the external tests and the tests constructed by the teachers. Thequestionnaires indicate that one of the fields which made the greatestcontribution to the teachers is the field of evaluation: constructing tests,with the greatest contribution attributed to raising awareness to good testitems; integration of a diversity of skills in test items; integration ofdifferent cognitive levels; improvement in the quality of the items anddiversity in test items. As indicated by Figure 1, the external assessmentframeworks that were mediated by the science teachers’ educationinfluenced their construction of tests. Sharing knowledge within theframework of a professional community is important for the PD processes


of the teachers and helps them transfer this knowledge into theirexperience in the classroom (Fullan, 2001; Wenzlaff & Wieseman, 2004).

All the teachers reported a contribution to their teamwork and to theirenrichment in knowledge and ideas. They reported a great contribution inthe field of sharing knowledge and ideas and in strengthening the teamworkbetween the science teachers in the school and the professional community.The site that accompanied the teacher education contributed to the teachersin the field of contents as well as in mutual enrichment between members ofthe community. The NSB (2008) reports positive attitudes of scienceteachers towards PD. These teachers graded the activities on subjectcontent and use of computers for instruction as “useful” or “very useful.”These findings reinforce previous researches on teachers’ PD whichdiscuss the characteristics of the PD that leads to a positive change inteaching–learning processes, such as the duration, teamwork, and contents(Cohen & Hill, 2001; Madigan, 2001; NSB, 2008).

The students’ Achievements

The results in science that were obtained from the two external evaluationframeworks, i.e. the GEMS (2004–2005) and the TIMSS (2003), werehigher after the teacher education than before. It is difficult to isolate thevariables and determine what influenced the students’ achievements. Inthe GEMS, the schools receive the achievements of their studentsaccording to class and according to the results in each subject in everyclass. In the external international evaluations, the results are national andit is difficult to isolate the results of the central district. We assume that,since the central district is the largest district in the country, its relativeweight is large and the test results can be projected on the district. The datawe received from the Ministry of Education indicate a decrease in theparticipation of science teachers, except in the central district where the PDwas carried out according to the model that we constructed. The findingsalso indicate that the students’ achievements in science in the central districtincreased during the years in which science teachers participated in anintensive PD program. The mean total increase in students’ achievements inscience in the central district is 14%. Another finding that reinforces theimprovement in the students’ achievements is the decrease in the number ofstudents with low achievements (22% to 8%) compared to an increase in thenumber of students with high achievements (10% to 45%; Ministry ofEducation, 2005). The students’ achievements in the TIMSS also improved.

According to Guskey’s (1998) model, we can evaluate the PDprogram. The criterion of what the participants learned indicates that


they acquired new knowledge in teaching–learning processes and in thefield of evaluation to a great or very great extent. The criterion thatexamines the extent to which the participants use the new knowledge andskills was found to be implemented in the teachers’ classrooms.Regarding the criterion of the students’ learning achievements, it can besaid that the teachers reported that changes took place in the learningprocesses. We examined only achievements in external tests and found animprovement. There is relatively little research that directly links PD tostudent learning. This is due, in part, to the difficulty in establishing aclear connection between the two. In the present research, it is also difficultto link between the PD and student achievements, beyond the fact that thePD improved the teachers’ teaching methods and that the students’ scoresin the external tests improved. Almost all studies on the relation betweenPD characteristics and improvements in student learning focus on studentachievements in mathematics and/or science (Cohen & Hill, 2000;Kennedy, 1998, 1999; Wenglinsky, 2002).

There is one criterion that we did not examine, i.e. the support of theorganization in integration and innovation. Implementation in theeducational institution is most probably supported by the Ministry ofEducation, since this PD program is funded by the ministry. Thus, thefour components of evaluating a PD program were definitely expressed inthis model.

CONCLUSION

The science teachers’ education that we developed is in agreement withother studies on the desired structure and contents of an effective PDprogram that leads to changes in the teaching–learning process. Theuniqueness of our model is in relying on broad external evaluationframeworks for reflecting the strengths and weaknesses in the learningcontents and skills and the teaching method as well as dictating the needsrequired for the science teachers’ PD. The needs that were taken intoaccount in the development of the teacher education were raised not bythe teachers but rather from the external assessment frameworks whichbecame the standards system for constructing the program. It is importantto plan the program such that it will maximize the effective change in theclassroom and will lead to an improvement in the students’ achievements.Large-scale assessment may afford the teachers effective PD whichincludes development of tests, indicators, and curricula to improveevaluation strategies in the classroom (NRC, 2003).


REFERENCES

Clewell, B. C., Campbell, P. B., Perlman, L. (2004). Review of evaluation studies ofmathematics and science curricula and professional development models. Submitted tothe GE Foundation. Washington, DC: Urban Institute.

Cohen, D. K., & Hill, H. C. (2000). Instructional policy and classroom performance: Themathematics reform in California. Teachers College Record, 102(2), 294–343.

Cohen, D. K., & Hill, H. C. (2001). Learning policy: When state education reform works.New Haven: Yale University Press.

Darling-Hammond, L. (1998a). Teacher learning that supports student learning.Educational Leadership, 55(5), 6–11.

Darling-Hammond, L. (1998b). Teachers and teaching: Testing policy hypotheses from aNational Commission Report. Educational Researcher, 27(1), 5–15.

Erickson, F. (1998). Qualitative research methods for science education. In B. J. Fraser, &K. G. Tobin (Eds.), International handbook of science education (pp. 1157–1173).Dordrecht, NL: Kluwer Academic.

Fullan, M. G. (2001). Leading in a culture of change. San Francisco: Jossey-Bass.Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., Youn, K. S. (2001). What makesprofessional development effective? Results from a national sample of teachers.American Educational Research Journal, 38, 915–945.

Glenn, J. (2000). Before it's too late. Washington, DC: US Department of Education.Guskey, T. R. (1998). The age of our accountability. Journal of Staff Development, 19(4),36–44.

Guskey, T. R. (2003). What makes professional development effective. Phi DeltaKappan, 84(10), 748–750.

Hoban, G. F. (2002). Teacher learning and educational change: A systems thinkingapproach. Buckingham: Open University Press.

Hoban, G. (Ed.) (2005). The missing links in teacher education design—developing amulti-linked conceptual framework. Dordrecht: Springer.

Kennedy, M. M. (1998). Form and substance in in-service teacher's education (Researchmonograph no. 13). Madison, WI: National Institute for Science Education, Universityof Wisconsin.

Kennedy, M. M. (1999). Form and substance in mathematics and science professionaldevelopment. NISE Brief, 3(2), 1–7. Madison, WI: National Institute for ScienceEducation, University of Wisconsin.

Lin, H. S. (2008). The impact and implications of large-scale assessments on scienceeducation. Paper presented at Conference of Asian Science Education, Kaohsiung,Taiwan.

Loucks-Horsley, S., Love, N., Stiles, K. E., Mundry, S., & Hewson, P. W. (2003).Designing professional development for teachers of science and mathematics.Thousand Oaks, CA: Corwin.

Madigan, K. (2001). Strategies for improving teacher quality. Washington, DC: AmericanBoard for Certification of Teacher Excellence.

Ministry of Education (2005). The education system in light of the GEMS—the centraldistrict. Jerusalem: Ministry of Education (In Hebrew).

National Research Council (NRC) (2003). Assessment in support of instruction andlearning: Bridging the gap between large-scale and classroom assessment—Workshopreport. Washington, DC: National Academies Press.


National Science Board (NSB) (2008). Science and engineering indicators 2008 (volume1, NSB 08-01; volume 2, NSB 08-01A). Arlington, VA: National Science Foundation.

Patton, M. Q. (1980). Qualitative evaluation. Beverly Hills, CA: Sage.Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers onfuture student academic achievement. Knoxville, TN: University of Tennessee.

Shulman, L. S. (1997). Communities of learners and communities of teachers.Monographs from the Mandel Institute (In Hebrew).

Stake, R. E. (2000). Case studies. In N. K. Denzin, & Y. S. Lincoln (Eds.). Handbook ofqualitative research (2nd ed., pp. 435–454). Thousand Oaks, CA: Sage.

TIMSS (1999). Retrieved March 12, 2009 from http://timssandpirls.bc.edu/timss1999i/science_achievement_report.html.

TIMSS (2003). RetrievedMarch 12, 2009 from http://timss.bc.edu/timss2003i/scienceD.html.Vannatta, R. A., & Fordham, N. (2004). Teacher dispositions as predictors of classroomtechnology use. Journal of Research on Technology in Education, 36(5), 253–272.

Viadero, D. (2005). Pressure builds for effective staff training. Education Week, 43(24),1, 18–19, 21.

Wenglinsky, H. (2002). How schools matter: The link between teacher classroompractices and student academic performance. Education Policy Analysis Archives, 10(2).Retrieved March 12, 2009, from http://epaa.asu.edu/epaa/v10n12/.

Wenzlaff, T. L., & Wieseman, K. C. (2004). Teachers need teachers to grow. TeacherEducation Quarterly, 31(2), 113–125.

Zhan, S. X., & Robertson, G. (2009). The implementation of task-based approach in aprofessional learning community: Why, what and how language teachers change.Retrieved March 12, 2009 from http://ocs.sfu.ca/aare/index.php/AARE_2008/AARE/paper/view/411.

Zuzovsky, R. (2005). Teachers’ qualifications and their impact on student achievement:Findings from TIMSS 2003 data for Israel. Retrieved November 19, 2009 from http://www.ierinstitute.org/IERI_Monograph_Volume_02_Chapter_02.pdf.

Aviva Klieger

Secondary Education Department and Natural Science DepartmentBeit Berl Academic CollegeDoar Beit Berl, 44905, IsraelE-mail: [email protected]

Nurit Bar-Yossef

Hakibbutzim College of Education and Ministry of EducationTel Aviv, IsraelE-mail: [email protected]


http://timssandpirls.bc.edu/timss1999i/science_achievement_report.html

http://timssandpirls.bc.edu/timss1999i/science_achievement_report.html

http://timss.bc.edu/timss2003i/scienceD.html

http://epaa.asu.edu/epaa/v10n12/

http://ocs.sfu.ca/aare/index.php/AARE_2008/AARE/paper/view/411

http://ocs.sfu.ca/aare/index.php/AARE_2008/AARE/paper/view/411

http://www.ierinstitute.org/IERI_Monograph_Volume_02_Chapter_02.pdf

http://www.ierinstitute.org/IERI_Monograph_Volume_02_Chapter_02.pdf

professional development of science teachers as a reflection of large-scale assessment

Documents