High School Science Teacher Engagement and Motivation in Formative Classroom Assessment

Andy Boyd Mathematics and Science Educator Almira/Coulee-Hartline High School Heritage University

Mike Trevisan Dean and Professor College of Education Washington State University

Purpose The purpose of this research was to conduct a beginning, investigative study of the current status of teacher engagement in and motivation for formative classroom assessment, for Washington state high school science teachers. In order to complete this research, a survey instrument developed by Stanton (2011) around self-determination theory, was adapted and used to measure high school science teachers’ satisfaction of motivational needs, self-determined motivation, and engagement regarding formative assessment of student learning. For an assessment to be formative, the data collected from the assessment needs to provide information so changes are made during the instruction period in which the assessment was given. The results of formative assessments need to lead to actual adjustments in instruction from the teacher or the way in which students are trying to learn the information being presented (Popham, 2008a; Stiggins, 2008). Data from this survey instrument were used to answer the following research questions for this study:

Research Question 1: What are the existing levels of engagement in formative classroom assessment for Washington state high school science teachers? Research Question 2: What are the existing levels of motivation for the use of formative classroom assessments for Washington state high school science teachers? Research Question 3: To what extent do demographic variables of Washington state high school science teachers affect their levels of engagement and motivation in formative classroom assessment?

Perspectives and theoretical framework Formative classroom assessment is an important component of the science inquiry process (e.g., Michaels, Shouse, & Schweingruber, 2008; Bass, Contant, & Carin, 2009) and is integral for improving student understanding (e.g., Atken & Coffey, 2003; Black & Harrison, 2001; Black & Wiliam, 1998a). Chiappetta (1997) argues that “teaching science as inquiry stresses active

student learning and the importance of understanding a science topic. Here content becomes a critical aspect of the inquiry.” (p. 23) Throughout this inquiry process, continual checking of student understanding is essential to monitor the progress of students. Classroom activities geared toward assessing student understanding are fundamental aspects of the teaching and learning of science concepts and impact almost every aspect of a classroom teacher (Brookhart, 1999; Brookhart, 2003; Brookhart, 2007; Popham, 2008a; Stiggins, 2008).

Multiple science education reform efforts describe the importance and effectiveness of formative assessments (Popham, 2008b; Stiggins, 2008; Keeley, 2008). Nationally, 25 states have a formal policy on the use of formative assessments by districts and schools (Altman, Lazarus, Quenemoen, Kearns, Quenemoen, & Thurlow, 2010). Given this national investment in professional development, gaining knowledge and insight on science teacher engagement in and motivation for formative assessment could be beneficial. This knowledge could help school districts further target professional development needs for teachers. Engagement and motivation for formative classroom assessment are critical variables to productively using formative assessment as a classroom instructional strategy. The basis for this study was adapted from the work of Stanton (2011) who studied engagement and motivation in formative classroom assessment among college level engineering faculty. Stanton (2011) defined engagement as being composed of a set of affective constructs (e.g. teacher interest/enjoyment in formative classroom assessment) and behavioral constructs (e.g. effort/importance of the formative classroom assessment). Motivation for formative classroom assessment was based on self-determination theory (Ryan & Deci, 2000) which identifies the constructs external regulation, introjected regulation, identified regulation, and intrinsic motivation as central components (Stanton, 2011). Figures 1 and 2 shows the engagement and motivation domains, respectively, the complete set of constructs, descriptions for each, and how the constructs relate to one another. National Science Education Standards (1996) identifies and communicates learning and performance goals appropriate for specific grade levels of students. Classroom Assessment and the National Science Educational Standards (2001) offer a general template for designing and integrating formative assessment into regular classroom practice. The template, along with the other supporting research (Cowie & Bell, 2001; Bell & Cowie, 2001; Brookhart, 2003; Clarke, 2005; Popham, 2008b; Stiggins, 2008; Keeley, 2009, etc.), provides a solid foundation and common language to help identify different classroom formative assessment techniques that teachers can implement and use. A major component for successful implementation of formative processes and strategies is the teacher’s engagement with these ideas and motivation for use. In Washington State, school districts are using professional development models and frameworks to promote the use of formative classroom assessments within their schools (Washington State Leadership and Assistance for Science Education Reform, 2010). This professional development has and continues to be conducted at the elementary, middle, and high school levels and for mathematics, science, reading, and language arts content. No work could be found that documents the extent to which science teachers are engaged in the formative assessment processes and strategies learned, nor their motivation for doing so. Methods, techniques and data sources This study used a survey instrument to gather data critical to addressing the research objectives. In the fall of 2010, an invitation to participate in the study was sent to 1181 high school science teachers, representing 90% of the population of secondary science teachers in Washington State. Of the 1181 emailed requests, a 15% sample of the population or 181 initial responses were gathered. This sample size is reasonable as per the busy nature of respondents, time and cost

constraints (Slavin, 2007; Desimone & Le Floch, 2004, Mertler, 2003). The survey was administered electronically using the software program, SurveyMonkey. The instrument used in this study is a two-component survey originally designed to assess engagement and motivation for use of formative classroom assessment among engineering education faculty (Stanton, 2011). Minor modifications were made to a small number of items to reflect the high school science education context. Figure 3 presents example items representing each construct to show a general layout of the instrument. The first component of the survey provides two classroom assessment scenarios. Teachers were required to provide open ended responses to the scenarios. The responses were used to determine the extent to which teachers sufficiently understand formative classroom assessment in the context of high school science classes. A 10-point scoring rubric provided by Stanton (2011) was used to evaluate the extent to which the response shows sufficient teacher understanding of formative classroom assessment. Based on the score, 141 teacher surveys were selected. 13 responses were removed because of missing data resulting in 128 responses for analysis. Demographics include the following: 81 teachers identified as female (63%) and 47 as male (37%); 123 identified as white (96%); 125 (97.6%) had taken 5 or more semesters of college science courses; 80 teachers (62.5%) reported teaching 11 or more years. The Office of the Superintendent of Public Instruction (OSPI) published a report on the current state of science education in Washington State stating 3,620 valid teaching certificates are associated with secondary science employment (OSPI, 2011). OSPI did not have specific secondary science educator demographic data; however, the state reported secondary classroom demographic data. Table 1 provides the complete set of demographic sample and population data to support the argument the characteristics are similar. The second component of the survey consists of 88-items that operationalize the constructs that make up engagement and motivation. Each item is rated with a 7-point Likert scale ranging from (1) “not very true” to (7) “very true”. These responses were aggregated at the construct and overall levels to address the research questions. Coefficient alpha reliability indices ranged from a low of 0.63 for the engagement construct, “choice of behavior” to a high of .90 for the

engagement construct, “value and usefulness.” Thus, there was sufficient reliability in the construct scores to warrant the statistical analyses. Results and/or substantiated conclusion or warrants for arguments/points of view Means and standard deviations were computed for all items and constructs. These data were aggregated and disaggregated in various ways to obtain overall engagement and motivation scores as well as scores for demographic variables. Ninety-five percent confidence intervals were also computed for each construct score. In general, teachers responded with moderate to moderately high ratings for most constructs that make-up engagement and motivation. Most statistical comparisons were not significant. Exceptions within the engagement construct of choice of behavior are noted. The number of years taught had a statistically significant difference in the mean reported score for engagement in formative assessment for the construct, choice of behavior, (F(6,118) = 2.697,

p = 0.017). A Tukey post-hoc test showed teachers with 26 or more years of experience provided significantly lower (4.44 ± 1.93, P = 0.007) ratings compared to teachers who taught 0 – 2 years (6.25 ± 0.20). In regards to gender, the construct choice of behavior in engagement in formative assessment, female teachers reported significantly higher ratings (5.58 ± 0.98) versus male teachers (4.96 ± 1.21) (t(125) = 3.152, P = 0.002). Although still preliminary, ratings of items from the affective and behavioral components of engagement show Washington state high school science teachers experience positive engagement in formative classroom assessment. The teachers responding found value and usefulness in formative assessment as a way to help them in their classrooms and improve student learning. These teachers report to be persistent in performing formative assessments. Teachers also report a feeling of satisfaction from the use of formative assessments as well as classifying it both interesting and enjoyable. Finally, these teachers intend to continue the use of formative assessments in their classrooms. Though statistical differences for engagement were found for gender and years of teaching experience, all ratings were moderately high or high. In part, findings can be explained by self-determination theory which suggests that women tend to report higher levels of self-determination motivation than men and that the more experienced a professional is, the less they feel the need to respond to external forces without accepting it as their own (Vallerand, 1997; Vallerand & Ratelle, 2002). Scientific or scholarly significance of the study or work Research has shown the importance of developing, implementing, and using formative classroom assessment as part of the inquiry instructional process. Not only does formative classroom assessment improve the instructional practices of the teacher, it also helps students understand their misconceptions around content topics. The use of formative classroom assessment also involves students more actively in their own learning through the specific feedback from their teacher. As shown in this study, overall, science teachers in Washington State who understand the formative classroom assessment process largely experience positive engagement in formative classroom assessment. These Washington state high school science teachers identify formative classroom assessment as an important component to the teaching and learning process. Teachers in this study who understand formative classroom assessment rate their interest, enjoyment, and satisfaction very high when using formative classroom assessments. Washington science teachers also report that these assessments are valuable and useful in their classrooms and more importantly high school science teachers feel formative classroom assessment practices are a responsibility of effective teachers. This study holds potential to fill an important gap in the current empirical literature concerning teachers’ engagement in and motivation for formative classroom assessment, particularly

Washington state high school science teachers. A large percentage of the Washington state high school science teachers who responded to the survey were able to give valid examples of formative classroom assessment techniques and activities. Expanded use of the survey instrument could be a means to replicate this study with different populations of teachers across the United States, potentially improving the generalizability of the findings.

Findings from the current study could be used to further bolster the argument for the merits of formative classroom assessment professional development, particularly at this time when budgets are constrained. The findings could also be used to develop and target professional development in meaningful ways to meet the needs of particular teachers. Formative classroom assessment is not only necessary to create effective teachers, but as research shows, it is crucial in every aspect of education. Understanding the connection between formative classroom assessments and learning progressions can enhance the description of how a student’s thinking develops over a period of time. Also, providing time for students to monitor

their own learning through feedback and reflection of their own thinking is critical in inquiry instruction.

References Altman, J. R., Lazarus, S. S., Quenemoen, R. F., Kearns, J., Quenemoen, M., & Thurlow, M. L. (2010). 2009 survey of states: Accomplishments and new issues at the end of a decade of change. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. American Association for the Advancement of Science. (1993). Benchmarks for science

literacy. New York, NY: Oxford University Press.

Atkin, J. M. & Coffey, J. E. (Eds) (2003). Everyday Assessment in the Science Classroom (pp.

41 – 59). Arlington, VA: NSTA Press.

Bass, J. E., Contant, T. L., & Carin, A. A. (2009). Teaching Science as Inquiry. Boston, MA: Person Education, Inc. Bell, B & Cowie, B. (2001). The characteristics of formative assessment in science education.

Science Education 85(5): 536 -553.

Black, P. J., & Harrison, C. (2001). Self- and peer-assessment and taking responsibility:

The science student’s role in formative assessment. School Science Review,83(302),

43 – 49.

Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in

Education: Principles, Policy & Practice, 5 (1), 7-74.

Bransford, J., Brown, A., and Cocking, R. (2000) How people learn: Brain, mind, experience

and school. Washington, DC: National Academy Press.

Brookhart, S. M. (1999). Teaching about communicating assessment results and grading.

Educational Measurement: Issues and Practice, 18 (1), 5-13.

Brookhart, S. M. (2003). Developing measurement theory for classroom assessment purposes

and uses. Educational Measurement: Issues and Practice, 22, 5–12.

Brookhart, S. M. (2007). Expanding views about formative classroom assessment: A review of

the literature. In J. H. McMillan (Ed.), Formative classroom assessment: Research,

theory and practice. New York, NY: Teachers College Press.

Chiappetta, E. (1997). Inquiry-based science. The Science Teacher, 65(7), 22-26.

Clarke, S. (2005). Formative assessment in the secondary classroom. London, UK: Hodder

Murray.

Cowie, B. & Bell, B. (1999). A model of formative assessment in science education, Assessment

in Education, 6(1), 101-116.

Desimone, L. M., Le Floch, K. C. (2004). Are we asking the right questions? Using cognitive

interviews to improve surveys in education research. Educational Evaluation and Policy

Analysis, 26(1), 1-22.

Dillman, D. A., Smyth, J. D., Christian, L. M. (2009). Internet, mail, and mixed-mode surveys:

The tailed design method. Hoboken, NJ: John Wiley & Sons, Inc.

Keeley, P. (2005). Science curriculum topic study: Bridging the gap between standards and

practice. Thousand Oaks, CA: Corwin Press.

Keeley, P. (2008) Science formative assessments: 75 practical strategies for linking assessment,

instruction, and learning. Arlington, VA: NSTA Press.

Mertler, C. A. (2003). Patterns of response and nonresponse from teachers to traditional and web

surveys. Practical Assessment, Research, & Evaluation, 8(22). Retrieved July 11, 2013

from http://PAREonline.net/getvn.asp?v=8&n=22.

Mertler, C.A. & Vannatta, R. A. (2005). Advanced and multivariate statistical method: Practical

application and interpretation. (3rd

ed.). Glendale, CA: Pyrczak Publishing.

Michaels, S., Shouse, A. W., & Schweingruber, H. A. (2008). Ready, Set, Science! Putting Research to Work in K-8 Science Classrooms. Board on Science Education, Center for Education, Division of Behavior and Social Sciences and Education. Washington, DC: The National Academies Press. Office of Superintendent of Public Instruction. (2011). Science and stem education in

Washington state: background and current landscape. Olympia, WA: OSPI.

Popham, W. J. (2008a). Classroom assessment: what teachers need to know (5th ed.). Boston,

MA: Allyn & Bacon.

Popham, W. J. (2008b). Transformative assessment. Alexandria, VA: Association for

Supervision and Curriculum Development.

McManus, S. (Ed.) (2008). Attributes of effective formative assessment. Washington, DC:

Council of Chief State School Officers.

National Research Council. (1996). National science education standards. Washington, DC:

National Academy Press.

National Research Council. (2001). Classroom assessment and the national science education

standards. Washington, DC: National Academy Press.

Shepard, L. A. (2000). The role of classroom assessment in teaching and learning. (CRESST

Technical Report No. 517). Los Angeles: University of California, National Center for

Research, on Evaluation, Standards, and Student Testing (CRESST).

Skinner, E. A., Furrer, C., Marchand, G., & Kindermann, T. (2008). Engagement and

disaffection in the classroom: Part of a larger motivational dynamic? Journal of

Educational Psychology, 100, 765 – 781.

Slavin, R.E. (2007). Educational research in an age of accountability. Boston, MA: Pearson

Education, Inc.

Stanton, K. C. (2011). Engineering Faculty Motivation for and Engagement in Formative

Assessment (Doctoral dissertation). Retrieved from http://scholar.lib.vt.edu/edt/

Stiggins, R. (2008). Student-involved classroom assessment (3rd ed.). Upper Saddle River, NJ:

Prentice-Hall.

Vallerand, R. J. (1997). Toward a hierarchical model of intrinsic and extrinsic motivation. In M.P. Zanna (Ed.), Advances in experimental social psychology (pp. 271-360). New York: Academic Press

Vallerand, R. J., & Ratelle, C. F. (2002). Intrinsic and extrinsic motivation: A hierarchical model. Dans E. L. Deci and R. M. Ryan (Eds.), The motivation and self-determination of behavior: Theoretical and applied issues. Rochester, NY: University of Rochester Press.

Washington State Leadership and Assistance for Science Education Reform. (2010). Alliances [Description of professional development]. Retrieved from http://www.wastatelaser.org/

Figure 1: Engagement Constructs

Domain Specific Construct Description

Seeing worth in formative

assessment use

Interest/Enjoyment

Positive emotion (value

and usefulness)

Inclination toward

formative assessment

Affect

Behavior

Satisfaction

Anxiety

Feeling satisfied with

formative assessment use

Expected to have a

negative relationship with

self-determined

motivation

Choice of behavior

Persistence

Intent

Engaged in using various

methods, collecting data,

analyzing data, and

applying results to

improve teaching and

learning

Staying on-task even

when faced with negative

results or difficulties

Intentions to use

formative assessment

Figure 2: Self-determination theory continuum (Ryan & Deci, 2000b, p.72)

Figure 3: Examples of questions from the survey instrument representing the constructs of engagement and motivation

Engagement

The following questions pertain to your engagement in formative assessment. Please indicate

how true each of the following statement is for you given your experiences doing assessment or

your plans to do assessment.

Not very true (1) 2 3 Somewhat true (4) 5 6 Very true (7)

1. I enjoy doing formative assessment very much.

2. I think that doing formative assessment is useful for improving my teaching.

3. I am sometimes anxious while reviewing assessment results.

4. I find myself choosing to do assessment over other teaching tasks.

5. I pay attention and respond when assessment results show concern for student learning.

6. I intend to learn more ways to assess my students’ learning.

7. I get a sense of fulfillment from applying assessment results to improve my teaching.

8. I intend to learn better ways to apply assessment results to improve my teaching.

9. When balancing my teaching duties, formative assessment is an important priority.

10. I put a lot of effort into assessing my students.

Motivation

The following questions concern your feelings about your formative assessment efforts or your

plans to use formative assessment, as well as your connection with other educators you know

who assess their students’ learning.

Not very true (1) 2 3 Somewhat true (4) 5 6 Very true (7)

1. I have learned the essential skills needed to assess student learning. 2. I would like to share assessment results with other educators. 3. There is not much opportunity for me to decide for myself how to go about assessing my students. 4. I have reason to believe I do a good job at assessment. 5. I feel formative assessment is valued by my colleagues. 6. I am able to choose assessment methods based on my own goals for teaching and learning. 7. I do not feel very competent when trying to assess my students’ learning. 8. I am free to express my ideas and opinions on how assessment should be done. 9. I feel formative assessment keeps me in touch with my students and their education. 10. Most times, I feel I’m effective at assessing my students’ learning. 11. If it came up, I would be willing to share both positive and negative assessment results with

other educators. 12. I feel like I can pretty much be myself when assessing my students. 13. I believe people at work would appreciate me for my assessment efforts. 14. I feel pressured to do assessment. 15. When I think about assessment, I often do not feel very capable. 16. When it comes to assessment, I have to do what I am told. 17. I feel like I have a lot of control in deciding how I assess my students’ learning.

Table 1: Demographics of the 128 secondary science teacher responses whose data was used in the analysis.

Category Response Count Percent of Sample

(95% Confidence interval)

Washington State

Secondary Teachers

Gender Female

Male

81

47

63% (56%, 71%)

37% (29%, 45%)

53.61%

46.39%

Ethnicity

(may choose

more than one)

American Indian or

Alaska Native

5

0.04%

0.83%

Asian 4 0.03% 2.08%

Black or African

American 2 0.02% 1.28%

Hispanic or Latino 2 0.02% 2.87%

Native Hawaiian or

Other Pacific Islander 1 0.01% 0.15%

White or Caucasian 123 96% 89.14%

Age Less than 22 years

22 – 29

30 – 39

40 – 49

50 – 59

Greater than 59

0

11

45

30

35

7

0%

9% (4%, 14%)

35% (27%, 43%)

23% (16%, 30%)

27% (20%, 35%)

6% (2%, 10%)

Average age- 47.5

years

Number of

college science

courses

None

1 semester

2 semesters

3 semesters

4 semesters

5 or more semesters

0

0

1

0

1

125

0%

0%

0.8% (0%, 2%)

0%

.08% (0%, 2%)

98% (96%, 100%)

Highest degree

attained

Bachelors

Masters

Doctorate

20

103

4

16% (10%, 22%)

81% (74%, 87%)

3% (1%, 6%)

Teacher with

masters or higher –

67.1%

Number of years

teaching

0 – 2 years

3 – 5 years

6 – 10 years

11 – 15 years

16 – 20 years

21 – 25 years

26 or more years

10

23

25

31

17

11

9

8% (4%, 13%)

18% (12%, 25%)

20% (13%, 27%)

25% (17%, 33%)

14% (8%, 19%)

9% (4%, 14%)

7% (3%, 12%)

Average number of

years teaching-

14.2 years

Grade level

taught

9th grade

10th grade

11th grade

12th grade

34

55

33

5

27% (19%, 35%)

43% (35%, 52%)

26% (19%, 34%)

4% (0.8%, 8%)

Subject taught Physical Science

Biology

Chemistry

Physics

AP (AP Biology, etc)

28

58

27

12

2

22% (15%, 29%)

46% (37%, 54%)

21% (14%, 28%)

9%, (5%, 15%)

2% (0%, 4%)

Number of

students per

period

Less than 15

15 – 25

26 – 35

More than 35

5

28

92

1

4% (0.8%, 7%)

22% (15%, 29%)

73% (66%, 81)

0.8% (0%, 2%)

High school

classification

1B

2B

1A

2A

3A

4A

4

9

19

18

32

37

3% (0.8%, 7%)

8% (3%, 13%)

16% (9%, 23%)

15% (9%, 21%)

27% (19%, 34%)

31% (23%, 40%)

Educational

Service District

(ESD)

ESD 101-Northeast

ESD 105

ESD 112

ESD 113

ESD 114 – Olympic

ESD 121 – Puget Sound

ESD 123

ESD 171 – North Central

ESD 189 - Northwest

8

9

17

12

7

36

12

7

14

7% (3%, 11%)

7% (3%, 12%)

14% (8%, 21%)

10% (5%, 16%)

6% (3%, 11%)

30% (21%, 39%)

10% (5%, 16%)

6% (2%, 10%)

12% (6%, 17%)