coming to know and understand alternative assessment in science

Journal of Science Teacher Education, 7(4), 267-282 0 1996 Kluwer Academic PubIishers. Printed in the Netherlands

Model Programs

Coming to Know and Understand Alternative Assessment in Science

Daniel P. Shepardson

School Mathematics and Science Center, Department of Curriculum and Instruction, 1442 Liberal Arts and Education Building, Purdue University, West Lafayette, Indiana 47907-1442, USA

Paul E. Adams

Department of physics, Fort Hays State University, Hays, Kansas 676014099, USA

Having a traditional versus alternative assessment assignment was beneficial in that it showed very well that some students didwell and some didn’t do well on the tests because of the format. The alternative assessment showed that students think very difSerently from each other and that all students who received the same informationat the same time allprocessed the informationdi#$erently . (Bill, INLAB participant)

This article describes the professional development framework utilized in the INLAB (Integrating Laboratory Instruction and Assessment) project for changing teacher practice. The INLAB model revolves around the establishment of an intellectual community of learners that challenges understandings about practice, promotes reflection upon practice, and encourages dialogue and collaboration among peers.

The INLAB projectis basedonthepremisethatleamingis aconstructive process. Thus, teachers must construct their own purpose and meaning of laboratory instruction and assessment as it relates to their classroom community and school culture. This construction is facilitated through

Tbis material is based upon work supported by tbe National Science Foundation (Grant No. TPE-9154840). Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect tbe views of the NSF.

DANIEL P. SHEPARDSON & PAUL E. ADAMS

reflection and the negotiation of a shared meaning among participants and project staff. Project participants, however, bring with them what Shulman (1987) referred to as “wisdom-of-practice” knowledge; that is, participants have existing knowledge of laboratory instruction and assessment constructed from their experiences in classrooms and schools. Because these existing conceptions contribute to the meanings constructed by teachers, teachers must be actively involved in the professional development process (Hart & Robottom, 1990) in order for the experience to become relevant to their existing curricular, instructional (Feely, 1986), and assessment approaches. These existing conceptions contribute significantly to the shared meanings constructed by both participants and project staff.

The Knowledge Base

Shulman (1987) identified an array of knowledge that interacts to form one’s understanding of teaching. The knowledge base that has received the most attention by science education is pedagogical content knowledge. What appears to be missing from this construct is teacher assessment knowledge; specifically, the ability to apply assessment knowledge to the development and implementation of assessment techniques, to integrate assessment techniques that are both pedagogical and content appropriate and pedagogical in nature, and to create learning situations. Thus, INLAB staff and participants negotiated an understanding of pedagogical content assessment knowledge. The assumptions behind pedagogical content assessment knowledge are that:

1. the science content we know influences the ways we teach (pedagogy) and assess, as well as what we teach and assess;

2. the pedagogy we know influences the science content we teach and what we assess; and

3. what we know about assessment influences how we teach (assessment as a way of teaching) and the science content we teach (assessment as a way of learning).

The concept map created by Don, an INLAB participant, illustrates the general construct of pedagogical content assessment knowledge (see Figure 1). Don’s concept map displays pedagogical understanding in that he indicates that teachers may use concept maps to plan instruction and that students may use concept maps as a means for organizing hrformation. From an assessment perspective, Don’s concept map illustrates that concept maps may be used to directly assess student knowledge. Finally, Don’s concept map emphasizes the use of concept maps as a tool for organizing

ALTERNATIVE ASSESSMENT 269

Figure 1. Don's concept map about concept maps.

270 DANIEL P. SHEPARDSON & PAUL E. ADAMS

content for students, as well as for teachers, to determine what is taught.

The Professional Development Framework

The following sections provide descriptions and insights as to our experience in effectively collaborating with teachers to facilitate change in assessment practice. Professional development activities must engage teachers in learning experiences that build their knowledge, understanding, and ability; thus, shifting professional development activities from technical training to an emphasis on theoretical and practical understandings of practice (National Research Council [NRC], 1996). Further, professional development activities should occur within the context of teachers’ classrooms and schools, involve interactions among peers and students, and promote inquiry and reflection upon practice (NRC, 1996).

The INLAB project operationalized the NRC professional development standards through the establishment of a dialogical, reflective community that shaped participants’ ways of knowing and understanding assessment practice. The professional development strategy utilized Shulman’s (1987) model of pedagogical reasoning and action:

1. modeling ways of knowing and understanding assessment by project staff;

2. comprehending ways of knowing and understanding assessment through collaborative dialogue among staff and participants;

3. transforming and applying ways of knowing and understanding assessment in the classroom;

4. reflecting upon the classroom application of ways of knowing and understanding assessment through writihg;

5. sharing ways of knowing and understanding of assessment with peers and project staff through small-group, collaborative dialogue;

6. reflecting upon collaborative dialogue, leading to a new comprehension of assessment; and

7. developing/revising alternative assessment tools based upon the new comprehension.

To promote an intellectual, reflective atmosphere, INLAB instructional strategies involved the presentation of ideas in order to provoke a constructive process for understanding assessment. For example, participants were introduced to alternative assessment by project staff who modeled the use of alternative forms of assessment during an instructional activity on pendulums: (a) written response to open-ended questions, used prior to and at the conclusion of the laboratory activity; (b) skills checklist, used during the laboratory activity; (c) interview questions, also used during the


laboratory activity; and (d) a traditional, multiple-choice test administered at the end of the pendulum activity. This pedagogical-assessment experience served as the foundation for discussing and analyzing alternative forms of assessment with participants, with staff and participantsexpressingpersonal beliefs and understandings about assessment. Participants, in small groups, also discussed, analyzed, and reflected upon the alternative forms of assessment from the perspective of the teacher and student. Modeling and comprehending occurred through active participation and collaborative dialogue in workshop settings.

Transforming and reflecting upon alternative assessment was promoted through participants reading scholarly articles on assessment, developing and implementing alternative assessment tasks, videotaping practice and collecting student assessment data or artifacts, and reflecting upon their assessment practice. The videotape of practice, the student data or artifacts (work examples), the developed assessment tasks, and the self-reflections served as the focal points for the collaborative dialogue. Sharing and reflecting upon collaborative dialogue occurred in follow-up workshops where participants shared and engaged in dialogue about their assessment tasks and practice in small groups of staff and peers. The small-group sharing led to further reflection upon assessment practice and a new comprehension of assessment for both presenter and group members. Developing/revising alternative assessment tasks occurred in small groups of peers and resulted in the application of new ways of knowing and understanding assessment. The process then recycled when participants implement their new ways of assessing and conducted action research on their assessment approach.

The process incorporated the NRC (1996) professional development standards by engaging participants in learning about assessment through activities that involved actual students and utilized real student work as the basis for thoughtful reflection, interaction with colleagues, and application of practice. Specifically, participants observed good classroom assessment practice; reviewed assessment instruments and tasks; aligned curriculum, instruction, and assessment; selected and developed assessment tasks; analyzed and interpreted assessment data; and collaborated with others to evaluate student work, as well as develop and refine assessment tasks (NRC, 1996).

Alternative Assessment Assignment: An Example

The alternative assessment assignment presented here is an example of


the professional development process. Following the workshop on ways of thiiing about assessment and models of alternative assessment techniques, participants were assigned the task of developing and reflecting upon two alternative assessmentprototypes. Theassignment: (a)providedparticipants with practice in developing and implementing alternative assessment techniques, (b) promoted collaborative dialogue about assessment practice, (c) assisted participants in understanding that different assessment techniques provide different pictures of student performance, and (d) demonstrated the need for alternative assessment techniques. (See Appendix A for a copy of the assignment.)

The introductory quotation by Bill, along with Tina’s reflection below, depicts the value of engaging participants in reflecting upon their alternative assessment practice.

The process of trying to come up with differentforms of assessment was worthwhile in that it caused me to stop and really think about how to do something differentfrom what I had been doing.

Tina’s response illustrates the importance of the assignment in promoting teachers’ thinking about assessing their students differently. Not only does Bill’s reply emphasize the need to think about assessing students differently but, and perhaps more importantly, the different picture one constructs of students by using different assessment tasks.

The three teacher examples that follow illustrate the importance of engaging teachers in the development and implementation of alternative forms of assessment. Further, the teachers’ comments about their initial alternative assessment approaches reveal the importance of engaging teachers in the reflecting process. By engaging participants in critical reflection, change in practice is more likely to occur as participants transform the way practice is experienced and understood (Robottom, 1987).

Although Mary made note of the difficulties in using alternative assessment formats, she also identified the positive aspects of alternative forms of assessment:

Having the observable skills checklist helped me to focus on those specifics . . . . I prefer the observations checklist because I am looking at what a student is doing. Even though I may not always be correct, the more I use that kind of assessment, the more knowledge I have of what the student can do.


Mary, by identifying the difftculties, has a basis from which to revise her assessment approach; by acknowledging the positive attributes of alternative assessment, she recognizes the value and need for alternative assessment.

Although Nick and Sue did not utilize their alternative assessment instrument with students, the assignment prompted them to develop an alternative form of assessment that would serve as a summary assessment. Further, their assessment requires students to conduct an eco-column investigation, making observations over five days and apply their understandings about energy flow to a new eco-column environment. The assessment utilizes an analytic scoring rubric to assess students content understandings, as well as their performance abilities, a multi-trait assessment task.

Yolanda’s first attempt at incorporating alternative assessment into her classroom was quite successful in that “students seemed to do very well on the assessment.” Althoughpleased withher students’ performances, Yolanda was able to identify a gap in students’ conceptual understanding. “They did not make the connection that water would be transferred from plants to animals or from animals to animals throughconsumption.” It is unlikely that a traditional, multiple-choice test would have made visible this conceptual void in students’ understanding. Thus, for Yolanda, the assignment enabled her to construct a more detailed picture of her students’ conceptual understanding. The reflection process also provided her with insight into ways of revising and improving her assessment task.

Concept Maps as an Assessment Tool

Teachers were introduced to concept maps as an assessment tool through staff modeling and selected readings (e.g., Dana et al., 199 1). Since the project emphasized concept mapping as an instructional planning and teaching tool, it seemed natural that the teachers would adopt concept mapping as a mode of assessment. Joan, an elementary teacher, expressed the following about the use of concept maps for assessment:

[I use concept maps] at the beginning [of a unit] to assess the background knowledge [the students] can recall, to see any missing links or misconceptions. [I use concept maps again] at the end [of a unit] to test what they know in a more open response way. [The students] are expressing their own links.

Joan exhibits an important understanding about the use of concept maps for assessment. It is a way to individualize instruction by “find[ing] out the


appropriate place to begin instruction.” This suggests the pedagogical- assessment value of concept maps. Further, the summative assessment aspects of concept mapping served to determine how students’ conceptual understandingchanged. Joan, byusingconceptmappingas apre-assessment task, is, in essence, investigating student conceptions to inform teaching practice. Don, a middle-school science teacher also indicated a similar use of concept maps to “assess students’ understanding of related ideas . . , it is easier to get a more complete answer than an open-ended essay.”

Jennifer not only used concept maps as apre- and summative assessment but also as a formative assessment:

I have the students [construct a] conceptmap so I can evaluate their understanding before and after the unit. I also have them [construct a] concept map during the unit so that they can continue to make connections between previous laws and concepts to current labs and ideas.

Jennifer uses concept mapping as an assessment that is seamless with instruction--a pedagogical-assessment tool. The concept map is used by students “to make connections between ideas” (a pedagogical tool) and as a means for “evaluat[ing] their understanding before and after the unit” (an assessment tool).

Action Research Assignment: Investigating the Use of Alternative Assessment

The purpose of the action research assignment was to engage participants in evaluating the impact of the INLAB project on their classroom practice. Participants designed and implemented an action research project to answer a question of interest to them. An aspect of the assignment included investigating the use of alternative assessment. The action research assignment provided participants an opportunity to link theory to classroom practice, become doers of research versus consumers of research, work collaboratively toward shared goals, and contribute to the knowledge base for understanding assessment practice (Perry-Sheldon & Allain, 1987).

The findings and reflections of Bill and Laura are presented here. Bill and Lauracollaborated on investigating “the success of students on traditional assessment questions compared to their success on alternative assessment questions.” They designed a summary assessment instrument that contained traditional and alternative assessment questions. For Bill and Laura,


traditional assessment questions included fill-in-the-blank and short answer questions; alternative assessment questions involved students correcting illustrations, interpreting illustrations, and drawing and explaining illustrations.

Bill and Laura looked at the responses from 110 students over a two- year period. They observed that students scored higher (68.4% correct) on alternative assessment questions compared to traditional assessment questions (31.5% correct). They also noted that:

Students who generally do well on a traditional style assessment showed more success on the traditional assessment questions. Those same students appeared to not do as well on the alternative assessment questions. Students who generally do poorly on a traditionalstyleassessmentshowedgreatersuccesson thealternative style assessment questions. Those same students continued to do poorly on the traditional assessment questions.

Based upon the data collected during their action research project, Bill and Laura drew the following conclusion about alternative and traditional assessment questions:

[There is a] need to use a combination of alternative and traditional style questions on assessments to reach a wide range of student learning styles. We do feel, however, that a higher concentration of alternative assessment questions tends to be more beneficial to the entire group of students than a higher concentration of traditional assessment questions.

For Bill and Laura, me action research assignment enabled them to investigate an assessment issue of relevance to their classrooms by providing them with a classroom context for understanding assessment. The action research assignment gave meaning to alternative forms of assessment, as well as provided empirical evidence for using alternative assessment questions in their classrooms. Further, the action research assisted others in understanding the assessment issues, perspectives, and contexts facing Bill and Laura. For example, we now know that an important issue for Bill and Laura is the ability of their students to perform successfully on summary assessment instruments. We also know that alternative assessment for Bill and Laura includes the use of alternative assessment questions like correcting and interpreting illustrations and drawing and explaining illustrations.


Participant Self-Reflection on Change in Assessment Practice

Participants’ understandings of assessment practice may be reflected in two general categories: (a) change in assessment practice and co) no change in assessment practice. Change in assessment practice is further subdivided into: (a) new knowledge or understanding of assessment practice; (b) new ways of assessing, with assessment viewed as a learning tool (seamless with instruction); and (c) new ways of assessing, with assessment related to, but discrete from laboratory teaching (aligned with instruction). The category of no change in assessment practice is divided into two subcategories: (a) new knowledge or understanding of assessment that aligns with current assessment practice, thus no change in practice, and (b) maintenance of traditional assessment practice.

Change in Assessment Practice

INLAB activities and assignments changed the assessment practice or understanding of 27 of the 30 participants. For 13 teachers, the process provided them with new ways of knowing and or understanding assessment. Susan’s and Jack’s responses illustrate the variety of ways participants now view assessment.

I used to think that laboratory assessment meant grading the answers that werefilled in the provided spaces on the blanks of preprinted lab sheets. I now realize that in order to assess students’ knowledge learned in the activity by relating it to other situations, they should also be able to explain (preferably verbally) what was learned in the lab activity and to share that learned information with others. (Susan)

Laboratory assessment was a new conceptfor me since I rarely did labs. I think that good lab assessments should assess many areas (other than reading skills) and, in many cases, can be more difficult than traditional forms. I am learning to use a scoring matrix more accurately but stillfind that it takes a great deal of teacher time. I dofeel thatI am viewing my students in a better light than Idid when all assessment was done through a traditional approach. (Jack)

An emphasis of JNLAB was the development of multi-trait assessment as a way of assessing students. Clearly, these individuals have come to not


only know alternative assessment as a new way of knowing but have also developed the requisite skills in utilizing these methods through the use of a scoring matrix (rubric).

Six participants came to understand assessment practice as being seamless with instruction. The responses of Betty and Terry reflect this perception.

I have also enjoyed the ideas of assessing the laboratory while in progress by assessing laboratory cooperation, staying on task, etc., and walking around asking certain questions while the students are doing the laboratory. In other words, having a scoring rubric for lab day. (Betty)

I give points for pieces of information given in a demonstration, scientific process, spot questioning, effort, growth. I use checklists, interview one-on-one test, and spot check. I give students new situations to use learned skills to explain in essay form (orally for LD) and actual physically doing things one-on-one. (Terry)

Another emphasis of INLAB was coming to know pedagogical assessment knowledge as a way for planning and implementing instruction. These individuals came to see assessment as part of instruction. That is, seamless with instruction in that they assess students during the laboratory or instructional activities. Further, they see assessment as multi-trait in that they assess students’ understandings as well as their cooperation and ability to use the science processes.

Yet, for some participants, INLAB provided new ways of knowing and understanding assessment that aligns assessment with laboratory teaching, but as a discrete activity. The responses of Mary and Jill illustrate this thinking about alternative assessment.

In the past, I really didn’t assess thefew lab activities that were done except with objective answers to afm questions. I do like the idea of using the scoring rubrics to assess work. I think it makes mefocus on what I want the students tofocus on and learn and it helps me stay focused. (Mary)

I am using rubrics this year to assess student work. I find it helps the students as well as myself. Because of the rubric, the student has a better idea of hw to complete the work. I have already donemini- interviews as students complete lab activities. I really didn’t


consider those as assessments, but now I see the application of mini- interviews as an assessment. (Jill)

These individuals describe using alternative assessment techniques that align with instruction by guiding the activity and assessing student performance; however, it is discrete from instruction in that the assessment technique is applied to students’ work at the end of instruction.

No Change in Assessment Practice

Altogether, only three participants’ responses reflected no change in assessment practice. For one participant, there was no change in practice as their current practice aligned with the alternative assessment philosophy taken by the project. Salty’s response depicts the alignment between the assessment approach of the project and her existing practice:

It hasn’t changed that much. I have always felt that individualized instruction and evaluation is key to the success and esteem of my students. INLAB has given me some additional ideas to try.

Although Becky’s and Jane’s responses indicated no change in assessment practice due to concerns, the concerns were also reflected by other participants who changed their assessment practice. Their responses, then, also illustrate the concerns, frustrations, and uncertainty some participants experienced with alternative assessment.

I think that there should be alternative forms of assessment. I still think I don’t know enough about different forms and how to be fair to different types and abilities of learners. I’m afraid that the more subjective our assessment becomes, the greater the potential for parents to cry foul . . . . I’m not as happy with my assessment as I used to be because some of the assessment ideas that I developed this summer have not been very practical as I’ve tried them. I feel I should change but I’m not sure ifI’m moving in the right direction. (Becky)

I’m still not happy with any lab assessment as far as being fair and complete. Again, I feel multiple assessment techniques should be used, but my concerns restrict my use. (Jane)

Although Becky and Jane feel they should change their assessment


practice, their responses echo a concern about fairness in assessing students that have prevented them from changing their assessment practice. Further, Becky’s reply elucidates a concern about the practicality of alternative forms of assessment--time, time to develop alternative forms of assessment, time to implement alternative forms of assessment in the classroom, and time to score student work.

Conclusion and Implication

The success of INLAB in changing teacher assessment practice was, in part, built upon the establishment of an intellectual community of learners. A community that encouraged and supported science teachers’ self-reflection, collaborative dialogue, and the negotiation of a shared understanding about assessment. Modeling and comprehending ways of assessing provided teachers with a vision of practice from which they could reflect upon their own classroom practice. INLAB assignments encouraged participant reflection and supported collaborative dialogue about assessment practice. Providing time during workshop sessions was essential in order for teachers to develop and revise their assessment tasks based upon their reflections and collaborativedialogue. The integration ofpedagogical, content, and assessment knowledge resulted in participants’ construction of alternative assessment techniques that aligned or were seamless with laboratory teaching.

The INLAB framework provides a successful model for operationalizing the NRC (1996) professional development standards by providing direct implications for establishing effective science teacher enhancement projects. Science teacher enhancement projects must engage teachers in reflection upon practice that involves them in analyzing practice in light of their classrooms, providing a classroom context for making meaning of practice. Reflecting on practice in the context of the classroom must take into account the learner and the teacher. Reflection in the context of the learner provides evidence to support change in practice, whereas reflection in the context of the teacher takes into consideration that teachers come to know and understand practice in different ways.

References

Dana, T. M., Lorsbach, A. W., Hook., & Briscoe, C. (1991). Students showing what they know: A look at alternative assessment. In G. Klum & S. M. Malcom (Eds.), Science assessment in the service of reform (pp. 331- 337). Washington, DC: American Association for the Advancement of


Science. Feely, J. (1986). Reading practices in schools: Linking theory with

practice. (ERIC Document Reproduction Service No. ED 269 734) Hart, E. P., & Robottom,T. M. (1990). The science-technology-society

movement in science education: A critique of the reform process. Journal of Research in Science Teaching, 27,575-588.

National Research Council. (1996). National science education standards. Washington, DC: National Academy Press.

Perry-Sheldon, B., & Allain, V. A. (1987). Using educational research in the classroom. Bloomington, IN: Phi DeltaKappaEducationalFoundation.

Robottom, T. M. (1987). Two paradigms of professional development in environmental education. The Environmentalist, 7,291-298.

Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, l-22.


Appendix A Project INLAB Alternative Assessment Assignment

Introduction/Purpose

Science educators are being called upon to reform how they assess students and to align assessment with their science curriculum and instructional approaches. The successful implementation of innovations in science teaching rests on the development and use of appropriate assessment techniques which are connected to the curricular and instructional innovation. Therefore, the purpose of this assignment is to engage you in developing, implementing, and reflecting upon alternative assessment techniques.

Procedures

You will develop, implement, and reflect upon two different alternative assessment techniques. To accomplish this, you will need to:

1. Select two assessment instruments that you currently use and will use to evaluate your students between now and the end of school.

2. Develop one alternative assessment prototype for each of the above assessment instruments. This means you willdeveloptwo different alternative assessment prototypes. The alternative assessment prototypes need not assess every item covered on your original (current) assessment instruments. As you develop your alternative assessment prototype, be sure to include the following:

- Student directions for completing the alternative assessment. Are the directions clear and concise? Is all necessary information provided?

- Scoring rubric or matrix which is used to judge student performance. Does the rubric reflect a range in student performance? Does the rubric state specific criteria for each level of performance?

- Provide all necessary information for administering the alternative assessment prototypes to students. Will other teachers be able to use your prototypes?

&&: This is a first attempt and you will need to revise your prototypes based on student responses. Alternative assessment techniques must be revised based on student responses in order to improve their reliability and validity.

282 DANIEL P. SI-IEPARDSON & PAUL E. ADAMS

3. Administer both your current assessment instruments and your alternative assessment prototypes to at least one class. Depending on the length of your assessment instruments, you may want to administer your current assessment instruments on one day and your alternative assessment prototypes the next day.

4. Photocopy the student responses for all assessment instruments (current and alternative assessment prototypes), along with clean copies of each assessment instrument. You will need to turn in the photocopies with your reflection paper.

5. Based on the results for one class, write a reflection paper which compares and contrasts student outcomes between your current assessment instruments and the alternative assessment prototypes. The following questions may assist you in your reflection:

- Did students exhibit a difference in their understanding? If so, in what ways? Why might this be? What is your evidence?

- In your way of understanding, which instrument (current or alternative) best reflected student learning? Why do you think so? What is your evidence?

- What do you see as the benefits and difficulties with developing and implementing alternative assessment techniques? Give reasons for your views?

- Based on student responses, what ways would you modify your prototypes to improve their reliability and validity?

6. Revise your prototypes based on student responses and your reflection. Be sure to photocopy and turn in your revised prototypes.

7. Prepare a short small-group presentation on your reflection paper. You should at least share examples of your alternative assessment prototypes, student evidence (results), and your general thoughts about the benefits and difftculties of alternative assessment techniques.

coming to know and understand alternative assessment in science

Documents