what's new in courseware? action research in teacher-student partnerships
TRANSCRIPT
This article was downloaded by: [190.206.4.11]On: 22 March 2014, At: 09:29Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office:Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Educational Action ResearchPublication details, including instructions for authors and subscriptioninformation:http://www.tandfonline.com/loi/reac20
What's new in courseware? actionresearch in teacher-student partnershipsJames Kusch a , Alex Pan b , Gigi Bohm c & Heiden Stein da University of Wisconsin, Madison, USAb University of New Jersey, USAc Union Grove Elementary School, USAd Cambridge Elementary School, USAPublished online: 07 Mar 2011.
To cite this article: James Kusch , Alex Pan , Gigi Bohm & Heiden Stein (1999) What's new incourseware? action research in teacher-student partnerships, Educational Action Research, 7:2, 259-272,DOI: 10.1080/09650799900200087
To link to this article: http://dx.doi.org/10.1080/09650799900200087
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis, ouragents, and our licensors make no representations or warranties whatsoever as to theaccuracy, completeness, or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, and are not the viewsof or endorsed by Taylor & Francis. The accuracy of the Content should not be relied uponand should be independently verified with primary sources of information. Taylor and Francisshall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses,damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly inconnection with, in relation to or arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any substantialor systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, ordistribution in any form to anyone is expressly forbidden. Terms & Conditions of access anduse can be found at http://www.tandfonline.com/page/terms-and-conditions
What’s New in Courseware? Action Research in Teacher–StudentPartnerships
JAMES W. KUSCHUniversity of Wisconsin, Madison, USAALEX PANUniversity of New Jersey, USAGIGI BOHMUnion Grove Elementary School, USAHEIDE STEINCambridge Elementary School, USA
ABSTRACT In an action research inquiry into the uses of teacher and student
partnerships to evaluate educational courseware, two elementary teachers
teamed up with two teacher educators to see how elementary students would
fare when asked to identify the qualities and limitations of courseware. Since the
participating students were challenged by their collaboration with other
students, and were engaged by their evident partnership with teachers, students
learned to develop critical strategies when they worked with educational
courseware. Since the participating teachers developed their assessments of
courseware in conjunction with students who were making explicitly
comparative evaluations, teachers learned that hey could rely upon and
incorporate student evaluations when they attempted to integrate courseware
into the curriculum.
Educational courseware is computer software that has been created to assist
teachers in the classroom. This report summarises what the authors learned
when we applied the philosophy and methods of action research to explore
ways that courseware can be incorporated into a reflective teacher student
partnership in elementary school classrooms. Specifically, since our
previous experience with educational courseware has revealed it to be
uneven in its instructional quality and curricular applicability, we applied
the insights of action research to a classroom-based evaluation of
courseware to ascertain whether reviews of teachers and student work with
courseware clarify the ways that such courseware can be useful to teachers
and students alike.
Our efforts were guided by the expectation that teachers can transform
their practice by studying the processes by which knowledge is both
mediated and produced by teachers and students (Zeichner & Noffke, 1998).
EVALUATING COURSEWARE
259
Educational Action Research, Volume 7, No. 2, 1999
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
As a consequence, we viewed the philosophy and methods of action research
as an integrated approach to recursive consideration of teaching practice as
it develops day-to-day in the classroom (Fischer, 1996; Zeichner, 1997).
Thus, the investigation started when two elementary school teachers who
were enrolled in a graduate course decided to team-up with two
university-based teacher educators to evaluate educational courseware in
their elementary school classrooms. One of the elementary teachers had 4
years of teaching experience; the other had 6 years of experience. What
follows is a summary of issues, methods and findings that emerged when
the elementary teachers created investigatory partnerships with their
students to carry out the study.
Purposes and Intended Benefits of Courseware Programs
In the United States, thousands of courseware programs are nationally
available to primary and secondary school teachers, and the problem of
evaluating all of these offerings is at least directly proportional to the
absolute number of products that vie for classroom trials. Six years ago, the
Educational Products Information Exchange (EPIE, 1993) noted that
approximately 10,000 software programs were on the market. Two years ago,
the Digest of Educational Statistics (1997) concluded that approximately
7000 titles were available from 300 different publishers. The offerings in
question have been developed by individuals, textbook and software
publishers, and professional associations of educators. The approaches of
these offerings include structured and unstructured mathematics
exploration using games, classical models and concepts, and simulations of
real-world problems and their solutions. The applications of these offerings
range from preschool mathematics readiness instruction through primary
school concept formation and skill practice, to full-scale curricula in algebra,
trigonometry, geometry and calculus.
Given the fact that the first courseware was developed for the old Apple
II series of microcomputers in the early 1980s, teachers are well into their
second decade of using and evaluating this kind of resource. The well-worn
claims about the benefits of courseware boil down to the following
assertations about the potential for individualised study and immediate
feedback:
� Students who use courseware are expected to be productive when they
work privately or in small groups, at their own pace.
� Students are expected to use courseware to explore curricular material in
greater depth than they would when following a printed text or series of
programmed worksheets.
� Students are expected to conduct their explorations with pre-programmed
checks for understanding at appropriate points in their explorations.
� Students are expected to frame hypotheses and then test those
hypotheses, all while they are using highly attractive and attention-getting
video technology.
JAMES W. KUSCH ET AL
260
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
Frustrations with the Quality and Applicability of Courseware Programs
While teachers respond with appropriate enthusiasm when they consider the
potential value of using educational courseware in classrooms, they have
reason to be concerned about the quality of some of the programs that are
currently available. Published reviews of courseware, informal exchanges of
faculty commentary and the personal experience of the authors of this study
reveal profound limitations in the pool of courseware offerings.
Courseware often emphasises style and operates at a superficial level
(Turkle, 1997). Such courseware achieves the goals of neither its users nor,
probably, its creators, since both parties should be interested in the clarity
with which programs communicate concepts. At times, sound, colour and
animation enhance the objectives of courseware, but frequently these same
factors fail to rescue mediocre ideas and mask marginally valuable content.
For example, in one elementary-level program that is used to teach counting
skills and the use of Arabic numbers (Millies Math House, 1995), the
program frustrates some of its users when it does not explain why it
proceeds or concludes as it does. Thus, students could be spared the
consternation that arises when their session ends abruptly with an arresting
animation (of a horse neighing, and eating an apple) before they have
learned how or why it was that they succeeded in getting to the end of the
game that they were playing.
When teachers cannot find courseware that fits or enhances a unit of
study, it is possible that the internal logic of available courseware is
fragmented, or that available courseware is poorly designed (Oppenheimer,
1997; Stoll, 1995). In such instances, it is likely that only a small portion of
a particular program is geared to the skills that a teacher wishes to develop
in students users. Thus, it is crucial for teachers to know how to customise
existing courseware exploiting the expected internal obility of programs to
move to the portions that best serve instructional objectives. For example, in
one middle-level program that is used to teach estimation and simulation
skills in mathematics and social studies (Oregon Trail, 1997), it would be
helpful if teacher could customise the program to permit students to work
with the components that they can understand, even when they do not have
the historical background or reading skills to grasp the whole program.
Thus, students need not be stymied when they encounter the part of the
program that announces that a character has contracted cholera a disease
of unknown nature and implications to most users.
Many educational programs rely upon drill and practice . Writing in
1992, Grouws commented on the application of computers in classrooms,
Most investigations of children s learning have involved seriously deficient
software and hardware on CAI drill and practice (Grouws, 1992). Indeed, 2
years ago the Educational Market Report concluded that 82% of the
mathematics programs that it reviewed relied on such methods (Blohm and
Associates, 1997). In such programs, there is a very weak link between real
life and program material. Skills that are presented in a drill and practice
EVALUATING COURSEWARE
261
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
format do not take students beyond rote learning, and fall short of the
potential value of courseware.
The potential benefit of on-line answers to user questions is often
reduced by variability in the ways that teachers seek to integrate courseware
into their curricula (Stoll, 1995). Programs that might work in one curricular
context can be relatively ineffective when applied in another context.
Stated in the affirmative, the value that teachers and students place
upon courseware depends largely upon the actual uses that they find for the
courseware. It is this contingent aspect of courseware evaluation that gave
rise to the action research project described in the passages that follow.
Observations that Guided this Inquiry
Regardless of the intrinsic quality or potential applicability of courseware, it
is clear from a teacher-educator s perspective that courseware is often
abused or poorly used in the school. During the frequent visits of the two
authors who work in pre-service education, we observe teachers who allow
students to use courseware on their own, with very little guidance.
Frequently, we observe students spending time with courseware as a reward
for either good behaviour or for completing homework. Since teachers
typically have only superficial knowledge of educational applications for
courseware, they tend to minimise the use of them or they merely allocate
them as a means to control behaviour.
From a teacher educator s perspective, there is an even more
fundamental problem with courseware that derives from the manner in
which it is crafted. Since most courseware is commercially produced and
frequently created by non-educators, it is often difficult for teachers to
integrate courseware into existing instructional patterns and to connect
practical thinking about courseware with the intended competencies, skills
and interests of their students. Teachers, as well as teacher educators, must
begin to focus their critical attention on the sources that develop and
distribute courseware, if they are to integrate such programs into the
particular purposes of classrooms. However, when this kind of attention is
directed to courseware, we should be careful to mind the observation drawn
from examination of conventional textbooks (Apple, 1992) that meanings
signify particular constructions of reality and ways of selecting and
organising knowledge. It is the responsibility of teacher educators and
classroom teachers to learn the sources and consequences of those
meanings.
The authors have concluded that teachers and students should explore
courseware together, so that they can reach a mutually-shared
understanding of how courseware can be used to re-orientate both teaching
and learning. Just as it is not generally assumed that what gets taught is
what gets learned, it should not be assumed that what is in courseware
actually comes out in an effective manner. Just as effectiveness in the use
of conventional instructional material depends upon teachers ability to
mediate and alter the material (Apple, 1996), effectiveness in the uses of
courseware requires teacher participation and intervention in the use of
JAMES W. KUSCH ET AL
262
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
courseware by students. In addition, when students bring their own desires
and biases into their use of courseware, it is likely that they will construct
meanings as actively as their teachers. What we are describing, then, is a
variety of teacher student partnership that merges the interpretations of
students with the interpretations of teachers when courseware is employed.
Partnership as a Curricular Innovation
What if teachers and their students had a dialogue and critical discussion
about courseware before it was actually used in the classroom? More
particularly, how might teachers organise dialogue and critical discussion
with students about courseware and yet have time to do all of the other
things required of them by their busy schedules?
The intersecting tasks of promoting collaboration between teachers and
students, and of using time and classroom resources effectively are among
the central problems of teaching and learning. Teachers could gain a shared
understanding of courseware if they were to collaborate with their students
in evaluating the courseware. The partnership between teacher and students
would give teachers an opportunity to gain valuable insights about the
meaning of courseware without interrupting normal classroom activities.
Additionally, the partnership between teacher and students would give
students an opportunity to develop answers to questions that are often
deemed irrelevant to students, such as questions about the origins and
purposes of courseware (Medawar, 1979). In the context of courseware, such
student questions concern its origins and purposes.
What happens in partnership learning? When teachers operate
courseware with individual students and when they have an active dialogue
to explore the different purposes and features of the programs, then the
teachers and students create a partnership in learning.
How exactly does a dialogue take place? Typically, teachers find a spot
where they can be alone with a student, or aside with a small number
(perhaps three or four) of their students. In elementary classrooms used in
this study, teachers use a designated area of carpet to meet with their
students. Once a student has worked through a particular courseware
package or lesson, the student is asked to describe on the Features Matrix
presented below the experience of using the courseware. Simultaneously,
the teacher records her observations and reflections in a journal. Then the
teacher and student both make comments on the courseware and the
learning that accompanied the experience.
Two samples of teacher student dialogue, taken from teacher journals,
show how this conversation actually works. The samples are drawn from the
experience of teachers involved in this research and pertain to discussion of
the courseware program Number Muncher (1990). Two students, Kristofer
and Delia, both aged seven years, shared their opinions about using Number
Muncher in the following dialogues.
EVALUATING COURSEWARE
263
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
Sample Dialogue 1
Teacher, working with Number Muncher , asks: Does Number
Muncher teach you anything you don t already know?
Student Kristofer: With Number Muncher, there s always
something I don t know.
Teacher: What s new with Number Muncher, then?
Student Kristofer: Something. Most of the time; but not always ... It
really doesn t help with doing math; but its great for practicing.
Comment: Kristofer s guarded statements about the limited
usefulness of the program direct the teacher toward the distinction
between learning new material in math and practicing previously
learned material.
Sample Dialogue 2
Teacher, working with Number Muncher , asks: What does the
program teach you?
Student Delia: It tells us what we ought to do next.
Teacher: And what in particular does it tell you to do?
Student Delia: It tells us that we ought to wait for you. [Pause]
If we re good we get to go on to something else.
Comment: The follow-up to Delia s first response tells the teacher
that there is at least part of the program in question that breaks
down the link between Delia and the program requiring the
teacher to intervene, perhaps to reconnect Delia to the flow of the
program.
Partnerships give participants a systematic relation that uses
teacher student dialogue to promote reflection on the personal meanings of
learning. In the course of a reflective exchange one person looks to the Other
rarely seeing oneself and learns what it is that the Other knows. As
Cixous writes, we are always blind; we see of ourselves what comes back to
us through (the difference of) the Other (Cixous & Calle-Gruber, 1997).
When students interpret the often ambiguous meanings of courseware
with their teachers as partners, both gain. On the one hand, we concur with
Scheffler who emphasises that teachers ought to place their curiosities and
claims in front of learners in order to develop the ability of students to
JAMES W. KUSCH ET AL
264
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
independently interpret how teachers sort out ambiguous meaning (1965,
pp. 11 12). Such teaching involves a rigorous attempt to induce students to
think reflexively. By using partnerships, teachers exemplify the critical
thinking that they strive to develop in students, combining tough-minded
instruction with a penchant for inquiry (Holmes Group, 1986, pp. 28 29). A
shared process of evaluating courseware transforms the role of teacher,
towards promoting thought and reflection on the part of the student, with
requests for argument and evidence in support of assertions. The teacher s
role is to make sure that the student has the opportunity to clarify the
problem, make observations in potentially profitable areas, from and test
hypotheses, and to reflect the results.
On the other hand, in partnerships students learn on their own to
perceive meaning in courseware. When students work in partnership with
their teacher, they may pick and choose, interpret, seek and impose order,
and devise guesses as they go through unfamiliar meanings together. As
students and teachers explore courseware together, students may find
temporary meanings or may construct guesses about where the courseware
may lead. Such guesses may, in fact, become what Medawar calls larval
theories or conclusions (1996, pp. 20 21). This process of learning by
students to extract meaning and develop theories in one context can be
applied by the students to other contexts at other times. The courseware
partnership, thus, provides students with an opportunity to develop their
capacities for insightful work in a technology-dependent environment.
Dialogues between teachers and their students, in fact, may move
students beyond conveying the kind of information that their teachers
expect them to treat. Research suggests that knowledge in courseware
cannot merely be transmitted directly from one knower to another; instead,
it must be built-up actively by the learner, in shared dialogue with others
(Driver et al, 1994). If teachers are to integrate courseware into existing
curricula, they would do well to listen to and incorporate the meanings that
children already have or seem likely to assign concerning courseware. In
this respect, such jointly-developed classroom tools as the feature matrix
treated later in this report provide a systematic means by which teachers
and students can reflect in partnership on different aspects of courseware.
The work described in this report is not merely a methodological device
for solving the practical problem of how courseware could be implemented in
classrooms. Instead, the action research into teacher student partnership
treated here provides a way to probe the meaning of courseware when that
meaning is contingent and transient (Winter, 1987). By probing the meaning
of courseware programs for instruction with students, teachers are likely to
learn reflexively about instruction in short, how each party thinks.
Research Methods
Our approach to this study applied the methods of action research (Wood,
1988; Noffke et al, 1994; Kusch, 1996), in which we reflected upon the issue
of courseware programs, created a plan for action, outlined a way to
EVALUATING COURSEWARE
265
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
implement the plan, and returned to the process of reflection so that we
could make our reconsidered findings public.
Reflection: first phase
Our initial reflections concerned the motivational aspects of courseware and
the implications of using such courseware in the classroom.
Children, as well as adults, need to be motivated or encouraged to
perform tasks that they might not otherwise undertake. As a consequence,
the teachers who introduce courseware into their classrooms have a special
interest in the ways that instruction in areas that may not be initially
appealing to students can be made motivational when student use
courseware. Thus, it would be useful to be able to specify both the relatively
obvious and non-obvious aspects of motivational courseware. Moreover, it
would be doubly useful if teachers knew not only what they consider to be
motivational, but what their students think about that characteristic of
educational software.
One difficulty with the process of evaluating courseware is that such
software resides on computers devices that most teachers believe to be
intrinsically attractive to some students, and correspondingly daunting to
other students. However, we need to ascertain the degree to which the
imputed attractions and fears that are generated by computer use operate in
practice. In addition, we need to develop a sense of change in student
responses to computers and courseware, since computers are losing some of
their novelty in school contexts.
For the purposes of this study, we conceive of motivation as the
common bond that links students and teachers to the same courseware
programs. Building upon this basic insight, we propose that motivational
courseware is:
� challenging when it helps students understand or develop awareness of
material that is beyond their initial level of understanding;
� applicable when it relates explicitly and directly to the day-to-day
experience of students, both in and out of the classroom;
� appealing when it develops student interest, by tying graphics, sound
and format to the concepts and ideas that are the subject of particular
courseware;
� standardized when it adheres to skills and standards established by
state and school curricular guides;
� coherent when it integrates facts that pertain to the curriculum that a
teacher is already covering.
Additionally, we believe that courseware is educationally motivational when
it does not contain aspects that are merely entertaining, violent or sexist.
JAMES W. KUSCH ET AL
266
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
Planning
Our Planning followed a four-step process that was designed to help
students eventually act upon their attitudes toward courseware programs
and their findings concerning those programs.
� First, we collected examples of the courseware programs that were
available in our school. Once collected, we developed a catalogue of the
contents of each courseware program, organising the contents by subject
category and concept.
� Secondly, we created an initial student interest survey that students and
teachers could use to document their interests at the start of the study.
Once documented, we (teachers) brainstormed with our students to learn
the qualities that they like in computer programs.
� Thirdly, the preferred qualities of courseware were then gathered into a
courseware feature matrix, to aid in comparing programs.
� Fourthly, we asked students to fill-in a worksheet that could be used to
document their subject area interests. The worksheets matched
courseware titles with subject interests, as organised by concept.
Action
The action or implementation phase of our study incorporated five steps that
we and our students followed, more-or-less iteratively. Throughout, however,
we interviewed and observed the students informally seeking to develop a
clear qualitative view of the process and the results of our partnership in
experimentation.
� First, we gave students the option of working by themselves or in pairs.
Nonetheless, throughout the study, all students elected to work in pairs.
Ultimately, the students organised themselves into 21 pairs. (To keep our
discussion clear, we have restricted the term partnership to the joint
working relationship between teacher and student; and we have adopted
the term collaboration to describe the working relationship that exists
within pairs and groups of students.)
� Secondly, we asked students to pick up a copy of the feature matrix.
Students were requested to choose one courseware program to work with
that matched their responses to the initial interest survey. Between the
two classes that served as the experimental environments for this study,
35 different examples of courseware were evaluated by students and
teachers. Of these programs, 13 were designated by courseware producers
mathematics, four were designated art, six were designated social studies
and 12 were designated language arts.
� Thirdly, we had students experiment with the programs. As they
experimented, they were asked to rate each program using the scale and
categories presented on the feature matrix. If students identified a quality
EVALUATING COURSEWARE
267
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
that was not listed on the matrix, they recorded the pertinent observations
separately.
� Fourthly, the process continued, with students filling out the feature
matrix for each courseware program, until all programs were reviewed.
� Fifthly, after all programs were evaluated and opinions were documented,
students got together with their teacher to draw-up and post a set of
conclusions that could be used for further study and reference. The
purpose of this posting was to promote integration over time of the links
between courseware and curriculum (Altrichter et al, 1993).
Reflection: second phase
Our reconsideration of these issues and our observations took us through all
of the documentary and qualitative material that we had developed in the
course of our study. The two elementary school teachers who provided the
classroom laboratories and experimental personnel summarised their
findings individually.
Teachers work with courseware. There were differences in the ways that the
two teachers surveyed student attitudes and understood how courseware
served to enhance student learning. The participating teachers attribute
those differences to variation sin the ways that they organise their
instruction using courseware and arrange their students in the classroom.
Both Betty and Suzanne pseudonyms, in each instance work with
elementary students who are drawn from relatively similar populations.
Betty is a Title 1 teacher, who teaches 37 students in grades 2 4. Most of
her pupils have been identified as remedial readers; however, eight have
been characterised as non-readers. Suzanne teaches 19 students in grade 4.
Of her pupils, four have been identified as having learning disabilities.
Before she began to teach her students to use courseware programs,
Betty began by reading and absorbing the meaning and objectives of the
available programs as stated in their documentation. When she developed
her overall assessment of the programs, she observed that the target ages
and enhancements of available courseware varied in the following respects:
� Courseware for younger students relied upon animation and sound, but
courseware for older students relied less upon such enhancements.
� Courseware that relied upon animation and sound tended to attract
users.
� Courseware that relied upon animation and sound offered little that was
useful for non-reader.
� Student evaluations of specific courseware varied according to their ability
to read, with non-readers tending to miss the purposes of some
courseware since they were unable to participate in its use on their own.
Suzanne based her overall assessment of the programs upon the degree to
which courseware related directly to the content that she covered in class.
JAMES W. KUSCH ET AL
268
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
� Most courseware provided drill and practice that reinforced the
computational work that she taught in mathematics. However, such
courseware added little that was new or different to her instruction.
Instead, the drill and practice programs tended toward repetitious use of
similar problems.
� Some courseware enabled her students to identify new instances of
concepts that they had previously learned. For example, Speedway Math
(1986) depicted toys in a toy shop, and asked students to identify a
portion of the toys of one variety and to give that portion a fractional
name. She discovered that the program s use of animation to remove or
highlight sets of toys enabled students to understand fractional amounts
in ways that were different and interesting.
� Other courseware successfully challenged her students to work with
material that was entirely new. For example, Dinosaur Discovery (1993)
presented an overview of different geological eras in which dinosaurs lived.
Although some students had known such terms as Jurassic or Cambrian
from previous exposure to books or movies, they had not known that the
terms referred to different periods of time until they worked with the
courseware. She observed that students tended to be drawn to programs
that challenged them and that held surprises in learning.
Students learning with courseware. Both teachers noted that there was a
dynamic character in the way that pairs of students evaluated courseware,
inasmuch as they reasoned and discussed their ideas about the interest
survey and feature matrix:
� Students tended to influence each other in the course of evaluation.
� Many students were very comfortable with the fact that they were hearing
a counterpart express different opinions and they bounced ideas
back-and-forth as to whether the content of courseware related to the
content that they had learned in their textbook.
� Where there were great differences in ability, however, the least able
students were less forthcoming in expressing their opinions.
� Students who were non-readers tended to give high evaluations to
courseware that contained the most attractive animation and sound.
However, all students were drawn to the programs that had the highest
quality visual displays.
� Students who were high achievers tended to seek out courseware that
challenged them.
Two aspects of courseware design emerged as important variables during
student evaluations.
First, clarity in courseware directions is critical to student success in
working with programs. Students benefit most when directions state
procedures clearly, using language that is readable. In Betty s case, clarity
and readability corresponded to grammatical usage, and the incidence of
words with more than two syllables. She observed that most students paid
considerable attention to directions and asked her for clarification when
EVALUATING COURSEWARE
269
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
they were uncertain. However, she also observed that students who were
non-readers tended to ignore directions and merely played with keys that
made the cursor move about until she noticed that they were stuck.
Secondly, the physical requirements of using keyboards can be
obstacles to success in working with courseware. Betty observed that
programs that ask children to use the combination I, J, K and M were the
most difficult for children in all grades to use, since they require mastery of
the left/right and up/down orientations. When children cannot reliably
distinguish left from right, it becomes difficult to use keyboard-based
courseware. Suzanne observed that the most user-friendly programs were
ones that operate without the use of function keys.
In general, students were honest if too critical in their evaluations of
courseware. They participated eagerly in the evaluation process, since they
felt that they were being recognised and rewarded for their opinions:
� Betty s students emphasised the real life application criterion in their
evaluations.
� Suzanne s students felt that the challenge criterion was most important.
� Each class rated graphics and sound as important; and nearly all
students observed that clarity in directions was important.
Conclusions
In the evaluation of courseware, a partnership of teachers and students
enables teachers to gain a great deal of useful information in an expeditious
and revealing manner, and permits students to develop their own critical
skills.
Teachers can use such partnerships to evaluate courseware in an
explicitly comparative mode, since the number of students and programs
involved can be structured by the teacher. More generally, teachers benefit
by developing their capacity to think reflexively about their teaching
practices, since student participation in the evaluation of courseware
requires that the teacher consider his or her own conclusions in light of
those developed by students.
Joint student teacher work with courseware encourages students to
delve as deeply as the courseware will permit, since the teacher is
sufficiently involved in the student s work that the teacher can provide
expert assistance when the student might otherwise shy away from
challenges. Thus, students learn how to study in partnership with a person
who has the broad experience and content knowledge of a teacher; and
students learn the generalizable skill of conducting research with
computer-based educational tools.
Students can use the partnership to clarify, refine and consolidate
their thinking. When students reviewed courseware, they gained a greater
sense of understanding about the courseware and about content that they
had already learned in classwork. As tested in this study, a partnership of
teachers and students helps students learn evaluation skills that they can
apply in two different contexts. First, once they learn to evaluate courseware,
JAMES W. KUSCH ET AL
270
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
they can apply the ideas incorporated in the feature matrix to other
courseware and other educational situations. The evaluative process is thus
replicable. Secondly, the evaluation model explored in this paper places
participation and critical thinking at the heart of instruction. The feature
matrix helps students to relate processes to their conceptual underpinnings,
and gives them a concrete and collaborative activity to discuss when
explaining and justifying their thinking.
The interpretations that students are compelled to make in the
partnership take them beyond the search for right answers. Good
courseware used in partnership can provide a forum in which students learn
to express opinions or tell stories. Since we concur with Stoll s observation
that children s educational software often assumes that children of a given
age have similar abilities and learning styles (Stoll, 1995), the partnership
tested by this study permit differences in ability and style to emerge
constructively.
In a teacher student partnership, teacher and student statements are
open to question, reaction and elaboration by others. This partnership sets a
new standard for evaluation that helps students explore the relation between
the things that are done in school and the things that are done out of school.
Although not explicitly studied in this project, it seems likely that student
participation in such a partnership will enable them to see more readily
where they need to focus their efforts to improve academically. At the very
least, students learn to evaluate school resources; and that process may
help students make the transition into people who evaluate their use of
school resources.
Correspondence
Dr James W. Kusch, 1706 Helena Street, Madison, WI 53704, USA
References
Apple, M.W. (1992) The text and cultural politics, Educational Researcher, 21(7),
pp. 4 11.
Apple, M.W. (1996) Cultural Politics and Education. New York: Teachers College Press.
Altrichter, H., Posch, P. & Somekh, B. (1993) Teachers Investigate Their Work: an
introduction to the methods of action research. London: Routledge.
Blohm, C. & Associates (1997) Educational Market Report. Washington: Software
Publishers Association.
Cixous, H. & Calle-Gruber, L. (1997) RootPrints: memory and life writing. London:
Routledge.
Digest of Educational Statistics (1997) National Center for Education Statistics, US
Department of Education Washington.
Dinosaur Discovery (1993) (version 1.5) [Computer software]. West Chester: Applied
Optical Media Corporation.
EVALUATING COURSEWARE
271
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4
Driver, R., Asoko, H., Leach, D., Mortimer, E. & Scott, P. (1994) Constructing scientific
knowledge in the classroom, Educational Researcher, 23(7), pp. 5 12.
Educational Products Information Exchange (1993) New York: EPIE.
Fischer, J. (1996) Open to ideas: developing a framework for your research, in
G. Burnaford, J. Fischer & D. Hobson (Eds) Teachers Doing Research: practical
possibilities. New Jersey: Lawrence Erlbaum.
Grouws, D. (Ed.) (1992) NCTM Handbook of Research on Mathematics Teaching and
Learning. New York: Macmillan.
Holmes Group (1990) Tomorrow s Schools: a report of the Holmes Group. East Lansing:
Holmes Group.
Kusch, J. (1996) Mining teacher practice with action research, Wisconsin Teacher of
Mathematics, Spring, pp. 17 23.
Medawar, P. (1979) Advice to a Young Scientist. New York. Oxford University Press.
Medawar, P. (1996) The Strange Case of the Spotted Mice. Oxford: Oxford University
Press.
Millies Math House (1995) (version 2.2) [Computer software]. Redmond: EdMark
Corporation.
Noffke, S., Mosher, L. & Maricle, C. (1994) Curriculum research together: writing our
work, in S. Hollingsworth & H. Sockett (Eds) Teacher Research and Educational
Reform, Ninety-Third Yearbook of the National Society for the Study of Education.
Chicago: University of Chicago Press.
Number Muncher (1990) (version 1.2) [Computer software]. Minneapolis: MECC.
Office of Technology Assessment (1988) Power on! New tools for teaching and learning
(OTA SET-379), in D. Grouws (Ed.) Handbook of Research on Mathematics
Teaching and Learning. New York: US Government Printing Office.
Oppenheimer, T. (1997) The computer delusion, Atlantic Monthly, 280(1), pp. 45 66.
Oregon Trail, 3rd edn. (1997) (version 1.2 for Windows and Macintosh) [computer
software]. Cambridge: The Learning Company.
Scheffler, I. (1967) Science and Subjectivity. New York: Bobbs-Merrill.
Stoll, C. (1995) Silicon Snake Oil: second thoughts on the information highway. New
York: Anchor Books.
Speedway Math (1986) (version for Apple II or higher) [computer software]. Saint Paul:
MECC.
Turkle, S. (1997) in Sex, lies, & avatars, in P. McCorduck (Ed.) Wired. San Francisco:
Wired USA Publishers.
Winter, R. (1987) Action Research and the Nature of Social Inquiry: professional
innovation and educational work. Aldershot: Gower.
Wood, P. (1988) Action research: a field perspective, Journal of Education for Teaching,
14, pp. 135 150.
Zeichner, K. (1997) Action research as a tool for educational and social reconstruction,
paper presented at the Annual Meeting of the Brazilian National Association of
Postgraduate Education and Educational Research (ANPED), Caxambu, Brasil.
Zeichner, K. & Noffke, S. (1998) Practitioner research, in V. Richardson (Ed.)
Handbook of Research on Teaching, 4th edn. Washington: American Educational
Research Association.
JAMES W. KUSCH ET AL
272
Dow
nloa
ded
by [
190.
206.
4.11
] at
09:
29 2
2 M
arch
201
4