Riverdeep: An Action Research Project

By Terry Scott and Kathy Briggs

California State University, Sacramento,

Spring 2003

1

Table of Contents

INTRODUCTION...............................................................................................................4

AREA OF FOCUS.............................................................................................................5

RESEARCH QUESTIONS...................................................................................................6

LITERATURE REVIEW......................................................................................................7

Literature Related to Action Research......................................................................7

Learning Theories as They Relate to Technology....................................................9

Multimedia...............................................................................................................13

Web-based Instruction............................................................................................15

Computer-Assisted Instruction................................................................................18

PROJECT DESCRIPTION.................................................................................................21

LIMITATIONS OF THE STUDY..........................................................................................24

FINDINGS.....................................................................................................................26

Data Collection........................................................................................................26

Research Questions...............................................................................................26

Data Analysis..........................................................................................................26

Table 1 - Student Progression Report from Destination Math.............................27

Table 2 – Student Math Proficiency Test Scores................................................30

Online Likert Scale Survey......................................................................................31

Table 3 - Student Survey 1-- Responses at beginning of study.........................31

Table 4 - Student Survey 2 – Responses at end of study..................................33

Table 5 - Teacher Survey Results.......................................................................36

Group Interviews.....................................................................................................38

2

Researcher Observations.......................................................................................39

DISCUSSION.................................................................................................................40

Reflection & Revision..............................................................................................40

Implications.............................................................................................................42

APPENDIX.....................................................................................................................44

Appendix #1 – Informed Consent............................................................................45

Appendix #2 – Student Survey #1...........................................................................46

Appendix #3 – Student Survey #2...........................................................................47

Appendix #4 – Teacher Survey...............................................................................48

Appendix #5 – Riverdeep Login screenshot...........................................................51

Appendix #6 – Riverdeep Module screenshot........................................................51

Appendix #7 – Riverdeep Tutorial screenshot........................................................52

Appendix #8 – Riverdeep Student Assessment screenshot...................................52

References.................................................................................................................53

3

Introduction

Can increases in student achievement be the result of computer-assisted modes

of delivery? Researchers have found that computer-assisted instruction enhances the

learning rate. Research has shown that students using computer-assisted instruction

seem to learn faster than students who use conventional instruction. For example, a

study conducted by Capper and Copple led to the conclusion that users of computer-

assisted instruction learned as much as 40 percent faster than those receiving

traditional teacher-directed instruction (Capper & Copple, 1985). Another study found

an increase in mathematics achievement using the INVEST system was greater than

the gains in those classrooms using traditional teaching approaches, particularly in the

areas of mathematical concepts and problem solving (Wilson, 1992; Moore, 1993).

“Through computers the use of multimedia has created novel modes of learning

and greatly contributed to the restructuring of instructional environments in schools”

(Relan & Gillani, 1997). With the presence of multimedia computers in today’s

classrooms, educators now face many more challenges. Instructors find themselves

with the responsibility of teaching in today’s sophisticated environment. Some

researchers, like Lewis (1999), supported the idea that in the 21st century, students

learn concepts at a higher level of thinking when they are interwoven with multimedia

products

Numerous curriculum producers are now making educational software. Their

school customers are purchasing their software products to use in classroom

instruction. By optimizing an individual learner’s strengths and talents, instruction

4

becomes specific to an individual learner’s needs. Custom-tailored learning

experiences resulted from individualized curriculum (Fraley & Vargas, 1975).

“In curriculum design, the instructor simultaneously needs to consider teaching

methods, materials, the nature of the subject area, and characteristics of the student

audience,” (Weston & Cranton, 1986, p.259). Innovative instructional practices focused

on teaching and learning strategies that make a difference in the daily practices in our

classrooms. This in turn translated into stronger student performance (McKenzie 2001).

The concern for competency in mathematics has led mathematics educators to attempt to clarify the processes underlying the acquisition of various mathematical skills. Within the past twenty years, novel techniques attempted to juxtapose conventional strategies for teaching mathematics. In the forefront of these are computer-based educational programs. (Walker, 1987).

The purpose of this Action Research Study is to enliven and enlighten current

discussion about educational classroom technologies. This project will explore

implementation of Riverdeep Destination Math software in a single eighth-grade math

class at Cardozo Middle School.

Area of Focus

The purpose of this study is to evaluate the use of Riverdeep Destination Math

Software in its implementation at Cardozo Middle School in the Riverbank Unified

School District. The District made a sizeable investment in Riverdeep Destination Math.

This study examines the effectiveness of Riverdeep Destination Math software and its

manner of use on math achievement scores. This study also seeks to determine the

effect of Riverdeep Destination Math software on student attitudes toward’ 8th grade

math at Cardozo Middle School. Results from both achievement and attitudinal data of

5

this study could help revise the future implementation of Riverdeep Destination Math at

Cardozo Middle School.

Competency tests created by the school site determine acceptable eighth grade-

level math proficiency at Cardozo Middle School. This study utilized pre-and-post test

scores from this competency test to measure changes in student achievement. This

Action Research also employed student Likert Surveys, small-group interviews, and

investigator observations to gather data.

This Action Research paper examines the use of Riverdeep Destination Math

software with a specific group of 8th grade middle-school students. Students in the

study group are those who failed Cardozo Middle School’s math minimum competency

test and involved in a math invention program after school.

As research by Sivin and Kachala (2002) suggested, the use of similar ILS-type

software, such as Plato or Accelerated Math, will enhance both student achievements in

math and increase positive student attitudes toward math.

Research Questions

The use of web-based, multimedia instructional materials has become a

noteworthy force in distance learning in upper education. Providing quality, Integrated

Learning System (ILS) types of products to K-12 students has only been a recent

occurrence.

The researchers, Terry Scott, a district technology coordinator, and Kathy Briggs,

a classroom computer literacy teacher, were curious about the efficacy of ILS-type

programs. The researchers sought to determine the effectiveness of daily corrective

and remedial use of Riverdeep Destination Math software on selected 8th grade middle

6

school math students. Therefore, these researchers explored the recent

implementation of Riverdeep Destination Math software at Cardozo Middle School. The

investigators focused on two research questions:

1. How will the use of Riverdeep Destination Math, when used as a frequently

employed remedial program, affect student achievement on math proficiency

scores?

2. How will the use of Riverdeep Destination Math, when used as a frequently

employed remedial program, affect student attitudes toward math?

Literature Review

Several areas of research are relevant to the investigation presented in this

Action Research project. Components of this Literature Review provide background

information for later perspectives on the effectiveness of Riverdeep’s Destination Math

software. This literature review consists of the following sections:

1. Literature Related to Action Research

2. Learning Theories as They Relate to Technology

3. Multimedia

4. Web-based Instruction

5. Computer Assisted Instruction

Literature Related to Action Research

Several authors have described Action Research in similar ways. One paper

described Action Research as “a process by which change and understanding can be

pursued at the one time” (Dick, 1997). This dual-purpose process, typically used by

educators, suggests repetitive action and reflection sequences over a specific time-

7

period to analyze a problem and propose solutions (Dick, 1997). Another authority

explained Action Research as a method to “gather information about the ways that their

particular schools operate, how they teach, and how well their students learn” (Mills,

2000). Calhoun (1993) suggested Action Research “captured the notion of disciplined

inquiry (research) in the context of focused efforts to improve the quality of an

organization and its performance (action)”.

Calhoun (1993) described and differentiated between three Action Research

scenarios of individual, collaborative, and school-wide types of research. This Action

Action research uses the collaborative or small group type. The central purpose of

collaborative research focuses on classroom improvement. Outside support for

collaborative Action Researchers frequently comes from higher education and data

utilized can be qualitative or quantitative (Calhoun, 1993). The project presented in this

paper uses the collaborative type of action research.

Mills (2000) described Action Research as a four-step process:

1. Identify an area of focus.

1. Collect data.

2. Analyze and interpret data.

3. Develop an action plan.

There are multiple benefits of Action Research. Calhoun (1993) expressed

consideration of Action Research as a “progress in professionalism” creating a source

of change in classroom practices. Mills (2000) emphasized that Action Research is

“largely about developing the professional disposition of teachers, that is, encouraging

teachers to be continuous learners – in their classrooms and in their practice.” As

8

articulated by Dick (1997), “action research intends to introduce change” in an

educational environment by supportive documentation.

The planning and execution of this Action Research project complies with current

educational principles and practices.

Learning Theories as They Relate to Technology

Support for the effective use of instructional technology by current learning

theories is noted. Interestingly, the two most contrasting learning theories of

Behaviorism and Constructivism both provide support, in differing contexts, to the use of

learning technologies.

Underwood argued convincingly that different learning theories apply to different

circumstances, depending on the type of knowledge desired: factual, procedural, and

conceptual knowledge may be learned better using different learning paradigms.

Underwood also emphasized that people sometimes had better memory of self-

generated information than information they receive passively. The encouragement of

self-generated knowledge, and personal exploration, would include actively sought

information, supported in this case, by Information Literacy techniques and strategies

used in Internet searching. In other cases, people remember as well, or better,

information provided rather than information they initiate (Underwood, 1994).

Skinner (1938) introduced the notion that a stimulus and response mechanism

can control behaviors, given a suitable sequence of reinforcing repetitions. Modern

educational behaviorists described learning as not much more than the acquiring of new

behavior by focusing on objectively observable behaviors and discounting mental

activities (On Purpose Associates, 1998). Jones (1993) described Skinner’s early

9

application of a teaching machine, which would reward a learner for every correct

response. Later, Skinner conceptualized a teaching machine for classroom use by

individual students, which would break learning into small steps, with appropriate

reinforcement at each level (Jones, 1993). Today’s computer-assisted instruction

software utilizes such small-step reinforcements at varying intensities.

Geisert and Futrell acknowledged that Behaviorism does have its place in

technology. They also suggested basic drill-and-practice software is suitable for

reinforcing skills already introduced and learned in a different context. The

development of automatic skills sets in math can benefit from drill-and-practice software

assistance. For example, drill-and-practice software can be conducive to the

memorization requirements of multiplication tables (Geisert and Futrell, 1990).

Black suggested caution regarding excessive dependence upon behaviorist

archetypes, in an avoidance of the presumption that humans are android-like.

Behaviorism does not account for all kinds of learning, since it simplifies and disregards

the activities of the mind in behavior changes (Black, 1995). Jones, et al, (1994)

remarked that complex learning behavior is hard to analyze in terms of the simplified

stimulus and response circumstances proposed by Behaviorists.

In contrast to Behaviorist theory, Piaget fundamentally described learning in

stages of cognitive development, as well as the two distinct, but non-exclusive,

processes of assimilation and accommodation. Assimilation is a learning condition that

is non-contradictory to the learner’s existing knowledge. The learning condition is

absorbed into the learner’s framework with little or no resistance. Accommodation

10

refers to changing the cognitive structure to make sense of the environment in an

adaptive process (Piaget, 1952).

The Constructivist model aligned with Piaget’s ideas, given that learners

“construct” their own understanding of the world they live in, by considering their own

experiences (On Purpose Associates, 1998). This alternative depiction of Piaget’s

assimilation and accommodation utilizes various rules and mental models engendered

by the learner to make sense of life encounters. Jones, et al, (1994) contended the

connection to life experiences create “engaged learners” who are able to be responsible

for their own learning goals and evaluations.

Gardner emphasized the significant role technology will have in future

educational environments. Prominence of the computer in education engenders

improvement in higher-order thinking skills. Individualization and engaged learning are

key Constructivist ideas. He succinctly expressed technology’s role in education.

All students may receive a curriculum tailored to their needs, learning style, pace and profile of mastery, and record of success with earlier materials and lessons. Indeed computer technology permits us to realize for the first time, progressive educational ideas of “personalization” and “active, hands-on learning” for students all over the world. (Gardner, 2000, pp.43-44)

Jones, et al, (1994) pointed to Project-Based Learning (PBL) with its challenging,

authentic, and multidisciplinary approaches. Such learning becomes conducive to the

use of open-end software (word processor, spreadsheets, programming, etc). Davis, et

al, (1997) reiterated that the authenticity of a Project Based Learning (PBL) task relates

to the effectiveness of learning. Integration with non-computer activities and relevance

11

of computer work to the curriculum all sustain instructional technology effectiveness

(Davis, et al, 1997).

Papert (1980) claimed an endorsement of both Piaget and Constructivism, by

advocating use of LOGO software for the development of mathematical thinking.

Students would relate LOGO structures to previously introduced concepts in order to

assimilate or accommodate those models. Papert (1980) argued LOGO software

becomes a “mathematical world” and “workspace” for authentic math problem solving.

Papert’s “ mathematical world” developed into a related computer activity especially for

use in Project Based Learning.

Does it Compute? The Relationship between Educational Technology and

Student Achievement in Mathematics (Wenglinsky, 1998) is a study of middle school

students utilizing drill-and-practice computer-assisted instruction (CAI) software. This

study provided an interesting assertion that such software, when used inappropriately,

may actually degrade a student’s score on the NAEP’s mathematics standardized test.

However, this study determined student success is still possible but only with

appropriately used mathematics technology, by well-trained teachers, with suitable

frequency, and associated with software using higher-order thinking skills. Wenglinsky

stated, “…when computers are used properly, they may serve as important tools for

improving student proficiency in mathematics, as well as the overall learning

environment in the school”.

Significant technology implications provided by Wenglinsky (1998) and

Christmann (1997) included the importance of properly trained teachers, especially in

high-poverty and urban schools. Does it Compute? (1998) stressed the focus to apply

12

technology for higher order thinking skills previously introduced elsewhere. The

importance of obtaining software for higher order thinking skills, such as problem

solving becomes critical as well. Some researchers also noted technology use has a

much greater effect on the middle school level rather than at younger grades

(Wenglinsky, 1998). Others emphasized a need for specific guidelines to teachers

where computers can be helpful and where they cannot (Christmann, 1997).

Schacter summarized, in The Impact of Education Technology on Student

Achievement: What the Most Current Research has to Say, that recent research

recapping the initial assertion of this section on Learning Theories. That is, both

Behaviorist-directed and Constructivist-directed technology may have a beneficial effect

on student achievement. “These studies show that …students with access to computer-

assisted instruction, or integrated learning systems technology, or simulations and

software that teaches higher order thinking, or collaborative networked technologies, or

design and programming technologies, show positive gains in achievement on

researcher constructed tests, standardized tests, and national tests” (Schacter, 2000).

Multimedia

The relationship between teacher and learner continues to change as technology

allows them to communicate in a variety of ways and access a wide range of resources.

The way the teacher and/or learner delivers, represents, accesses, and manipulates

information is unlike that of any other time in the history of education (Hedberg, Brown,

& Arrighi, 1997). With interactive multimedia, information communicates in a variety of

visual and verbal forms. As a software user, a learner’s actions encompass a full range

of activities offered by software designers. It may be passive guided direction in

13

prescriptive environments or simulations with active gathering and reconstruction of

multimedia resources. With immediate feedback from the interactive multimedia

program, the learner situates into a rich, real world, problem-solving learning

environment (Moursund, 2001). Additionally, the learner’s educational experience using

real life situations can be an inhibitor, which is not the instructional designer’s intent

(Hedberg, Brown, & Arrighi, 1997).

The information in a multimedia world is new and novel. However simple this

may sound, it is an important consideration when evaluating the selection of educational

products. The viewing of a map, or listening and interacting with speech files on CD are

examples of ways in which multimedia can provide an understanding of problems not

evident in text-based description of problems and issues.

By transforming the computer into an instructional tool in a classroom, the

computer will become the most popular tool (Brown, 1998). Multimedia environments

allow users to explore and undertake a range of tasks that closely mirror those of the

real world. In this way, you do not have to be constrained by verbal descriptions of

visual activities. When students are able to convert learning into a world in which the

leaning processes naturally unfolds, higher levels of cognition are attained (Hedberg,

Brown, & Arrighi, 1997).

Because attention tends to lapse some ten to eighteen minutes into a typical

classroom lecture, teachers need to find ways to engage students into the classroom

lecture. Video and web-resources reengage students. Brief digital sound and video

clips can accentuate a point and add an element of surprise to the lecture causing

students to pay closer attention (Stone, 1999). “Traditional instruction has been

14

considered one of the major causes of a dysfunctional and even obsolete educational

system. Through computers the use of multimedia has created novel modes of learning

and greatly contributed to the restructuring of instructional environments in schools”

(Relan & Gilliani, 1997).

Students learn at different speeds. Regular, immediate feedback facilitates the

learning process. The effective classroom exploits individualized learning techniques.

Individualized learning examples include students working with prepared materials at

their own pace and receiving information as to their progress in regular intervals. Using

the multimedia computer in the classroom fosters individualized learning techniques.

Web-based Instruction

Considering the browser-based capabilities of Riverdeep Destination Math, a

review of literature related to web-based instruction (WBI) can be beneficial. In a survey

report of educators throughout the nation, Sivin-Kachala emphasized the potential K-12

audience for web-based instructional systems and the need for significant

improvements to the design attributes beyond bland HTML. This nation-wide sampling

of district administrators and school principals indicated highest interest in cost-

effective, online student preparation for high stakes testing, online class planning

resources for teachers, and online professional development (Sivin-Kachala, 2002).

The World Wide Web acts as both a repository of knowledge and its distribution

mechanism. As ubiquitous Internet access increases, availability of knowledge

becomes more widespread. Gardner (2000) concisely described this phenomenon.

Knowledge is also now seen as distributed. That it, it does not reside exclusively within the head of an individual; rather it emerges jointly from one’s own perspective, the perspectives of other individuals, and the

15

information derived from available human and technological resources. (p.98)

Language differences are becoming less formidable barriers to the

transfer of knowledge on the Internet. With the advent of language translation

sites such as Babblefish, dynamic conversion of web-page language is now

possible. While linguists criticize the accuracy of such rudimentary translation

processes, such capabilities enhance and foster the distribution of knowledge

across the Internet.

The browser-based Riverdeep Destination Math is effective on the Internet or on

an Intranet. Riverbank Unified School District utilizes both methods. The Wide Area

Network (WAN) permits all school sites immediate access to Riverdeep Destination

Math Software. Internet connectivity allows students and teachers to access the

software from outside the boundaries of the district WAN.

Researchers subscribed to the careful significance of web-based design:

“Venturing into this new dimension will require thoughtful analysis and investigation of

how to use the Web's potential in concert with instructional design principles,” (Ritchie

and Hoffman, 2000). Directly related to planning and design, effects on subject matter

and student motivation could well be profound with an appropriate presentation.

Recommendations also included an active student involvement process and efficient

response feedback for greater effectiveness in using web-based software (Ritchie and

Hoffman, 2000).

Jones and Liu identified specific web-based design characteristics without which

“…web-based instructional environment have no impact on student achievement…”

16

While the study focused on higher education, many of this study’s suggested design

activities might be significant for K-12 software as well. They claimed the most crucial

design activities include a dedication to instructional objectives, recognition of learner

needs, and a high degree of visualization. Although less to do with design

characteristics and thus less controllable, student goal orientation and student

perception skills can be factors for student success (Jones and Liu, 2001).

Henke described the importance of sound web-based instruction and effective

web design issues. Top structural design mistakes to avoid include:

1. Frames which inhibit book marking,

2. Constantly moving animations causing reader distractibility,

3. Overly complex URLs fostering typing errors,

4. Long scrolling pages confounding readers,

5. Difficult and confusing navigation techniques,

6. Non-standard link colors making link determination difficult,

7. Outdated web site information frustrating users (Henke, 1997).

A significant contribution to software design, including WBI design, comes from

Jones and Okey in their article, Interface Design for Computer-based Learning

Environments. Effectively collating and summarizing instructional software design

mechanisms, the study illustrated a detailed inventory of effective components. Using

five basic concepts, with significant sub-concepts, the authors provide meaningful

guidelines to software designers. These five basic concepts include browsing, media

integration, metaphors, information access, and unfamiliar territory (Jones and Okey,

1995).

17

Web-based instruction accessibility issues can be concerns. Standardized,

structural web accessibility issues, especially for people with disabilities, describe basic

features and components for useable content on a web page or site. An example of

these disability-related issues would be the excessive use of flashing objects, which

could trigger seizures (Henke, 1997). There are also issues related to equity of access,

providing Information Literacy for all students, not limiting it to certain groups. Also,

there is the realistic issue of the number of students having home computers and having

Internet access.

The World Wide Web Consortium), in their Web Content Accessibility Guidelines,

delineated the guiding principles of practical web page components, which conform to

ubiquitous access requirements. These principles include

1. Alternatives to auditory and visual content,

2. Issues of color,

3. Appropriate use of tables,

4. Designing for device-independence,

5. Providing contextual information,

6. Navigational charactertistics (World Wide Wed Consortium, 1999).

Computer-Assisted Instruction

Computer-assisted instruction most often refers to drill-and-practice, tutorial, or

simulation activities. Drill-and-practice and tutorial activities are the two most commonly

used computer-based instructional strategies employed in K-12 schools. Individualized

learning delivered by computer-assisted instruction can increase a student’s productivity

in the classroom. Students are also able to achieve “higher quality” learning, allowing

18

students to learn materials at their own pace. Students are also able to study subjects

interesting to them, and teachers are able to individualize the instruction to meet the

needs of each student in the classroom (Walker, 1987). By being able to select from

various forms of multimedia, including video, animation, audio, text, animation, graphs

and equations, students can receive material in the most meaningful form that matches

their particular learning style thus facilitating higher levels of comprehension (Lewis,

1999). “The concern for competency in mathematics has led mathematics educators to

attempt to clarify the processes underlying the acquisition of various mathematical skills.

Within the past 20 years, there have been attempts to juxtapose novel techniques into

conventional strategies for teaching mathematics. In the forefront of these are

computer-based educational programs,” (Walker).

School districts are now adding computer-assisted instruction to help improve

achievement in both basic skills, and highly specialized areas of instruction.

The trend for adopting computer-assisted instruction is rooted in theoretical as well as pragmatic educational foundations. Much of our theoretical insights into cognitive development of mathematical skills have psychological theory basis. The structuralism of Piaget and Bruner is an example. Additionally, Skinner’s notion of operant conditioning has been instrumental in the buildup of arguments in favor of program of learning and computer-assisted instruction (Walker, 1987, p.562).

Schools have found that by including computer-assisted instruction in the

instructional environment, students will receive the following benefits:

1. Frequent feedback to learners,

2. Tutorial readings,

3. Individual pacing,

4. Individual programming,

19

5. Clarity of presentation,

6. Motivational factors (Walker).

Computer-assisted instruction has many advantages. It allows educators to teach

higher-level, specialized courses and provides additional class time for higher-level

instruction. The computer assigns basic skills development and provides opportunities

for the creative development of innovative curriculum materials (Walker).

Additionally, the tutorial mode of instruction involves the presentation of new

materials directly from the computer. Instruction provides the context of solving

problems. Students learn new material with monitoring as they progress through a

program. Increasingly complex material follows increased proficiency (Walker).

After each new concept introduction, the student works a number of problems designed

to put that concept to use. The tutorial provides immediate feedback and guidance on

incorrect and non-strategic steps. The computerized tutor compares information entered

by the student to determine a correct or incorrect response. If the input matches a

correct rule, the tutor is silent or complementary and waits for further input. If the input

is determined to be an error, the tutor interrupts with advice. Thus, the feedback is

immediate and necessary instruction given both in general terms and in context of the

current problem. The tutor also provides guidance to the student as they complete the

exercise. The student can request clarification of a current part of the problem and ask

for the next step in the solution. In addition, if the student has sufficient difficulty

including the particular part of the problem, the tutor will intervene. “Given its adaptive

instructional capabilities, the computer is viewed as displaying versatility in on-line drill-

and-practice, providing individualized instructional prescriptions, giving immediate

20

feedback from its diagnosis of individual errors, and assigning remediation” (Rockart,

1973).

Research supports computer-assisted instruction as a supplement to traditional

teacher-directed instruction. Achievement effects are superior to those obtained with

traditional instruction alone. These findings hold true for students at different ages and

abilities and for learning in different curricular areas. Dalton and Hannifin’s research

(1988) indicated that "while both traditional and computer based delivery systems have

valuable roles in supporting instruction, they are of greatest value when complementing

one another" (Dalton & Hannifin, 1988).

Project Description

The Riverbank Unified School District made a decision, in July of 2002, to

acquire Riverdeep Destination Math software for $35,000. Cardozo Middle School

students use Riverdeep to attain math skills needed to pass the required district math

proficiency.

Installation of the Destination Math program took place on the school server and

teachers trained a single day using the math program in a classroom setting. Teachers

assigned student modules after the first training. Following initial implementation, the

district technology coordinator determined that the program had technical problems.

The district technology coordinator corrected these technical issues over time.

Technical and practical considerations caused the district technology coordinator to

move the entire program to a dedicated server. Because it took four months to correct

all of the technical issues, the teachers disregarded Riverdeep and their initial training.

Riverdeep provided a second training to staff members to help re-acquire the skills

21

necessary to implement the Destination Math program in a classroom, In view of the

significant amount of money invested in this program, the researchers felt it was

important to explore how this program would help the Riverbank Unified School District.

The purpose of the study is to evaluate use of Riverdeep Destination Math on 8th

grade student achievement in math and on 8th grade student attitudes towards math

education.

Riverdeep Destination Math is a comprehensive and sequenced software

product utilizing multimedia to present mathematical issues related to real life situations.

The product teaches basic math skills, math reasoning, conceptual understanding, and

problem solving.

There is full audio support with visuals. On screen manipulates are employed to

help students master math concepts. A web-server dispenses HTML pages exploiting

Macromedia Flash, Java Virtual Machine, and QuickTime movies to student and

teacher Internet browsers. The software is also available to students with home access

through the school district’s web page. The software contains teacher management

tools and assessment tools correlated to California State Standards for mathematics.

An embedded scope and sequence exists for each module along with student

worksheets.

In each sequenced lesson, there are several multimedia-based tutorials, followed

by an on-screen assessment. If a student makes a predetermined passing score on the

assessment, he or she advances to the next lesson in that particular scope and

sequence. If a student does not achieve a passing score, tutorials occur again prior to

retaking the assessment.

22

The students in the after-school intervention program utilized the Riverdeep

Destination Math software section called Course III, Intermediate Mathematics. This

particular module has tutorials and assessments designed with the following learning

objectives:

1. Basic addition, subtraction, multiplication and division of integers,

2. Basic addition, subtraction, multiplication and division of fractions,

3. Basic addition, subtraction, multiplication and division of decimals,

4. Basic problem solving techniques related to equations,

5. Basic scientific notation skills.

The performance requirements of the Cardozo Middle School’s eighth Grade Math

Competency test heavily emphasize these specific objectives. Utilizing the teacher

management tool, an instructor can determine assessment scores, time on each task,

as well as pre-assign specific levels to individual students.

This particular study uses a group of 8th grade students in a math intervention

program during after-school hours at Cardozo Middle School. Students are volunteers

selected by their inability to pass the school’s math proficiency test. A noticeable

limitation is that attendance is strongly encouraged but not mandatory. The program

potentially mitigates student retention.

Because students were not required to attend this math intervention program

during after-school hours, over half of the students dropped out. A recommendation for

subsequent years’ implementation includes mandatory attendance. Increased

integration of Destination Math into the daily classroom environment would also

acknowledge a greater commitment to utilize the program effectively.

23

During the study period student subjects spent Wednesday and Thursday

afternoons utilizing Destination Math for 60 minutes under the supervision of a teacher.

Attitudinal surveys were conducted with both students and teachers involved in

the study. Researchers and instructors refined initial paper surveys. In an effort to

simplify analysis of student survey data, researchers utilized a browser-based survey

software product. Students took the online survey immediately prior to the start of the

study period, February 27, 2003, and at the end of the study period, April 4, 2003.

Teachers took the paper survey during the week of April 7, 2003. Copies of student

surveys are in Appendix 2 and Appendix 3. The online versions of the student surveys

are located at http://www.riverbank.k12.ca.us/survey/TakeSurvey.asp?

DisplayHeader=Yes&SurveyID=107 and http://www.riverbank.k12.ca.us/survey/

TakeSurvey.asp?DisplayHeader=Yes&SurveyID=108. A copy of the teacher survey is

in Appendix 4. Results of the student surveys are in Tables 3 and 4. Results of the

teacher survey are in Table 5.

Maintenance of strict confidentiality occurred with all student participant names,

surveys, and assessments. Appendix 1 contains a copy of the Informed Consent form

signed by all persons involved in the study.

Limitations of the Study

The Action Research study was limited in the following aspects.

Student data in this study is limited due to a reduced number of students

who consistently attended the after-school math program. The Riverdeep

Destination Math after-school intervention class was a volunteer math

program. The voluntary nature of the program dramatically reduced the

24

number of student subjects. Over half of the students who participated in

the after-school program dropped out after only two weeks.

The content of the instructional unit required alignment with the Riverbank

Unified School District 8th grade math proficiency as opposed to the

California Math standards. Teachers and researchers spend unnecessary

time manually matching appropriate standards to student assignments.

Staff training for Destination Math occurred when the program was not

fully functional on the Riverbank Unified School District server. Despite

later correction of these technical issues, staff interest and training

momentum waned.

Cardozo Middle School math teachers received insufficient planning time

during implementation of Riverdeep Destination Math. Appropriate

implementation procedures were never completely developed. In addition

to implementation problems in daily classroom use, the after-school

program in this study suffered a lack of direction as well.

25

Findings

Data Collection

The researchers utilized several methods of data gathering including:

1. Online Destination Math assessment modules and reports (in Table 1),

2. Online Likert scale surveys (in Tables 2 and 3),

3. Group interviews,

4. Researcher observations.

Compiled data permitted making limited statements regarding the effectiveness of

Destination Math software as well as changes in student attitudes toward math.

Research Questions

1. How will the use of Riverdeep Destination Math affect student achievement on

math achievement scores?

2. How will the use of Riverdeep Destination Math affect student attitudes toward

math?

Data Analysis

Determinations regarding the effects on student achievement use built-in

assessment and reporting mechanisms in Destination Math. These embedded reports

permitted achievement growth measurements during the research period from March 3-

April 4, 2003. These reporting mechanisms also enabled the researchers to measure

time on task. Results in Table1 were findings using these reports. The researchers

determined the focus would be on students who had 90% or better attendance (less

than 50%) in this voluntary intervention program. Twenty-seven Cardozo Middle School

students initially participated in this program to acquire the skills necessary to pass the

26

math proficiency. Students were not required to attend this after-school program and

instructors were unable to assign any meaningful consequences for non-attendance.

Implications to students included that non-attendance would place them in jeopardy of

not acquiring the skills necessary to pass the math proficiency. Despite reviewing a

retention policy based upon passing school proficiencies along with signed student and

parent acknowledgements, only limited attendance took place.

Table 1 - Student Progression Report from Destination Math

Student

Number

Time on each

module

Modules

completed

Number of times modules were

taken and failed

8647 < 40 minutes 2 0

8612 > 40 minutes,

but < 80

2 1

8636 <40 minutes 2 0

8690 > 120 minutes 3 1

8634 > 40 minutes,

but < 80

2 0

8623 > 40 minutes,

but < 80

2 0

8698 > 120 minutes 3 0

8693 > 40 minutes,

but < 80

2 1

8610 <40 minutes 2 0

8675 > 40 minutes, 2 1

27

but < 80

8600 > 40 minutes,

but < 80

2 1

8630 > 120 minutes 3 1

An obvious conclusion from Table 1 above is that there is a noticeable

association between the time students actually spent using Destination Math and the

number of modules they completed. The more time spent on task using the Destination

Math program, the greater the number of modules students completed. All three

students who spent less than 40 minutes on task using this program did not fail any

modules and completed two modules. Students who spent more than 40 minutes and

less than 80 minutes on task completed two modules but four of the six students failed

one module. Students who spent more than 120 minutes on task using the Destination

Math program, completed three modules, but two of the three students failed one

module. After looking over the data, the researchers felt the reason some students

spent more time on task was that they had to retake a module. This potentially would

lend itself to the conclusion that if a student had to retake a module they would then

spend more time on task.

The researchers also looked at how students scores changed on the math

proficiency test taken at the beginning of the year and then again in April. Students who

spent more than 80 minutes but less than 120 minutes on task using the Destination

Math program raised their math proficiency scores between 15 and 25 points. Students

who spent more than 40 minutes but less than 80 minutes on task raised their math

28

proficiency scores between 5 and 12 points. Students who spent less than 40 minutes

on task using the Destination Math program raised their scores between 8 and 12

points. This leads the researchers to state there is a positive correlation between

students using the Destination Math program and passing the math proficiency. The

researchers feel it is important to have further studies into the correlation between the

Destination Math program and the Riverbank Unified School District’s math proficiency

test.

The researchers also looked at why students did not pass the math proficiency

after spending time in an after-school tutorial program and using the Destination Math

program. Data in Tables 1 and 2 indicate the more time spent using the Destination

Math program results in an increased score on the Cardozo Middle School math

proficiency test. Sufficient on-task time requirements enabling students to pass the

math proficiency are the subject of further research. It was also determined that many

of these students had noticeably low scores the first time they took the math proficiency.

This problem is another indicator that Destination Math implementation should have

occurred earlier in the school year.

The researchers were pleased to note that all of the students who participated in

this study did increase their math scores on the math proficiency test. Did the students

enhance their scores enough to pass the math proficiency? No, but many of them

came very close to passing. Given the short use of Destination Math, it is difficult to

determine the contribution the software may have contributed to the test improved

scores. In the future, teachers will need to start this program earlier in the year and have

29

students working on specific modules to help them to acquire the skills necessary to pass the

math proficiency.

Table 2 – Student Math Proficiency Test Scores

Student

Number

Score on

the Math

Proficiency

test taken at

the

beginning

of the

school year.

Score on

the Math

Proficiency

test taken

in April.

Number of points

the Math

Proficiency

scores have

risen.

Students who

passed the Math

Proficiency.

8647 45 55 10 Did not pass.

8612 54 59 5 Did not pass.

8636 47 55 8 Did not pass.

8690 35 50 15 Did not pass.

8634 45 52 7 Did not pass.

8623 49 57 8 Did not pass.

8698 36 61 25 Did not pass.

8693 56 68 12 Did not pass.

8610 55 67 12 Did not pass.

8675 47 56 9 Did not pass.

8600 48 60 12 Did not pass.

8630 40 58 18 Did not pass.

30

Online Likert Scale Survey

The researchers developed two simple student surveys utilizing Likert Scales to

determine any attitudinal changes toward math. A scale of 1 to 4 indicates the following

values: 1=strongly disagree; 2=disagree; 3=agree; 4=strongly agree. Using this set of

values, the higher score is indicative of a more positive agreement with the statement.

Student Survey 1 (see Appendix 2) generated data from the beginning of the

study. Student Survey 2 (see Appendix 3) generated data from the end of the study. A

teacher survey determined teacher opinions regarding the use of Destination Math at

the end of the study. The online location for both student surveys is

http://www.riverbank.k12.ca.us/survey/Default.asp, facilitating simplified data analysis.

Tables 3, 4, and 5 show findings from those surveys.

Table 3 - Student Survey 1-- Responses at beginning of study

Question

Strongly

Agree

= 4

Agree

= 3

Disagree

= 2

Strongly

Disagree

= 1 Score

1. I like math. 1 2 4 5 1.92

2. I think I am good at math 0 1 7 4 1.75

3. I learned more math this

year than I did last year 2 1 4 5 2.00

4. I spent more time on math

this year than I did last year 5 2 2 3 2.75

5. The pace of this math class 1 3 3 5 2.00

31

is just right

           

Overall Student Survey #1

Average         2.08

Student Survey 1 in Table 3 above showed students had some negativity toward

math in general on questions 1 and 2. These two questions also related to a student’s

self-esteem toward math. In small group interviews, many of the students made

statements consistent with the belief that they were poor in math. Related to this belief

was the student conviction that their poor math skills were the primary reason for being

in this math intervention program.

Scores on questions 3 and 5 indicated in Student Survey 1 in Table 3 above

suggests ambivalent responses. As to whether they learned more math this year than

last year, overall students were notably uncertain. In addition, students were not sure

about the pacing of their current class. The pacing ambivalence was borne out in the

small group interviews as well. There were shrugs and blank looks when asked if the

class was going too fast for them.

The question indicating the most positive score in Student Survey 1 in Table 3

above was number four, regarding how much time they felt they were spending on math

this year. Given that they were going beyond their normal classroom work to come to

this after-school math program, it seems reasonable that they would see themselves as

spending more time in their pursuit of passing the math proficiency.

32

The overall score (2.08) of Student Survey 1 in Table 2 above is marginally

positive. However, as stated previously, these particular respondents are non-

proficiency-passing students who achieved a 90% attendance rate. These attendance

conditions could be construed to mean that these particular students are already open

to the potential usefulness of this math intervention program.

Table 4 - Student Survey 2 – Responses at end of study

Question

Strongly

Agree

=4

Agree

= 3

Disagree

= 2

Strongly

Disagree

= 1 Score

+/-

from

Survey

#1

1. I like math. 2 3 4 3 2.33 +0.42

2. I think I am good at

math 2 2 5 3 2.25 +0.50

3. I learned more math

this year than I did last

year 2 3 2 5 2.17 +0.17

4. I spent more time on

math this year than I did

last year 4 3 4 1 2.83 +0.08

5. The pace of this math

class is just right 3 2 2 5 2.25 +0.25

Average from questions

in Student Survey 1 2.37

+0.28

33

6. I like math better this

year than last year 5 2 3 1 2.75

7. It was easy to learn

how to use the computer 4 2 2 4 2.50

8. I learn math better with

the computer instead of

only with a book 7 3 2 0 3.42

9. I feel confident that I

can pass the tests that

the computer gives me 4 3 1 4 2.58

Average from questions

6 to 9 2.81

           

Overall Student Survey

#2 Average         2.56

By comparison with the initial survey, questions 1 and 2 in Student Survey 2 in

Table 4 above show score increases. While not huge jumps in scores (+.42 and +.50

respectively), this is an indication that student attitudes toward math in general, and

their self-confidence toward math, has strengthened. Notably, the scores on these two

questions showed the greatest increases of any of the original five questions from the

initial survey.

34

Questions 3 and 5 in the initial survey which indicated uncertainty or

ambivalence, show small increases (+.17 and +.25 respectively) as well. This could be

indicative of a notable progression toward positive feelings about the pacing of the class

and their accumulation of math knowledge and skills.

Question 4, regarding the amount of time spent on math, indicated the least

score increase out of all the questions from the initial survey. This question had scored

highest on the initial survey and continued to score highest on the secondary survey.

This may be because the positive nature of this question, from the students’

perspective, was already virtually as high as it could go.

The overall average score of questions 1 to 5 in this subsequent Student Survey

2 as shown in Table 4 above, when compared with the initial survey, showed

conspicuous growth (+.28). In addition to indicating a more positive appreciation of the

math intervention program, the increased score could also be pinpointing students’

outlook on their daily math classes.

Remaining questions in Student Survey 2 (6 to 9) are indicative of technology in

general, and how Riverdeep Destination Math in specific, might affect their attitudes

toward math. In this effort to accommodate the technology differences, the average

score in this small group of questions was much higher (2.81) than the original set of

five questions.

Not surprisingly, question number 8 regarding the use of a computer for math

assignments, scored the highest of any survey question. The novelty effect of computer

use may influence this question, but responses are largely consistent with student

opinions given in the small group interviews.

35

Factoring in questions 6 to 9 of Student Survey 2 may skew a suitable

comparison with the initial student survey. However, after embedding these questions

into the overall average, there is a sizeable increase in the overall score, as seen in

Table 5 below. Even while recognizing that it is statistically inappropriate to that joining

of data, the resulting score does reinforce an overall impression of more positive

student attitudes during the study period.

Table 5 - Teacher Survey Results

Question

Strongly

Agree

= 4

Agree

= 3

Disagree

= 2

Strongly

Disagree

= 1 Score

1. My students are learning

basic math skills better this

year. 0 2 0 0 3.00

2. My students are learning

higher-order thinking and

problem-solving skills better

this year. 1 1 0 0 3.50

3. My students are progressing

through math topics faster this

year 0 1 1 0 2.50

4. My students are more

confident in math this year 0 1 1 0 2.50

5. My students spend more 1 0 1 0 3.00

36

time doing math this year

6. My students math time is

more productive this year 0 2 0 0 3.00

7. My students are helping

each other more and working

more cooperatively this year. 0 1 1 0 2.50

8. I have fewer discipline

problems in math class this

year. 0 2 0 0 3.00

9. I am better able to deal with

my students’ different ability

levels this year. 0 0 2 0 2.00

10. I am better able to

diagnose and correct individual

student difficulties this year. 0 1 1 0 2.50

11. The information provided

by Destination Math enables

me to teach more effectively

than in previous years 1 1 0 0 3.50

12. I spend less time grading

papers and keeping records

this year 0 0 2 0 2.00

13. I spend more time teaching 0 2 0 0 3.00

37

and helping individual students

this year

           

Average         2.77

While only two teachers took the survey, there are positive score indicators in the

Teacher Survey in Table 5 above. As part of the transition to using Riverdeep

Destination Math this school year, the school has taken some unique measures to aid

successful implementation. For background information, there should be recognition

that the school has equipped all math classrooms with five new, networked student

computers, a new, networked teacher computer, and a high-speed printer. Each math

teacher has had a minimum of three full release days for Destination Math staff

development. The two teachers who participated in this particular study have had an

additional two days of release time specifically for planning and the hands-on

implementation of the plans.

Most notable among the higher scores in the Teacher Survey in Table 5 above

are questions 2 and 11. One could correlate the two questions as the primary

contribution of the technology, both hardware and software, to an overall belief that

students are making progress toward higher-order thinking and problem-solving skills.

Group Interviews

When queried what they liked best about the use of Destination Math to help

them improve their math skills, many student comments focused on the details of the

program’s use of sound and animation, rather than specific improvement techniques for

38

learning math. Students said it was “fun” or “cool” indicating that they enjoyed it. It is

difficult to say how much of this particular emotion was due to the novelty of using the

computer and whether it would wear off as time went along. Although these students

were doing this work after-school, they noted that they would much prefer using the

computer and Riverdeep during class time instead of doing “bookwork.”

Researcher Observations

The researchers were able to observe this after-school program on several

occasions. In the observation notes, it was determined that students were able to focus

for about 20 to 30 minutes at a sitting prior to becoming distracted. The students wore

headsets to hear the computer sound better rather than using external speakers.

Indicative of active student engagement was the quiet room during Destination

Math time. If a student had a problem, he raised his hand to get the teacher’s attention.

Most student problems related to technical issues, rather than an understanding of the

process needed to properly perform the module. It seemed the students had already

learned the software navigation process in previous sessions.

According to teacher comments recorded in researcher notes, initially Riverdeep

was not the most of stable of technical environments. Occasionally the software caused

a “looping” process remedied only by restarting the computer. The teachers indicated

fewer technical issues since software and hardware upgrades in early January.

Teachers also noted the high dropout rate of students. When queried about this,

the two teachers agreed that a voluntary program would not seem to be effective in

getting students to this intervention program.

39

Discussion

Reflection & Revision

Student research started in March 2003 and concluded in April 2003. Using the

feedback received from students and teachers, the researchers were able to make

modifications that would help in the future when using the Destination Math program.

Some of the modifications suggested and implemented were as follows:

1. Destination Math software was put on its own server rather than running as an

application on the school server used for file and printer sharing. This allowed

the program to run much smoother and faster, distributing web contents more

efficiently. On the previous server, web content delivery speed was inconsistent

due to an overworked processor, resource requirements of SQL, and Riverdeep

software problems. Re-installation of Riverdeep on a single server coincided

with a software upgrade, eliminating other Riveredeep technical issues. A

planned Linux web-proxy server implemented at each school increases

Riverdeep web content delivery speed across the WAN by caching web pages

locally. Riverdeep performance to both local and distant desktops improved with

these changes.

2. School site meetings determined a need to start the math intervention program

much earlier in the school year. The after-school intervention program began

only after the school year was already two-thirds completed during the 2002-

2003 school year. Additional time a student spends in the math after-school

program, and using Riverdeep, will better supplement classroom efforts to

develop skills needed to pass the math proficiency.

40

3. Teachers will receive additional training on effective Riverdeep use, as technical

issues negated previous training. Ongoing instructional support and planning

time provide increased familiarization with the product, processes and

procedures. The school has also committed increased release time for a pilot

group of teachers to develop a practical classroom model for Riverdeep use prior

to school-wide implementation. The pilot group will then act as local mentors to

all Cardozo Middle School math teachers.

The researchers for this project were conducting research to answer the

following questions:

1. How will the use of Destination Math affect student achievement on math

achievement scores? The researchers noted student participants took the

Riverbank Unified School District math proficiency a second time and still did not

pass the proficiency. The students’ scores did increase but were still not

sufficient to pass the proficiency.

2. How will the use of the Destination Math program affect student attitudes toward

math? Student attitudes towards math have changed as indicated by the

conducted research. The researchers found that students describe the learning

satisfaction as favorable and agreed that they enjoyed working with the

Destination Math program versus learning the same material from a textbook.

Action Plan Summary

In view of the limited period and rapid snap shot of Destination Math, one of the

researcher’s goals is to reuse this Action Research project and process for the next

school year, better determining product efficacy. The considerable expense of this

41

learning environment indicates that the Cardozo Middle School is committed to the

continued use of this software product. There have been, and continue to be, additional

computer hardware purchases for all math classrooms to make full use of this product.

Planning for ongoing staff development to increase staff use of Destination Math is

essential.

The use of this software as a browser-based, Intranet or Internet accessed,

learning tool, is unique for this district. It is the perspective of the researchers that any

future large-scale learning environment purchases should be web-based to provide

simplified and ubiquitous access to all constituents. LAN-based products, by

comparison, are not as effectively suited to service a variety of schools and student

grade levels.

Implications

Completion of the study and examination of the results suggests a positive

correlation between students using Destination Math and an increase in their math

proficiency test scores. Use of Destination Math also supports a potential for more

positive student attitudes toward math. However, the limited quantity and quality of the

study data requires further verification by another more rigorous study of Riverdeep

Destination Math.

Results of this study can be important because of the American public’s high

expectations involving technology curriculum integration and the substantive data

supporting its effectiveness.

The population from which the study sample was drawn was relatively narrow.

The participants were all members of an after-school, voluntary math program. The

42

district should actively study a larger number of students to judge the software’s

efficacy.

Riverbank Unified School District spent $35,000 to purchase Riverdeep

Destination Math software and $40,000 to purchase computers for math classrooms.

Students can access this software from all school sites as well as from homes. Initial

implementation of this software program provided three days of staff development.

However, technical problems negated this initial training. Considering the significant

funds spent on software and hardware, the district needs to continue additional staff

development assisting teachers to implement Riverdeep into classroom and after-

school settings.

Interventions mandated by the No Child Left Behind (NCLB) Act of 2001 require

schools to act aggressively to alleviate student failure (Brady, 2003). Riverdeep

Destination Math, appropriately implemented, can be an aggressive attempt to mitigate

student mathematics failure. The district needs to make a determined effort for an

appropriate Riverdeep implementation in coming years.

The accumulation of all student data, regardless of source, is essential for

“adequate yearly improvement” indicators and as aids to Data Driven Decision Making

(Brady, 2003). As full integration of Destination Math at Cardozo Middle School occurs,

data derived from the software will be an additional tool in helping determine future

curriculum decisions.

43

Appendix

The following pages include appendices related to documentation of student and

staff data input, as well as screen shots of student use of Riverdeep Destination Math.

As currently implemented by the Riverbank Unified School District, Riverdeep

Destination Math software is accessible both locally and by Internet at:

http://riverdeep.riverbank.k12.ca.us/riverdeep/lms/login/login1.asp.

44

Appendix #1 – Informed Consent

INFORMED CONSENT FORM FOR PARTICIPANTS

You are invited to participate in a study being conducted by Terry Scott and Kathy Briggs, master level students from Sacramento State University and employees of Riverbank Unified School District.

The project focuses on using the computer program Accelerated Math to help students learn math concepts. The researchers are particularly interested to discover if technology is helping students to understand and learn the math skills needed to pass the math tests that are required in the math classes.

If you decide to participate, you will be asked to take part in an interview and survey. These will be conducted at a convenient time and place for you. The interview and survey should take one hour of your time

Participation in the project is completely voluntary. If you do not want to participate in the project, you may withdraw at any time.

Your confidentiality will be protected throughout the study.  Any audiotapes of interviews and any other data obtained from you will be kept confidential and will not be viewed by anyone but the researchers. All audio or videotapes will be retained in a locked cabinet or other locked storage area. The tapes will be erased at the completion of the project.

There are no anticipated benefits or risks to you as a participant, aside from helping us have a better understanding of how technology can help students learn math concepts.

If you have any questions about the research project, you can call Terry Scott at 869-2538 or Kathy Briggs at 869-1891.

Thank you for your participation!

If you do not want your child to participate in this research project please sign this form and return it to Mr. Cox or Ms. Smith.

I do not want my child to participate in this research project:

____________________________________________ ________________Parent Signature Date

____________________________________________ ________________Student Signature Date

45

Appendix #2 – Student Survey #1

Riverdeep Survey

Student Name__________________________________________________________Grade_________________ Check One: ____Boy ____GirlTeacher ______________________________________________________________

Please indicate how much you agree with the following statements by circling your response.

1. I like math.Strongly agree Agree Disagree Strongly Disagree

2. I think I am good at math. Strongly agree Agree Disagree Strongly Disagree

3. I learned more math this year than I did last year.Strongly agree Agree Disagree Strongly Disagree

4. I spent more time on math this year than I did last year.Strongly agree Agree Disagree Strongly Disagree

5. The pace of this math class is just right.Strongly agree Agree Disagree Strongly Disagree

6. I average the following number of hours on math homework each week.Less than 1 2 3 4 5 More than 5

46

Appendix #3 – Student Survey #2Riverdeep Survey

Student Name__________________________________________________________Grade_________________ Check One: ____Boy ____GirlTeacher ______________________________________________________________

Please indicate how much you agree with the following statements by circling your response.

7. I like math.Strongly agree Agree Disagree Strongly Disagree

8. I think I am good at math. Strongly agree Agree Disagree Strongly Disagree

9. I learned more math this year than I did last year.Strongly agree Agree Disagree Strongly Disagree

10. I spent more time on math this year than I did last year.Strongly agree Agree Disagree Strongly Disagree

11.The pace of this math class is just right.Strongly agree Agree Disagree Strongly Disagree

12. I average the following number of hours on math homework each week.Less than 1 2 3 4 5 More than 5

13. I like math better this year than last year.Strongly agree Agree Disagree Strongly Disagree

14. It was easy to learn how to use the computer.Strongly agree Agree Disagree Strongly Disagree

15. I learn math better with the computer instead of only with a book.Strongly agree Agree Disagree Strongly Disagree

16. I feel confident that I can pass the tests that the computer gives me.Strongly agree Agree Disagree Strongly Disagree

47

Appendix #4 – Teacher SurveyRiverdeep Survey

Teacher

Name________________________________________________________________Date_________________ Grade ___________

Please compare this year’s teaching experience using Accelerated Math with your past math teaching experiences. Feel free to elaborate on your responses by writing comments, using additional paper if necessary.

17.My students are learning basic math skills better this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

18.My students are learning higher-order thinking and problem-solving skills better this year.

Strongly agree Agree Disagree Strongly Disagree Don’t know

19.My students are progressing through math topics faster this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

20.My students are more confident in math this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

21.My students enjoy math more this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

22.My students are more motivated to work at math this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

23.My students take more responsibility for their math work this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

24.My students spend more time doing math this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

25.My students math time is more productive this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

26. My students are helping each other more and working more cooperatively this year.

Strongly agree Agree Disagree Strongly Disagree Don’t know

27. I have fewer discipline problems in math class this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

48

28. I am better able to deal with my students’ different ability levels this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

29. I am better able to diagnose and correct individual student difficulties this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

30.The information provided by Destination Math enables me to teach more effectively than in previous years.

Strongly agree Agree Disagree Strongly Disagree Don’t know

31. I spend less time grading papers and keeping records this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

32. I spend more time teaching and helping individual students this year.Strongly agree Agree Disagree Strongly Disagree Don’t know

33. Did you change the way you teach math because of using Accelerated Math? Please explain.

34.Did you keep your whole class together in their work using Accelerated Math, allow students to work at their own rates through the objectives, or have another system? Please explain.

35.Do you thin Accelerated Math had a positive effect on girls’ achievement in math, their attitude towards math, or their confidence? Please explain.

36.How would you describe student interactions in your math class? For example, were students helping each other informally? Were they working in assigned groups? Please explain.

37.Teachers using Accelerated Math have reported that students learn math through various combinations of whole-class lessons, one-on-one explanations, small group instruction, students learning on their own, students working cooperatively, or other means. How do students learn math in your class?

49

38. If your students spend more time doing math this year, is this due to an increase in the math period time, more efficient use of class time, or some other factor? Please explain.

39.Does Accelerated Math help prepare your students for high-stakes testing? Please explain. (This question may not apply to you.)

40.Please list the Accelerated Math reports that you find most valuable and briefly explain how you use them.

Please write any comments you may have:

50

Appendix #5 – Riverdeep Login screenshot

Appendix #6 – Riverdeep Module screenshot

51

Appendix #7 – Riverdeep Tutorial screenshot

Appendix #8 – Riverdeep Student Assessment screenshot

52

References

Black, E. (1995). Behaviourism As A Learning Theory . Houston: University of Houston

Clear Lake. Retrieved March 3, 2003, from: http://129.7.160.115/INST5931/

Behaviorism.html

Brady, R. (2003). Can Failing Schools Be Fixed? Thomas B. Fordham Foundation.

Retrieved April 19, 2003, from: http://www.edexcellence.net/

library/failing_schools/failingschools.html

Brown, S. (1998). “Presentation Software and the Single Computer”, Learning and

Leading with Technology. V. 26 no. 2

Calhoun, E. (1993). Action research: Three approaches. Educational Leadership,

51(2) , 62-65

Capper, J.& Copple, C. (1985). Computer Use in Education: Research Review and

Instructional Implications. Washington, D.C.: Center for Research into Practice,

Christmann, E., Badgett, J., & Lucking, R. (1997). Progressive comparison of the effects

of computer-assisted instruction on the academic achievement of secondary

students. Journal of Research on Computing in Education, 29, 325-336.

Cotton, K. (1992). Computer-Assisted Instruction: School Improvement Research

Series. Series V, Close-Up 10.

Dalton, D.W. and Hannifin, M.J. (1988). “The Effects of the Computer-Assisted and

Traditional Mastery Methods on Computation Accuracy and Attitudes". Journal

of Educational Research 82/1: 27 – 33

Davis, N. et al. (1997). Can quality in learning be enhanced through the use of IT? In

Somekh, B. and Davis, N. (Eds.) Using Information Technology effectively in

53

Teaching and Learning: Studies in Pre-Service and In-Service Teacher

Education London and New York: Routledge.

Dick, B. (1997) Action learning and action research. Retrieved May 3, 2003, from

http://www.scu.edu.au/schools/gcm/ar/arp/actlearn.html

Farmer, L. (1999). The Cost of Getting Digitized. Book Report, vol 18 Issue 3 p49

Fraley, Lawrence E. and Vargas, Ernest A. (1975). Academic Tradition and Instructional

Technology: Journal of Higher Education, Vol. 46, Issue 1 p1-15.

Gardner, H. (2000). The disciplined mind: Beyond facts and standardized tests, the K-

12 education that every child deserves. New York: Penguin Books. 

Geisert, P. and Futrell, M. (1990). Teachers, Computers, and Curriculum :

Microcomputers in the Classroom Boston: Allyn and Bacon.

Hedberg, J., & Brown, C. (1997). Interactive Multimedia and Web-Based Learning:

Similarities and Differences. In Khan, B. (Ed.), Web-Based Instruction (pp. 47-

58). New Jersey: Educational Technology Publications, Inc.

Henke, H. (1997). Evaluating Web-Based Instruction Design. Retrieved March 2, 2003,

from: http://www.scis.nova.edu/~henkeh/story1.htm

Jones, A. and Mercer, N. (1993). Theories of learning and information technology. In

Scrimshaw, P. (Ed.) Language, Classrooms and Computers London and New

York: Routledge.

Jones, B., Valdez, G., Norakowski, J. and Rasmussen, C. (1994). Designing Learning

and Technology for Educational Reform: Meaningful Engaged Learning [online].

North Central Educational Laboratory. Retrieved February, 22, 2003, from:

http://www.ncrel.org/sdrs/engaged.html

54

Jones, C. & Liu, M. (2001). Web-Based Instruction: The Effect of Design Considerations

on Learner Perceptions and Achievement. Presented at the World Conference

on Educational Multimedia & Hypermedia (Ed Media 2001), Tempara, Finland.

Jones. M.G., and Okey, J.R. (1995). Interface Design for Computer-based Learning

Environments. Retrieved March 10, from: http://www.gsu.edu/~wwwitr/

docs/idguide/

Khan, B.H., (Ed.) (1997). Web-Based Instruction. Educational Technology Publications,

Englewood Cliffs, New Jersey, page 6.

Kleiman, G.M. (2000). Myths and realities about technology in K-12 schools. In D.T.

Gordon (Ed.), The digital classroom (pp. 7-18). Cambridge, MA: Harvard

Education Letter

Kulik, J.A. (1994). Meta-analytic studies of findings on computer-based instruction. In

E.L. Baker and H.F. O'Neil, Jr. (Eds.), "Technology assessment in education and

training." Hillsdale, NJ: Lawrence Erlbaum.

Lewis, M. (1999). Computer-Based Training: New Technologies for Improved

Effectiveness. Heating/Piping/Air Conditioning Engineering. V.71 no. 12

McKenzie, J. (2001). Head of the Class: How Teachers Learn Technology Best.

Retrieved February 18, 2003, from: http:// www.electronic-school.com/

2001/01/010112.html

Mills, G. (2000). Action research: A guide for the teacher researcher. Upper Saddle

River, NY: Prentice-Hall.

Moore, A. (1993). "Computer-assisted Instruction (ILS) for Adults." Paper presented at

the Annual International Conference of the National Institute for the Staff and

55

Organizational Development on Teaching Excellence and Conference

Administrators, Austin, TX. (ED 377 897)

Moursund, D.G. (2001). “Highly Interactive Computing in Teaching and Learning.”

Learning and Leading with Technology v. 28 no 7

On Purpose Associates. (1998). About Learning - Theories - How do people learn?

Constructivism. New York: Funderstanding. Retrieved March 1, 2003, from:

http://www.funderstanding.com/learning_theory_how1.html

Papert, S. (1980). Mindstorms: children, computers and powerful ideas Brighton:

Harvester Press.

Piaget, J. (1952). The Origins of Intelligence in Young Children New York: IUP

Quality Education Data (QED). (1999-2000). Quality education data’s technology

purchasing forecast (5th ed.). Denver, CO: QED.

Relan, A., & Gillani, B.B. (1997). Web-Based Instruction and the Traditional Classroom:

Similarities and Differences. In B.H. Kahn (Ed.), Web-Based Instruction. (pp. 41-

46). Englewood Cliffs, NJ: Educational Technology Publications.

Ritchie, C and Hoffman, B. (2000). Using Instructional Design Principles To Amplify

Learning On The World Wide Web San Diego State University. Retrieved March

2, 2003, from: http://edweb.sdsu.edu/clrit/learningtree/DCD/WWWInstrdesign/

WWWInstrDesign.html

Rockart, John F. (1973). A Method for the Integrated use of Learning Resources in

Education: Journal of Higher Education, Vol. 44, Issue 4 (April, 1973) 280-298.

56

Schacter, J. (2000). The Impact of Education Technology on Student Achievement:

What the Most Current Research Has to Say. Milken Exchange on Education

Technology. Santa Monica, CA.

Sivin-Kachala, J. (2002). Final Survey Report: Online Courses and Other Types of

Online Learning for High School Students. Interactive Educational Systems

Design. New York, NY. Retrieved February, 20, 2003, from:

http://apexlearning.com/results/IESD_ExecSum_Web.pdf

Skinner, B. (1938). The Behavior of Organisms: an experimental analysis New York:

Appleton-Century-Crofts.

Stone, L. (1999). Multimedia Instruction Methods. The Journal of Economic Education.

V. 30 no3

Underwood, J. (1994). Introduction: Where are we now and where are we going? In

Underwood, J. (Ed.) Computer Based Learning London: David Fulton Publishers.

Walker, Elaine M. (1987). Understanding Minorities Students’ Mathematics Learning

Gains in Computer-Assisted Instruction: Journal of Negro Education, Vol. 56,

Issue 4 (Autumn, 1987) 557 – 569

Web-based Education Commission. (2000). The power of the Internet for learning:

Moving from promise to practice. Report of the Web-based Education

Commission to the President and the Congress of the United States, December,

2000.

Wenglinsky, H. (1998). Does it compute? The relationship between educational

technology and student achievement in mathematics. Educational Testing

57

Service Network. Retrieved January 19, 2003, from: http://www.ets.org/

research/pic/dic/preack.html.

Weston, Cynthia and Cranton, P. A. (1986). Selecting Instructional Strategies: Journal

of Higher Education, Vol. 57, Issue 3 (May-June, 1986) 259-288.

Wilson, A.M. (1992). "The INVEST Program: A Computer-Based System for Adult

Academic Upgrading. A Pilot Study." Research report, Cumberland Campus of

Nova Scotia Community College. (ED 377 896)

World Wide Web Consortium. Web Content Accessibility Guidelines 1.0 from the

(W3C). (1999). Retrieved March 11, 2003, from: http://www.w3.org/TR/1999/

WAI-WEBCONTENT-19990324/#Guidelines

58