THE EVALUATION AND REVIEW OF EDUCATIONAL SOFTWARE: A

CASE STUDY OF ENGLISH FORM FOUR TEACHING COURSEWARE

HASUBAZILAH BINTI HASAN

A thesis submitted in fulfillment of the requirements for the award of the degree of

Bachelor of Science and Education (TESL)

Faculty of Education

University of Technology Malaysia

April, 2010

ii

iii

� Dedicated to �

My beloved parents;

My supportive sisters and brothers;

My wonderful nieces and nephews;

Friends and all the amazing people around me.

iv

ACKNOWLEDGEMENT

First and foremost, I would like to extend my deepest gratitude to Allah S.W.T

for His blessings in making it possible for me to complete this thesis. Without His help,

it would have been impossible for me to endure this challenge.

I would like to express my most sincere gratitude and appreciation to my

supervisor , Puan Khairuzilah Khalil for her guidance, advice and endless support

throughout the process of completing this study.

My sincere thanks to Dr Azizah Rejab and Dr Adlina Samad for their

encouraging comments in making this study a success.

Finally I would like to express my love and gratitude to may family especially

my beloved parents, Hasan b Umar and Ramlah bt Abd Wahab and my sisters, Hasnida,

Hasnita, Hasuzila and Hasuhamizah. My deepest love and appreciation to my fellow

TESLian classmates, thank you so much. It has been a wonderful 4-year journey.

v

ABSTRACT

The use of courseware as one of the teaching materials has become popular recently due

to the rapid development of computer technology in Malaysia. The ministry of education

has also distributed a set of courseware that can help to teach literature better. This study

investigates the characteristics of the courseware used to teach literature (short stories),

with the focus given on its presentation scheme, and the restrictions faced by teachers in

using the courseware in teaching and learning process. In addition to that, this study also

tried to look from students’ point of view on the usage of this courseware. The research

was conducted on 40 students of SMK Kompleks Sultan Abu Bakar, Johor and 5

English teachers who are familiar with the content of the courseware. Respondents were

required to answer the questionnaire distributed to them and for teachers; they were also

required to evaluate the presentation scheme of the courseware. The data obtained in this

study were analyzed using qualitative research methodology. The result shows that the

students have very positive attitudes towards the use of the courseware in classroom

(mean = 2.82), the presentation scheme is appropriate (mean = 2.85) and few significant

problems faced by teachers in using it (mean = 2.77).Implications of the findings and

suggestions for future research are further explained.

vi

ABSTRAK

Penggunaan koswer sebagai salah satu bahan mengajar agak popular

kebelakangan ini selari dengan perkembangan teknologi komputer yang giat

membangun di Malaysia. Kementerian Pelajaran juga telah mengedarkan set-set koswer

yang boleh membantu mengajar komponen sastera Bahasa Inggeris dengan lebih baik.

Kajian ini mengenal pasti ciri-ciri yang terdapat pada koswer yang diedarkan ke sekolah

dengan skema persembahannya sebagai focus utama, dan juga halangan yang dihadapi

oleh para guru dalam menggunakan koswer ini. Selain itu, kajian ini juga mengenalpasti

pandangan pelajar terhadap penggunaannya didalam kelas semasa proses pembelajaran.

Kajian telah dijalankan terhadap 40 orang pelajar dari SMK Kompleks Sultan Abu

Bakar, Johor dan 5 orang guru bahasa Inggeris yang telah didedahkan dengan isi

kandungan koswer tersebut. Data yang diperoleh dari kajian ini telah diproses secara

kualitatif. Keputusan menunjukkan para pelajar mempunyai tanggapan yang positif

terhadap penggunaan koswer ini didalam kelas (min= 2.82), skema persembahan

keseluruhannya adalah memuaskan (min= 2.85) dan beberapa halangan yang signifikan

dihadapi oleh para guru didalam menggunakan koswer ini( min= 2.77). Implikasi dari

dapatan dan juga cadangan untuk kajian pada masa akan datang juga turut disertakan.

vii

TABLE OF CONTENTS

CHAPTER TITLE PAGE

Supervisor’s Approval

Title Page

Student’s Declaration ii

Dedication iii

Acknowledgement iv

Abstract v

Abstrak vi

Table of Contents vii

List of Tables xi

List of Figures xii

List of Appendices xiv

1 INTRODUCTION

1.1 Introduction 1

1.2 Background of the Study 1

1.3 Statement of Problem 5

1.4 Purpose of the study 7

1.5 Objectives 8

1.6 Research Questions 8

viii

1.7 Significance of the study 9

1.8 Scope of the study 10

1.9 Definition of Term 10

2 LITERATURE REVIEW

2.1 Introduction 12

2.2 Computer in Education 12

2.2.1 Education: why computers? 13

2.2.2 The myth of the educational computers 14

2.2.3 Technology & Cognition: Tools for mind 15

2.3 The introduction of CALL 16

2.3.1 The Development of CALL 16

2.3.2 CALL courseware in the classroom 18

2.4 Educational Courseware Standards 20

2.4.1 The evaluation of educational courseware 20

2.4.2 Techniques & approaches in evaluation 21

2.4.2.1 The evaluation framework 21

2.4.2.2 The Heuristic approach 22

2.4.2.3 The Checklist approach 23

2.5 Previous Studies 24

ix

3 RESEARCH METHODOLOGY

3.1 Introduction 27

3.2 Research Design 27

3.3 Subject of study 28

3.4 Research Instrument 29

3.4.1 Courseware Checklist 29

3.4.2 Questionnaire sets 30

3.5 Pilot Study 30

3.6 Research procedure 31

3.6.1 Administration of courseware checklist 32

3.6.2 Administration of Questionnaire 32

3.7 Data analysis 33

3.7.1 Analysis of courseware checklist 33

3.7.2 analysis of Questionnaire 34

4 FINDINGS AND DISCUSSION

4.1 Introduction 35

4.2 Characteristics of form four selected shorts

stories courseware 37

4.3 Teachers’ views on courseware’s presentation

scheme 39

4.4 The restriction faced by the teachers 49

4.4.1 Restrictions from teachers 49

4.4.2 Restrictions from the courseware 52

4.5 Students’ views on the courseware 57

x

5 CONCLUSION

5.0 Introduction 65

5.1 Summary of the study 65

5.2 Pedagogical implications of the study 67

5.3 Limitation of the study 68

5.4 Suggestion for future research 69

5.5. Conclusion 70

REFERENCES 71

APPENDICES A-D 75-86

xi

LIST OF TABLES

TABLE NUM. TOPIC PAGE

4.1.1 Indicators 36

4.3.1 Respondents’ demographic info 39

4.5.0 Respondents’ demographic info 57

xii

LIST OF FIGURES

FIGURE NUM. TOPIC PAGE

4.3.1 Software characteristic Percentage of views on 40

4.3.2 Percentage of views on the graphic 41

4.3.3 Percentage of views on the font 42

4.3.4 Percentage of views on text 43

4.3.5 Percentage of views on the navigation system 44

4.3.6 Percentage of views on audio 45

4.3.7 Percentage of views on other elements 46

4.3.9 Overall mean value for courseware presentation

scheme

47

4.4.1.1(a) Percentage of restrictions from teachers 50

4.4.1.1(b) Percentage of restrictions from teachers 51

4.4.1.2(a) Percentage of restrictions from the courseware 52

4.4.1.2(b) Percentage of restrictions from the courseware 53

4.4.1.2(c) Percentage of restrictions from the courseware 54

4.4.1.3 Percentage of restrictions from others 55

4.4.1.4 Overall mean value for restrictions faced by teacher 56

4.5.1 Mean score for multimedia elements 58

4.5.2 (a) Percentage of the courseware for learning/exam

purposes

59

4.5.2 (a) Percentage of the courseware for learning/exam 60

xiii

purposes

4.5.2 (c) Percentage of the courseware for learning/exam

purposes

61

4.5.3 Percentage of the courseware layout 62

4.5.4 Overall mean value of students’ perspective 63

xiv

LIST OF APPENDICES

APPENDIX TITLE PAGE

A Teacher’s Questionnaire 76

B Students’ Questionnaire 82

C Evaluation Checklist 85

D EPRD’s Approval Letter 86

1

CHAPTER 1

INTRODUCTION

1.1 Introduction

This chapter provides an insight to the background, purpose and significance of the

study on the evaluation and review of educational software: a case study of English form

four teaching courseware. Definitions of terms are also specified to further assist in

understanding this project paper.

1.2 Background of the study.

English education in Malaysia has evolved tremendously starting on the opening

of the first school in Penang during the British colonial era. Literature has been regarded

as one of the core subjects in education and that scenario remains unchanged ever since

2

for years, even after the end of the colonialism. However in 1970’s, a drastic shift has

taken place where English was taught only as a second language. The significant

transformation was primarily due to the implementation of the Education Enactment Bill in

1971 (Idrus, 2009). Starting on the day on, English is no longer the medium of instruction

in schools and subjects that used to be taught using English were changed to Malay. This

change has resulted in a sharp drop in the number of students opting to do literature as

an examination subject.

In 1990, the subject has been integrated within the English language program that

aimed to promote English proficiency and the study of literature in general, beginning at

the lower secondary level (Subramaniam, 2003). The program, called The Class Reader

has been launched in order to expand reading skills and to establish elements of

literature into language teaching. However it failed to fulfill the aim set on its launching

as it is not a compulsory part of the English language curriculum. The integration of

literary component within English language lesson in 2001 has brought back hope for

literature where starting on that year, the literature has been put on the syllabus and

students will be tested on them during PMR and SPM. This integrated component within

English language lesson is aimed at enhancing students' language proficiency, and at the

same time geared for the purpose of generating the aesthetic part of the language that is

personal response from student (Hwang & Embi, 2007).

Since that day, the government has never looked back and keeps on improving on

the policies by preparing and distributing the courseware to be used in teaching

literature. According to Subramaniam (2003) literature should be taught for all these 16

purposes:

• Pleasure

• Fostering good reading habits

• Offering challenges hence achievements

• Providing genuine communication of ideas

3

• Offering linguistics model

• Developing comprehension and motivation

• Balancing receptive and productive skills

• Providing reach and diverse exposure to language

• Improving concentration

• Encouraging reflection

• Allowing contact with native speakers’ world

• Improving vocabulary, syntax and structure

• Allowing learning from what is not taught

• Increasing exposure to language in a limited time.

• Leading to language acquisition.

CALL courseware that has been distributed to the schools nationwide is

undoubtedly interactive, complete with notes and exercises and more important, serving

a platform for the students to have fun while studying. Based on the list, it can be

concluded that the use of CALL courseware developed by the government for the

teaching literature purposes is the best choice in aiding the teaching since it fulfill almost

half of the criteria. The use of courseware in classroom is very beneficial since it can be

used as a powerful tool in grabbing students’ attention. For the short story part, it comes

in the form of animation and that is believed to help a lot in understanding the whole

story and at the same time very enjoyable for the students.

In spite of all the advantages bring by the courseware usage in teaching

literature; most of the teachers still do not use the technology even though all the

facilities have been prepared for them. Most of them still feel comfortable to teach using

the traditional way just like what they had been taught in the past. Recent studies have

reported that students were seen to be passive and were unable to respond critically and

literature lessons were often too teacher-centered and thus, labeling teachers to be dull

4

and less creative (Gurnam Kaur, 2003; Siti Norliana, 2003; Suriya Kumar, 2004) This is

because “substantial and effective professional development is rare, and many teachers

naturally gravitate towards the more familiar methods they remember from their own

experience as students” (Sparks, 1998). Hence, the coursewares provided by the

government are not fully utilized to the maximum and most of the school just keep them

rather than use them. This is such a big waste since a vast amount of money have been

invested in producing the CDs

However, with the sudden change in education policies where ETeMS is no

longer be used starting from 2012, the syllabus for English language subject is also

changed, including the literature part. As a kick start, only form four and form one

students will be using the new syllabus launched so all the previous courseware

provided, including the samples used in this study will also soon be obsolete. Looking

from the bright side of it, the change of the syllabus also means that the government will

come up with a new set of courseware to replace the older version. Any adaptation and

improvement can be done by studying the previous usage of the courseware, like the

strengths of them and also their weaknesses. Thus, this is the right time to evaluate the

CDs so the next set of them will be more attractive and effective to be used in the

classroom.

Software evaluation can be done using any methods as utilized by researchers in

the previous study. It ranged from the simplest one, which is the common checklist

method as improved by Elissavet and Economides (2003) to a different, better way of

evaluation as proposed by Hubbard (1987) through his ‘framework approach’

Does not matter what ever approach used by the evaluator, most of the

assessment contains these elements : the hardware needed to run the program, the

presence and quality of documentation and printed support materials, the topic area and

5

content of the program, its ease of use and reliability of operation, and the use of color,

graphics and sound in the program ( Bayram & Nous, 2004).What count is that by

evaluating the courseware, the teacher can decide whether the software’s objectives are

in line with the objectives already set for that classroom.

1.3 Statement of problem

Education trends nowadays have been shaped majorly by today’s rapid

technological advancement, particularly through multimedia-based learning

(Muthukumar, 2004). As a developing country, Malaysia also has step forward in

education field with the integration of computer in teaching and learning. As a part of

strategy in bridging the digital gap between students of urban areas with those who

studying in remote area, which has been the main concern for the government lately, the

use of technology in the form of educational courseware has become the most potential

way of executing it.

Not only they have been distributed as there was a change in education policy

where science subjects were required to be taught in English, and the implementation of

literature component in English language classroom, the government has came up with

this strategy to make sure that all these subjects will be well taught by the teachers.

These series of educational CALL courseware made specifically not only to ease

teachers’ burden in preparing the materials for the entire topic stated in the syllabus, but

also play a role as a very effective tool in teaching and learning. As stated by Price et al

(1996), computer is regarded as a powerful tool to decrease workload. Quoting Salwani

et al (2008) “technological innovations such as the development of an interactive,

comprehensive and multidimensional educational courseware as an effective learning

tool to assist educators in teaching and learning process would be an advantage”

6

Focusing on the English courseware that also provides literature component in

each CD, it is found that only some of English teachers use this courseware in teaching

simply because the school is not well equipped with the appropriate facilities such as

LCD projectors and computers, and some other teachers just find it useless in increasing

students’ interest in the classroom, while others claimed that they need to sacrifice a lot

of time just to set them up. It should not be a problem at this moment because as

compared to few years back, the cost of multimedia computers now worth every penny

if we were to look at its usefulness (Lomik & Punik, 1999).

Using courseware in the classroom can be frustrated sometimes. This is due to

the fact that it is impossible to find a perfect educational courseware that can be utilized

to all level of students. Besides that, not all educational courseware are developed based

on the current needs of both teachers and students. Hence, it is suggested that teachers

“carry out a full evaluation to reveal whether the developers have tackled a subject

according to its educational objectives” (Sahari et all, 2009).Evaluation can be made

according to teachers’ preference, because there are selections of method that can be

chosen by the teachers.

According to Rivera (1992), many students feel that educational courseware

“slow, repetitive, very similar and boring” although computers serve as facilities that

“can provide lessons that are fast-paced, interactive, friendly and multimedia-based”.

This might be caused by thousands of reason, but as first impression counts, the

presentation scheme of the courseware may have some influence towards this

phenomenon. It is because “content and design are the two most important elements of

good software” (Diptinto & Turner, 1995; Small & Grabowski, 1992).

Usually, the major goal of assessing instructional software is to find out “if it can

be used effectively in a particular classroom” (Reiser & Kegelmann, 1994). In order to

7

achieve the goal, teachers can adopt any of the assessment approaches available.

Evaluation checklist is the most popular one since it requires less time to be completed

as opposed to other approaches. The most important thing here is that the aim of doing

the evaluation can be achieved.

1.4 Purpose of the study

The purpose of the study is to evaluate the characteristics of the educational

courseware used for teaching form four literature in the classroom. In addition to that,

this study also tried to look not only on its’ presentation scheme but as teaching involves

both students and teachers, thus their views on the usage are also crucial to be look at.

This series of courseware will be actually obsolete starting from 2012 due to sudden

change in national education policy, but the checklist used in this study can still be used

to evaluate other educational software too. Besides that, the government will be surely

come up with a new version of the courseware, in line with the new curriculum

specification launched. Thus, any adjustment can be made to the new version of the

courseware to make sure that they will help the teachers in grabbing students’ attention

during the class and eventually make learning in classroom fun and attractive.

The tasks of evaluating educational environments as well as finding and

comparing the evaluations are difficult for teachers. However it still needs to be done

because teachers usually seek educational software they feel most likely to produce

positive learning outcomes (Hawley et al., 1997).

8

1.5 Objectives

This research was conducted to fulfill the objectives followed:

i. To evaluate the characteristics of form four selected short stories literature

courseware.

ii. To identify the teachers’ view on the presentation schemes of form 4 short

stories literature courseware towards teachers’ usage in literature classroom

iii. To identify the restrictions faced by teachers in using the form 4 selected

short stories literature courseware in the classroom.

iv. To investigates the students’ views on the form four selected short stories

literature courseware.

1.6 Research Questions

The research is guided by the following four research questions, which are:

i. What are the characteristics of form 4 selected short stories literature

courseware?

ii. What are the teachers’ views on the presentation scheme of the form 4

selected short stories literature courseware?

iii. What are the restrictions faced by the teacher in using the form four selected

short stories literature courseware in the classroom?

iv. What are the students’ views on the form 4 selected short stories literature

courseware?

9

1.7 Significance Of The Study

The main purpose of this study is to evaluate the characteristics of educational

CALL software and to investigate whether its presentation scheme influences the usage

of this courseware in the classroom. Besides that, this study also tries to look into the

limitations faced by the teachers in integrating the courseware in their teaching. Thus it

is hoped that the findings from this study will benefit the government in making any

amendment needed to the new sets of CDs, especially for English subject as the new

syllabus has been launched this year.

Hence, the new version will be way better than the previous version, which is the

one that has been used as a subject of this study. It is also hoped that it can be used as

reference for the developer of educational CALL courseware on the important elements

required in a courseware and how this technology can be tuned so that the goal of

developing it can be successfully achieved.

1.8 Scope of the Study

This study is on the issue of the evaluation of educational CALL courseware for

teaching English, focusing on users’ views on its presentation scheme, the limitations of

it usage as well as the limitations, strengths and weaknesses to be integrated in the

classroom. As the government has distributed various sets of courseware covering

English and science subjects, the set of form four teaching courseware has been selected.

To make it easier to be analyzed, the focus then has been narrowed down only to the

literature (short stories) part, particularly on its presentation scheme.

10

This study involved the participation of 40 students of SMK Kompleks Sultan

Abu Bakar , Gelang Patah, Johor Bahru with different level of proficiency and 2 English

teachers from the same school. For the courseware evaluation’s purposes, 3 more

teachers from different schools took part to evaluate the courseware. The data collected

were then analyzed based on the results from the questionnaires distributed to the

targeted respondents earlier.

1.9 Definition of Term

The following terms have been defined in relation to the study that will be carried

out so that there will not be any misleading preconceptions

1.9.1 CALL courseware

An abbreviation of Computer aided language learning courseware. In this study,

the courseware is referring to the set of courseware distributed by the government to

help teachers in school to teach English form 4, including the literature component

better. There are 6 CDs altogether and each CD contains one or two poem and short

stories.

11

1.9.2 Novel Paradigm

A situated approach to predictive evaluation. This paradigm generates context

specific evaluation issues by considering the interactions between the three principal

factors associated with the development and use of educational software — the teacher,

the designer and the student(s).