development of competency-based web learning material and ... · structure of cblm is composed of...

Development of Competency-Based

Web Learning Material and Effect

Evaluation of Self-Directed Learning

Aptitudes on Learning Achievements

Chi-Cheng Chang*National Taipei University of Technology, Taiwan ROC

This study aims to develop and evaluate competency-based web learning material (CBWLM) for

the college practicum Microprocessor Laboratory. After using the CBWLM for 8 weeks, this study

investigates CBWL’s learning effects and self-directed learning aptitudes (SDLAs) as well as

exploring the influence of SDLA on learning effects based on the sample of 38 students. The results

of this study indicate that over half of the students achieve the mastery level after using CBWLM.

SDLAs of the mid-CBWL and post-CBWL do not influence learning effects.

Research Background

Competency-based learning (CBL), a non-linear, systematic, and self-learning

process, allows learners to individually study each unit of the learning material

depending on their own competency. Furthermore, it allows learners to perform

repeated learning, remedial learning until they master each unit’s knowledge and skill

in order to achieve required competency standards. CBL possesses the nature of

systematic process, mastery learning, emphasizing individual differences and self-

directed learning. Such a learning method stressing competency standards is

specifically suitable for technical learning and training. Web learning is an inevitable

trend in the future. However, it has a weakness—there are limitations to the

presentation and learning methods for technical subjects. If we apply the concept of

CBL to web learning, both will complement each other and improve web technical

learning effects. As indicated by Lin and Liao (2000), the concept of competency-

based learning material (CBLM) can enhance the effect of web learning materials. To

sum up the argument mentioned above, it’s quite clear that competency-based web

learning (CBWL) or competency-based web learning material (CBWLM) ought to

*Institute of Technological and Vocational Education, National Taipei University of Technology,No. 1, Section 3, Chung-Hsiao E. Road, Taipei, Taiwan 106. Email: [email protected]

Interactive Learning Environments

Vol. 14, No. 3, December 2006, pp. 265 – 286

ISSN 1049-4820 (print)/ISSN 1744-5191 (online)/06/030265-22 � 2006 Taylor & Francis

DOI: 10.1080/10494820600954112

have a certain level of demand and feasibility. As there are few studies combining the

practices regarding CBL and web learning, CBWL or CBWLM becomes a topic

worthy of studying.

Westera and Sloep (1998) developed so-called ‘‘The Virtual Company’’, a

distributed distance learning environment built upon the notions of CBL and

constructivist learning, but the system seems not to bear supporting mechanisms of

CBL. Lin (2002) developed the web learning material of the competency-based

‘‘Digital Logic’’ course in the vocational senior high school by taking an ordinary web

learning system as a platform where CBLM was placed. However, there is no control

mechanism for CBL processes between each unit of the teaching material, which

seems not to conform to the nature of competency-based individual self-learning. Ho,

Hsu, and Tao (1999) referred to CBL in their web learning material, but the material

does not appear to match the concept and spirit of CBL. Most studies in the world

simply propose the idea of applying the competency-based concept in web learning

initially; no CBWLM’s substantial output is produced. Kirschner (2000), for

instance, discussed what instructional designers (ID) need to do when designing

electronic CBL environments. Woelk (2002) delivered a conception of a compe-

tency-based e-learning just-in-time system; Ritzen and Kosters (2002) also proposed

an idea of a competency-based web-based portfolio system. Kirschner (2000) further

argued that no full-fledged ID models are yet available to provide clear prescriptions

for how to design electronic CBL environments. Therefore, design and development

of electronic CBL environments may become highly cost-ineffective. For this reason,

there are some discussible issues: What on earth should a CBWLM consist of? And,

what are its learning effects? Theses study results can provide references for the

design and implementation of CBWLM, and will help enhance the effects of CBWL.

Elshout-Mohr, Oostdam, Snoek, and Dietze (2000) stated, CBL provides learners

with opportunities to regulate and manage their own learning. Wielenga (2000)

argued, competency-oriented learning should offer students an environment in which

they are given the responsibility to monitor their own learning processes. These

arguments imply self-directed learning is the characteristic of CBL. Hanna, Dudka,

and Runlee (2000) argued that students are required to possess self-directed learning

characteristics for successful web learning. This implies that web learning accentuates

active and self-directed learning. Richards, Dooley, and Lindner (2004) conceived

that web learning courses draw on the ability of learners to be self-directed, therefore,

web courses should focus on the self-directed nature of learners. Learners of

e-learning must experience self-directed learning (Ghaoui & Janvier, 2004; Ryan,

Scott, Freean, & Patel, 2000). Conrad and Donaldson (2004) argued that a learner

on the web must establish comfort with a higher level of self-direction than in a

traditional classroom. They further asserted that the learner would quit the online

course in frustration if this comfort level was not reached.

Self-directed learning aptitude (SDLA), derived from Guglielmino (1977), is a

learning behaviour allowing students to continue learning on their own initiative. The

SDLA consists of six factor aspects: Effective Learning, Fondness for Learning,

Learning Motivation, Active Learning, Independent Learning, and Creative

266 C.-C. Chang

Learning. Because web learning is becoming more prevalent in colleges, an issue thus

arises. What is the SDLA generated by college students through CBL on the web?

Moreover, Hanna et al. (2000) argued that assignments, projects, tests, and activities

might help students consider the important self-direction elements of online learning.

It means that assignments and activities may be the incitements to self-directed

learning behaviours. Thus, the other issue arising is that whether or not SDLA

reflects on the outcomes of assignments, projects, or tests. Namely, does SDLA

influence any learning effect? Are learning effects different among students with

varied SDLA? They are all valuable issues for further discussion.

Research Purposes

Based on the referred research background and motivation, this research aims to

develop a CBWLM for the practicum Microprocessor Laboratory of the Electronic

Department in college as well as to evaluate CBWL’s learning effects and SDLAs,

and to probe the influence of SDLA on learning effects with an empirical study.

Detailed research objectives associated with their corresponding research questions

and statistical methods are shown in Table 1.

The learning effect referred to in this study constitutes two aspects, the self-

perceived learning effect and the outcome of an achievement assessment, derived

Table 1. Research objectives, research questions and statistical method

Research objectives Research questions Statistical method

1. To evaluate learning

effects of CBWLM.

1.1. What are the self-perceived

learning effects after students

have used CBWLM?

Means

1.2. What are the outcomes of

achievement assessment after

students have used CBWLM?

2. To evaluate SDLAs for

different stages of CBWL.

2.1. What is the SDLA of students

before CBWL?

Means


during CBWL?


after CBWL?

3. To explore influences of

SDLA on learning effects

for CBWL.

3.1. Is there any statistically

significant difference in learning

effects among the students with

various SDLAs during CBWL?

One-way MANOVA

(dependent variables are

self-perceived learning

effect and outcome of

3.2. Is there any statistically

significant difference in learning

effects among the students with

various SDLAs after CBWL?

achievement assessment)

Competency-Based Web Learning Material 267

from the first and second level based on Kirkpatrick’s (1996) training effect

evaluation (Noe, 1999). The self-perceived learning effect captures average scores of

students’ comments through the questionnaire of a Likert 5-point rating scale; the

score of achievement assessment includes the laboratory assessment of four CBWLM

units (representing 50%) plus paper-pencil test (representing the other 50%). SDLA

consists of various factor aspects such as Effective Learning, Fondness for Learning,

Learning Motivation, and Active Learning, etc.

Literature Review

Competency-Based Learning Materials

Competency-based learning material is based upon a competency profile derived

from the competency analysis. As Westera and Sloep (1998) have stated, a CBL

environment should be based on authentic profiles that cover a set of

competencies. Woelk (2002) mentioned a competency gap analysis, identifying

what competencies the workers lack to effectively carry out their jobs, that is, based

on what the workers need to know and what they already know. In respect of

knowledge, skill, and attitude, CBLM stipulates the learning objectives and

organizes the units of the learning material with the substantial behaviour

objectives. Each unit’s learning steps and competency requirements should be

described in detail, and behaviour objectives of each competency should be

specified as well. These behaviour objectives not only explain the measurable

learning outcomes but also represent the standards to be achieved (Weng, 1996).

According to the guide of competency-based training material delivered by the

Division of Employee Training, Department of Labor (2005), Taiwan, the

structure of CBLM is composed of cover pages, learning guidelines, introduction,

definitions, learning objectives, learning activities, and testing. According to Tseng

(1999), this study prepares and induces mandatory contents and goals of CBLM as

shown in Table 2.

Furthermore, according to Woelk and Lefrere (2002) learner profile and

competency ontology may be included in the CBL or performance-based learning

environment. Learner profile refers to a description of the skills and of an individual.

The learner profile for a worker or learner may include, for instance, preferences,

experiences, and assessment of the worker’s competencies. Competency ontology is a

description including the required competencies for specified job tasks and

relationships among these competencies (Woelk & Lefrere, 2002, pp. 92, 95). It is

primarily to capture the competencies that a worker or learner must possess to solve

problems in his or her working processes. Competency ontology described by Woelk

and Lefrere may be regarded as so-called competency profile included in the

CBWLM developed by this study. Woelk and Lefrere further indicated that the

competency ontology might provide reasoning about competencies. Generally

speaking, the learner profile must be able to make reference to competency ontology,

learning objects, and working processes of employees.

268 C.-C. Chang

Self-Directed Learning Aptitude

Self-directed learning means learning something proactively, independently, and

patiently; being responsible to learn; learning which is a challenge; a self-training ability;

high curiosity; intense impetus to learning; self-assurance; enabling a fundamental

learning skill; scheduling time for learning; and planning the integral learning and

enjoying learning toward an objective (Teng, 1995). Self-directed learning may help

adapt to changing environment and enhance creativity (Ramsey & Couch, 1994).

Self-directed learning readiness (SDLR) questionnaire, developed by Guglielmino

(1977), measures learner or worker readiness for self-directed learning. The SDLR

questionnaire, completed several times through the Delphi approach, has been

verified to possess sufficient validity and reliability. It is based on a Likert 5-point

scale covering 58 items that yield a total score for identifying self-directed readiness.

After Guglielmino’s SDLR questionnaire was presented, there were some scholars

applying it to measuring SDLR or self-directed learning aptitude (SDLA) for various

students, and there were some scholars implementing studies regarding the reliability

and validity of the SDLR questionnaire (Long, 1987; Reynolds, 1986).

Table 2. Contents of CBLM and their goals

Contents Goals

1. Code and title of

learning material

1. The code and topic are disclosed on the cover page of each unit of the

learning material.

2. Introduction 2. Specify highlights and importance of the unit of the learning material to

inspire learning interests of students.

3. Learning objectives 3. List the fundamental occupation capability that students should provide

after completing the unit in a behaviour objective approach.

4. Pretest 4. Test students in their achievement level before learning; students

reaching the level are allowed to discard this unit of the learning

material.

5. Learning tasks 5. Based on the behaviour objectives, test results prior to learning, and

depending on the capability degree of students, teacher assigns students

the selective tasks such as learning activities, learning contents, and

assignments, etc.

6. Instructional media 6. To improve learning effects, it’s recommended to use teaching tools or

educational media that are favourable for students learning.

7. Learning activities 7. The learning activity can be divided into two parts: knowledge and skill,

and, occasionally, some tests.

8. References 8. Students may refer to relevant data if they find problems or intend to

have an in-depth discussion or further learning contents during the

learning process.

9. Posttest 9. Allow students to undertake the test after completing the unit of the

learning material for evaluation of learning effects.


Eight factor aspects were included in the SDLR questionnaire: openness to

learning opportunities, self-concept as an effective learner, initiative and indepen-

dence in learning, informed acceptance of responsibility for one’s own learning, love

of learning, creativity, positive orientation to the future, ability to use basic study skill

and problem-solving skills (Bonham, 1989; Guglielmino, 1977; Hung, 1995). These

eight factor aspects underlying the SDLR ascertain learner readiness for self-directed

learning and have been identified and further applied in many later studies (McCune

& Guglielmino, 1992; Siaw, 2002).

Guglielmino’s SDLR questionnaire has been translated into several languages around

the world (Guglielmino & Guglielmino, 1991), for instance, translated into Chinese by

Teng (1995) and revised by some experts in Taiwan. Consequently the number of the

questions was reduced from 58 to 55 and the original eight factor aspects were slashed

down to six aspects based on the consideration for cultural differences. The four factor

aspects of the Chinese-based SDLR, able to measure the degree of self-directed learning

aptitude are (1) Effective Learning, Fondness for Learning, (2) Learning Motivation,

Active Learning, (3) Independent Learning, and (4) Creative Learning.

Development of Competency-Based Web Learning Material

The CBWLM contents are based on the ‘‘competency profile’’ (required

competencies for a technician of microprocessor control, including 16 responsi-

bilities), announced by the Division of Employee Training, Department of Labor

(2005) in Taiwan, plus the proposal delivered by the instructor for the curriculum

progress. Finally, we decided to take the Task 1 (to understand basic architecture of

MCS-51 Microprocessor) and Task 4 (to familiarize with control and application of

MCS-51 Instruction) in Task 6 (Microprocessor Control) of the preceding 16

responsibilities as the CBWLM competency standard. Table 3 shows the competency

profile of a technician of microprocessor control (the sixth responsibility).

Based on the referred two competency standards (for Task 1 and Task 2), two

behaviour objectives in the CBWLM were formulated. They are, respectively, to

familiarize with attribute of output ports of MCS-51 and application of LED, and to

familiarize with attribute of output ports of MCS-51 and application of seven-section

displayer. Four learning units of the CBWLM were designed based on the two behaviour

objectives. The first behaviour covers three units of the CBWLM: ‘‘Practice of Single-

Lamp Right-Transferal’’, ‘‘Practice of Advertising Lamp’’, and ‘‘Practice of Thunder-

bolt Lamp’’. The second behaviour represents only one unit of the learning material,

‘‘Seven-Section Displayer of One Order’’. These four units of the learning material are

mainly designed to teach students the ‘‘Application of MCS-51 Output Port’’.

Contents and their descriptions of one unit of CBWLM based on the practicum

Microprocessor Laboratory are as follows:

. Material title: Subject of the unit.

. Competency profile: A set of competency standards which can be achieved after

completing the CBWLM.

270 C.-C. Chang

. Learning objectives: Competency levels which should be reached after completing

the unit.

. Learning guideline: Teaching students how to use CBWLM.

. Content description: Describing contents of the unit.

. Pretest: Learning the content of the unit only if passing the test.

. Test feedback: Listing wrongly answered questions and correct answers of the

pretest.

. Remedial learning (1): Students failing in the pretest need to take remedial

learning (1). After having learned the content of remedial learning (1), they

should attend the pretest again, and learn the laboratory knowledge only if they

pass the test.

. Laboratory knowledge: Laboratory knowledge associated with skill

operations.

. Laboratory demonstration: Teaching students skill operations.

. Posttest: Entering into the next unit if passing the test.

. Test feedback: Listing wrongly answered questions and correct answers of the

posttest.

. Remedial learning (2): Students failing in the posttest need to take remedial

learning (2). After having learned the content of remedial learning (2), they

Table 3. Competency profile of a technician of microprocessor control

Competencies of each task for the 6th responsibility (microprocessor control)

*Task 1. Tounderstand basic

architecture ofMCS-51Microprocessor

Task 2. To understand

basic architecture of

EM-78

Microprocessor

Task 3. To understand

basic architecture of

PIC Microprocessor

*Task 4. Tofamiliarize with

control andapplication ofMCS-51 instruction

Task 5. To be able to

manipulate basic I/O

of PIC


manipulate

interruption control

of PIC


manipulate

timepiece control of

PIC


manipulate

electrical circuit

control of PIC


manipulate

timepiece control of

MCS-51


manipulate electrical

circuit of MCS-51


manipulate MCS-51

LED


manipulate PIC


use PIC to produce

product


manipulate EM78

signal ship


manipulate basic I/O

control of EM78


manipulate

interruption control

of EM78


manipulate

timepiece of EM78


manipulate electrical

circuit of EM78


use EM78 to

produce product

Note: *Learning contents of Task 1 and Task 4 are included in the CBWLM developed by this

study.


should attend the posttest again, and enter into the next unit only if they pass the

test.

. Learning log: Recording the students’ online behaviours of using CBWLM.

Methods

Subjects

The subject of this study consists of 38 undergraduate students in the practicum

Microprocessor Laboratory, a sophomore course of the Department of Electronic

Engineering at a college located in northern Taiwan. This course, elective for one

semester, is a practicum that requires more learning by doing. After implementation

of the CBWLM for 8 weeks, we evaluate students’ learning effect through the

‘‘questionnaire for self-perceived learning effect’’ survey. We investigate the SDLA

before, during, and after the use of the CBWLM and further explore influences of

SDLA on learning effects.

After completing the course, these students are supposed to achieve some technical

standards. This is really correspondent to the rationale of CBL and thus becomes a

content of CBWL. Since the students are majoring in electronic or computer fields

they have a high level of computer use and access to the Internet.

Process

The study lasts 8 weeks. Before the CBWL course begins, the students complete the

pretest of the ‘‘SDLA questionnaire’’. After the CBWL course ends, the students

complete the posttest of the ‘‘SDLA questionnaire’’ and the ‘‘questionnaire for self-

perceived learning effects’’. The instructor first teaches students how to use CBWLM

before this study. The process of the CBWLM implemented in classroom teaching is

that the instructor, after part of the theory teaching, asks the students to access the

CBWLM website and practice writing in MCS-51 Microprocessor language;

the students, following the required procedure of CBWL, systematically learn the

CBWLM unit by unit. The procedure of CBWL followed by students is based on the

procedure of CBL proposed by Kuo (2000). In the procedure of CBWL, shown

in Figure 1, the students are to:

. Select a unit to self-learn and read learning objectives and laboratory contents.

. Take the pretest.

. Take remedial learning (1) if failing to pass the test.

. Learn laboratory knowledge and watch demonstration of the unit if passing the

pretest.

. Take the posttest after completing the learning of laboratory knowledge and

demonstration.

. Take remedial learning (2) until he or her succeeds if failing in the posttest.

. Enter into the next learning unit.

272 C.-C. Chang

The relationships among the research variables in correspondence to the research

questions proposed above are listed as follows and shown in Figure 2.

. Learning effects, a dependent variable, is composed of two aspects, namely, the

self-perceived learning effect and the outcome of the achievement assessment.

. Self-directed learning aptitude (SDLA), an independent variable, is divided into

three stages (before, during, and after CBWL). The SDLA consists of four factor

aspects: Effective Learning, Fondness for Learning, Learning Motivation, and

Active Learning.

Descriptive statistics is used to analyse the self-perceived learning effect of CBWL

and investigate SDLAs for three stages of CBWL. One-way MANOVA is used is to

determine if there is any statistically significant difference among learning effects for

various SDLAs during CBWL (and also after CBWL).

Figure 1. Procedure of competency-based web learning


Fig

ure

2.

Rel

atio

nsh

ips

amo

ng

rese

arch

vari

able

s

274 C.-C. Chang

Methods of Data Gathering

The study uses the questionnaire survey in the following stages to collect relevant

data.

Scoring of achievement assessment. According to Elshout-Mohr et al. (2000),

assessment of CBL cannot only be focused on the use of standard tests and fixed

criteria. There should also be an emphasis on the fairness of an assessment that allows

students to demonstrate relevant competencies at the end of a longer learning period.

Westera and Sloep (1998) claimed that students’ performances should be assessed

within practical, life-like contexts. Thus, grading of CBL achievement might integrate

tests, work, the learning process, and progress. In this study, the score of achievement

assessment is the summation of the laboratory assessment (representing 50%) and a

paper-pencil test (representing 50%); both are graded by the teacher. The laboratory

assessment consists of five checking aspects in terms of accuracy of laboratory

outcomes, amplitude of laboratory outcomes, creativity of laboratory outcomes,

correction of laboratory procedure, and attitude toward doing the laboratory. The

total score of the laboratory assessment is 100 points, that is, 20 points for each

checking aspect. The paper-pencil test includes 25 questions in fill-in-the-blank

format based on a scale of 100 points. The questions on the paper-pencil test cover

knowledge in four CBWLM units. Reliabilities of the laboratory assessment and

paper-pencil test are acceptable since they are all retrieved from assessment bases in

the teaching guide for instructors.

Evaluation of self-perceived learning effects. On finishing CBWL, the study uses the

questionnaire for self-perceived learning effects to detect perceptive learning effects

for students after using CBWLM. Key contents in the questionnaire are used to

assess whether the functions of CBWLM will enhance students’ ‘‘MCS-51

Microprocessor’’ knowledge and practical ability. The questionnaire contains 10

questions of a Likert 5-point rating scale and one opening question. The data

acquired is deemed a self-perceived learning effect.

Examination of SDLAs before, during, and after CBWL. The Chinese version of the

SDLA questionnaire, with some minor amendments by Teng (1995), is used in this

study to measure the extent to which individuals perceive themselves as processing

abilities and attitudes frequently associated with self-directed aptitudes in CBWL

experiences. A week prior to CBWL, during CBWL, and a week after CBWL we

implement the pretest, midtest and posttest of the SDLA questionnaire for the

students, so as to evaluate students’ SDLAs before, during, and after using CBWLM.

Validity and Reliability of Instruments

Validity and reliability analysis of SDLA questionnaire. Since the SDLA questionnaire,

modified by Teng (1995), is intended to be used for the measurement of web


learning students, the ulterior questions need to be moderately modified in

accordance with web learning nature. We modified the presentation style of the

questions in the SDLA questionnaire and sent them to a web-learning scholar, a CBL

expert, and an instructor for review. Since the purpose of the initial SDLA

questionnaire was more applicable to workers and is now used for college students in

this study, some terms used in the questions were modified. Since the Chinese-based

SDLA questionnaire, initially derived from Guglielmino (1977) and later modified by

Teng, has been revised by this study through expert review to achieve so-called expert

validity, it is really not necessary to redo the validity analysis by a pilot test.

After removing 17 questions in two factor aspects for inadequate Cronbach’s avalues (see the next section for details), 37 items in the SDLA questionnaire were

analysed by using a Likert 5-point rating scale that yielded a total score for self-

directed learning aptitude. As shown on the left-hand side in Table 4, five levels of

the SDLR constructed by Guglielmino and Guglielmino (1991) can be identified

according to their overall or mean scores. Based on the same rate of overall score or

mean scores, five levels and their ranges of scores for the SDLA in this study are

shown on the right-hand side in Table 4.

The study uses Cronbach’s a coefficient to test the reliability of the SDLA

questionnaire. Table 5 listed the internal reliability of the six factor aspects, where acoefficients lie between 0.6 and 0.9. It’s obvious that, besides Creative Learning and

Independent Learning, a values of the remaining factor aspects are all higher than

0.7. As a result, those two factor aspects are removed.

Validity and reliability analysis of learning effect questionnaire. We randomly select 30

students from the 38 students participating in the study to undertake the pilot test of

the questionnaire for self-perceived learning effects. We have received 28 effective

questionnaires in total. These students have properly checked the answers to the

questions. We have also modified ambiguous questions. Originally this study would

plan to use the method of factor analysis for building the validity. This method,

however, is discarded, because a small number of samples do not lead to genuine

validity (to build the validity via factor analysis usually requires over a hundred

samples).

Table 4. Five levels of the SDLA for students based on overall and mean scores

Guglielmino’s study This study

Scale by

overall scores

Scale by

mean scores

Term of

SDLA level

Scale by

overall scores

Scale by

mean scores

Term of

SDLA level

58 – 176 1 – 3.04 low 37 – 112 1 – 3.03 very low

177 – 201 3.05 – 3.47 below average 113 – 128 3.04 – 3.46 low

202 – 226 3.48 – 3.90 average 129 – 144 3.47 – 3.90 average

227 – 251 3.91 – 4.33 above average 145 – 160 3.91 – 4.32 high

252 – 290 4.34 – 5 high 161 – 185 4.33 – 5 very high

276 C.-C. Chang

Tab

le5

.R

elia

bilit

yo

fp

ilo

tte

sto

fS

DL

Aq

ues

tio

nn

aire

(n¼

28

)

Rel

iab

ilit

y

Eff

ecti

ve

lear

nin

g

(11

qu

esti

on

s)

Fo

nd

nes

sfo

r

lear

nin

g

(9q

ues

tio

ns)

Lea

rnin

g

mo

tiva

tio

n

(7q

ues

tio

ns)

Act

ive

lear

nin

g

(10

qu

esti

on

s)

Ind

epen

den

t

lear

nin

g

(13

qu

esti

on

s)

Cre

ativ

e

lear

nin

g

(4q

ues

tio

ns)

Cro

nb

ach

’sa

0.9

10

.73

0.7

20

.89

0.6

70

.63


For the pilot test, we gain a high value of Cronbach’s a coefficient that is equal to

0.92. It implies a sufficient reliability for the learning effect questionnaire.

Results and Discussions

Perceptive Learning Effects

After finishing competency-based web learning, we use the questionnaire for self-

perceived learning effects to implement measurement for the students and receive 38

effective questionnaires, no void questionnaires. The results are shown in Table 6.

In Table 6, the overall students’ learning effect (3.55 points) is classified as a

competent level. This indicates that the students are mostly satisfied with CBWLM.

In terms of the points, each question scores over 3 points. Question 5, ‘‘benefits of

feedbacks of the pretest and posttest’’, scores the highest (3.71 points), and Question

8, ‘‘online discussion is helpful to resolve curriculum problems’’ scores the lowest

(3.18 points), which indicates the highest effect of the ‘‘feedback of test’’ while the

effect of online discussion requires enhancement.

Table 6. Self-perceived learning effects (n¼ 38)

Self-perceived learning effects Mean SD

1. Does the mechanism, ‘‘pretest’’ and ‘‘remedial learning (1)’’, enable you

to consolidate your prerequisite knowledge before learning the CBWLM?

3.66 0.67

2. Does the mechanism, ‘‘posttest’’ and ‘‘remedial learning (2)’’, enable you

to master the laboratory knowledge of the learning material?

3.55 0.76

3. Does the visualized simulation of ‘‘Laboratory Knowledge’’ allow you to

have clear understanding of the executive outcomes of MCS-51

Microprocessor?

3.68 0.66

4. Does the streaming media material of ‘‘Laboratory Demonstration’’ allow

you to have more understanding of the practical product of MCS-51

Microprocessor?

3.66 0.71

5. Does the feedback from the pretest and posttest allow you to have clear

understanding of laboratory knowledge?

3.71 0.98

6. Is the procedure of CBWL [Pretest ! Remedial learning (1) ! Posttest

! Remedial learning (2)] helpful to you in learning MCS-51

Microprocessor?

3.66 0.75

7. Generally, are contents of CBWLM able to enhance your knowledge and

capability of MCS-51 Microprocessor?

3.66 0.67

8. Is online discussion forum with classmates helpful to you while you are

encountering problems in learning MCS-51 Microprocessor?

3.18 0.87

9. Are website-linking resources helpful to you in completing the

MCS-51 Microprocessor project?

3.3 0.87

10. Are online-handout resources helpful to you in completing the

MCS-51 Microprocessor project?

3.42 0.86

Overall 3.55 0.54

Note: The scoring is based on a 5-point rating scale with a total of 50 points.

278 C.-C. Chang

Scores of Achievement Assessment

After totalling the scores of the laboratory assessment (50%) and paper-pencil

test (50%), we obtain the scores of achievement assessment. In Figure 3, it is

clear that all of the students’ scores of achievement pass 60 points. If we take 80

points (100 points based) as the mastery learning standard, over a half of the

students (20) achieve certain competency standards (over 80 points), but there

are less than half of the students (18) failing to reach mastery standards (below

80 points). Finally, the students achieve mastery standards (a mean of 81.1

points).

Self-Directed Learning Aptitude

At a time before, during (a month later), and a week after CBWL, the study

implements measurements for the students by using the SDLA questionnaire and

receives 38 questionnaires without any void questionnaires.

Pretest of SDLA questionnaire. In Table 7, before the CBWL intervention, the overall

students’ SDLA (mean¼ 3.50 points) is classified as an average level. Factor aspect of

Fondness for Learning, classified as an average level, scores the highest (3.75 points),

and Effective Learning, classified as a low level, scores the lowest (3.27 points). As

regards the deviation between each factor aspect and its mean value, Fondness for

Learning has the largest deviation (standard deviation¼ 0.75) while Effective

Learning has the smallest one (standard deviation¼ 0.52).

Figure 3. Scores of achievement assessment (n¼ 38, M¼ 81.1, SD¼ 7.05)


Midtest of SDLA questionnaire. In Table 8, during the CBWL intervention, the overall

students’ SDLA (mean¼ 3.53) is classified as an average level. Factor aspect of

Fondness for Learning, classified as an average level, scores the highest (mean¼ 3.86

points) while Effective Learning, classified as a low level, scores the lowest

(mean¼ 3.27 points). As regards the deviation between each factor aspect and its

mean, Fondness for Learning has the largest deviation (standard deviation¼ 0.75)

while Effective Learning has the smallest one (standard deviation¼ 0.63).

Posttest of SDLA questionnaire. In Table 9, after the CBWL intervention, the overall

students’ SDLA (mean¼ 3.47 points) is classified as an average level. Factor aspect of

Fondness for Learning, classified as an average level, scores the highest (mean¼ 3.57

points) while Active Learning, classified as a low level, scores the lowest (mean¼ 3.27

points). As regards the deviation between each factor aspect and its mean, Fondness

for Learning has the largest deviation (standard deviation¼ 0.65) while Active

Learning has the smallest one (standard deviation¼ 0.52).

Concerning the SDLAs measured before, during, and after CBWL, the midtest of

SDLA has the highest result, the pretest of SDLA follows, and the posttest of SDLA

Table 7. Pretest of SDLA questionnaire (n¼ 38)

Factor aspects Minimum Maximum Mean SD Level of SDLA

Fondness for Learning (9 questions) 21 45 3.75 0.75 Average

Effective Learning (11 questions) 24 48 3.27 0.52 Low

Learning Motivation (7 questions) 17 34 3.62 0.69 Average

Active Learning (10 questions) 23 45 3.43 0.53 Low

Overall (37 questions) 85 160 3.50 0.54 Average

Note: The scoring is based on a 5-point rating scale with a total of 185 points.

The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1�m� 3.03

(very low), 3.04�m� 3.46 (low), 3.47�m� 3.90 (average), 3.91�m� 4.32 (high), and

4.33�m� (very high).

Table 8. Midtest of SDLA questionnaire (n¼ 38)



Effective Learning (11 questions) 23 53 3.27 0.63 Low


Active Learning (10 questions) 20 49 3.49 0.69 Average


Note: The scoring is based on a 5-point rating scale with a total of 185 points.The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1�m� 3.03


4.33�m�(very high).

280 C.-C. Chang

generates the lowest outcome by small differences. If comparing by level norm, there

is no changing level among the three tests of SDLA.

Influences of SDLAs of Mid-CBWL and Post-CBWL on Learning Effects

Based on the quartile, we divide ‘‘SDLA of mid-CBWL’’ into ‘‘low’’ (first 25%),

‘‘middle’’ (50%, middle), and ‘‘high’’ group (last 25%), and perform one-way

MANOVA together with learning effects (self-perceived learning effect and outcome

of achievement assessment). From Table 10 it is clear that there is no significant

difference in learning effects among the students of the three groups of ‘‘SDLA

during CBWL’’, showing the students’ ‘‘SDLA of mid-CBWL’’ will not influence

their learning effects.

From Table 11 it is clear that there is no significant difference in learning effects

among the students of the three groups of ‘‘SDLA after CBWL’’, showing the

students’ ‘‘SDLA of post-CBWL’’ will not influence their learning effects.

Conclusions and Implications

After CBWL has been used in classroom teaching and after-class review for the past 8

weeks, each student’s score of the achievement assessment is higher than 60 points

(100 points based). If we take 80 points as the mastery level, over a half of the

students (20 students) obtain a certain competency standard, less than half of them

(18 students) fail to pass the mastery level. For this reason, CBWL is generally

capable of enabling the students to achieve the mastery learning level and may further

identify ‘‘CBL is mastery learning’’ (Kang, 1997; Yang, 1998).

Comparison among SDLAs of pre-CBWL, mid-CBWL, and post-CBWL, SDLA

of mid-CBWL scores the highest, SDLA of pre-CBWL follows, and SDLA of

post-CBWL scores the lowest by slight differences. As regards factor aspects of

SDLA, Fondness for Learning is the highest for the three stages of CBWL, Effective

Learning is the lowest for pre-CBWL and mid-CBWL, and Active Learning is the

Table 9. Posttest of SDLA questionnaire (n¼ 38)



Effective Learning (11 questions) 22 50 3.48 0.63 Average


Active Learning (10 questions) 23 44 3.40 0.52 Low


Note: The scoring is based on a 5-point rating scale with a total of 185 points.The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1�m� 3.03


4.33�m� (very high).


Tab

le1

0.

On

e-w

ayM

AN

OV

Ao

nle

arn

ing

effe

cts

for

vari

ou

sgro

up

so

f‘‘

SD

LA

of

mid

-CB

WL

’’(n¼

38

)

So

urc

eso

f

vari

atio

nD

egre

e

Wilk’s

�va

lue

(sig

nifi

can

tle

vel)

Lea

rnin

g

effe

cts

Deg

ree

Su

mo

f

squ

are

(SS

)

Mea

no

f

SS

(MS

)F

valu

e

Sig

nifi

can

t

leve

l

Bet

wee

ngro

up

s(S

DL

A)

10

.95

(0.4

1)

Sel

f-p

erce

ived

lear

nin

gef

fect

22

3.8

61

1.9

30

.38

0.6

9

Ach

ieve

men

tas

sess

men

t2

2.6

11

.30

50

.03

0.9

8

Wit

hin

gro

up

s(r

esid

ual

)3

5S

elf-

per

ceiv

edle

arn

ing

effe

ct3

51

09

1.6

53

1.1

9

Ach

ieve

men

tas

sess

men

t3

51

83

4.2

85

2.4

1

To

tal

37

Sel

f-p

erce

ived

lear

nin

gef

fect

37

11

15

.51

Ach

ieve

men

tas

sess

men

t3

71

83

6.8

9

282 C.-C. Chang

Tab

le1

1.

On

e-w

ayM

AN

OV

Ao

nle

arn

ing

effe

cts

for

vari

ou

sgro

up

so

f‘‘

SD

LA

of

po

st-C

BW

L’’

(n¼

38

)

So

urc

eso

fva

riat

ion

Deg

ree

Wilk’s

�va

lue

(sig

nifi

can

tle

vel)

Lea

rnin

gef

fect

sD

egre

e

Su

mo

f

squ

are

(SS

)

Mea

no

f

SS

(MS

)F

valu

e

Sig

nifi

can

t

leve

l

Bet

wee

ngro

up

s(S

DL

A)

10

.93

(0.3

8)

Sel

f-p

erce

ived

lear

nin

gef

fect

26

3.0

71

1.9

31

.05

0.3

6

Ach

ieve

men

tas

sess

men

t2

26

.83

1.3

05

0.2

60

.77

Wit

hin

gro

up

s(r

esid

ual

)3

5S

elf-

per

ceiv

edle

arn

ing

effe

ct3

51

05

2.4

43

0.0

7

Ach

ieve

men

tas

sess

men

t3

51

81

0.0

75

1.7

1

To

tal

37

Sel

f-p

erce

ived

lear

nin

gef

fect

37

11

15

.51

Ach

ieve

men

tas

sess

men

t3

71

83

6.8

9


lowest for post-CBWL. After using further inferential statistics, we find that CBWL

will enhance the factor, Fondness for Learning, in SDLA.

As indicated in the study result of Chung (2001), the students’ learning effects were

correlative with SDLA after web learning. But we find, in this study, that there is no

significant difference in learning effects among the students of various groups of

SDLA during CBWL, likewise having the same result for ‘‘after CBWL’’. These

imply that SDLAs of the mid-CBWL and post-CBWL do not influence learning

effects. The discrepancy between this study result and that of Chung needs a further

examination.

Subject to the instructor’s limitations, this study is targeted at a class with

few students, instead of two classes (or groups), for the contrastive experimental

study. It is recommended to perform the comparative study based on experimental

and controlled groups in the future, and to eliminate some variables that may

affect the study potentially, for a better understanding of the difference between

learning effects of CBWL and traditional CBL, CBWL and traditional classroom

learning.

References

Bonham, L. A. (1989). Self-directed learning: Emerging theory & practice. In H. Long (Ed.),

Self-directed orientation toward learning: A learning style? (pp. 55 – 68). Norman, OK:

Oklahoma Research Center for Continuing Professional and Higher Education, University

of Oklahoma.

Chung, Y. (2001). Influences of self-directed learning aptitude on asynchronous distance learning.

Unpublished master’s thesis, National Kaoshiung Normal University, Taiwan.

Conrad, R., & Donaldson, J. (2004). Engaging the online learner: Activities and resources for creative

instruction. San Francisco: Jossey-Bass.

Division of Employee and Training. (2005). Competency-based training. Retrieved November 30,

2005, from http://cbt.evta.gov.tw/

Elshout-Mohr, M., Oostdam, R., Snoek, M., & Dietze, A. (2000). Assessment in a competency-

oriented dynamic curriculum. Retrieved November 30, 2005, from http://www.efa.nl/

publicaties/publiexplo/english/index.html

Ghaoui, C., & Janvier, W. (2004). Smart proflexlearn: An intuitive approach to virtual learning

environment. In C. Ghaoui (Ed.), E-education applications: Human factors and innovative

approach (pp. 66 – 83). Hershey, PA: Information Science Publishing.

Guglielmino, L. (1977). Development of self-directed learning readiness scale. Dissertation Abstracts

International, 38, 6467A.

Guglielmino, L., & Guglielmino, P. (1991). Learning preference assessment facilitator guide. King of

Prussia, PA: Organizational Design and Development, Inc.

Hanna, D. E., Dudka, M. G., & Runlee, S. C. (2000). 147 practical tips for teaching online groups:

Essentials of web-based education. Madison, WI: Atwood.

Ho, C., Hsu, P., & Tao, C. (1999). Distance teaching for vocational distance and Internet training

material. Employee and Training, 17(4), 11 – 23.

Hung, S. (1995). Relationships between self-directed learning aptitude and learning achievement.

Unpublished master’s thesis, National Taiwan Normal University, Taipei, Taiwan.

Kang, T. (1997). Theory and curriculum development of competency-based vocational training.

Employee and Training, 15(6), 3 – 9.

284 C.-C. Chang

Kirkpatrick, D. (1996). Evaluation. In R. Craig (Ed.), The ASTD training and development handbook.

New York: McGraw-Hill.

Kirschner, P. A. (2000). Design strategies for competency based learning. Heerlen, The Netherlands:

Open University of The Netherlands.

Kuo, C. (2000). Procedure and management of CBT. Journal of Human Resources Development, 4,

28 – 33.

Lin, C., & Liao, M. (2000). Competency-based training of distance instruction. In Proceedings

of Conference on Application of Internet (pp. 392 – 399). Tainan, Taiwan: Cheng-Kung

University.

Lin, F. (2002). Planning and construction of competency-based instructional website: A case of ‘‘Digital

Logic’’ course in vocational technical high school. Unpublished master’s thesis, National Chang-

Hua University of Education, Taiwan.

Long, H. (1987). Item analysis of Guglielmino’s self-directed learning readiness scale. International

Journal of Lifelong Education, 6, 331 – 336.

McCune, S., & Guglielmino, L. (1992, January). Factor analytic study of the self-directed learning

readiness scale. Paper presented at the Sixth International Symposium on Adult Self-Directed

Learning, West Palm Beach, FL.

Noe, R. (1999). Employee training and development. New York: McGraw-Hill.

Ramsey, V., & Couch, P. (1994). Beyond self-directed learning: A partnership model of teaching

and learning. Journal of Management Education, 18(2), 15 – 20.

Reynolds, M. (1986). The directedness and motivational orientations of adult part-time students at a

community college. Unpublished doctoral dissertation, Syracuse University, NY.

Richards, L., Dooley, K., & Lindner, J. (2004). Online course design principles. In C. Howard,

K. Schenk, & R. Discenza (Eds.), Distance learning and university effectiveness: Changing educational

paradigms for online learning (pp. 99 – 118). Hershey, PA: Information Science Publishing.

Ritzen, M., & Kosters, J. (2002). Combining different aims in a portfolio system: A web-based

portfolio and the various ways in which it can serve the student. In C. Montgomerie & J. Viteli

(Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommu-

nications (pp. 1552 – 1557). Norfolk, VA: AACE.

Ryan, S., Scott, B., Freean, H., & Patel, D. (2000). The virtual university: The internet and resource-

based learning. London: Kogan Page.

Siaw, S. C. (2002). Fostering self-directed learning readiness by way of PBL intervention in

business education. In Online Proceedings of 2nd Asia-Pacific Conference on Problem-

Based Learning, Singapore. Retrieved November 30, 2005, from http://www.tp.edu.sg/

pblconference/

Teng, Y. (1995). Adult instruction and self-directed learning. Taipei, Taiwan: Wu-Nan Publication.

Ting, T. (1996). Relationships between self-directed learning readiness, computer attitude and computer

achievement for students of open university. Unpublished master’s thesis, National Chung-Cheng

University, Taiwan.

Tseng, K. (1999). Development of instructional material. In W. Chiang (Ed.), Introduction of

vocational technical education. Taipei, Taiwan: NTNU Publication.

Weng, S. (1996). Development of vocational competency-based training material: A case of industrial

electronic curriculum. Unpublished doctoral dissertation, National Taiwan Normal University,

Taipei, Taiwan.

Westera, W., & Sloep, P. (1998). The virtual company: Toward a self-directed, competence-based

learning environment in distance education. Educational Technology, 38(1), 32 – 36.

Wielenga, D. (2000). Proving competency: Integrative assessment and web-based portfolio system

in a dynamic curriculum. Retrieved November 30, 2005, from http://www.efa.nl/publicaties/

ass-eng.html

Woelk, D. (2002). E-learning, semantic web services and competency ontologies. In P. Barker &

S. Rebelsky (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and

Telecommunications (pp. 2077 – 2078). Norfolk, VA: AACE.


Woelk, D., & Lefrere, P. (2002). Technology for performance-based lifelong learning. In

R. Kinshuk et al. (Eds.), Proceedings of International Conference on Computers in Education

(pp. 92 – 96). Los, Alamitos: IEEE Computer Society.

Yang, C. (1998). Theory and practice of vocational and technical education. Taipei, Taiwan: Fu-Hsing

Publication.

286 C.-C. Chang

development of competency-based web learning material and ... · structure of cblm is composed of...

Documents