digital course ware evaluation rubric - tan

5th National eLearning Conference

Taking IT to the Next Level: The Team Approach to eLearning Program Development

A Digital Courseware

Evaluation Rubric for CICT Content Development Projects

Maria Melizza D. TanHEA, CICT-HCDG

Project Coordinator, eSkwela

[email protected]

Commission on Information and Communications Technology

AGENDA

� CICT’s Content Development Strategy

• BECTA Quality Principles� Digital Courseware Evaluation Rubric:

the eSkwela Experience


HCDG’s Programs and Projects

� national ICT competency standards + certifications

� ICT4E: transformative role of ICT in teaching and learning

� ICT competency training for government personnel

� eSociety: digital inclusion + digital communities


constant clamor from the field

Comprehensive Approach

Best Practices

+ Standards

Content

ApplicationsInfrastructure

Training

Community Involvement

Coordination + Ownership +

Sustainability


Digital Content Development

“After the first decade of ICT4D, a major lesson learned by the

development assistance sector, particularly the UN agencies

and the World Bank, is that investments on content and

capability building should be significantly larger than

investments on infrastructure (Walsh 2006). According to some

estimates, for every dollar invested on ICT4D, ten cents should

go to infrastructure. Another ten cents should go to software.

Still another ten cents should go to training. But the remaining

70 cents should be spent on content development.”

− DepEd’s Strategic Framework on ICT4E under BESRA


Content Development Strategy

� Developing digital content that is appropriate, relevant, accessible, and meaningful to Filipinos is a priority of CICT.• Interactive, engaging, motivating

• High Quality

• Well-designed

• Well-produced

• Easily replicated at low cost for greater access

� Focus Areas:• Educational Multimedia Content

• Digital Cultural Content


Content Development Strategy

1. Mapping the content landscape

2. Developing a Content Development

Policy Framework

3. Building the necessary technical

architecture for comprehensive

national access

4. Training: development & use


Digital Content Devt phase 1

� Mapping activities

� Multi-media Learning Materials for Teachers (Pre & In)

� Computer-Based Training Modules (CBTs) for English Proficiency

� Web Competitions

� Content and Application Development for Rural Communities and Enterprises: Digitization of Replicable Research Outputs of Partner SUCs

� Jeepney ni JenJen –-Digitization of Tuklas Sining

� eHealth: Health information for emergencies website

� Indigenous People’s (IP) Knowledge

� skoool.ph

� eSkwela elearning modules

� Islam in the Philippines


QA: BECTA Principles

� British Educational Communications

and Technology Agency (BECTA)

Quality Principles for Digital Learning

Resources

1. Core Pedagogic Principles

2. Core Design Principles

http://partners.becta.org.uk/index.php?section=sa&catcode=_sa_cs_cf_03


BECTA Principles

� Core Pedagogic Principles

1. Match to the curriculum

2. Inclusion and access

3. Learner engagement, challenge, and motivation

4. Effective learning

5. Assessment to support learning

6. Robust summative assessment

7. Innovative approaches

8. Ease of use

� Core Design Principles

1. Digital learning resource design

2. Robustness and support

3. Human-computer interaction

4. Quality and Suitability of assets

5. Accessibility

6. Interoperability

7. Testing and verification

8. Effective communication


eSkwela

� to use ICTs in broadening access to

basic education by tapping into

appropriate ICT-supported models to

improve the quality of learning and

enhancing the quality of teaching in the

non-formal education setting

� target: out-of-school youth and adults


eSkwela

Materials Development

Stakeholders’

Training

Community

Support for

Program

Sustainability

Infrastructure

Deployment

eSkwelaeSkwela


eSkwela Content Development

� roots: Smokey Mountain

� conversion of existing ALS print modules

into digital modules

• Print modules: BALS + SEAMEO-INNOTECH• Curriculum Review done by BALS (2005)


eSkwelaeSkwelaelearning

modules (A&E)(beta)

BALS Session Guides & other

materials

eSkwelaLearning

Management System(alpha)

Learner outputs/ portfolios

+Teacher-learner interactions

+Teacher-teacher & learner-learner interactions

Involvement of Learning Community

+Livelihood Programs


Module Review for eSkwela

� assumptions & constraints:

• content of print modules already reviewed• HCDG-EDO-BALS combined effort: devt of version 1

• English as medium of instruction

• review process requirement• verbal discussions/agreements: problematic• search for existing evaluation form


Module Review Experience

� Rubric: a work-in-progress

� Briefing and discussion with reviewers/ raters• Retooling on ICT-enhanced instructional design

• i.e. Session Guides also need to be revised!• Clarifications on item definitions/ descriptions

• Targets: at least 3 per rubric item• Not summed up/averaged

• Clarifications on developer limitations and scope of work

� Clearer standards/ expectations for developers• Currently in the module enhancement stage (1st 20)

• Due to time constraints: “minimums” were set; better to have priority levels for detailed recommendations


Evaluation Rubric

� Assumptions:

• The learners are not impaired in any way (i.e. not

disabled or psychologically dysfunctional).

• The modules are always accessible to the learners.

� Acknowledgements:

• Some rubric definitions were taken from publicly-

available rubric-makers on Writing and Web Design,

located at Rubistar (http://rubistar.4teachers.org).

� Initial Review: Dr. Lolit Suplido-Westergaard


Evaluation Rubric: Content

1234

Module content

does not match

the set

curricular

standards.

Module content

is loosely

matched to set

curricular

standards.

Majority of the

module content is

matched to /

appropriate to set

curricular

standards but

may have one or

two elements that

are not

appropriate.

The entire module

is clearly matched

to /appropriate to

set curricular

standards.

Matched to /

Appropriate

for the

Curriculum



1234

The module

lacks a purpose

and theme.

The purpose

and theme of

the module is

somewhat

muddy or

vague.

The module has a

clearly stated

purpose and

theme, but may

have one or two

elements that do

not seem to be

related to it.

The module has

a well-stated

purpose and

theme that is

maintained

consistently

throughout the

site.

Focus



1234

There was no

clear or logical

organizational

structure.

Content is

logically

organized for

the most part.

Uses headings

or bulleted lists

to organize, but

the overall

organization of

topics appears

flawed.

Content is well

organized using

headings or

bulleted lists to

group related

material.

Subtopics

naturally

progress into

succeeding

ones.

Organization



1234

Content is

minimal OR

there are

several factual

errors.

Includes

essential

information

about the topic

but there are 1-

2 factual errors.

Includes

essential

knowledge

about the topic.

Subject

knowledge

appears to be

good – no

factual errors

detected.

Covers topic

in-depth with

details, extra

information,

and examples.

Subject

knowledge is

excellent – no

factual errors

detected.

Thoroughness

and Accuracy



1234

The author has

provided only the

minimum amount

of information and

has not

transformed the

information to

make it more

interesting to the

audience (e.g.,

has only provided

a list of links to the

content of others).

The author has

put lots of

information in

the module but

there is little

evidence that

the person tried

to present the

information in

an interesting

way.

The author

has tried to

make the

content of this

module

interesting to

the people for

whom it is

intended.

The author has

made an

exceptional

attempt to make

the content of

this module

interesting to the

people for whom

it is intended.

Interest



1234

There is only

one form of

assessment,

and it is

ineffective or

inappropriate to

the content.

Forms of

assessment

(formative and

summative) are

few, barely

effective or

inappropriate to

the content.

Forms of

assessment

(formative and

summative) are

limited, but

effective and

appropriate to

the content.

Forms of

assessment

(formative and

summative) are

varied, effective

and appropriate

to the content.

Quality of

Assessment



1234

Module is

lacking several

key elements

and has

inaccuracies

that make it a

poor elearning

module.

Module is

missing more

than two key

elements in its

use of ICT vis-à-

vis content to

enhance

learning. It

would make an

incomplete

elearning

module.

Module

generally makes

use of ICT

effectively vis-à-

vis content to

enhance

learning but is

lacking one or

two key

elements. It is

an adequate

elearning

module.

Module

consistently

makes use of

ICT effectively

vis-à-vis content

to enhance

learning. It is a

highly effective

elearning

module.

Effectiveness



1234

Very little or

no source

information

was collected.

Source information

collected for

graphics, facts, and

quotes, but not

documented in the

desired format.

Source information

collected for all

graphics, facts,

and quotes. Most

documented in the

desired format.

Source information

collected for all

graphics, facts, and

quotes. All

documented in the

desired format.

Sources



1234

Borrowed

materials are

not properly

documented

OR material

was published

without

obtaining

permission

from copyright

owner.

Fair use guidelines

are followed with

clear, easy-to-locate

and accurate

citations for most

borrowed material.

No material is

included from

sources that state

that publication is

not allowed unless

permission has

been obtained from

copyright owner.

Fair use guidelines

are followed with

clear, easy-to-

locate and

accurate citations

for almost all

borrowed material.

No material is

included from

sources that state

that publication is

not allowed unless

permission has

been obtained

from copyright

owner.

Fair use guidelines

are followed with

clear, easy-to-

locate and accurate

citations for all

borrowed material.

No material is

included from

sources that state

that publication is

not allowed unless

permission has

been obtained from

copyright owner.

Copyright


Evaluation Rubric: Design

1234

Use of font,

color,

graphics,

video,

animation, etc.

but these often

distract from

the

presentation

content.

Makes use of

font, color,

graphics,

video,

animation, etc.

but

occasionally

these detract

from the

presentation

content.

Makes good

use of font,

color,

graphics,

video,

animation, etc.

to enhance to

presentation.

Makes

excellent use

of font, color,

graphics,

video,

animation, etc.

to enhance the

presentation.

Aesthetics



1234

Multimedia

elements seem

to be used

randomly, are of

poor quality or

distract the

learner from the

content.

Multimedia

elements are

relevant and

appropriate to

the module, are

of good quality.

Multimedia

elements are

related and

appropriate to

the module, are

of good quality

and enhance the

learner's interest

or

understanding.

Multimedia

elements are

relevant and

appropriate to

the module, are

of high quality

and enhance the

learner's interest

or

understanding.

Multimedia

(Graphics,

Sound, Video,

etc.)



1234

Module does not

contain any form

of learner

interactivity nor

features for

immediate

feedback.

Module allows a

minimum degree

of interactivity by

manipulating

objects in

animations/

simulations,

models, data

sets, or online

assessment.

Module allows

an acceptable

degree of

interactivity by

manipulating

objects in

animations/

simulations,

models, data

sets, or online

assessment.

Module provides

immediate

feedback.

Module allows

learner a high

degree of

interactivity by

manipulating

objects in

animations/

simulations,

models, data

sets, or online

assessment.

Module provides

immediate

feedback.

Interactivity



1234

Less than

75%of the links

are clearly

labeled,

consistently

placed, work

properly. A user

typically feels

lost.

Most links

(75%-89%)

links are

clearly

labeled,

consistently

placed, and

work properly.

Some needed

links seem to

be missing. A

user

sometimes

gets lost.

Almost all (90%-

99%) links are

clearly labeled,

consistently

placed, and

work properly. A

user rarely

becomes lost.

All buttons and

links are clearly

labeled,

consistently

placed, and work

properly.

A user does not

become lost.Navigation



No Help or

FAQ function.

Help and/or

FAQ is hard to

find. Content is

hardly

understandabl

e or obviously

lacking.

Help and/ or

FAQ is

accessible,

understandable

and mostly

comprehensive

.

Help and/or

FAQ is easily

accessible,

understandable

for the users

and

comprehensive.

User Support

1234

http://balsontheweb.wikispaces.com

5th National eLearning Conference

Taking IT to the Next Level: The Team Approach to eLearning Program Development

Thank you.

[email protected]@ncc.gov.ph

digital course ware evaluation rubric - tan

Documents