summary of two evaluation studies

Amal AL-Balushi (66755)Khadeejah AL-Shidhani (61344)

Cheryl Bullock and John Ory, American Journal of

Evaluation 2000; 21; 315

The research first reviews the literature and describes the methods used in a myriad of evaluation studies in instructional technology.

The review result on the following three conclusions:1. Multiple data collection methods have been used to collect

information from multiple sources;2. Various evaluation models or approaches have been followed;

and3. There are a common set of problems and concerns about past

evaluations of technology-enhanced instruction.

Then it provides a concrete example of evaluating a campus-wide learning technology effort (the SCALE Project) at the University of Illinois at Urbana-Champaign. This evaluation spanned three years and used multiple methods.

Objectives:• Evaluating the impact of ALN

(Asynchronous Learning Networks) on professors and students

• Understanding the economic implications of ALN.

Approach: To answer the evaluation questions, the research

adopted the mixed-method approach. Following a document review, both qualitative

data and quantitative data were collected. Quantitative data took the form of survey results,

records of use, achievement gain scores, and an extensive cost-benefit analysis (which is reported in a subsequent article)

Qualitative data primarily involved interviews

Student surveys Post-course instructor surveys Computer support personnel surveys: Student and TA (teaching assistant)

group interviews Instructor interviews: Gains in student achievement: Course conferences:

Cheryl Bullock and John Ory, American Journal of Evaluation 2000; 21; 315

Linda Schwartz, Sharon Clark, Mary Cossarin, & Jim Rudolph

Evaluating wikis is very different from evaluating

vendor supplied or proprietary programs which

have a fixed set of features. All wikis can potentially adopt all of the features

found in other wikis, simply by accessing and

customizing the source code. This report describes the basic characteristics and

features of wikis and attempts to provide an easy

approach to selecting wiki features.

There is a comparison between WikiWiki Web (the

first wiki) and Seed Wiki (a WYSIWYG wiki).

Feature categories included:• Source code.

• Wiki management.

• Page formatting.

• Access control.

• Communications.

• Support.

• Advanced features.

Editable by major browsers (IE, Netscape).

WYSIWYG editing.

HTML support.

Text editing (italics, font size, color).

Image insertion.

Hyperlink insertion.

Tables.

Lists (numbered, bulleted, hierarchical).

Media insertion (streaming audio/video).

Search.

Spell-check.

Emoticons.

Blogging.

Polling.

Calendar.

RSS.

Link checking.

Drawing tools.

Equation editor.

Synchronous text messaging.

Features rarely incorporated in wikis are: equation

editor, synchronous text messaging, link

checking, and drawing whiteboard (some do have

drawing tools). If coursework requires these

capabilities s, an integrated collaborative software

program may be a better choice.

http://cde.athabascau.ca/softeval/reports/R270311.pdf