Module 2.1 Design Document Penny Reed /Heidi Weber

Topic: Earth and Space Science, Grade 2.

I. Analysis Phase

A. Background Information

The focus of this Web-based Learning System(WBLS) is to expand student

knowledge about the phases of the moon during science class. It is designed for

second grade students with a variety of learning styles, cognitive abilities, socio-

economic backgrounds, and average computer skills. The objective of this WBLS

is to provide students an opportunity to view current photos of the moon from any

computer at any time. The learning objectives include:

• Describe how the moon appears each day

• Identify the five main phases of the moon

• Predict the next moon phase

• Demonstrate the correct order of the phases of the moon

B. Problem Analysis

Students need to learn the material to have a better understanding how the solar

system works and to understand why they may see the moon appear differently

each day. Due to weather conditions, earlier bedtimes of second graders, and time

of sunset, a WBLS is beneficial for ‘observing’ and recording phases of the moon.

Websites allow students to view actual and current phases of the moon.

The gap between actual student learning and optimal learning is that these

students have come to second grade with a first grade science background. The

actual learning is that student’s prior knowledge is that the moon moves across

the sky and that the moon looks different at different times. The optimal learning

is that students will be able to describe how the moon appears different each day,

identify the five main phases of the moon, predict the next moon phase, and

demonstrate the correct order of the phases of the moon.

By using this WBLS students will be able to observe the moon’s phases anytime,

anywhere despite weather conditions, or other obstacles in observing the moon.

Students will also be able to participate in an on-line discussion responding to

instructor/peer posted questions and peer responses. The WBLS will be

supported through teacher-led direct instruction, class discussion, and various

hands-on activities. A WBLS is appropriate to solve this gap because it is a

highly motivational teaching strategy that incorporates credible websites, that are

accessible through public school Internet connections.

C. Instructional Situation Analysis

• Learning Goal and Outcome Level

Given websites, personal observation chart, and other supporting information, the

students will be able to describe how the moon appears each day, identify the five

main phases of the moon, predict the next moon phase, and demonstrate the

correct order of the moon phases.

Table 1 – Categories of Learning Descriptions

Category of

Learning

Learned

Capability

Verb

Explanation Example

Knowledge View Recall learned material,

exploring

Observe at the moon on a

nightly basis

Comprehension Describe Grasp the meaning of

new concepts

Describe which part of the

moon is lit (using first

quarter, half, third quarter,

full moon, new moon)

Illustrate Compare the illuminated

shape of the moon from one

night to the next by

completing a drawing

Application Demonstrate Use information in a

new concrete situation

Students will be able to

accurately name the four

main phases of the moon on

an observation chart (new,

full, ¼, ½, ¾)

Predict Students accurately predict

the shape of the illuminated

portion of the moon for the

next night

Synthesize Create Organize information

in the correct order

Creating a mini-book about

the moon, putting the phases

in the correct order

Evaluation Justify Judge value and

purpose with relevant

criteria

Student justifies the order of

his/her completed booklet

Attitudes Complete Assume responsibility

to complete the project

Complete daily drawing

either at home or at school

Motor Skills Fine motor

skills

Able to perform skills

on an individual basis

Create a mini-book, cutting,

pasting, coloring, fold

• Learning Context

The purpose of this analysis is to analyze the environment and the elements that

are included in the environment.

The WBLS will be used in an elementary school setting. The WBLS can be

accessed either in individual classroom settings, the school library computer lab,

or the school computer lab (20 desktop Dell computers). The table below shows

the elements for the WBLS to be a successful learning event for students.

Main Elements Aspects

Organizational

Infrastructure

Computers are Dell desktops, located either

in individual classrooms, the school library,

or the school computer lab. All computers

are connected to the Internet, using

Microsoft’s Internet Explorer. The WBLS

is owned by the creators, but can be

accessed by students, and managed by the

staff (teachers) as it is housed on the

school’s server.

Allocation and competencies

of personnel

Instructor is currently a member of the

District Technology Committee, is

attending a local university to obtain a

Master’s of Instructional Technology, and

has an elementary certification. The district

technical support staff is encouraging and

supportive when new applications of

technology are implemented. The school

administrative team is eager to showcase the

integration of technology in the classroom.

Learner location and

technology

Students are located in the same physical

location as the instructor. The district is

located in a suburban district in northeast

Ohio. The technology requirements for this

WBLS include a computer with Internet

connection.

• Learner Background

In this section of the analysis phase, the learners, or users, of the WBLS are

described. An in-depth analysis of the learner is conducted so that the WBLS is

relevant and interesting for the learner.

Main Element in

Learner Analysis

Aspects

General

Characteristics • A fairly even number of boys and girls

• Class size 20-25 students

• Second graders ranging in age from is 7-8 years old

• Have computer, keyboarding, and Internet

experience.

• Comparably diverse socio-economic backgrounds

• At least half read on grade level, others range from

one to three grade levels above or below grade

level

• Learning styles and abilities are diverse

Motivation • In the beginning of the project, motivation is high.

Students enjoy using the computer, matching the

phases of the moon to the description

• As the project progresses, students may lose

interest, therefore it will be important to add a new

dimension periodically to re-engage students.

• Students are interested in the environment and the

world around them

Prior knowledge,

Communication

skills,

Technology Skills

• Students know that the moon looks different at

various times throughout the month

• Students have a basic knowledge of how to

communicate ideas effectively using adverbs,

adjectives and verbs correctly

• Students have a basic knowledge of the computer.

Abilities and

Disabilities • Abilities and disabilities will be taken into

consideration are accounted for and WBLS can be

accommodated according to a student’s needs.

Other learner

characteristics • This WBLS takes into consideration the three main

learning styles – visual, auditory, and kinesthetic

• Educational enrichment needs can be incorporated

as needed

• For learners who may need concepts repeated,

presented in a different way, or are absent from the

classroom, the WBLS can meet these needs

Learning Content

The purpose of this section is to analyze the content the learner is to learn. This

includes the sequence of the presentation, what skills and subskills are needed by

the learner in order to complete the project.

------------------------------------------------------------------------------------------------------------

Phases of the Moon

1.0 Movement of the

moon in relation to

the earth and sun

(background

information)

1.1 Show United

Streaming video,

read books to

provide basic

knowledge

2.0 Observation

Calendar

2.1 Explain how to

complete the observation

calendar, illustrations, and

labeling the phase

2.2 As a class, view a

website showing the current

phase of the moon and

complete the first drawing

and label

3.0 Communication

3.1.1 Classroom discussion

3.1.2 On-line discussion

forum

Recognition that the

moon moves across

the sky

Recognize that the

moon looks different

at different times

Task-Objective-Assessment Item Blueprint (TOAB)

Learning Task Item and Number Objective Outcome Assessment Item

Step 1.0 F2F Interaction 1.1 Demonstrate movement of

the moon in relation to the earth

and sun (background

information)

1.2 Show United Streaming

video, read books to provide

basic knowledge

Defined Concept

Intellectual Skill

Verbal Information

Step 2.0 Observation Calendar 2.1 Explain how to complete the

observation calendar,

illustrations and labeling the

phase

2.2 View websites complete first

drawing of the moon phase

Concrete Concept

Motor Skill

Define Concepts

Step 3.0 Communication

3.1 Classroom Discussion

3.2 On-line discussion forums

Intellectual Skill

Verbal Information

Step 4.0 Library Corner

4.1 View video on why the

moon appears to change shape

4.2 Read a Reader’s Theater

about the moon phases

Concrete Concepts

Verbal Information

Planning for Evaluation of WBLS

Topic: Earth and Space Science, Grade 2

II. Evaluation Planning Stage The purpose of this stage is to plan for formative and summative evaluation of the

WBLS product. Formative evaluation is conducted during the design and development

phase and provides feedback to the designer. This feedback can give the designer

suggestions for improvement prior to full implementation of the WBLS and occurs in a

recursive loop allowing for many modifications in the design. The summative evaluation

is conducted after the full implementation of the WBLS with the user.

Evaluation Criteria Explanation Data Source

Effectiveness (mastery of

goals and success of WBI)

Determine if actual learning

has occurred with the

WBLS

Student mini-book

Observation Calendar

Discussion participation

Efficiency (delivered in a The WBLS was developed Document time to prepare

timely, or cost saving

manner)

well before the unit was to

be taught

materials and evaluation of

student products

Documentation of student

time on WBLS activities

Appeal (gain and

maintain learner attention

and interest; usability

The WBLS was easy to use

for second graders; the

colors were appropriate and

text size and color were

pleasing. New components

were introduced throughout

the project to maintain

student interest. Determine

ease of navigation of

bookmarked sites and

discussion forum

Instructor and student

opinions, review by peers;

interview of students to

determine degree of

difficulty in accessing

websites and participating

in the on-line discussion

forum

• Formative Evaluation Plan

Formative evaluation is conducted during the design and development phase of the

WBLS. This evaluation will take place as needed during the design/development stage.

The purpose is to provide feedback to the designer for improvement of the design, ease of

use, and ensure that it is visually appealing. This stage is important to the overall

effectiveness of the product as it gives the designer an opportunity to work out any bugs.

The designer will conduct formative evaluation during the WBLS design phase with

fellow teachers (regular and special education), friends, and family members. These

evaluators will review the WBLS and respond to the designer in an interview using the

questionnaire.

Formative evaluation in the final WBLS prototype development will take place

several times using small group and one-to-one tryout methods. Members of the small

group will typically include the target audience, but due to availability of students this

group would include the instructor’s family and/or other teachers or friends willing to

participate. Age level participants will be used in a one-to-one tryout to ascertain the

suitability and gain their perspective of the WBLS.

• List of Materials to be Examined (what is being evaluated?)

� Design Plans

• Objectives

• Assessment Items

• Clustered Objectives

• Instructional Strategies

• Motivational Strategies

� Prototype of WBLS

• Preliminary storyboards

• Interface (how the WBLS will look)

• Navigational features

• WBLS prototype

Formative Evaluation Plan:

Who are the stakeholders?

Primary – Designer, Instructor

Secondary – Students, Administration, Parents

What is being evaluated?

The design of the WBLS, the websites, discussion forum, materials

Who are the evaluators and reviewers?

Grade level teachers, special education teachers, friends willing to help

What are the evaluation methods? Questionnaire and Interview (see next section)

What types of decisions need to be made as the WBLS design plan and

prototype is develop and revised? Are the suggestions for revision valid and will the suggestions make the

WBLS more effective.

Formative Evaluation Plan Questionnaire

Evaluation Criteria Sample Questions Methods

and Tools

Effectiveness

• Goals

• Content

• Technology

• Message

Design

Does the goal match the ODE indicator?

Are the goals and objectives clear and achievable?

Are the goals and objectives appropriate for the

method of delivery?

Do the activities promote learning?

Is the information complete, covering the content

properly?

Are sites accessible through the school server

(filters)?

Was the time frame of the WBLS appropriate?

Are directions clear and concise?

Interview

Interview

Interview

/Observation

Interview/

Observation

Efficiency

• Goals

• Content

• Technology

• Message

Design

Are goals clear and concise?

Is it appropriate to our grade level?

Is content information clearly and concisely

presented?

Are websites and discussion forum easy to navigate?

Interview

Interview

Interview

Interview

Is the structure of the content clear?

Appeal

• Goals

• Content

• Technology

• Message

Design

Are students aware of the goals of the WBLS?

Is the content interesting?

Are there any typographical, punctuation, grammar,

or spelling errors?

Is navigation easy?

Is the vocabulary appropriate for the user?

Are there limited distractions on the sites?

Are the colors, typeface, and font emphasis used

appropriately to enhance learning?

Does it have good navigational design?

Are the graphics relevant and appropriate to the

WBLS?

Interview/

Observation

Interview/

Observation

Interview

Observation

Interview /

Observation

Results of the formative assessment will be delivered to the designer verbally

through an informal interview.

• Preliminary Summative Evaluation Plan

Summative evaluation takes place after the final implementation and students have

completed the WBLS. The purpose is to gather data to determine the effectiveness,

efficiency, and appeal of the final product. Summative evaluators will be fellow teachers

(regular ed and special education), students and the instructor. The method for fellow

teachers and students will take place in an interview/discussion format. The instructor

will evaluate the WBLS based on periodic observation and continual reflection.

• List of Materials to be Examined (what is being evaluated?)

o Design Plans

• Objectives

• Assessment Items

• Clustered Objectives

• Instructional Strategies

• Motivational Strategies

� Final Version of WBLS

• Interface (how the WBLS looks)

• Navigational features

• WBLS

Summative Evaluation Planning Questionnaire

Instructional Strategy Plan WBLS

Topic: Earth and Space Science, Grade 2

This section of the design document is the Instructional Strategy Plan. This section describes the instructional and motivational strategies that will be used throughout the WBLS. These strategies will be used during the month-long unit. These strategies have not been broken down into a lesson-by-lesson format.

I. Unit Overview

Description Instructional Strategies for phases of the moon unit

Orientation to Learning • Assist learner recall of prior knowledge, skills and experiences

Anticipation Guide – to access prior knowledge

Discuss results Anticipation Guide (classroom discussion)

Review Moodle use for answering questions and discussion with classmates and teacher

• Provide an overview of the unit

This WBLS is blended instruction between face-to-face direct instruction and self-directed learning using the web-based instructional units. Students will be viewing and responding to educational videos; reading trade books, listening to teacher read

Evaluation Criteria Sample Questions Data Sources

Effectiveness

How did the final product

meet the objectives?

What did the learner like?

What did the learner find

easy or difficult to use?

Mini-book

Observation Calendar

Discussions

Interview (informal)

Efficiency

Were the learners able to

participate in and complete

the activities in the time

allotted?

Teacher observation

Percentage of completed

activities

Informal interview with

users

Appeal

Will students want to

participate in another

WBLS?

Teacher observation

Percentage of completed

activities

Informal interview with

users

alouds, partner reading, individual reading; completing graphic organizers; journal writing about the earth, moon, and stars; letter writing (moon to the earth or the earth to the moon) accessing interactive websites, completing personal moon observation chart; performing hands on activities; interacting with classmates and teacher through Moodle (Learning Management System); entrance slips, exit slips; Reader's Theater; demonstrations; experiments; web searches formative assessments, summative assessment at the of the unit

• State the Learning Objective Students will be able to: Given websites, personal observation chart, and other supporting information, the students will be able to describe how the moon appears each day, identify the five main phases of the moon, predict the next moon phase, and demonstrate the correct order of the moon phases.

• Explain relevance of instruction

The gap between actual student learning and optimal learning is that these students have come to second grade with a first grade science background. The actual learning is that student’s prior knowledge is that the moon moves across the sky and that the moon looks different at different times. The optimal learning is that students will be able to describe how the moon appears different each day, identify the five main phases of the moon, predict the next moon phase, and demonstrate the correct order of the phases of the moon. Throughout this process, students will have a broader knowledge base about relationship between the sun, the moon, and the earth. This concept will be transferred to students through the use of a teacher-led, large group game called Rivet.

• Provide directions on how to start, navigate, and proceed through the instruction

• Letter to parents explaining the unit and what students will be expected to do.

• Model how to complete the Observation Chart.

• Demonstrate how to access websites organized on mybookmarks.com.

• Show the end product (Moon Phases Booklet).

• Teacher uses Interwrite board to navigate to first discussion question in Moodle, modeling how to respond in the

discussion forum. • On a daily basis, students will be

provided with instructions and expectations for that day’s activities.

II. Instruction on the Content

Instruction on the content is the area of the design document that shows what instructional strategies will be used during the presentation of the content.

Instruction on the content subcomponents

Instructional Strategies for Moon Phases Instructional Unit

Present Content Information

• Direct instruction through presentations using text, audio, multimedia, streaming audio/visual, lectures, etc.

• Demonstration led by teacher how moon reflects sunlight, how moon revolves and rotates around the earth always showing the same side to the earth

• Hands-on experiments • Modeling by instructor how to complete observation chart and how to log in and complete discussions in Moodle

• Graphic Organizers such as Venn diagrams, I Am poem,

t-charts, The Most Important Thing about… • Teacher-led discussions • Student-led discussions • In-class reading assignments • Web searches for information • Simulations on the Internet (mybookmarks.com website)

• Collaborative, cooperative, and independent learning situations

Provide Learning Cues • Moodle discussion with teacher created prompt

• Remind students to observe the moon nightly and complete the observation chart

• Before, during, after reading strategies

• Mnemonic (Left/last; right/first • On-line games (mybookmarks.com website)

• Web research • Journal writing • Moodle discussions • On-line vocabulary practice • Projects- completion of observation chart, mini book, and summary of readings

• Photos of the Moon • Virtual Field Trip (mybookmarks.com website)

• Journals response journals • Graphic Organizers

Present Opportunities for Practice

• Class discussion – student to student feedback

• Moodle discussion threads • Teacher feedback on activities • Self-correcting feedback on Internet activities

Provide feedback on the practice performance

• Review previous learning throughout the unit

• Summarize readings/activities at end of each lesson (exit slips)

• End lesson with preview of the next topic or task

• Use text, video, audio wrap-ups of the lesson

• Instruction comments on progress of on-line discussion forum

Provide review and close the unit of instruction

• Review game such as Rivet or tic tac toe in collaborative group, no summative

assessment

III. Measurement of Learning

Measurement of Learning outlines the methods used to assess student learning. These methods can take the form of diagnostic, formative, or summative assessments.

Measurement of Learning Subcomponents

Instructional Strategies for Moon Phases Instruction

Assess performance or progress • An informal pretest is given to students in the form of the Anticipation Guide prior to instruction to determine student background knowledge.

• Individual pre and post-learning will be assessed via a KWL graphic organizer prior to reading activities

• Exit and Entrance slips • Require completion of mini-book project

• Require completion of observation chart

• Checklists and rubrics to score reading, writing, and discussion participation

Advise learners of scores • There is no summative posttest

as second grade science is intended for exposure to concepts, not mastery

• Provide project guidelines for learners to self-check their work.

IV. Summary and Close

This section of the design document provides for a summary and closure of the learning. This component allows for the closure of learning, reviewing the work, and transferring the learning to new and different situations.

Summary and Close Overview of the Instructional Strategies for Moon Phases

Provide for opportunities for retention of learning

• Summarize and review key concepts regularly and often.

• Think-Pair-Share • Lunar Café Activity (small groups share knowledge retained throughout the unit, by recording facts on chart paper in a timed method. Students then rotate to the next chart to read peer’s comments and their own facts. This continues on until the rotation is complete.

Provide for remediation of unmet objectives

• Summarizing the lessons • Due to this being exposure and not mastery, remediation is not required, but the student not meeting the objectives is encouraged to review the products, websites, or reading materials

Enhance and enrich learning • Students want to continue the observation chart for an additional month

• Provide website list for students to access at home with parent permission

• Suggest additional reading materials

• Encourage continued learning through the use of school or public library resources

• Provide an open discussion forum for the sharing of new knowledge through Moodle

Task Analysis The task analysis portion of the design document takes the learning strategies presented in the previous section and turns them into tasks that the learners or users of the WBLS (Lunar Lab) will perform in order to complete this unit. There are several parts to the Task Analysis – a prioritized task list showing the core, important, and nice to have features – a functional requirement list again with a core, important, and nice to have list of requirements based on the importance or

frequency of use – a task flow diagram showing which tasks are on which screen in the WBLS.

Part I – General Task List

Initial Task List

Using the Lunar Lab WBLS the learner will be able to:

♦ Log into/out of Lunar Lab WBLS in Moodle

♦ Read text

♦ Navigate through Lunar Lab efficiently

♦ View and respond to discussion forum

♦ View videos

♦ View photos

♦ Post questions to instructor

♦ Access, open, and view links to outside resources

A. Prioritized Task List

(Key)

� Core Functionality

� Important Functionality

o Nice to have Functionality

Frequent by Many

� Log in and out of Lunar Lab WBLS

� Read Text

� Navigate through Lunar Lab efficiently

Frequent by Few

o Access, open, and view links to outside resources

I. Occasional by Many � View and respond to discussion forum

o View Photos

Occasional by Few

o Post questions to instructor

o View Videos

Part II – Functional Requirements

Initial Functional Requirements List

• Ability to log in and out of system

• Ability to scroll

• Ability to left click on mouse to select item

• Ability to right click

• Ability to navigate

• Ability to view text

• Ability to open, view, and access links

• Ability to type

• Ability to respond to questions posed by peers and

instructor

• Ability to compose a new post

• Ability to view video clips

• Ability to view graphics

• Ability to use spell check

• Ability to share enrichment materials with class

B. Function Requirement List

(key)

� Core Functionality

� Important Functionality

o Nice to have Functionality

Main Screen

� Ability to log in and out of system

� Ability to scroll

� Ability to navigate

� Ability to view text

� Ability to open, view and access links

� Ability to left click on mouse to select item

� Ability to view graphics

Discussion Forum (Moodle Link) Screen

(from Lunar Lab to Highland Local Schools Homepage, this is a

series of links to access Moodle for security reasons)

� Ability to scroll

� Ability to left click on mouse to select item

� Ability to navigate

� Ability to view text

(Highland Local Schools Homepage to Online

Academy Screen)

� Ability to scroll

� Ability to left click on mouse to select item

� Ability to navigate

� Ability to view text

(Online Academy to Log-in Page Screen)

� Ability to type in user name and password

� Ability to left click on mouse to select item

� Ability to navigate

� Ability to view text

(Log-in Screen to Class Page Screen)

� Ability to scroll

� Ability to left click on mouse to select item

� Ability to navigate

� Ability to view text

� Ability to open, view and access links

(Class Page Screen to Discussion Topic Screen)

� Ability to scroll

� Ability to left click on mouse to select item

� Ability to navigate

� Ability to view text

� Ability to open, view and access links

(Discussion Topic Screen to Add a new discussion

topic screen or response screen)

� Ability to navigate

� Ability to view text

� Ability to type

� Ability to respond to questions posed by peers and

instructor

� Ability to compose a new post

� Ability to use spell check

� Ability to scroll

(discussion topic screen to reply screen)

� Ability to right click

� Ability to navigate

� Ability to view text

� Ability to type

� Ability to respond to questions posed by peers and

instructor

� Ability to compose a new post

� Ability to scroll

� Ability to use spell check

Library Screen

� Ability to scroll

� Ability to navigate

� Ability to view text

My Bookmarks Screen

� Ability to scroll

� Ability to left click on mouse to select item

� Ability to navigate

� Ability to view text

� Ability to open, view and access links

� Ability to view graphics

� Ability to view video clips

Please see next page for Task Flow Diagram

C. Task Flow Diagram for Lunar Lab WBLS

Moodle On-Line

Discussion Forum

Moodle is access

through Highland Local

Schools Main Website

due to the need for on-

line security

My Bookmarks • Select links to

websites outside of

WBLS

• Select websites

used in WBLS

instruction

• Select video links,

interactive games,

virtual field trip,

digital photos

Main Page Screen

• Select Moodle On-Line

Discussion Forum

• Select Book Library link

• Select My Bookmarks

link

• View Fun Facts

• Read Unit overview

Book Library

View titles of books

related to unit

Module 3.3 Scenario

Task for both scenarios: Students are to check the phase of the moon and complete

the observation chart showing the phase of the moon for that day, post their

response to the question posted on the class discussion forum. The posted question

is, “Given today’s moon phase, when do you predict the next new moon will

happen?”

Scenario #1 – Student is in a hurry and wants to be as efficient as possible in his or her

work effort

Background:

Student A knows that it is permissible to complete this task in class the following day

because the teacher cannot require students to perform computer-based tasks at home.

During science class, when the teacher shows the previous evening’s moon phase, student

A quickly traces her nickel and shades in the illuminated portion using her white pencil.

Student A listens to class discussion and then correctly labels the current phase. After

this, the class goes to the computer lab to complete the Moodle discussion.

Structure of Tasks:

Student A walks to the computer lab with the teacher. Student waits until the teacher

assigns her to a computer. Student sits down. Turns on the computer. Waits for it to

boot up. Logs in to the system using the class username and password. Logs onto

Internet. Types in the Lunar Lab URL. Double left clicks the Moodle Discussion Forum

icon. The next screen to appear is the Highland Local Schools homepage. Student left

clicks on the On-Line Academy tab at the top of the homepage. The next screen to

appear is the “Available Courses in On-Line Academy.” Student A scrolls down to

locate her class. Student left clicks (once) on correct class name. The next screen is the

student log in screen. Student types in user name, hits the tab key to move cursor to

password field. Raises hand, cannot remember password. Teacher gives Student A the

password. Types in password. Using the mouse, student puts cursor over log in button

and left clicks (once). On the next screen the student reads the discussion topics, finding

the one to be completed that day. Using the mouse, the student moves the cursor to that

topic and left clicks (once) on that topic. On the next screen, the student reads the

question and can see who has participated so far in the discussion. Using the mouse the

student places the mouse over the “Add a new discussion topic” button. Left click

(once). Cursor appears in the subject field and student types the subject – “The next new

moon.” Student moves cursor with the mouse to the first line of the composing field and

begins to type a response to the posted question. Since student did not view the moon the

previous night, or play close attention to the classroom discussion, it takes quite a while

for Student A to form the response. Once the response is complete, the student uses the

mouse to place the cursor on the right scroll bar and while holding the left button down

on the mouse, and scrolls down to the bottom of the page. Student moves the cursor to

the “Post to Forum” button using the mouse and left clicks (one) to post the reply to the

forum. On the next screen student moves the cursor to the “Continue” button, left clicks

(one) to return to the discussion page. Student scrolls to the bottom of the page using the

right scroll bar and left mouse button. Then she moves the cursor to the “log out” button

(beside her name), left clicks (one) and logs out of the class page. Then she moves the

cursor to the top right of the Highland On-line Academy page and left clicks the red “X”

to exit out of the academy. Using the mouse, student places the cursor on the red “X” to

exit out of the Highland homepage. Using the mouse, student places the mouse over the

start button, left clicks on it. Using the mouse, student places the mouse over the shut

down button, left clicks on it. Using the mouse, student places the mouse over the shut

down button, left clicks on it to shut down the computer.

II. Scenario # 2- Student wants to have an efficient and effective collaborative work

with his/her teammates in the WBLS and achieving at a high level of performance.

Background:

Student B has always put forth his/her best effort in fully completing class assignments

neatly in order to get the most out of their education. When presented with today’s class

assignment, Student B was excited to complete the work!

At home, Student B makes a point to observe the moon in person by going outside during

dark hours and viewing the moon. After observing the way the moon appears in the sky,

Student B takes out the moon observation chart and completes the drawing for that day.

Students B then colors in the illuminated portion of the moon as seen and relates this to

what has been learned in class about the different phases. Student B believes that the

current moon phase is a waxing moon in the first quarter. After labeling the phase

underneath the drawing of the moon on the calendar, Student B logs onto the class

website to check the current moon phase. Student B navigates from the main page to the

mybookmarks.com site to select the website previously posted by the classroom teacher.

Student B accesses the website, http://www.briancasey.org/artifacts/astro/moon.cgi,

opens and views the current moon phase and confirms that what was observed and drawn

was a waning crescent. Student B also notices that the website says the current moon

phase is 13% full and states that it is 2 1/2 days before the New Moon. Student B is

excited to return to school and share this new knowledge with classmates and the teacher.

Structure of Tasks:

Student B walks to the computer lab with the teacher. Student quickly chooses a

computer and sits down. Turns on the computer. Waits for it to boot up. Logs in to the

system using the class username and password. Logs onto Internet. Types in the Lunar

Lab URL. Double left clicks the Moodle Discussion Forum icon. The next screen to

appear is the Highland Local Schools homepage. Student left clicks on the On-Line

Academy tab at the top of the homepage. The next screen to appear is called “Available

Courses in On-Line Academy.” Student scrolls down to locate her class. Student left

clicks (once) on correct class name. Next screen is the student log in screen. Student

types in user name, hits the tab key to move cursor password field. Types in password.

Using the mouse, student puts cursor over log in button and left clicks (one). On the next

screen the student reads the discussion topics, finding the one to be completed that day.

Using the mouse, the student moves the cursor to that topic and left clicks (once) on that

topic. On the next screen, the student reads the question and can see who has participated

so far in the discussion. Using the mouse the student places the mouse over the “Add a

new discussion topic” button. Left click (once). Cursor appears in the subject field and

student types the subject – “The next new moon.” Student moves cursor with the mouse

to the composing field and begins to type her response to the posted question. Since

Student B accessed the website the previous night, the answered response was arrived at

quickly and the student was able to efficiently complete the assignment. Student shared

additional information learned from the research, enriching the learning experience of

other students. Once the response is complete, using the mouse and the holding the left

button down on the mouse, student places the cursor on the right scroll bar and scroll

down to the bottom of the page. Student moves the cursor to the “Post to Forum” button

using the mouse and left clicks (once) to post the reply to the forum. On the next screen

student moves the cursor to the “Continue” button, left clicks (once) to return to the

discussion page. Student then views other class members who have replied and using the

mouse places the cursor over the discussion topic of a classmate and left clicks. This

brings up the discussion response of the classmate for student b to read. Student B views

text, carefully reading classmate’s response and chooses to reply, creating a collaborative

learning environment. To reply, Student B moves the cursor to the word reply on the

right side of the response page left clicks. This brings up the composition page. Student

places cursor in the composing field using the mouse. Student types a thoughtful and

thorough response to classmate’s posting. Student B poses a question to the classmate to

continue the collaborative discussion. Student B moves the cursor to the spell check icon

in the tool bar and left clicks (one) the icon. Student views the task window and makes

the decision to either accept or ignore the suggested spellings. Student moves the cursor

to the “Post to Forum” button using the mouse and left clicks (once) to post the reply to

the forum. On the next screen student moves the cursor to the “Continue” button, left

clicks (once) to return to the discussion page. Student B repeats this process until the end

of the class period creating many collaborative learning opportunities with classmates.

This becomes an effective web-based learning system and the student is achieving at a

high level of performance. Student scrolls to the bottom of the page using the right scroll

bar and left mouse button. Then she moves the cursor to the “log out” button (beside her

name), left clicks (one) and logs out of the class page. Then she moves the cursor to the

top right of the Highland On-line Academy page and left clicks the red “X” to exit out of

the academy. Using the mouse, student places the cursor on the red “X” to exit out of the

Highland homepage. Using the mouse, student places the mouse over the start button,

left clicks on it. Using the mouse, student places the mouse over the shut down button,

left clicks on it. Using the mouse, student places the mouse over the shut down button,

left clicks on it to shut down the computer.

Summary:

We decided to use the same task for both scenarios so that we could compare the tasks

that the students would do. The scenarios are detailed, so that readers can understand

what the users are doing throughout the task. The objective of “predicting when the next

new moon will occur” is reached. This is a sub-objective of the main objectives, “Given

websites, personal observation chart, and other supporting information, the students will

be able to describe how the moon appears each day, identify the five main phases of the

moon, predict the next moon phase, and demonstrate the correct order of the moon

phases”.

Module 3.4 Specification Report

The link to our Wiki page is:

http://mrsweber.pbworks.com

Module 4.1 – Expert and User Review

Part I. Evaluation Planning Stage/with modifications for formative

*Modifications made to this document after the design/development

stage prior to field testing with potential users, content experts, and web

design experts are in italics.

The purpose of this stage is to plan for formative and summative evaluation of the

WBLS product. Formative evaluation is conducted during the design and development

phase and provides feedback to the designer. This feedback can give the designer

suggestions for improvement prior to full implementation of the WBLS and occurs in a

recursive loop allowing for many modifications in the design. The summative evaluation

is conducted after the full implementation of the WBLS with the user.

Evaluation Criteria Explanation Data Source

Effectiveness (mastery of

goals and success of WBI)

Determine if actual learning

has occurred with the

WBLS

Student mini-book

Observation Calendar

Discussion participation

Efficiency (delivered in a

timely, or cost saving

manner)

The WBLS was developed

well before the unit was to

be taught

Document time to prepare

materials and evaluation of

student products

Documentation of student

time on WBLS activities

Appeal (gain and

maintain learner attention

and interest; usability

The WBLS was easy to use

for second graders; the

colors were appropriate and

text size and color were

pleasing. New components

were introduced throughout

the project to maintain

student interest. Determine

ease of navigation of

bookmarked sites and

discussion forum

Instructor and student

opinions, review by peers;

interview of students to

determine degree of

difficulty in accessing

websites and participating

in the on-line discussion

forum

• Formative Evaluation Plan

Formative evaluation is conducted during the design and development phase of the

WBLS. Feedback is given to the designer for improvement to the design to make it

easier to use or more visually appealing. This stage is important to the overall

effectiveness of the product as it gives the designer an opportunity to work out any bugs.

Formative evaluation will take place several times with a one-to-one tryout and small

group. Members of these groups will include the instructor’s family and/or other teachers

or friends willing to participate.

A final formative evaluation took place prior to full implementation to determine if the

WBLS met the goals of the designers, was visually appealing, was easy to navigate, if the

content was appropriate for the users, and if the design was appropriate.

Formative Evaluation Plan:

Who are the stakeholders?

Primary – Designer, Instructor

Secondary – Students, Administration, Parents

What is being evaluated?

The design of the WBLS, the websites, discussion forum, materials

Who are the evaluators and reviewers?

Grade level teachers, special education teachers, friends willing to help

What are the evaluation methods?

Questionnaire (see next section)

What types of decisions need to be made as the WBLS design plan and

prototype are develop and revised?

Are the suggestions for revision valid and will the suggestions make the

WBLS more effective.

Formative Evaluation Plan Questionnaire/Formative Evaluation Questions that were

added to this are in italics

Evaluation Criteria Sample Questions Methods

and Tools

Effectiveness

• Goals

• Content

• Technology

• Message

Design

Does the goal match the ODE indicator?

Are the goals and objectives clear and achievable?

Are the goals and objectives appropriate for the

method of delivery?

Do the activities promote learning?

Is the information complete, covering the content

properly?

Are sites accessible through the school server

(filters)?

Are the sites accessible through the WBLS?

Was the time frame of the WBLS appropriate?

Are directions clear and concise?

Interview

Interview

Observation

Observation

Field Test

Interview

Efficiency

• Goals

• Content

• Technology

• Message

Design

Are goals clear and concise?

Do you understand what to do for each of the lab

activities?

Is it appropriate to our grade level?

Is content information clearly and concisely

presented?

Are websites and discussion forum easy to navigate?

Is the structure of the content clear?

Interview

Interview

Observation

Interview

Observation

Interview

Appeal

• Goals

Are students aware of the goals of the WBLS?

Interview

• Content

• Technology

• Message

Design

Is the content interesting?

Are there any typographical, punctuation, grammar,

or spelling errors?

Is navigation easy?

Do you like the colors on the pages?

Did you like the pictures?

Do the pictures help you understand what to do?

Should the graphics be on the other pages too?

Is the vocabulary appropriate for the user?

Are there limited distractions on the sites?

Are the colors, typeface, font emphasis used

appropriated to enhance learning?

Does it have good navigational design?

Interview

Observation

Interview

Observation

Interview

Observation

The designer/developer will receive the results of the formative evaluation

through a written summary of the interviews and observation of the users, content

experts, and web design experts.

Background of Users:

The WBLS was implemented with 2 females age 8. They just completed second grade.

Both were familiar with Moodle and both had the ability to type and use basic

navigational skills.

Background of Content Experts:

One content expert is a male, age 50. He is an amateur astronomer and has a telescope

in his back yard. Another is a female, age 49 and is currently a teacher with certification

in general science and biology 7-12. The third expert is a college student studying in the

food and agriculture and environmental sciences department, and is 20 years old.

Background of Web Design Expert:

One web design expert is a female, age 27. She is a master’s student seeking her

master’s in instructional technology. The second web design expert is a

computer/technology teacher in a suburban district. She is 45 years old and has been

designing web pages for 2 years. The final web design expert is 18 years old, and has

been using the Internet for many years. He is conversant in navigation and design of web

sites, and will be majoring in computer technology in the fall.

Part 2. Relevant Documents/Guides/Instruction

A. Questions for Web Design Expert Review:

Are the tasks easy to understand?

Are the directions on the WBLS easy to understand?

Do the links work?

Is the layout visually appealing?

Are there any typos, grammar errors, punctuation, or spelling errors?

B. Questions for Content Expert Review

Are websites appropriate for second grade?

Is the content on the websites correct?

Are the book titles appropriate?

C. Questions for Users Review

Were the directions easy to follow?

Did you like the pictures, did they help you understand what to do?

Are the colors okay, or would you like it to be more colorful?

Did you understand what to do for each activity?

Should there be graphics on each page that is part of the Wiki?

Anecdotal Observations (notes taken while students are navigating

through the WBLS).

3. Results/Findings from the Expert Reviews a. Web Design Experts

Interview/Observation Results (conducted simultaneously)

• Design is clean, kid friendly

• Site is easy to navigate

• Move the Lunar Library and My Bookmarks links closer together so

there is not so much white space

• Add a return to front (home) page link on the library page and the

reflection page so users can easily return to the home page

• Change TeacherTube video to open in a separate page

• Add OR to Lab 1 so students know they can do either or both activities

• Add directions how to get to the appropriate site for lab 1 (1,2,3 or

bullets

• Add directions how to get to the appropriate site for lab 2 (1,2,3 or

bullets

• Add wording on lab 4 (library activities) to tell students how to get

back to Lunar Lab

• Spelling error on Reflection Page – should be ‘choose’

• For students on IEPs, you could print off the front page and have them

check off the boxes as they complete a task.

• Change wording at the bottom of the home page, I thought you could

click on it

• Tell students how to log off the lab

• I like how the bookmarks page had other activities to do, not just the

sites that you wanted them to go to.

• Like the graphics

• Just the right balance of text and graphics

• Like the set up with the boxes for the activities

b. Content Experts

Interview/Observation Results (Conducted simultaneously)

• Found error on moon fun fact in the side bar

• Directions are easy to follow

• The site was attractive and easy to find the content

• The websites chosen were appropriate for second grade students

• Websites on the bookmarks site had varied information for the diverse

learners in second grade

• Library books listed were adequate

• Comment: “When I logged on to the site, I was confused as to what to

do. You should add something on the front page so parents/students

know what you expect them to do with your site.

c. Potential Users

Interview Results:

� Both students liked the pictures, but one did not like the bookworm

picture.

� Both students like the colors, one wanted more colors on the main

page though.

� Neither found anything confusing about the site.

� Both understood the lab activities

� There were two links that did not work

� There should be graphics on all the pages on the Wiki

Observation Results:

� When reading the directions aloud, the highlighted information

was spoken in a louder tone, so she understood that this was

important information

� Because of their familiarity with my bookmarks, they wanted to go

to all the sites, see if there were any games (this was a distraction)

� Two of the website links did not work –my bookmarks and the

reader’s theatre

� Both students were able to navigate to Moodle through the various

screens due to privacy/security concerns.

� Both students were familiar with typing and basic navigational

skills necessary to complete the WBLS effectively and efficiently.

� One student was confused by the orange moon at the top and

wanted to click on it.

4. Ideas for Modification for the Design Products

Based on feedback received from the experts and users we will modify the WBLS

as follows:

� Fix the two links that did not work

� Add the bookworm graphic to the library page

� Add a graphic to the Moon Message page

� We will not add more colors or graphics to the main (front) page as we

feel this will be too distracting.

� Add wording to the lab 1 and lab 2 to explain to open the moon phases

folder

� Change wording at the bottom of the front page from “End Lunar Lab”

to “When you have completed Lab 5, you are finished with the Lunar

Lab Activities. You did it! Remember to log out of this site by

clicking on the words “sign out” in the upper right hand corner.

� Move two graphics and links (library and bookmarks) closer to the

center to make the triangle more compact and reduce white space

� Fix spelling error

� Fixed TeacherTube link so it opens in its own window outside of

Lunar Lab

� Added wording on Lunar Library to tell students how to return to

Lunar Lab when the are finished viewing the video or reading the

reader’s theatre

� Moved links in moon phases folder to be first or second choice

depending on lab

� Add wording to Labs 1 and 2 to direct students on how to access the

appropriate link

� Add the “click here to return to home(front) page” on the library and

reflection pages

� On lab 1 add the word or so students would know they could do either

activity

� Change the daily fun fact to be less confusing when it is read

� Added wording to front page to explain what to do with the sites,

activities to avoid confusion.

Module 4.2 Usability Test

1. Usability Test Plan

The usability test took place at the home of Penny Reed on June 29, 2009

from 1:00 to 3:00 pm. There were two participants, both were eight years

old. Both were familiar with Moodle, MyBookmarks, had basic

keyboarding and navigational skills. These participants were selected

based on their availability, proximity to the testing site, and felt

comfortable with the instructor and the testing site.

The learning goal for the usability test is to discover the usefulness,

learnability, efficiency, and satisfaction level of the WBLS. There were

several computers with Internet access available for the users, which were

connected to the Internet and ready for them to use. To prepare the uses,

we gave them a brief lesson of the objectives for using the site. We also

discussed with them why they were chosen to test out the site.

During the usability test, data was collected through observation and an

interview. We took anecdotal records of our observations and their

comments. After students finished the activities, interviewed each user

individually so that their answers would not be tainted.

2. Findings/Results of the Usability Test

� From observations we found that the link to the Reader’s Theater

and Mybookmarks didn’t function.

� From observations we found that the students were able to

remember how to navigate to the Moodle page.

� From observations we found that the students remembered how to

sign into and out of Moodle.

� From observations we found that students remembered how to

navigate the mybookmarks site (once we put them on the site)

� From observations we found that there was little need for them to

seek assistance to either navigate through the site or complete the

activities.

� From observation we found that the intended pace of the class was

appropriate for the age level and time given.

� From observation we found that the readability of the content was

appropriate for this grade level.

� From the observation we found that one student wanted to click the

orange moon thinking that it was a clickable area.

� From observation we found that students wanted to access the

interactive games instead of sites necessary for the activity.

� During the observation the designers realized that they neglected to

put their names on the front page of the WBLS.

� From observed behavior, we discovered that the students were

eager to participate in the usability test. They sat down at the

computer and quickly sat down to work, exploring the site, testing

out the links.

� From the interview, we learned that the topic was interesting to

them.

� From the interview, we learned that the site was appealing (design

and colors were appropriate). There were limited graphics and no

distractions.

� From the interview, we learned that students liked the individual

aspect of the WBLS.

� From the interview, we learned that students wanted to have

graphics on all the pages instead of just the home page.

3. Suggestions for Improvement

� Based on the results of this usability test we will:

o Fix the links that did not work

o Add a graphic to the library page

o Add a graphic to the moon message page

o Add the designers names to the front page of the WBLS

� Based on the observed behaviors of the participants we

discovered that the WBLS did not create frustration and

successfully met our objectives.

� Strengths:

o Students were eager to complete the task

o The site was appealing and easy to navigate

o Users were familiar with several aspects of the site

(Moodle and mybookmarks)

o Students were efficient in using the site, accomplishing

the age appropriate tasks with ease.

� Weakness

o The weakness was the limited number of users for the

test.

o Another weakness was performing this test during the

summer. Had this been implemented during the school

year, students would have had the benefit of face to

face instruction.

Because of the time frame, the test was conducted before complete testing of the

prototype was accomplished

Module 4.3 Implementation Plan and Summative Evaluation

a) Implementation Plan

The purpose of the Implementation Plan is to determine the timeline, allocate the

duties of personnel, and outline learning tasks and associated facilitation

strategies. This plan is based on the designers/instructor experiences and

suggestions gained from the usability test. The plan allows for the designers to

reflect upon the design if the learning objectives have been achieved, if a learning

community has been created and sustained in the WBLS.

i) Time Schedule of the Implementation

• Design and development of activities including maintaining the

Wiki and Moodle sites will be completed during the

instructor’s planning times and/or own time-

Both sites can be accessed from the instructor’s home

Estimated time commitment required is 15 hours

Instructor Duties Estimated Time

Requirement

Set up student

usernames/passwords

0.5 hour

Develop and/or collect online

materials, activities, and

discussion questions

7 hours

Teach course (content, learning

tasks, instruction on navigating)

2.5 hours

Provide feedback on activities

both off and on line

4 hours

Miscellaneous

activities/troubleshoot technical

problems

1 hour

Total hours for unit 15 hours

• Budget concerns will not need to be addressed since the

District has already implemented a Moodle site as part of the

school’s website and the class Wiki site is a free Internet tool-

this would also include technical support at no cost

• Lunar Lab activities are projected to cover a four week time

period due to the implementation of the project to observe and

record moon phases for one month

• All other tasks in Lunar Lab are designed to be completed

during two weeks

ii) Personnel and Duties Allocation

WBI Instructor:

• Design, develop, and maintain class Wiki website

• Implement and maintain District Moodle classroom website

• Develop personal biography section-introduce the instructor

• Provide unit objectives, course goals, and assignment

requirements

• Maintain site information to keep it current and that all links

are active

• Establish rapport and maintain communication between

learners and instructor

• Review content to check for accuracy and completeness

• Provide students with user names, passwords, and

instructions on accessing and navigating Wiki and Moodle

sites

• Design place, time, and topics for discussion

Administrative and Technical Personnel

• Provide the infrastructure needed to institute and maintain

District Moodle site

• Maintain network infrastructure

• Maintain computer equipment including computers,

monitors, web cams, microphones, mouse, LCD projectors,

and all connecting wiring

• Provide WBI instructor with technology support for

instruction

• Update software/hardware when necessary

• Enroll class members as participants on the class Moodle site

iii) Learning Tasks V.S. Facilitation/Strategies

The WBI is based on for a second grade class of 28 students

• Initial instruction of the WBI will take place during regular

class time

III. Learning Tasks IV. Facilitation Strategies Describe movement of the

moon in relation to the earth and

sun (learn content knowledge)

Pose questions for accessing prior

knowledge and as a discussion starter;

demonstrate with models/students the

earth-sun-moon movements, encourage

class discussion with additional

questions; summarize main points

Learn content knowledge Pose question before viewing

UnitedStreaming video to provide a

purpose on content to be discussed;

provide quiet uninterrupted viewing of

video; follow up with class discussion to

summarize major concepts covered

Read books to gain basic

content knowledge about the

moon phases

Read grade level fiction and nonfiction

books to stimulate class discussion and

acquire moon content knowledge;

provide grade level books for students to

read independently or in small groups;

provide opportunities for discussion

and/or reflection about knowledge

learned through readings; provide

graphic organizers to organize content

knowledge

Demonstrate how to access and

navigate Lunar Lab website,

activities, and how to participate

Model behavior and directions

necessary for accessing and using the

WBLS through the use of a LCD

in discussion forums projector and Internet websites; give

learners opportunity to ask questions;

inform participants of expectations for

completing activities; provide time for

learners to practice accessing websites

and activities to avoid possible

frustrations later when working

independently

Complete the observation

calendar, illustration, and

labeling the phase

Explain how to complete the

observation calendar, model one day’s

response by viewing website together,

completing drawing of the moon, and

labeling the phase

View website to complete first

drawing of the moon phase

Provide a observation chart with current

moon phase completed for student self-

checking

Complete responses in online

class discussion forums (Wiki

discussion & Moodle

discussion-threaded responses)

Asynchronous discussion-keep on track

and relevant by providing students with

guidelines for participating and by using

password protected forums; state &

model expectations for responses;

provide opportunities that allow learners

to assist each other with technical

support; personalize feedback by

responding to individual postings; use

threaded discussion (Moodle) to permit

participants to share experiences and ask

questions among themselves; provide

netiquette guidelines for the discussion

responses

View video on why the moon

appears to change shape

Provide a discussion response to verify

viewing of video

Read the Reader’s Theater

about moon phases

Provide own paper copy of Reader’s

Theater with character part highlighted;

form small groups to practice for live

performances

Work collaboratively and as

part of a group

Allow for a mix of small-group and

whole-group activities for peer support

on problem-solving tasks; share

cooperation strategies

Have opportunities to

differentiate in learning

activities

Include independent tasks; additional

activities; extra informational sources;

be flexible with students abilities and

responses

Acquire additional technology

skills

Provide opportunities for interacting

with various websites (Moodle, Wiki,

Mybookmarks); applying a variety of

keyboarding skills; practice login and

logging out procedures and keeping

passwords secure

Practice time management skills Provide ample time to complete grade

appropriate tasks and inform students of

how long the assignment should take;

allow for extra opportunities to finish

work if needed; Provide links that work

Summative Evaluation F2F

Class Discussion (reflection)

Correct any misconceptions about the

moon that may have occurred during the

online learning & activities; summarize

major concepts in discussions and

readings as a way to follow up on unit

objectives; ask students to reflect on unit

to share opinions of learning through

this type of activity; provide an

opportunity for students to share if they

enjoyed completing this type of activity

and would they want to learn this way in

future lessons

b. Summative Evaluation Plan

The summative evaluation takes place after the final implementation of the Lunar Lab

WBLS. The purpose of the summative evaluation is to determine the effectiveness of the

final product, if learning goals have been met, if stakeholders were satisfied with its

outcomes, if value has been added, and if students would like to do this type of learning

strategy again. The plan also lays the groundwork for the collection of data and its

analysis and the reporting of the findings to the stakeholders.

i) Summative Plan

Who are the Stakeholders? Primary: The Designers/Instructors

Secondary: The School District

The Students

What is being evaluated? Student Learning in Lunar Lab

Through the context of effectiveness,

efficiency and appeal

Who are the evaluators? o The designers (internal)

o Other 2nd grade teachers,

principal(external)

o The users of Lunar Lab (2nd

graders)

What are the evaluation methods and tools o Informal interview, survey,

observation

o Products produced during face-to-

face class meetings

o Discussion forum postings

How are data collected and analyzed? o Interview data collected with use of

tape recorder and transcribed

(students)

o Pen and paper survey and interview

(teachers, administrator)

o Products are graded by the

instructor and discussed with students

to clear up any misconceptions

o Microsoft Excel will be used to

analyze the data. Common threads

among short answer responses will be

looked for

How are evaluation results reported? o Designers discuss summative

evaluations to determine effectiveness,

efficiency and appeal of Lunar Lab

o Using Excel, tables and charts of

responses will be made to determine

frequency of responses, trends in the

data, and any outliers

ii) Questions to ask Evaluators during Summative Evaluation

The following table lists the questions that to be asked during the summative evaluation

of Lunar Lab. There are three areas of consideration – Instructional Goals and Content,

Learners and Instructors, and Context.

Area of Consideration Questions to be asked

Instructional Goals and Content Effectiveness

o Could students accurately predict when a moon phase

would occur?

o Could students order the phases of the moon

correctly?

o Were students able to communicate their thoughts

concisely in the discussion forums to demonstrate

learning?

o Can students properly identify the five phases of the

moon?

Efficiency

o Were the goals of Lunar Lab achieved?

o Could students access Lunar Lab from school/home

and understand the activities?

Appeal

o Do students want to participate in another WBLS

learning activity?

Learners and Instructors Effectiveness

o Were the stakeholders pleased with the outcomes,

products, and activities?

o Do users feel Lunar Lab was worth the time spent on

it?

o Were the activities and informational websites

informative?

Efficiency

o Was the Wiki easy to use?

o Were there some design features that made the Wiki

difficult to use?

o Was Moodle easy to use?

o Did Moodle lend itself to collaborative learning?

Appeal

o Did users find Lunar Lab interesting?

o Would users want to participate in another WBLS

learning activity?

o Did the graphics help you understand what to do?

Context Effectiveness

o Was the computer lab and/or library a good place to

work on this?

o Did you like doing Lunar Lab as a station in the

classroom?

o If you used it at home, did it work as well as it did at

school?

o If you used it at home, were you as successful in

completing the activities as you were at school?

Efficiency

o Was the site easy to access?

o Did all the links open?

o Were you able to navigate back to Lunar Lab from

the external links?

Appeal

o Were the authors clearly identified?

o Did the graphics and photos enhance or detract Lunar

Lab?