Training university instructors to use online communication tools in their courses

ADDIE approach, Final Project

December, 2014

Emtinan Alqurashi, Ed Kocher, Marcia Rapchak, Chunhua Xiong

Introduction to ADDIE

ADDIE is instruction system design that is composed of five steps. The name ADDIE is

an acronym for the five components of the theoretical model: analysis, design, development,

implementation, and evaluation. During the analysis step, the problem is defined, sources of the

problem considered, and potential solutions are generated. The instructional goals determined in

the analysis phase lead to the design phase. In the design step of the ADDIE model, the learning

objectives, test items, instructional plans and sequence are determined so that during the next

phase, Development, the lesson plans and materials are prepared for implementation.

Implementation is the process of delivering the instruction into the learning environment. The

learning environment could be in the classroom, laboratory, or in an online format. The final

term of the acronym ADDIE is evaluation. It is important to note that the ADDIE model makes

provision for evaluation in process (formative) and evaluation at the end of the process

(summative). The formative option makes it possible to make adjustments during each phase of

the process based on information and discoveries gained during process. The diagram below

illustrates the relationships between and among the five phases of the ADDIE model.

2

Rationale for ADDIE

When our instructional theory working group (Emtinan, Ed, Marcia, and Chunhua) began

to discuss our potential topic, we agreed that predictably, university professors have deep content

knowledge and advanced research skills and interests. However, our experiences inform us that

many college level teachers have not studied pedagogy, learning theory, and instructional

strategies. Often, professors tend to teach the way they were taught, but each year new learners

enter the higher education environment with different backgrounds, learning styles, and

expectations. Absent continuous effort from professors to develop relevant instructional

strategies, over time, disconnects between teachers and learners can develop. With that in mind,

we determined that it is worthwhile to develop a curriculum that would help university teachers

3

to become more effective using online tools, for both face to face and online learning

environments.

Once our group agreed on our higher education topic, our next challenge was to choose

an appropriate theoretical model to develop the project. In selecting from four possible choices,

ADDIE, Dick and Carey, Kemp, and Backward Design, the group chose the ADDIE model.

Here’s why:

University teachers tend to have small amounts discretionary time, so we concluded that

simplicity is desirable. The ADDIE model is recognized as an appropriate model for adult

learners. In considering that our proposed course is voluntary and work related, it appears that a

compact direct model is very effective for such training. ADDIE provides for formative

evaluation at each phase, providing an opportunity for adjustment as we build each step, and this

is also a model that has been shown to work well for online training, which is the approach we

have decided to take for our adult learners (instructors at Duquesne). In addition, we believe that

using the ADDIE design model will allow us to meet AECT Standard 1, Element 1,

Collaborative Practice “Candidates will collaborate with their peers and subject matter experts to

analyze learners, develop and design instruction and evaluate its impact on learners.”

4

Analysis

Introduction

The analysis phase must be conducted before beginning to design and develop training

and instructions in order to understand the need of the university and the learners. The main goal

of the analysis phase is “to identify the probable causes for a performance gap” (Branch, 2009, p.

23). We will follow the procedures that (Branch, 2009, p.24) has listed:

1. Validate the Performance Gap,

2. Determine Instructional Goals,

3. Analyze Learners,

4. Audit Available resources,

5. Recommend Potential Delivery System,

6. Compose a Project Management Plan.

We will then decide if the in planned instruction will solve the problem in order to move on to

the next phase of this project.

Problem statement

Many instructors at the university level do not recognize the benefit of using online

communication tools to provide another modality of connecting with students. However, even

5

for instructors teaching face-to-face, online communication tools can increase student

engagement and learning.

This course is intended to bridge the gap between the current level of instructors’

knowledge, skills and confidence in using technology (the actual performance), and the level

they have to achieve to teach effectively in both face to face and online instruction (the desired

performance).

Learning outcome

The main learning outcome of this course is for instructors to learn to use online

communication tools effectively in both face-to-face and online instruction in order to enhance

interaction and engagement between students themselves and between the student and the

instructor.

Goals and objectives

By the end of this course, learners will be able to:

● Create voice emails and announcements to effectively communicate with students.

● Use the assignment feedback tools in Blackboard to effectively provide feedback on

course performance to students.

● Create discussion boards and voice boards to engage in conversation with students.

● Use Collaborate to effectively communicate with students.

Instructional analysis

6

The following table illustrates the subordinate skills required for each of the stated tasks.

Tasks Subordinate skills

Announcements and Voice Emails 1. Create an announcement

2. Record Voice Email

Discussion Board and Voice Board 1. Create forum

2. Create thread

Assignment Feedback Tools 1. Leave general feedback using Blackboard’s grading function

2. Use document commenting feature with Crocodoc

Collaborate 1. Set up a Collaborate Room

2. Upload a presentation to Collaborate

3. Use the communication functions in Collaborate (chat, video, audio).

Learners analysis

There will be 10 university Instructors from different schools and departments taking this

course. They are voluntarily taking this course. Two instructors will be from the ESL department

in the School of Education. Two instructors will be from the Public History department in the

school of Liberal Arts. Three instructors will be from the Accounting department in the Business

School. Two instructors will be from the Leadership department in the school of Leadership and

7

Professional Advancements. The last instructor is from the Child Psychology department in the

School of Education. They range from beginning instructors to experienced.

The instructors have different backgrounds and skills in terms of using technology in

classroom. To be specific, they all have basic technology knowledge including how to use

Internet to search for information, how to email, and how to operate a personal computer. They

will have completed the Blackboard certification offered through Duquesne, but they will not be

very experienced in using Blackboard tools for their courses.

Since this course is voluntary, we expect that the learners will have a higher level of

motivation than those taking compulsory training. By considering the characteristics of learners,

we are meeting the AECT Standard 3, Element 6, Diversity of Learners “Candidates facilitate a

learning community that empowers learners with diverse backgrounds, characteristics, and

abilities.”

Context analysis

The course will be delivered online using Blackboard, a Learning Management System.

The lessons will be self-guided, but students will be expected to complete each lesson each

week. The training course is divided into 4 lessons and each lesson will take about an hour to

complete. Students must have computers, Internet access, and a valid multi-pass account to

access Blackboard and the learning module. By using appropriate tools and resources for the

needs of adult learners, we are meeting AECT Standard 1, Element 2, Using “Candidates

demonstrate the ability to select and use technological resources and processes to support student

learning and to enhance their pedagogy.”

8

9

Project management Gantt chart

Phase Task leaders Date

Analysis Instructional designers November, 15

Design of instruction Instructional designers November, 18

Development Instructional designers November, 21

Formative evaluation Instructional designers/ Faculty

November, 26

Implementation Instructional designers/ Faculty

November, 30

Summative evaluation Instructional designers/ Faculty

December, 3

Faculty = three of the ten university instructors (the learners)

10

Design of instruction

Introduction

This phase comes after the analysis. It is developed “to verify the desired performance

and the appropriate testing methods” (Branch, 2009, p. 59) that would allow the instructional

designers to determine if learners have achieved the desired skills. It aims to prepare a plan to

close the performance gap caused by the lack of knowledge and skills. We will follow the

procedures that (Branch, 2009, p.60) has listed:

1. Conduct a task inventory,

2. Compose performance objectives,

3. Generate testing strategies.

Course description

This course is a 4-week introduction to using Blackboard communication tools. The

intended audience is university instructors. Learners will be asked to watch demonstrations,

practice the skills learned, and reflect on how they might use the tools introduced in the course.

Each lesson will increase in complexity, building on the learners’ schemas (Piaget, 1962)

regarding online communication tools so that they develop their skills and understanding of how

to use those skills. By scaffolding (as developed by Vygotsky) through modeling, learners will

be able to transfer these skills to their own courses.

Course objectives

11

The main learning outcome of this course is for instructors to learn to use online

communication tools effectively in both face-to-face and online instruction in order to enhance

interaction and engagement between students themselves and between the student and the

instructor.

By the end of this course, learners will be able to:

● Use voice emails and announcements to effectively communicate with students.

● Use the assignment feedback tools in Blackboard to effectively provide feedback on

course performance to students.

● Use discussion boards and voice boards to engage in conversation with students.

● Use Collaborate to effectively communicate with students.

Outline of lessons and units

Week 1 / Lesson 1: Announcements and Voice Emails

Lesson objective: By the end of the lesson, learners will be able to create voice emails and

announcements to effectively communicate with students.

Unit 1: Announcements

This unit will introduce students to creating announcements on Blackboard, including setting

dates and sending the announcement as an email.

12

Unit 2: Voice Emails

This unit will cover recording a voice email to be sent to students.

Week 2/ Lesson 2: Discussion Board and Voice Board

13

Lesson objective: By the end of the lesson, learners will be able to create discussion boards and

voice boards to engage in conversation with students.

Unit 1: Discussion Boards

This unit will cover creating discussion board forums and threads to communicate with students.

Unit 2: Voice Boards

This unit will address creating a voice board and posting to the voice board.

Week 3/ Lesson 3: Assignment Feedback Tools

Lesson objective: By the end of the lesson, learners will be able to use the assignment feedback

tools in Blackboard to effectively provide feedback on course performance to students.

Unit 1: Grading and General Feedback

This unit will address providing feedback for students in the grading feedback box.

14

Unit 2: Document annotations with Crocodoc

This unit will cover providing in-line annotations with Crocodoc for more thorough and specific

feedback to students.

15

In-line grading of assignments. Image courtesy of Blackboard (2013).

Week 4/ Lesson 4: Collaborate

16

Lesson objective: By the end of the lesson, learners will be able to use Collaborate to effectively

communicate with students.

Unit 1: Create a Collaborate Room

This unit will show students how to create a room in Blackboard Collaborate, including how to

make the room available.

Unit 2: Upload a presentation to Collaborate

This unit will show learners how to upload a presentation to Collaborate.

Unit 3: Use Collaborate Communication Tools

17

This unit will demonstrate how to use the audio, video, and chat functions in collaborate.

Summary of course lesson and objectives

Lessons Objectives

By the end of the lesson, learners will be able to ...

Units

Announcements and Voice Emails

create voice emails and announcements to effectively communicate with students.

1. Announcements

2. Voice Emails

Discussion Board and Voice Board

create discussion boards and voice boards to engage in conversation with students.

1. Discussion Boards

2. Voice Boards

Assignment Feedback Tools use the assignment feedback tools in Blackboard to effectively provide feedback on course performance to students.

1. Grading and General Feedback

2. Document annotations with Crocodoc

18

Collaborate use Collaborate to effectively communicate with students.

1. Create a collaborate Room

2. Upload a presentation to Collaborate

3. Use Collaborate Communication Tools

Instructional strategy

This course is being delivered online and is based on Dick and Carey’s Chronological

Sequence of Instruction (Dick, Carey & Carey, 2015). Blackboard is used to design the course

content and deliver the learning materials.

A. Pre-instructional activities

1. Gain attention and motivate learners: Learners will be informed of the benefits and

applicability of what they learn to their courses. University faculty at Duquesne wish to serve

their students the best they can, so making the benefits of Blackboard communication tools clear

to the learners will motivate them and gain their attention.

2. Describe objectives: Letting students know the objectives of the lesson allows them to

better construct mental models of what they are learning.

3. Describe and promote recall of prerequisite skills: Recall from long-term memory will

help learners in dealing with more complex information that builds upon prior skills.

B. Content presentation

19

1. Content: Content will typically be presented as screenshots, interactive PowerPoints,

and screencasting videos. Lessons will be presented in a variety of modalities to better reach a

diversity of learners.

2. Learning guidance : Learners will be directed to attend to particular aspects of each

lesson through guiding questions. Content will be segmented so students may proceed at their

own pace (Mayer, 2005).

C. Learner participation

1. Practice: Learners will be asked to apply what they have learned in brief assignments

where they practice using Blackboard communication tools.

2. Feedback: Learners will reflect on what they have learned, and instructors will respond

to their questions and provide feedback on their ideas.

D. Assessment

1. Entry skills test: Learners will be asked to complete a brief skills test that covers basic

information about Blackboard.

2. Pretest: Before instruction, learners will complete a pretest to determine their prior

understanding and knowledge of the course content.

3. Posttest: After instruction, learners will complete a posttest on the objectives of the

course to determine mastery of the course concepts.

E. Follow-through activities

20

1. Memory aids for retention: These will include printable handouts for learners to take

with them in other learning environments as needed.

2. Transfer considerations - Learners will be expected to apply their knowledge from

each lesson to their own courses on Blackboard.

(Dick, Carey, & Carey, 2015, pp. 182-183)

Criteria for evaluating students performance

Formative Assessment

The main purpose of this assessment is to provide learners with feedback to enhance their

knowledge and skills. Discussion board questions will ask students to reflect on what they have

learned after each lesson. One-to-one and a small-group questionnaires are designed to evaluate

and revise the course in order to collect data before implementation. There is no formal

evaluation in this training course in order to successfully complete it.

Summative Assessment

A summative evaluation will be available for learners to do if they choose to. The process

aims to collect data after implementation. Learners will be asked to use each tool in a final

activity.

21

Development

Introduction

The development phase is based on the materials gathered during the analysis and design

phases. The main goal of this phase is “to generate and validate selected learning resources” (p.

84). In this phase, we will create learning resources for the planned training course. This phase

will provide insights into the curriculum by focusing on a single lesson from the course (Branch,

2009). We will follow the procedures that (Branch, 2009, p.84) has listed:

1. Generate content,

2. Select or develop supporting media,

3. Develop guidance for the student,

4. Develop guidance for the teacher.

After completing this phase, we will conduct formative evaluation of the instructional content

allows improvements to the design before the instruction is implemented on a large scale.

Sample lesson: Lesson 1 - Announcements and Voice Emails

The lesson we choose includes the learning objectives, AECT standards, learning

strategy, and assessment. The duration of this lesson is no more than an hour. The lesson is

delivered online via the Learning Management System (i.e. Blackboard).

Learning objectives

22

By the end of the lesson, learners will be able to create voice emails and announcements to

effectively communicate with students.

AECT standards

Standard 1, Element 1, Creating “Candidates demonstrate the ability to create instructional

materials and learning environments using a variety of systems approaches.”

● Supported with the instructional materials created and the effective use of the Blackboard

environment to support training.

Standard 3, Element 6, Diversity of Learners: “Candidates facilitate a learning community that

empowers learners with diverse backgrounds, characteristics, and abilities.”

● Supported with multimodal presentations and transcripts of videos for learners with

diverse abilities, skill levels, and learning preferences.

Learning strategy

Pre-instructional Activities: Learners will receive brief training on how to use the Blackboard

course site and how to navigate through the lesson. The learning outcomes for the lesson will be

supplied, and learners will be informed of some benefits of using announcements and voice

emails.

Content presentation: Learners will watch two instructional videos with a transcript for the

hearing impaired, in accordance with AECT Standard 3, Element 6. The first video will

demonstrate how to create an announcement on Blackboard using screencasting software. The

second video will demonstrate how to create a voice email in Blackboard. These will be brief

23

and will adhere to Mayer’s (2005) multimedia principles. These will also be available as

screenshots that show step-by-step how students can create an announcement and voice email.

Learner participation: Statistics tracking will track the learners who watch each video. Learners

will also be expected to engage in a discussion board reflection about how they could use

announcements and voice emails effectively in their course, and they will need to respond to

another learner on the discussion board as well.

Assessment: Formative assessment will occur as students reflect on and ask questions about the

lesson content. Instructors will monitor this discussion and intervene whenever a learner does not

understand or needs more information. At the end of the lesson, learners will send a voice email

to the class, which will ensure that students can complete the objectives of this lesson.

Discussion Board Forums for this lesson:

Assignment for this lesson:

Send a Voice Email to the entire class (students and instructors). Introduce yourself and why you

are taking this course in the message. Your message should be one to two minutes long.

Follow-through activities: Learners will be expected to make an announcement in Blackboard in

their Duquesne course sites so that they are applying what they learn in an authentic setting.

24

25

Formative evaluation

Introduction

Formative evaluation of the instructional content allows improvements to the design

before the instruction is implemented on a large scale. This will be in support of AECT Standard

1, Element 3 Assessing/Evaluation “Candidates demonstrate the ability to assess the effective

integration of appropriate technologies and instructional materials,” AECT Standard 3, Element

3 Assessing/Evaluating “Candidates use multiple assessment strategies to collect data for

informing decisions to improve instructional practice and student results,” and AECT Standard 4,

Element 4 Assessing/Evaluation “Candidates design and implement assessment and evaluation

plans that align to learning goals and instructional activities.”

Instructional lesson to be formatively evaluated

The lesson that will be formatively evaluated is Lesson 1: Announcements and Voice

Emails. By the end of the lesson, learners will be able to create voice emails and announcements

to effectively communicate with students.

Formative assessment goals

We wish to make sure that

1. Materials are clear and understandable by the learners.

2. The content we have included in the course supports the learning outcomes.

26

3. Subordinate skills for each lesson are organized logically and support the learning

outcomes.

4. The materials are valuable to the learners and their instructional needs.

Self-review trial

We plan to conduct a self-review trial for the instructional materials. We will be

reviewing the course materials and the learning resources as first-time users. When reviewing the

planned course, it is important to consider the purpose and the goals of the formative assessment

and see if they meet our expectations. In self-review trial, we will evaluate, analyze, examine

errors in the instructional materials.

One-to-one trial

In one-to-one trials, three learners will be selected to complete the lesson being evaluated

and to discuss their experience with the instructional designers. This allows designers to test the

clarity, impact, and feasibility of the instruction (Dick, Carey, & Carey, 2015, p. 288). The

learners will be selected based on their entry-skills scores and pre-test scores so that there are

learners who have below, average, and above average skills. Learners will complete the lesson

while discussing issues of clarity, difficulties in navigation, and errors in the instructional

materials with the instructional designers. After this, they will complete the posttest as well as an

evaluation on their affective response to the instruction. Instructional designers will ask them to

explain their responses on the posttest to determine any performance gaps that still exist, as well

as any issues with the posttest design.

Small group trial

27

After the one-to-one evaluation, changes will be made to the instructional materials.

Students will complete the lesson as they are intended - as self-directed instruction. Eight

learners will be chosen based on their pre-test scores to have a heterogenous group.

Data collection, analysis and reporting for all formative assessment activities

Pretests, posttests, and follow-up questionnaires will be used in the one-to-one and small

group trial. Quantitative and qualitative data will be reported to improve the instructional

materials.

Sample report outline

The report will be presented in the following format:

I. Introduction

II. Purposes of formative evaluation

a. Evaluation objectives

b. Description of sample lesson

III. Evaluation Methodology

a. Participants in one-to-one trial and small group trial (including participant selection information)

b. Instruments (qualitative interview protocols, pretest and posttest, student attitude surveys)

IV. Results

a. Analysis of data

b. Overall findings

V. Conclusions and recommendations for revising the lesson

28

Implementation

Introduction

The initial implementation will occur after formative evaluation, but ADDIE allows for

an iterative process where revisions will be necessary after a summative evaluation of the course.

Within this phase, the instructional designers will be supporting and administering the

instruction.

Implementation process

The instructional designers will be facilitating the course in support of AECT Standard 2,

Element 4: Management “Candidates will manage appropriate technological processes and

resources to provide supportive learning communities, create flexible and diverse learning

environments, and develop and demonstrate content pedagogy.“ The process of facilitation will

be as follows:

1. Testing of course software and launching of the course site within Blackboard.

2. Enroll interested university faculty into the course.

3. Test links, videos, and Blackboard tests throughout the duration of the course to make sure these

are functioning properly.

4. Facilitate the course by monitoring discussion boards, providing additional materials where

needed, and providing feedback on assignments.

29

Evaluation: Summative evaluation

Introduction

The summative evaluation is the final phase in this project. It aims to evaluate and assess

the quality of the instruction after implementation. In this phase, we will determine what need to

be evaluated and then select the appropriate instrument (Branch, 2009). The summative

evaluation will be conducted by members of design team and external content experts who are

familiar with the online instruction.

According to Dick, Carey, and Carey (2015), a summative evaluation has two main

phases: expert judgment and impact analysis. Combining the features of this course with the

nature of online instruction, the summative assessment will focus on two aspects respectively:

1. By evaluating the completeness and accuracy of course materials and the instructional strategy

contained in the course design in the expert judgment phase, evaluators will determine whether

the course meets the defined instructional needs and goals.

2. In the impact analysis, evaluators will review the effect of the instruction on the learner and

their performance through outcome analysis as well as assess instructor and student attitudes.

The specific approaches is as follows:

A. Content Analysis

During the analysis phase in this project, we have stated that our course will facilitate the

connection between instructors and their students by using online communication tools.

30

Therefore, at the end of implementation, we will judge if the content of our course is accurate

and complete. First, we will engage a content expert as an evaluator in our team to produce a

document with the analysis of instructional goal and subordinate skills. Then, by checking the

result of analysis, evaluators will review and judge how about the accuracy and completeness of

the instruction materials, tests, and sequence of information. Based on data analysis, we should

answer the questions as following:

How clear are the goal and the main objectives of this course?

How accurate and current is the information contained in this course?

How logical is the sequence of information and activities in this course?

Are course assessments congruent with the goals and objectives in this course?

B. Design Analysis

Quality instruction should gain the attention of the learner (Dick, Carey, & Carey, 2015).

For the design of this course, the summative evaluation will focus on the assessment of

motivation and the instructional strategy.

First, we use the ARCS model (Keller, 1987) to evaluate the potential of this course for

motivating learners and learners’ perceptions of how interested they were in learning the online

communication tools.

Summative Evaluation Based on Principles of Motivation

Instructions: Please rate your level of agreement with the following questions on a scale from 1 to 5, where 1= strongly disagree; 2= disagree; 3 = neutral; 4 = agree; 6= strongly agree.

ARCS Motivation Areas of Concern Evaluator Evaluator

31

Model 1 2

Attention Do the strategies used to gain and maintain the learners’ attention accomplish the goal?

Relevance Is the course relevant for the skills of online communication tools?

Are the learners sufficiently convinced of this relevancy?

Confidence Will the learners likely be confident at the outset and throughout the instruction so that they can succeed?

Are the learners easily able to transition from the course into the performance context?

Satisfaction Are the learners likely to be satisfied from the learning experience?

Additional Comments:

Second, a checklist for evaluating the instructional strategies may be presented in the

summative evaluation. In order to judge the presence of absence of each strategy component,

evaluators should list all learning components and then to check if all of them exist in the course

material. Also, evaluators can mark each strategy component to assess the perceived

motivational value of them for the intended learners.

The checklist for evaluating the learning components of the instructional strategies appears

below:

32

Learning Components

Pre-instructional

A. Initial motivationB. ObjectivesC. Entry skills

Information presentation

A. Organizational structures

● Headings

● Tables and illustrations

B. Elaborations

● Analogies/synonyms

● Prompts to imagine/consider

● Examples and non-examples

● Relevant characteristics of examples

● Summaries/reviews

Learner participation

A. Relevant practiceB. Feedback

● Answers

● Example solutions

● Common errors and mistakes

Follow-Through Activities

A. Memory aidsB. Transfer strategy

This checklist is from Dick, Carey, & Carey (2015, p. 360).

C. Instructional effectiveness

33

The first component of field trial phase in the summative, outcomes analysis involves

determining the effect of instruction on learners’ skills (Dick, Carey, & Carey, 2015). A

questionnaire, Duquesne’s Student Evaluation Survey 2.0 (2013), for learners will be given to

evaluate the effectiveness of the instruction. The questionnaire will use a 5-point Likert scale as

follow: 5=strongly agree, 4=agree, 3=neutral, 2=disagree, 1=strongly disagree. The survey will

be available online using Blue, Duquesne’s survey system, before the end of the course. The

table below shows the statements from Duquesne University’s SES 2.0 (2013). Once completed,

we will fill out the mean score and the comparison with the school mean score.

Questions Mean Score

School Mean Score

The instructor helped me to understand the material in this course.

The instructor was enthusiastic about teaching.

The assignments were helpful in acquiring a better understanding of course objectives.

The instructor treated students with respect.

The instructor encouraged students to seek help if needed.

The instructor created a learning environment in which students felt comfortable asking questions.

The instructor made it clear how student learning would be assessed.

The instructor returned graded materials within an appropriate time frame.

34

The instructor helped me to understand the relevance of this course.

Assistance from the instructor was readily available if I sought help.

The instructor provided constructive feedback on course assignments and exams.

The instructor responded to my communications in a timely manner.

The instructor used a variety of instructional strategies.

The instructor was well prepared for the course.

The instructor stimulated my thinking.

The instructor’s explanations were clear.

Communication with the instructor was helpful.

The instructor was concerned with whether or not the students learned the material.

The objectives of the course were well explained.

D. Instructional efficiency

The determination of instructional efficiency of this course is mainly to assess if the

proposed time is adequate for completion. We will collect the data to complete this part of

summative evaluation from two ways. First, a brief survey will be conducted by the instructors

by using the table below. Second, the data will be obtained through the Blackboard system,

35

which will track the amount of time a student spends on any given learning module and

assignment. So the total amount of time invested in preparing students to complete the course

can be collected. The analysis will determine whether the time spent on the course for both

students and instructors is reasonable.

Efficiency Satisfaction Survey

Instructions: Please rate your level of satisfaction with the following aspects of the course from 1 to 5.

Time spent setting up additional course materials or making assignments available in Blackboard.

Little 1 2 3 4 5 Very satisfied

Time required to prepare for instruction. Little 1 2 3 4 5 Very satisfied

Time spent answering student questions. Little 1 2 3 4 5 Very satisfied

Time required to assess students’ participation in the course.

Little 1 2 3 4 5 Very satisfied

Time spent corresponding with Blackboard and technical staff.

Little 1 2 3 4 5 Very satisfied

E. Benefits of instruction

To determine the benefits of this course is an assessment related to whether the course

has achieved the all instructional objectives. Our course is intended to bridge the gap between the

current level of instructors’ knowledge, skills and confidence in using technology (the actual

performance), and the level they have to achieve to teach effectively in both face to face and

online instruction (the desired performance). We will analyze the actual performance as well as

the desired performance by evaluating participation data from Blackboard system and using the

36

post-instructional learner survey. The table of survey can be found below. Students will also be

asked to complete a final assignment where they use multiple tools from the course.

In addition, formative assessment in this course may be considered to determine if the

learners obtain the essential knowledge and skills, and as a reference material to analyze the gaps

between actual and desired learner performance.

Post-Instructional Learner Survey

Please indicate your level of agreement with the following statements on a scale of 1-5, where 5= strongly agree; 4= agree; 3 = neutral; 2 = disagree; 1= strongly disagree. You may elaborate on your responses by adding a comment.

Statement Rating Additional Comments

1. I understand the purpose of this course.

2. I understand the benefits of using online communication tools in my course.

3. I feel that I can navigate to the online communication tools learned in this course on my own.

4. I am able to feedback on course performance to students through the online communication tools.

5. I understand the usage of different types of online communication tools provided in this course.

6. I feel confident that I can communicate effectively with my students by using the online communication tools.

7. I believe this course was a good use of my time.

8. I plan to use the online communication tools in my own courses.

37

9. I feel the time I spent working on this course was sufficient for my learning.

10. I believe the instructional material provided in this course will make my experience taking the skills of using online communication tools easier.

Sample Summative evaluation report

The instructional designers will write the final report by using the data collected from the survey.

The report will be presented in the following format:

I. Abstract

II. Purposes of summative evaluation

a. Instructional effectiveness

b. Instructional efficiency

c. Benefits of instruction

III. Evaluation Methodology

a. Participants

b. Instruments

IV. Results

a. Analysis of data

b. Overall findings

V. Conclusions and recommendations

Completing this summative assessment will allow us to determine whether we should continue

with this course, or whether it must be modified or even abandoned if unsuccessful. We believe

that our careful design and consideration in using the ADDIE model, however, will ensure a

38

successful course that is beneficial to university professors and will increase appropriate use of

communication technologies in Blackboard.

39

References

AECT standards (2011). Retrieved from Duquesne University

http://www.duq.edu/Documents/instructional-technology/_pdf/AECT_Proposed_Standar

ds_6-2011.pdf

Branch, R. (2009). Instructional design: the ADDIE approach. New York: Springer.

Dick, W., Carey, L., & Carey, J. (2015). The systematic design of instruction. 8th ed. Upper

Saddle River, New Jersey: Pearson.

Inline grading for assignments (2013). Retrieved from Blackboard

https://help.blackboard.com/en-us/Learn/9.1_SP_10_and_SP_11/Instructor/

070_Assignments/003_Grading_Assignments/Inline_Grading_For_Assignments

Keller, J. M. (1987). Development and use of the ARCS model of instructional design.

Journal of instructional development, 10(3), 2-10.

Mayer, R. E. (2005). The handbook of multimedia learning. New York, NY: Cambridge

University Press.

Piaget, J. (1962). The stages of the intellectual development of the child. Bulletin of the

Menninger Clinic, 26, 120-128.

SES 2.0 (2013, December). DORI. Retrieved from Duquesne University’s website:

https://portal.duq.edu/intranet/Documents/intranet/academicaffairs/_pdf/

SES_2.0_Final.pdf

40