to use or not to use 0 running head: to use or not to use … · 2011-12-14 · “moodle” as...

52
To Use or Not to Use 0 Running head: To Use or Not to Use Measuring Resistance and Usage Intensity of Moodle based Learning Management System by Faculty and Students of the School of Business at Kwantlen Polytechnic University. Nishan Perera # 301105451 EDUC 815: Field Project Dr. Michelle Nilson Simon Fraser University July 17, 2010

Upload: others

Post on 24-May-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 0

Running head: To Use or Not to Use

Measuring Resistance and Usage Intensity of Moodle based Learning Management System

by Faculty and Students of the School of Business

at Kwantlen Polytechnic University.

Nishan Perera # 301105451

EDUC 815: Field Project

Dr. Michelle Nilson

Simon Fraser University

July 17, 2010

Page 2: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 1

Table of Contents

Abstract …………………………………………………………….. 02

Background …………………………………………………………….. 04

Research Problem …………………………………………………………….. 04

Research Questions …………………………………………………………….. 06

Objectives of the Study …………………………………………………………….. 06

Significance of the Study …………………………………………………………….. 06

Scope of the Study …………………………………………………………….. 07

Literature Review …………………………………………………………….. 08

1) Literature on Learning Management Systems

2) Literature on Resistance Factors Associated in using Technology

3) Literature on Intensity of Use of Learning Management Systems

Methods and Procedures …………………………………………………………….. 15

1) Conceptualization

2) Variables and Concepts

3) Hypotheses

4) Operationalization

5) Methodology

Limitations of the Study …………………………………………………………….. 20

Findings …………………………………………………………….. 20

1) User Profiles/Technology Use

2) Faculty Perspective

3) Student Perspective

4) Faculty vs. Students

Discussion and Implications …………………………………………………………….. 25

References …………………………………………………………….. 29

Appendices

Appendix I - Faculty Questionnaire …………………………………….. 32

Appendix II - Student Questionnaire …………………………………….. 36

Appendix III - Profile/Use of Technology ……………………………….. 40

Appendix IV - Faculty Findings …………………………………….. 41

Appendix V - Student Findings …………………………………….. 46

Page 3: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 2

Abstract

In 2006, Kwantlen Polytechnic University (KPU) after a careful in-depth study introduced

“Moodle” as their sole learning management system. It was envisaged that Moodle will be used by

faculty and students within the University to enhance teaching and learning. Three years later,

statistics indicated that , only 39% of the faculty members of the School of Business used Moodle

for teaching purposes. The problem under investigation revolves around understanding reasons for

the lower rate of usage of Moodle for teaching and learning by faculty and students in the School of

Business. Three research questions were raised in taking this study forward. The first was to

understand the nature of the relationship between resistance to use and the usage intensity of

Moodle based learning management system by faculty members, the second was to identify the

nature of the relationship between resistance to use and the usage intensity of Moodle based

learning management system by students and the third was to compare student experience in using

Moodle based learning management systems with faculty.

In reviewing literature on resistance and usage intensity of learning management systems

(LMS), Michelle and May (2009) identified four types of resistance categories that faculty members

typically depict in using technology. As for student’s point of view, Abbad et al (2009) highlights

perceived usefulness, perceived ease of use and other external factors that may lead to resistance of

use. There were many other models that were reviewed. However, these two models were used in

this study to measure resistance by faculty and students respectively. Janossy & Hover (2008)

model on evaluation of LMS identifies five types of usage intensities. This model was used as a

base to measure usage intensity of Moodle by faculty and students in the School of Business.

Page 4: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 3

In designing the study, the positive and negative relationships between resistance and intensity

between faculty and students were conceptualized. Four hypotheses were developed to test positive

and a negative relationship for each group. In collecting data, two questionnaires were developed,

one for faculty members and one for students. A non random sampling technique was used. 50% of

the faculty members (80 faculty) and 15% of the students (465 students) in the School of Business

were surveyed.

In analyzing data, correlation between resistance and usage intensity was tested. A one tail test

was carried out to test the hypotheses at a 95% confidence level. In answering the first research

question, as for faculty, there was a strong negative correlation between resistance and usage

intensity of Moodle. In the hypothesis testing this was verified by rejecting the null hypothesis. The

biggest contributor for faculty resistance was attributed from resistance to change the current

practices of teaching. In the qualitative data analysis, perceptions of excessive time that would be

required to use a LMS were cited as main reasons for this resistance. In answering the second

research question, as for students, a positive correlation between resistance and usage intensity of

Moodle was identified. In the Hypothesis testing this relationship was verified by rejecting the null

hypothesis at a 95% confidence level. The overall resistance to use Moodle was very much less. In

the qualitative analysis, students cited the lack of technical support and system issues as reasons for

their non use, if any. The third research question was to have a comparison between the two. In the

analysis of the usage and profile information, there was little difference between the two except for

age, time spent on internet, resistance and intensity levels.

This study would be useful to inform practice in many ways. The institution spends more

resources on providing technical support but none in terms of changing attitudes towards use of

LMS to enhance teaching and learning. This study clearly indicates the need to change the current

Page 5: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 4

professional development strategy. It informs policy in terms of setting guidelines of expectations

from faculty and students in using technology for teaching and learning purposes. It informs theory

in many different ways. Although the discussion of resistance to use technology has been

documented since 1920 (Cuban, 1986), there is not enough emphasis on methods and procedures on

changing faculty attitudes towards use of technology. This would be an area that future research

could address in filling this knowledge gap.

Background

In 2006, Kwantlen Polytechnic University (KPU) after a careful in-depth study introduced

“Moodle” as their sole learning management system replacing its own lotus notes based system. The

taskforce that was put together identified that Moodle would best support Kwantlen’s culture of

delivering applied courses to its students and extending learning beyond the classroom. It was

envisaged that Moodle will be used by a majority of faculty members within the University in

achieving this objective.

Research Problem

Three years after introducing Moodle, while some have embraced this, the majority of

faculty members do not seem to use Moodle to support their teaching. Presently 39% of the faculty

members of the School of Business use Moodle to teach face to face, online and partially online

courses. The breakdown of these statistics is presented in table 1. The problem under investigation

revolves around understanding reasons for the lower rate of usage of Moodle for teaching and

learning within the School of Business. Michelle and May (2009) states in their study that while in

recent years many institutions have embraced online learning, the increase in faculty acceptance of

technology has been lagging behind. In the same study, these authors identify four types of

resistance categories that faculty members typically depict in using technology. As for the student’s

point of view, Valenta (2001) identified several forms of factors that may lead to resistance by

Page 6: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 5

students in using technology. Abbad et al (2009) identifies several categories that students would

typically follow in adopting technology, based on the technology acceptance model developed by

Davis (1986).

Table 1

Total Courses Offered Vs. Moodle Usage

Department Courses Offered Moodle Course %

CBSY 32 27 84%

CISY 7 7 100%

INFO 4 4 100%

ACCT 96 32 33%

BUQU 34 0 0%

ECON 41 15 37%

CMNS 28 19 68%

CPSC 16 12 75%

ENTR 28 15 54%

FNSR 1 0 0%

HRMT 11 5 45%

BUSI 62 29 47%

LFLA 8 0 0%

MRKT 40 14 35%

Totals 365 141 39%

Source : Moodle Statistics in Kwantlen Polytechnic University

https://elearning.Kwantlen.ca/courses/course/index.php

In order to narrow down the problem, the study would attempt to understand to what extent

resistance to use Moodle, by faculty and students, will impact the intensity of its usage. While

acknowledging that there could be many other factors that would lead to a lower usage rate, the

author is interested to identify to what extent resistance affects the usage and if so what types of

resistance factors have a bigger impact.

Page 7: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 6

Research Questions

The following research questions are framed to address the identified research problem.

1. What is the nature of the relationship between resistance to use and the usage intensity of

Moodle based learning management system by faculty members within the School of

Business at KPU?

2. What is the nature of the relationship between resistance to use and the usage intensity of

Moodle based learning management system by students within the School of Business at

KPU?

3. How does the student experience in using Moodle based learning management systems

compare with faculty usage?

Objectives of the Study

Based on the above research questions, the purpose of this study is to identify whether there

is a relationship between resistance and usage intensity of Moodle learning management systems by

faculty members and students within the School of Business. The study will also attempt to identify

the nature of relationship that exists between these variables and will highlight how the findings will

inform policy, practice and theory associated in using learning management systems for teaching

and learning purposes.

Significance of this Study

In taking this study forward, a question that could be raised is, why the institution should

emphasis the use of technology for teaching purposes? The answer lies in how technology would

impact student learning. Kimble (1999) in reviewing literature that highlights the impact technology

on learning, points to a series of studies that have shown the positive impact of technology on

learning while acknowledging that there are also other studies to highlight the opposite. Schacter

Page 8: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 7

(1999) synthesizes the findings of several meta analysis studies on the impact of technology on

student achievement, and concludes that, on average students who use technology perform better in

assessments, students tend to learn more in less time, instructors can extend the discussions beyond

the classroom, technology supports interaction and student participation, leading to a higher degree

of learning etc. As for the use of learning management systems for teaching, and in particular

Moodle as a platform, the study would identify whether this is a cause for any resistance.

Cuban (1986) in his study of documenting the classroom use of technology since 1920,

highlights that from the early days when institutions were attempting to use films, radio, television

and eventually computers in the classroom for teaching purposes, teachers showed resistance

towards using them for their teaching purposes. While some of the resistance typologies identified

in the current study may be old, at the turn of the 21st century, understanding to what degree the

resistance to use technology (and different versions of it) have on usage intensity would be very

useful for policy makers to initiate strategy to manage it. It will also have a profound impact on the

development of professional development initiatives within organizations to address this resistance.

Currently, the professional development initiatives at Kwantlen are mostly directed towards

providing technical support to implement learning management systems for teaching. If this study

directs that resistance to use technology due to several typologies of resistance factors, then specific

professional development and support initiatives could be further identified to address these

concerns.

Scope of the Study

The scope of the study is limited to the School of Business in KPU for manageability

purposes. It is further limited in identifying the faculty and student’s perspective in using Moodle,

leaving out other stakeholders that use Moodle for other administrative reasons. There could also be

many other factors that could be associated with usage intensity of Moodle for teaching and

Page 9: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 8

learning. The study is limited only to identifying how resistance factors would impact the intensity

of usage of Moodle within the defined faculty of KPU.

Literature Review

The literature review will be carried out in several segments. The first section will introduce

learning management systems in general, its benefits and different types available. The second part

of the review will present various factors identified in literature that leads to resistance by faculty

and students in using technology and learning managements systems. Factors associated with

faculty members and students are identified separately. The third component of the review will

introduce different models found in literature that would be useful to understand the usage intensity

of a learning management system.

1) Learning Management Systems

This section will introduce learning management systems (LMS), best practices and issues

related to the adoption of LMS within a higher education context. While there are many definitions

associated with learning management systems, Dehindo and Odunaike (2008), defines LMS as “a

software program or integrated platform that contains a series of web based tools to support a

number of activities and course management procedures”. The use of web based learning

management systems used for instruction is on the rise due to many features that they provide to

support teaching and learning. LMS could be used for a fully online class, to deliver a mix

mode/blended learning sessions or in a full face to face classroom for different pedagogical reasons.

Bouhnik (2008) et.al, as quoted in Dehindo and Odunaike (2008), has identified several distinctive

benefits of using LMS for both learners and instructors. The freedom to decide when the lesson is to

be learnt, lack of dependence on the instructor, the freedom for learners to express thoughts and the

accessibility of course materials were cited as some key advantages. On a primary level, LMS

could be used as an administrative tool to manage a course and at a higher level as it could be used

Page 10: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 9

as an advantage teaching tool with a lot of pedagogical value supporting faculty to allow different

areas of content, teaching philosophies and instructional styles.

There are many types of LMS. They are Web Course Tools (WebCT), Web Course Home

Page (WebCH), Blackboard Learning Management Systems, System for Multimedia Integrated

Learning (SMILE), Moodle, Desire2Learn, Angel and many other platforms. While each of these

systems have their own differences, many are built to support similar usage needs. Feldstein (2006),

as quoted in Black (2007) et. al, states that despite the fact there are many learning management

systems and each of them have some form of a better performing basic functionality, they all have

one thing in common; the offer of a one size fill all option.

Moodle based LMS is the center of this study. Moodle was originally an acronym for

Modular Object-Oriented Dynamic Learning Environment. Moodle was developed to encourage

interaction between students and to improve learning through the construction of ideas and concrete

“things” for others to experience. Moodle is an open source project, which means the Moodle

course management software is free to download. Furthermore, the Moodle open source project has

a community of supporters and developers who are instrumental in Moodles continued growth.

(Moodle Org’ 2010)

2) Resistance Factors Associated in Using Technology

Various literature that has highlighted the resistance and adaptation related factors associated

with using technology/ learning management systems/online teaching will be reviewed below. They

are presented as separate sections, as factors affecting faculty resistance and factors affecting

student resistance below.

Page 11: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 10

a) Faculty Members

Cuban (1986), in his synthesis of the use of classroom technology highlights various factors

associated with faculty resistance in using technology over the last 90 years. Although the use of

radio, film, television, machines and eventually the computer evolved over a period of time, the

types of resistance factors that faculty generally demonstrated remained the same. Some of those

factors were skill issues, accessibility issues, willingness to change issues and also the lack of

understanding in using the technology for teaching purposes.

The study carried out by Mitchell & May (2009) in identifying attitudes affecting online

learning implementation in higher education institutions, categorizes the faculty resistance to use

technology into four main themes. They are

• Intellectual reluctance: perceptions about the degree to which online learning is

consistent with their professional values and norms;

• Support: perceptions that their efforts are valued by the institution and that there is

general and specific support;

• Change: perceptions of degree of instability caused by changes in their institution, and to

their job; and

• Cost-benefit: perceptions of online learning as benefits outweighing costs.

This study clearly identifies the main categories to cluster broader themes of resistance

factors.

Walker (2005) presents several factors of resistance based on the framework used in the technology

acceptance model originally presented by Davis (1986). These factors relate to faculty intensions in

using web based instructional components for teaching purposes. These resistance factors include

Page 12: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 11

• Usefulness: This relates to the faculty perceptions on the usefulness of the technology

component in achieving the set learning objective.

• Computer background: The ability of the instructor to use the equipment and programs

associated in using the technology.

• Organizational support: provided by the institution, department as well as from peers in

using technology for teaching.

• Ease of use: how user friendly the technology is to operate in a practical manner?

Goodson (2005) categorizes faculty resistance into two main typologies in using e-learning

based technologies. They are

• Type # 1 resistance factors: These include defensiveness shown by faculty members

based on fear, ignorance and unawareness in using technology. They mainly express

their defensiveness by being unwilling to examine assumptions that underlie behaviours

or fairly tested assumptions associated in using such technologies with others who have

different views about effective practice. He characterizes these resistance factors as

being stubborn and blocking new initiatives related to technology use.

• Type # 2 resistance factors: According to Goodson these would be defensiveness that is

created based on interests of learning. Faculty members who exhibit this type of

defensiveness are more willing to test their positions only if those practices could lead to

a much better practice. This resistance is mostly associated with their perceptions of their

professional competence as teachers.

Goodson states that faculty members who show type # 1 resistance factors should be left

alone since attempting to get them to embrace technology may only create more resistance. As for

Page 13: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 12

type # 2, these faculty members’ resistance factors could be overcome by creating situations where

their learning outcomes of using technology could lead to positive results.

Moser (2007) presents the faculty e-learning behaviour process and a continuum of

behavioural factors. He suggests that if these are supported at each stage, faculty members will

adopt using technology faster and in a positive manner.

b) Students

Abbad et. al (2009) presents several factors affecting student adoption of e-learning systems.

They use the technology acceptance model (Davis, 1986) as their base theory and highlights several

important factors that students may lead to resisting the use of e-learning systems for learning

purposes. They are

• Perceived usefulness: These include how the students perceive how useful the technology

based system is in supporting their learning purposes.

• Perceived ease of use: Relates to student perceptions of how easy for them to use the

technology based system in terms of accessibility, navigation, following instructions etc.

• External factors: These include a wide variety of other factors. They are - technical support

availability, student’s internet/computer experience, subjective norms in terms of who

influences the student to use these systems, the level of interactivity with the system and self

efficacy criteria.

Valenta et al (2001) identifies positive and negative student attitudes and learning associated

with technology based education in a distance learning perspective. Some of the negative factors

that led to resistance to use as identified in their study includes

• Limitations on interactivity

• Technological problems

• Increased workload

Page 14: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 13

• Lack of logistical support (administrative and technical)

• Costs (equipment, online phone charges, etc.)

The above review would highlight the resistance factors associated with using technology

both by faculty and students at large.

3) Intensity of the use of Learning Management Systems

This segment will highlight various models that would help understand how to evaluate the

usage intensity of a LMS.

Janossy & Hover (2008) presents a proposed model for evaluating C/LMS usage. This model

identifies five levels of usage intensity of a learning management system and categories them as

follows.

• Level 0 – No C/LMS use

• Level 1 – Document distribution

• Level 2 – Work submission, feedback

• Level 3 – Online testing, asynchronous discussion, feedback

• Level 4 – Recording for review, real time distance participation

Each level of usage is further categorized into several intensity levels totaling 13 categories spread

across these 5 levels. This model is the first of its kind that presents a very clear categorization, with

a clear criterion set to establish usage intensity. This was originally developed for a WebCT based

LMS but includes provisions to apply this to any LMS. Janossy & Hover indicates that this model

will help faculty members and students to understand their current usage and would be beneficial in

many ways. Faculty could evaluate whether there is a direct impact of higher usage and the

performance of their students. This could also be used as a bench mark to further improve moving

up the usage levels and initiate professional development to further support this effort.

Page 15: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 14

Moore (1996) identifies three typologies of interactions that are applicable for distance

learning education interactions. These different interaction frameworks include

• Learner – Content Interaction – where the learner users the LMS to access content

• Learner – Instructor Interaction - this is argued as an essential component for the learner

to interact with the instructors especially in a fully online environment.

• Learner – Learner Interaction – recognized as a third important dimension to support a

more network enabled educational environment.

Tuovinen & Quinn (2003) introduces a fourth interaction typology to Moore’s (1996) model

based on resource based learning paradigm as suggested by Rowntree (1994). This fourth

interaction includes

• Instructor – Content Interaction – allowing the instructor the design and improve course

content over a period of time.

Using the above interaction framework, Tuovinen & Quinn (2003) presents five

configurations of LMS uses which are presented as a continuum as follows.

• LMS supported plus face-to-face/interactive video teaching (IVT);

• LMS supported plus print;

• LMS supported plus mixed media (CD);

• LMS supported plus mixed media (Website); and

• Fully online on LMS.

They have also provided different forms of support that needs to be provided to students at

each level of the configurations in supporting the use of the LMS.

Page 16: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 15

Further to the above models that provide insights in categorizing different usage intensity of

learning management systems, Downing et al (2007) presents yet another categorization of LMS in

an online learning environment as

• Minimal use

• Supplemental use

• Integral use

• Central use and

• Exclusive use

Kim & Lee (2008) present their version of usage intensity of learning management systems

as

• LMS for instructional management

• LMS for interaction

• LMS for evaluation

• LMS for information guidance

There are many other models presented in literature that would help identify usage intensity

further. The above models reviewed are a few but useful models that highlight different formats of

usage intensity that would help to develop a framework to evaluate the level and the intensity of

usage of a LMS used for teaching and learning purposes.

Methods and Procedures

The following sections will further highlight methods and procedures associated in finding answers

to the identified research questions.

1) Conceptualization

Based on the literature review and the research question identified, the following

conceptualization is presented to test several hypotheses.

Page 17: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 16

Figure 01

Conceptualization

2) Variables and Concepts

The independent variable would be the resistance factors in using Moodle as shown by

faculty members and students separately. As for faculty members, the four main categories

identified by Mitchell & May (2009) is used to further capture the nature of resistance. As for

students, the factors identified by Abbad et. al (2009), originally cited by Davis (1986), would be

used for further analysis.

The dependent variable would be usage intensity of Moodle based LMS. In order to identify

the usage intensity both by faculty members and students, the components of the five level usage

intensity model proposed by Janossy & Hover (2008) will be subjected for testing.

These variables will be further operationalized into a questionnaire which is presented in

appendix I and II.

3) Hypotheses

The following hypotheses will be tested in this study.

H1 – There is a positive relationship between resistance and intensity to use Moodle based learning

management system by faculty within the School of Business at KPU. That is, as resistance increase

intensity to use increases.

Resistance

factors in

using

LMS/Moodle

Intensity of

using

LMS/Moodle

LMS

Page 18: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 17

H0 – There is no positive relationship between resistance and intensity to use Moodle based

learning management system by faculty within the School of Business at KPU.

H2 – There is a positive relationship between resistance and intensity to use Moodle based learning

management system by students within the School of Business at KPU. That is, as resistance

increase intensity to use increases.

H0 – There is no positive relationship between resistance and intensity to use Moodle based

learning management system by students within the School of Business at KPU.

H3 – There is a negative relationship between resistance and intensity to use Moodle based learning

management system by faculty within the School of Business at KPU. That is, as resistance increase

intensity to use decreases.

H0 – There is no negative relationship between resistance and intensity to use Moodle based

learning management system by faculty within the School of Business at KPU.

H4 – There is a negative relationship between resistance and intensity to use Moodle based learning

management system by students within the School of Business at KPU. That is, as resistance

increase intensity to use decreases.

H0 – There is no negative relationship between resistance and intensity to use Moodle based

learning management system by students within the School of Business at KPU.

4) Operationalization

The following chart will indicate how each of the elements of the research question and variables

are informed by literature and how the associated instruments to test them are built around it.

Page 19: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 18

Table 02

Operationalization of Variables

Research

Question

Variables Selected Theory/Citation Instrument

What is the nature

of relationship

between

resistance to use

and the usage

intensity of

Moodle by

faculty within

the school of

Business at KPU

Resistance

Mitchel, B & May, G , (2009) ‘Attitudes

Affecting Online Learning

Implementation in Higher Education

Institutions’, Journal of Distance

Education, VOL. 23, No. 1, 71-88

Faculty

questionnaire is

developed based on

this model. See

appendix I

Usage

Intensity

Janossy, J. & Hover, T. (2008). Proposed

Model for Evaluating C/LMS Usage,

Proceedings of Society for Information

Technology & Teacher Education

International Conference 2008 (pp. 2979-

2986). Chesapeake, VA

What is the nature

of relationship

between

resistance to use

and the usage

intensity of

Moodle by

students within

the school of

Business at KPU

Resistance

Abbad, M , Morris, D & Nahlik, C

(2009) , Looking under the Bonnet:

Factors Affecting Student Adoption of E-

Learning Systems, International Review of

Research in Open and Distance Learning

Volume 10, Number 2.

Student

questionnaire is

developed based on

this model. See

appendix II

Usage

Intensity

Janossy, J. & Hover, T. (2008). Proposed

Model for Evaluating C/LMS Usage,

Proceedings of Society for Information

Technology & Teacher Education

International Conference 2008 (pp. 2979-

2986). Chesapeake, VA

Page 20: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 19

5) Methodology

a) Data Collection

Data will be collected through a survey among the identified groups of the study. They include

• Faculty members who teach in different departments in the School of Business.

• Students who are currently taking courses offered by the School of Business.

b) Selection of the Sample

The following table will highlight the population and the sample that is planned for the

survey.

Table 03

Sampling Plan

Group Population Size Sample Size Percentage

Faculty members 150 76 50%

Students 3100 465 15%

The sample will be selected on a non random basis. Faculty members from departments in

the School of Business will be selected for the sample. The researcher will attend the monthly

department meetings and will obtain from faculty members, those who are willing to take part in the

study, a survey. As for the student survey, students from several courses at different levels will be

selected for the study based on the willingness of the faculty members to allow the researcher to

visit their classrooms.

c) Questionnaire Design

The reader is requested to refer to appendix I & II for the two research instruments

developed to gather data.

d) Method of Data Analysis

The data analysis will be carried out in several sections. The first section of data analysis

will provide profile and usage of technology of the users. This analysis will be further used to carry

out cross tabulations with the main findings. The second component will be to identify the

Page 21: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 20

resistance factors for faculty members and will be identified based on each of the four main

categories as identified by Michell, May (2009) and for students based on the study of Abbad &

Nahlik (2009). Thirdly the intensity of Moodle usage will be identified based on the five levels of

the model identified by Janossy & Hover, (2008).

The main analysis would be to identify the relationship between resistance factors and the

usage intensity. Correlation between these variables at a confidence level of 95% and one tail

hypotheses tests will be carried out to test the null hypotheses in proving the identified hypotheses.

Limitations of the Study

There are several limitations associated with this study. They are as follows.

The findings of this study will reflect only the perspective of faculty and students of the

School of Business. However, resource allocations to support the learning management system are

not based at faculty level but at university level. The implications for policy and practice identified

cannot be enforced at faculty level. While this study may inform theory at the faculty level its

relevance for policy and practice at an institutional level may be limited.

Usage intensity is affected by many factors. In this study the focus would be to identify how

usage resistance may have an impact on the intensity of the use of Moodle based learning

management system. This finding may not provide a complete result since there could be many

other factors that could explain the intensity, other than resistance and may be of limited use for

policy and practice.

Findings

The findings of this study will be presented in several sections. Firstly, this section will

highlight common profile related information of both the faculty members and students. Then

findings to answer the three research questions will be highlighted.

Page 22: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 21

1) User Profiles/Technology Usage

The reader is directed to appendix III of this paper. A careful review of this appendix will

indicate that faculty members and students were significantly not very different in terms of their

technology usage in a few areas. In terms of their profile, the average length of a faculty member's

teaching experience in the post secondary education institutions was 15 years and experience

working at Kwantlen was a little over 10 years. As for students, the length of time in the post

secondary system was on average over 3 years and at Kwantlen a little over 2 1/2 years. In terms of

age, 85% of students were in the age range of 16 to 25 while 67% of faculty members were

between ages 36 to 55.

Time spent on Moodle per week among students and faculty (who used Moodle) was 2.87

hours and 7.84 hours, respectively. In terms of both students and faculty members, self rating on

their computer usage and internet usage skills was almost the same. This was also the case with the

number of hours spent with a computer both by students and faculty members. However, students

spent almost twice the time as much as faculty on browsing the internet. Over 75% of faculty

members and students had used a different learning management other than Moodle before.

The general conclusion from the above is that in terms of attitude towards technology or

usage of technology, students and faculty members were not different in a significant manner.

The following sections will highlight the findings associated with the three research questions.

2) Faculty Perspective

The first research question was to understand the nature of the relationship between

resistance to use and the usage intensity of Moodle based learning management system by faculty

members within the School of Business at KPU. The reader is directed to appendix IV of this paper

for the findings associated with this research question.

Page 23: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 22

In understanding the nature of the relationship between resistance to use and usage intensity

of Moodle by faculty members, the correlation between the two variables were calculated. (section

01 in appendix IV). A strong negative correlation of -0.8625 between the above two variables was

noted. This finding made hypothesis one which assumed a positive relationship between resistance

to use and usage intensity invalid and was not subjected to further test. In order to test the

significance of this negative correlation between resistance to use and usage intensity, the null

hypothesis of hypothesis three was subjected to a one tail significance test at a 95% confidence

level. (See section 02 in appendix IV). The difference between sample means of resistance to use

and usage intensity (12.00) exceeded the critical value. Therefore, the difference between resistance

to use and usage intensity of Moodle based learning management systems by faculty members was

significant leading to the rejection of the null hypothesis and the acceptance of Hypothesis three.

This held the assertion that there was a negative relationship between resistance to usage and usage

intensity of Moodle based learning management systems by faculty members at a 95% confidence

level. This essentially answered the research question one set for this study.

In order to further understand the nature of resistance faculty members posed towards using

Moodle for teaching purposes, a further analysis of different typologies of resistance factors as well

as qualitative reasons for this resistance was further analyzed. (see section 03 and 04 in appendix

IV). The resistance factors suggested by Mitchell & May (2009) were essentially tested. Among

faculty members who did not use Moodle, resistance to change was the number one reason for non

use. Also, many faculty member's perceived use of Moodle led to extra time and effort on their part.

This was further reinforced by the qualitative statements presented by them which highlighted lack

of time and support issues. Many faculty members cited that their departments as well as an

institution as a whole did not support the use of Moodle.

Page 24: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 23

As for usage intensity of Moodle by faculty members (see section 5 in appendix IV) who

used Moodle, a large percentage of faculty members were using Moodle as a document distribution

mode which was rated as level 01 use by Janossy & Hover (2008). The level 02 use which related to

work submission and feedback through Moodle was below the 30% mark and Level 03 which

related to online testing, asynchronous discussion and feedback was in the 25% mark. There was

hardly any level 04 use, which relates to real time and remote review.

3) Student Perspective

The second research question was to understand the nature of the relationship between

resistance to use and the usage intensity of Moodle based learning management system by students

within the School of Business at KPU. The reader is directed to appendix V of this paper for the

findings associated with this research questions.

In understanding the nature of the relationship between resistance to use and usage intensity

of Moodle by students, the correlation analysis indicated a weak positive correlation of +0.4167

between the two variables. (See appendix V, section 01)

This findings made hypothesis four which assumed a negative relationship between

resistance to use and usage intensity invalid and was not subjected to further test. In order to test the

significance of this positive correlation between resistance to use and usage intensity, the null

hypothesis of hypothesis two was subjected to a one tail significance test at a 95% confidence level.

(See section 02 in appendix V). The difference between sample means of resistance to use and

usage intensity (14.255) exceeded the critical value. Therefore, the difference between resistance to

use and usage intensity of Moodle based learning management systems by students was significant,

leading to the rejection of the null hypothesis and the acceptance of Hypothesis four. This held the

assertion that there was a positive relationship between resistance to usage and usage intensity of

Page 25: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 24

Moodle based learning management systems by students at a 95% confidence level. This essentially

answered the research question two.

The reader is directed to appendix V section 03, 04 and 05 to see further analysis carried out

to understand the nature of resistance factors associated with students in using Moodle. Overall

student perceptions of resistance to use was heavily influenced by lack of technical support, support

by people who influenced the use of Moodle (Faculty members) and the perceived ease of using the

system. A further analysis of the qualitative statements mainly highlighted technical issues related to

the system such as slowness, system accessibility issues, and most importantly, the poor use and

design of Moodle courses by instructors. The latter was cited as one of the main causes of reluctance

to use.

Overall the fact that resistance and intensity to use showing a weak positive correlation may

suggest that that resistance to use was not the main reasons for poor usage intensity of Moodle. This

will be further highlighted under the discussion area.

As for usage intensity (see section 5 in appendix 5), student usage intensity levels followed

the faculty usage intensity levels of Moodle. This is understandable since student use of Moodle is

restricted to what faculty members assign in Moodle. However, level 2 and 3 use was quite popular

among students as many used it to download documents and check grades posted on each of their

assignment marks.

4) Faculty Experience Vs. Student Experience

In answering the third research question on how the student experience in using Moodle with

faculty use, the following are a few observations.

As highlighted earlier in this section, in terms of using technology and skills levels between

the two groups, there was no significant grounds to claim that any distinctive differences existed

between the groups.

Page 26: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 25

In terms of using Moodle, faculty resistance to use were mainly originating through their

own perceptions about the time associated in using the system and their lack of resistance to deviate

from current teaching practices. Issues related to technical support in using the system was

relatively low. Whereas students cited technical support and system issues as a major factor for

usage resistance. Student claim on poor design of Moodle courses are further reinforced by many

faculty members using Moodle mainly as a document distribution system.

Overall, students seem to acknowledge the use of Moodle based systems while faculty

members who do not use Moodle seem to resist it due to resistance to change and their own

perceptions associated with support for using the system to advance teaching. Students seem to

show less resistance to use Moodle while faculty members seem to show a higher level of resistance

to use Moodle for teaching courses.

Discussions and Implications

Based on the above findings, it is evident that faculty resistance to change their current

teaching practices seem to be one of the biggest barriers in them using Moodle to supplement their

teaching. Faculty perceptions about the lack of support from their own departments and the

institution as a whole towards using this system seem to be significant. This would explain why

almost 1/2 of the faculty members in the sample do not favour using Moodle to supplement their

teaching.

It was highlighted earlier that students seem to have a different perception about some of the

factors associated with resistance. Many of them seem to associated lack of technical support and

system related issues to be major barriers. There seem to be little resistance towards the perceived

usefulness of the system making students accepting Moodle as a learning tool.

In terms of the current use of the system, among faculty who use Moodle, a large majority

predominantly use the system as an administrative tool forgoing the rich pedagogical modules that it

Page 27: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 26

seem to provide. This raises a question whether faculty really understand the intended application

that Moodle seem to offer. From the student's point of view, the findings also seem to highlight that

students are willing to use more level four applications (See section 6 in appendix V) for their

learning purposes. This indicates that there is a higher preference for learning outside the classroom

even when they enrol to a face to face course. 83% of students favoured a blended course rather

than a mere face to face or a fully online course. This further supports the view that students are

looking for a face to face experience while expecting some conveniences that technology support

through remote learning.

The above would have many implications to theory, policies and practice across the

institutions.

The biggest implication to practice is how the findings of this study highlights a severe short

coming in the current professional development efforts within the institution in supporting Moodle

based training efforts. 95% of all professional development programs offered by the Centre for

Academic Growth at Kwantlen seem to provide technical knowledge in terms on how to set up a

Moodle site and its technical applications. There are virtually no programs offered in terms of how

Moodle could be used to teach, highlighting its pedagogical value. Also, there are no strategies to

address faculty reluctance to use Moodle for teaching. This study clearly highlights the problem is

not mainly about faculty not knowing how to use technology but their reluctance to use Moodle

specifically for teaching due to various perceptions about its time consuming nature of applications.

As a heavy Moodle user, the author could highlight that this is not the case with Moodle but clearly

it is a problem associated with how the system has not been promoted as a friendly learning

management system. To start with, specific programs needs to be developed to address the fears and

the insecurities of faculty members in using learning management systems for teaching purposes.

Also programs needs to be developed to highlight the pedagogical use of various modules in

Page 28: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 27

Moodle that may support rich learning experiences. Also, offering programs of this nature is simply

not enough if the concept is not promoted to the department and the faculty itself about the

advantages of using learning management systems. Kimble (1999) in a number of studies has shown

differences in students performance levels correlated with technology use for learning purposes. The

appointment of mentors to support faculty members would be another idea that would help address

the resistance to change issues through the use of influential change agents.

This study would also highlight some implications for policy within the institution. Clearly

there seem to be differences in course delivery among faculty members who use Moodle verses

others who do not. The question is not about which method is superior but the inconsistencies that

students experience with users verses non using faculty members, as well as faculty members who

use Moodle in different levels. This should also create various issues on faculty professional

development as well as work load issues. Clear policies may need to be formulated to manage these

discrepancies. Also at policy level, it would also be relevant to evaluate how best is Moodle meeting

the needs of the institution. Over 75% of all users stating that they have used another learning

management system other than Moodle earlier may also indicate that Moodle may not be meeting

some expectations of some faculty members. Perhaps it may be time to review the policy of using

Moodle as the sole platform to support learning management systems in the institution and also look

at a multiple platform model that could be used within the budgetary constraints of the institution.

In terms of implications to theory, resistance to use technology in the classroom has been

documented since 1920 (Cuban, 1986). This is an old problem that has been repeating itself for

closer to a century. While there are many theories and models that have identified reasons for

resistance to use technology, learning management systems, there are not too much literature,

research that address how resistance to use technology, learning management systems and Moodle

in particular could be managed. There are many theories that support how resistance to change

Page 29: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 28

should be managed but resistance to use technology could be unique in many ways that would

require different intervention strategies. This is an important gap that needs to be filled so that this

age old discussion could be managed more efficiently. This would certainly be an area that further

research would open opportunities for exploration.

In conclusion, the author of this study attempted to understand how resistance to Moodle

based learning management systems would have an impact on usage intensity levels on the said

system itself. While resistance could be an important independent variable, there could be many

other variables that may affect usage. To further enrich this study, a multivariate study of how

multiple factors would affect usage intensity would help to understand how important resistance

would be verses other factors. This will give important insights for institutions to help improve its

usage levels to offset some of the operational costs that they may incur in running and maintain

these expensive systems. This is yet another possible potential area for future research.

Page 30: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 29

References

Abbad, M , Morris, D & Nahlik, C (2009) , Looking under the Bonnet: Factors Affecting Student

Adoption of E-Learning Systems, International Review of Research in Open and Distance Learning

Volume 10, Number 2.

Bergquist, W, Pawlak, K. (2008), Engaging the six cultures of the academy : Jossey Bass, John

Wiley & Sons

Blacj, E. W., Beck, D., Dawson, K., Jinks, S., & DiPietro, M. (2007). The other side of the LMS:

Considering implementation and use in the adoption of an LMS in online and blended learning

environments. TechTreds, 51(2), 35-39.

Blass E, & A. Davis, (2003), Building on solid foundations: establishing criteria for e-learning

development, Journal of Further and Higher Education, Vol.27, No.3, pp. 227-245

Britain, S. & Liber, O (1999), A framework for pedagogical evaluation of e-learning environments,

JISC Technology Applications Programme, Report No. 41.

Cuban, L. (1986). Teachers and Machines: The classroom use of technology since 1920. pp. 1-71

Davis, F. D. (1986). A technology acceptance model for empirically testing new end-user information

systems: Theory and results. Doctoral dissertation, Sloan School of Management, Massachusetts

Institute of Technology

Dehindo, J (2008), The need, use and best practices for the implementation of Learning

Management Systems in organizations and Higher Education institutions, Information system

educators conference, Nashville, 2008, V25

Page 31: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 30

Downing, Kevin J., Lam, Tsz-fung, Kwong, Theresa, Downing, Woo-kyung and Chan, Sui-wah

(2007) 'Creating interaction in online learning: a case study', ALT-J, 15: 3, 201 — 215

Geva-May, I (2010). An Operational Approach to Policy Analysis: The Craft. Boston: Kluwer

(1997, 2001 with Wildavsky, A.); New York:Palgrave Macmillan, 2nd ed.: Chapter 1 to 4

Goodson, R, (2005), Faculty Resistance: E-Learning, A working paper presented at the University

of Phoenix, Online Learning Conference

Hinson, J. M., & LaPrairie, K. N. (2005). Learning to teach online: Promoting success through

professional development. Community College Journal of Research and Practice, 29, 483-493.

Kim, S & Lee, M (2008), Validation of an evaluation model for learning management systems

Journal of Computer Assisted Learning (2008), 24, 284–294

Janossy, J. & Hover, T. (2008). Proposed Model for Evaluating C/LMS Usage, Proceedings of

Society for Information Technology & Teacher Education International Conference 2008 (pp.

2979-2986). Chesapeake, VA: AACE Retrieved from

http://www.editlib.org.proxy.lib.sfu.ca/p/27681

Mitchel, B & May, G , (2009) ‘Attitudes Affecting Online Learning Implementation in Higher

Education Institutions’, Journal of Distance Education, VOL. 23, No. 1, 71-88

Moser, F.Z. 2007. “Faculty adoption of educational technology” EDUCAUSE quarterly, no 1. pp.

66-9. Available online at: http://www.educause.edu/ir/library/pdf/eqm07111.pdf accessed 10

June 2007

Page 32: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

To Use or Not to Use 31

Nhlanhla, M, (2005), e-Learning and learning management systems (LMS) in a changing higher

education environment, ‘ e-Innovation Academy’

Peruski, L., Mishra, P. & Rosaen, C. (2006). Technology diffusion in the culture of higher

education. In C. Crawford et al. (Eds.), Proceedings of Society for Information Technology &

Teacher Education International Conference 2006 (pp. 1731-1736

Rogers, D. L. (2000). A paradigm shift: Technology integration for higher education in the new

millennium. Educational Technology Review, 13, 19-27,33.

Schacter, J. (1999, Feb). The impact of education technology on student achievement: What the most

current research has to say. Milken Family Foundation, Santa Monica, CA

Tuovinen, J & Quinn, P , (2003), The Application of an Interactions Framework in the Design and

Adoption of a Learning Management System, Annual Conference of the Australasian Society

for Computers in Learning in Tertiary Education.

Valenta, A., Therriault, D., Dieter, M., & Mrtek, R. (2001). Identifying student attitudes and

learning styles in distance education. Journal of Asynchronous Learning Networks, 5,

111-127.

Walker, G. (2005). Faculty intentions to use Web enhanced instructional components.

Retrieved from Dissertations & Theses: Full Text database. (AAT 3150454)

Page 33: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

32

APPENDIX I - Faculty Questionnaire

Dear Colleagues,

Thank you for volunteering to take part in this important research to help understand how we, as faculty, use or intend to use learning management

systems (Moodle) to support our teaching. This study is carried out as a partial fulfillment of the Masters in Education degree that I am currently

completing at the Simon Fraser University. This survey takes approximately 10 to 15 minutes to complete. All answers in this questionnaire will

be treated in the strictest confidence. Your participation is completely voluntary; you may stop at any time without any consequences.

1. How long have you been an instructor at college or university level? ____________________

2. How long have you been teaching at Kwantlen? ____________________________________

3. Have you used any technology based learning management system for teaching? (e.g. Web CT, blackboard, Moodle etc) YES_____NO _____

4. Have you used Moodle for teaching? YES_____NO _____ (If your answer is YES, please continue , if NO please go to question 8)

5. How many sections have you taught using Moodle during a year? ________________________________

6. For a year, how frequently do you use Moodle for teaching? (Number of courses per semester) ___________________________________

7. Each week, how much of time do you spend using Moodle (At home or at work)? ___________________

8. How do you rate your overall computer skills? Beginner Expert

1 2 3 4 5

9. How do you rate your skills in using the internet? Beginner Expert

1 2 3 4 5

10. Each week, how much time (in hours) do you spend on a computer? (for work, for pleasure, any other) _____________________

11. Each week, how much time (in hours) do you spend browsing the internet? _________________

12. Please select your age category

16 – 25 26-35 36-45 46-55 56 & 65 66 and above

For Official

Use Only

FACULTY Respondent

Number

Page 34: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

33

13. Please rate to which extent you agree or disagree with the following statements:

Statements Strongly

Disagree

Disagree Agree Strongly

Agree

I don’t

Know

Using Moodle to teach can never be as effective as traditional instruction.

Students don’t learn as much through a Moodle as they learn through other methods.

Moodle is suitable in teaching online courses but not face to face courses.

Using Moodle will improve the quality of instruction.

Moodle supports my style of teaching.

Moodle could not be used to teach subject matter associated with my courses.

Use of Moodle for teaching is seen as a priority at my institution.

My department does not see using Moodle as a priority.

I have experience using a different learning management system for teaching.

I have the required computer literacy skills to develop a Moodle course.

I am satisfied with the level of technical support provided to use Moodle.

My institution does not provide training on how to use Moodle for effective teaching.

Using Moodle is very time consuming for faculty.

Using Moodle may require extra effort from faculty.

Integrating Moodle to my teaching will not change my teaching style significantly.

I am comfortable with the way I teach and don’t see reasons to switch to Moodle.

Using Moodle will not make my job harder.

I don’t have the proper training to use Moodle effectively.

The benefit of using Moodle vs. the time spent with it brings a return.

I fear that I will lose all student data and files when I use Moodle.

Moodle is not the correct learning management system for me.

My institution does have the resources to support the use of Moodle for teaching.

Page 35: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

34

14. What has been a barrier for you to use Moodle for teaching purposes?

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

15. What has facilitated your use of Moodle for teaching? (If you have not used Moodle before, please skip this question.)

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

16. Please rate each of the following statements in terms of how you use Moodle for your teaching purposes. (If you have not used Moodle

before, please skip this question).

Statements on how you use Moodle for Teaching Never Rarely

Occasionally Frequently Very

Frequently

Use Moodle to make course announcements.

Use Moodle to distribute word, power point, pdf, and other documents related to

the course.

Use Moodle to list hyperlinks related to the course.

Use the standard features in the Moodle template with minimum customization.

Page 36: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

35

Statements on how you use Moodle for Teaching Never Rarely

Occasionally Frequently Very

Frequently

Customize the Moodle template to an extent leaving some standard features.

Customize the Moodle template to a very large extent without leaving any

standard features making it visually attractive.

Students can submit their work through the Moodle site but feedback is not given

through the Moodle site.

Students can submit their work through the Moodle site and also feedback is

received through the Moodle site.

Students can only see grades for assignments submitted through the Moodle site

electronically.

Students can see grades for all assignments through the grade book and records grades for all quizzes, homework, midterm exams, and projects including work not submitted electronically.

Students can take at least some quizzes or tests via the Moodle site. They are

provided with scores and feedback immediately after completing the test.

Instructor posts discussion topics, students respond to it and to other student

postings.

Instructor posts discussion topics and also moderates and judges the extent to

which students participates in these discussions.

Students can listen to the conduct of the class lecture after the class session

concludes through recorded proceedings.

Students can view and hear what is happening in the class in real time as well as

later review recorded proceedings.

Students can interact with the class from a remote location in real time as well

as later review recorded proceedings.

Thank you very much for your valuable time in completing this questionnaire.

Page 37: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

36

APPENDIX II - Student Questionnaire

Hello Students,

Thank you for volunteering to take part in this important research. The following questionnaire is presented with the objective of gathering data in understanding how students use or intend to use learning management systems (Moodle) to support their learning activities. I am currently a graduate student at the Simon Fraser University completing the Masters in Education degree and this research study is a requirement for me to complete this degree. This survey will take approximately 10 to 15 minutes to complete. All answers in this questionnaire will be treated in the strictest confidence. Your participation is completely voluntary; you may stop the survey at any time. Your responses will have no consequences to you, and will not have any impact on your mark in this class.

1. How long have you been a student at the college or university level? _______________________

2. How long have you been a student at Kwantlen? ____________________________________

3. Have you ever chosen to take a course because you knew:

• It was offered online? YES ______NO __________

• It was not offered online? YES ______NO __________

• It was offered mix mode? (Partially online and face to face) YES ______NO __________

4. Have you used any technology based learning management system for learning? (e.g. Web CT, Blackboard, Moodle,etc.) YES ___NO___

5. Have you used Moodle before? YES ______NO _______(If your answer is YES, please continue , if NO please go to question 8

6. How many courses have you taken that use Moodle during the last year (since January 2009)? ________________

7. Each week, how much time do you spend using Moodle (include at home and at school)? ____________________

8. How will you rate your overall computer skills? Beginner Expert

1 2 3 4 5

9. How will you rate your skills in using the

internet?

Beginner Expert

1 2 3 4 5

10. Each week, how much time do you spend with a computer (at with, at home, at school etc)? _____________________

11. Each week, how much of time do you spend browsing the internet? _________________

12. Please select your age category

16 – 25 26-35 36-45 46-55 56 & 65 66 and above

For Official

Use Only

STUDENT Respondent

Number

Page 38: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

37

13. Please rate to which extent you agree or disagree with the following statements:

Statements Strongly

Disagree

Disagree Agree Strongly

Agree

I don’t

Know

Using Moodle will allow me to accomplish learning tasks more quickly.

Getting the information from Moodle is not easy.

Using Moodle would enhance my effectiveness in learning.

I am confident using Moodle as long as someone shows me how to do it.

A hotline is available when there is technical problem.

My interaction with Moodle is clear and understandable.

I don’t spend many hours using a computer.

Overall, I find that Moodle is not easy to use.

I intend to use Moodle to study.

I intend to study other subjects through Moodle.

Using Moodle would not increase my productivity in learning.

I don’t intend to increase my use of the Moodle in the future.

I frequently use the Internet.

Moodle does enable interactive communications between instructors and students.

E-mail enquiries can be made when there is technical problem.

I am not confident using Moodle when there is no one around to show me how to use it.

The communicational tools (e-mail, chat room, forum, etc) in Moodle are effective.

My instructors think that I should use Moodle for learning.

Moodle enables interactive communications among students.

People who are important to me think that I should use Moodle for learning.

People who influence my behaviour think that I should use Moodle for learning.

I am confident of using Moodle even if I have never used such a system before.

Having used Moodle, I would recommend it to my colleagues to use it for study purposes.

Page 39: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

38

14. What have been barriers for you to use Moodle for learning purposes?

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

15. What has facilitated your use of Moodle? (If you have not used Moodle before, please skip this question)

____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________

16. Please rate each of the following statements in column A based on the questions in Columns B & C. (If you have not used Moodle before,

please skip this question)

Column A

Column B Column C

How frequently do you use the features listed in

Column A in your Moodle courses

Would you like have

these features included in

your Moodle courses

Never Rarely Frequently When

Graded

Not

Assigned

Yes No

Use Moodle to see course announcements.

Use Moodle to download documents related to the course.

Use Moodle to use hyperlinks and other resources related to the

course.

Use the standard features (calendar, upcoming events, etc.)

provided in the Moodle site which may or may not be directly

relevant to the course.

Page 40: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

39

Column A

Column B Column C

How frequently do you use the features listed in

Column A in your Moodle courses

Would you like have

these features included in

your Moodle Courses

Never Rarely Frequently When

Graded

Not

Assigned

Yes No

Use some of the specific features (discussion forums, chat

rooms etc) provided within the Moodle site relevant to the course.

Use all of the specific features provided within the Moodle site

relevant to the course.

Use the Moodle site to submit assignments but to receive

feedback in classroom.

Use the Moodle site to submit assignments and to receive

feedback through Moodle itself.

See grades for assignments submitted through the Moodle site

electronically.

See grades for all assignments through the grade book and

records grades for all quizzes, homework, midterm exams, and

projects including work not submitted electronically.

Take at least some quizzes or tests via the Moodle site and

receive with scores and feedback immediately after completing

the test.

Instructor posting discussion topics, students respond to it and

to other student postings.

Can listen to the conduct of the class lecture after the class

session concludes through recorded proceedings

Can view and hear what is happening in the class in real time as

well as later review recorded proceedings.

Can interact with the class from a remote location in real time as

well as later review recorded proceedings.

Thank you very much for your valuable time in completing this questionnaire.

Page 41: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

40

Appendix III - Profile/Use of Technology

Description Student Faculty

Length of time as student/teacher 3.12 years 15.37 years

Length of time involved with Kwantlen 2.73 years 10.29 years

Previous Use of LMS based technology

Yes 86% 73%

No 14% 27%

Number of courses taken, taught using Moodle during the last

one year - among users 3.79 7.83

Time spent using Moodle (per week) 2.87 hours 7.84 hours

Self rating on their overall computer skills (1 - very low, 5 - very

high)

3.92 3.86

Self rating on skills in using the internet? (1 - very low, 5 - very

high)

4.07 4.03

Time spent with a computer (per week) 25.75 hours 29.68 hours

Time spent with browsing the net (per week) 15.32 hours 8.57 hours

Age distribution

16-25 85% 0%

26-35 11% 5%

36-45 2% 33%

46-55 0% 34%

56-65 0% 26%

66 and above 0% 1%

Page 42: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

41

Appendix IV - Faculty Findings

1. Correlation between Usage Resistance and Usage Intensity

2. Significance

Hypothesis Testing - H0 (H3)

H3 – There is a negative relationship between resistance and intensity to use Moodle based learning

management system by faculty within the School of Business at KPU. That is as resistance increase,

intensity to use decreases.

H0 – There is no negative relationship between resistance and intensity to use Moodle based learning

management system by faculty within the School of Business at KPU.

Page 43: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

42

Testing the H0 in accepting/rejecting H3 using significance test

H0 - There is no negative relationship between resistance and intensity to use Moodle based learning

management system by faculty within the School of Business at KPU.

H3 - There is a negative relationship between resistance and intensity to use Moodle based

learning management system by faculty within the School of Business at KPU. That is as

resistance increase, intensity to use decreases

Confidence level 95%

Nature of the test One tail

Standard error

σ (x1-x2) = σx12 σx2

2

n n

σ (x1-x2) = 0.972 15.60

2

73 73

= 3.3465

Confidence 95% = 1.96

Critical value = 0 + 1.96 (3.3465)

= + 6.559

The difference

between sample

means

resistance and

usage

Resistance = 2.45

Intensity = 14.45

Difference = 2.45 - 14.45 = 12.00

The difference between sample means of resistance to use and usage intensity (12.00) exceeds the

critical value. The difference between resistance to use and usage intensity of Moodle based learning

management systems by faculty members is significant.

Hence, H0 is rejected with the acceptance of H3.

The assertion (H0) that there is no negative relationship between resistance to use and intensity

to use Moodle based learning management system by faculty within the School of Business at

KPU cannot be held at 95% confidence level leading to the acceptance of H3

Source – survey data

+

Where

σx1 = Standard dev around resistance

mean

+

Page 44: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

43

3. Resistance Factors By Faculty Members

Page 45: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

44

4. Other Qualitative Reasons for Resistance to Use

Page 46: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

45

5. Usage Intensity of Moodle by Faculty Members

Page 47: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

46

Appendix V - Student Findings

1. Correlation between Usage Resistance and Usage Intensity

2. Significance Hypothesis Testing H0 (H2)

H2 – There is a positive relationship between resistance and intensity to use Moodle based learning

management system by students within the School of Business at KPU. That is as resistance increase,

intensity to use increases.

H0 – There is no positive relationship between resistance and intensity to use Moodle based learning

management system by students within the School of Business at KPU.

Page 48: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

47

Testing the H0 in accepting/rejecting H2 using significance test

H0 - There is no positive relationship between resistance and intensity to use Moodle based learning

management system by students within the School of Business at KPU

H2 - There is a positive relationship between resistance and intensity to use Moodle based learning

management system by students within the School of Business at KPU. That is as resistance increase,

intensity to use increases.

Confidence level 95%

Nature of the test One tail

Standard error

σ (x1-x2) = σx12 σx2

2

n n

σ (x1-x2) = 0.632 9.59

2

462 462

= 0.199

Confidence 95% = 1.96

Critical value = 0 + 1.96 (0.199)

= + 0.3912

The difference

between sample

means

resistance and

usage

Resistance = 1.475

Intensity = 15.73

Difference = 1.475 - 15.73 = 14.255

The difference between sample means of resistance to use and usage intensity (14.255) exceeds the

critical value. The difference between resistance to use and usage intensity of Moodle based learning

management systems by students is significant.

Hence, H0 is rejected with the acceptance of H2.

The assertion (H0) that there is no negative relationship between resistance to use and intensity

to use Moodle based learning management system by students within the School of Business at

KPU cannot be held at 95% confidence level leading to the acceptance of H2

Source – survey data

+

Where

σx1 = Standard dev around resistance

mean

+

Page 49: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

48

3. Resistance Factors By Students

Page 50: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

49

4. Other Qualitative Reasons for Resistance to Use

Page 51: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

50

5. Usage Intensity of Moodle by Students

Page 52: To Use or Not to Use 0 Running head: To Use or Not to Use … · 2011-12-14 · “Moodle” as their sole learning management system replacing its own lotus notes based system. The

51

6. Current Use verses Future Intention to Use Moodle by Students