the formulation of an isdt for e-learning

67
http:// eros.cqu.edu.au/ The formulation of an The formulation of an ISDT for e-learning ISDT for e-learning David Jones (CQU) Shirley Gregor (ANU) http://cq-pan.cqu.edu.au/david- jones/

Upload: david-jones

Post on 04-Jun-2015

483 views

Category:

Education


1 download

DESCRIPTION

Presentation for Jones and Gregor (2006)Jones, D., & Gregor, S. (2006). The formulation of an Information Systems Design Theory for E-Learning. Paper presented at the First International Conference on Design Science Research in Information Systems and Technology, Claremont, CA.

TRANSCRIPT

Page 1: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

The formulation of an ISDT The formulation of an ISDT for e-learningfor e-learning

David Jones (CQU)Shirley Gregor (ANU)

http://cq-pan.cqu.edu.au/david-jones/

Page 2: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

The QuestionThe Question

How do you design and support an information system for e-learning within

an institution of higher education?

First asked in 1996

Still an open question…..Still an open question…..

Page 3: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

It is importantIt is important

The importance of information and communication technologies and e-learning in promoting open, distance and flexible education in contemporary universities cannot be denied (deFreitas & Oliver, 2005).

Overall % of schools identifying online educationas a critical long-term strategy grew from 49%in 2003 to 56% in 2005. (Allen & Seaman, 2005)

Page 4: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

There are problemsThere are problems

Vendor identified, best-practice institutionsreport around 50% faculty adoption(Sausner, 2005)

Most HEIs are still struggling to engage a significant percentage of students and staff in e-learning (Salmon, 2005)

a relatively stable minority of Chief AcademicOfficers believe that their faculty fully accept the value and legitimacy of online education (Allen & Seaman 2005)

Page 5: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

There are problemsThere are problems

…not particularly innovative, may limit the incentive to innovate, limited in quality and may restrict the ability to integrate with other systems

(Alexander, 2001; Anonymous, 2004; Paulsen, 2002)

…Web-based learning lacks a disciplined, systematic approach to the development process.….much of its construction is carried out without a true analysis of therequirements that are proper to Web-based learning (Hamid, 2002).

Page 6: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

It is difficultIt is difficult

..complex systems incorporating a variety of organisational, administrative, instructional and technological components (Avgeriou, Papasalouros, Retalis, Skordalakis, 2003)

Absence of theory for e-Learning IS (Jones, Gregor and Lynch, 2003)

Scholars in Information Systems can offer vision on structures and processes to effectively implement technology-mediated learning initiatives (Alavi & Leidner, 2001)

Page 7: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Page 8: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Proposed contributionProposed contribution

• ISDT that provides 1 answer to the question– Greater adoption, customization, differentiation

• Based on 10 year action research cycle involving "real" system(1200+ staff, 22,000+ students)

• Demonstrates novel use of ISDT– To understand/improve EXISTING information

system

Page 9: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

What we didWhat we did

Page 10: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

PlanPlan

ActActReflectReflect

WatchWatch

1996-2003 3 generationsof action research

2002- Formulation of Information SystemsDesign Theory

WebfuseWebfuse24 research publications24 research publications

5 research publications5 research

publications

Page 11: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

What we'll do todayWhat we'll do today

• Laid out the context • Describe three generations–Why and what (The ISDT)– Instantiation (Webfuse) and results

• Reflect– Value of ISDT applied to existing systems–What can be IS design science research?

Page 12: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 1: 1996-1999Generation 1: 1996-1999Build it and they will comeBuild it and they will come

Page 13: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

WhyWhy

• 1996 - the web is a "good thing"

• Design– Single academic one teaching free term

• Aim to support–Web-based learning and teaching– Faculty website

Page 14: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

What - the ISDTWhat - the ISDT

Page 15: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

KernelKernelTheoriesTheories

Not explicitlystatedNot explicitlystated

DE/educationliterature &experience

DE/educationliterature &experience

Hypermediatemplates, s/w wrappers

Hypermediatemplates, s/w wrappers

Flexibility &ease of useFlexibility &ease of use

"Waterfall""Waterfall"TemplatesTemplates

Possible to construct a usable systemWill be more flexible

Design Design MethodMethod

Meta-Meta-requirementsrequirements

Meta-designMeta-design

TestableTestablehypotheseshypotheses

Page 16: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Meta-Meta-requirementsrequirements

KernelKernelTheoriesTheories

Not explicitlystatedNot explicitlystated

DE/educationliterature &experience

DE/educationliterature &experience

Hypermediatemplates, s/w wrappers

Hypermediatemplates, s/w wrappers

"Waterfall""Waterfall"TemplatesTemplates

Possible to construct a usable systemWill be more flexiblePossible to construct a usable systemWill be more flexible

Design Design MethodMethod

Meta-designMeta-design

TestableTestablehypotheseshypotheses

What's needed?Flexibility &ease of useFlexibility &ease of use

Page 17: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Meta-requirementsMeta-requirements• flexibility and the ability to adapt to change• platform independence• use of Internet and other widely accepted standards• not reinventing the wheel• maximising the choice and flexibility provided to

students and teachers• providing the tools not the rules• minimising online time• minimising the new skills required to use the system• being freely available

(Jones & Buchanan, 1996;McCormack & Jones, 1997)

•Support for variety and change•No specific functional requirements•No support for specific learningtheory

•Basic support for • Information distribution•Communication•Assessment•Management•Design

(McCormack & Jones, 1996; Hansen and Frick, 1997)

Page 18: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Flexibility &ease of useFlexibility &ease of use

Meta-Meta-requirementsrequirements

KernelKernelTheoriesTheories

Not explicitlystatedNot explicitlystated

DE/educationliterature &experience

DE/educationliterature &experience

Hypermediatemplates, s/w wrappers

Hypermediatemplates, s/w wrappers

Flexibility &ease-of-use

"Waterfall""Waterfall"TemplatesTemplates

Possible to construct a usable systemWill be more flexiblePossible to construct a usable systemWill be more flexible

Design Design MethodMethod

Meta-designMeta-design

TestableTestablehypotheseshypotheses

Product

Page 19: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Each course website starts witha default skeleton of

web pages

There can be different skeletons

Page 20: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Content + conversion + appearance

Page 21: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Content + conversion + appearance

Page 22: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Edit link

Page 23: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Page 24: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Templates as wrappersTemplates as wrappers

Users

Discussion forum Chat room Online quizzes

Page 25: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Templates as wrappersTemplates as wrappers

Users

Discussion forum Chat room Online quizzes

Page 26: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Flexibility &ease of useFlexibility &ease of use

Meta-Meta-requirementsrequirements

KernelKernelTheoriesTheories

Not explicitlystatedNot explicitlystated

DE/educationliterature &experience

DE/educationliterature &experience

Hypermediatemplates, s/w wrappers

Hypermediatemplates, s/w wrappers

"Waterfall""Waterfall"TemplatesTemplates

Possible to construct a usable systemWill be more flexiblePossible to construct a usable systemWill be more flexible

Design Design MethodMethod

Meta-designMeta-design

TestableTestablehypotheseshypotheses

Process

Page 27: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Flexibility &ease of useFlexibility &ease of use

Meta-Meta-requirementsrequirements

KernelKernelTheoriesTheories

Not explicitlystatedNot explicitlystated

DE/educationliterature &experience

DE/educationliterature &experience

Hypermediatemplates, s/w wrappers

Hypermediatemplates, s/w wrappers

"Waterfall""Waterfall"TemplatesTemplates

Possible to construct a usable systemWill be more flexiblePossible to construct a usable systemWill be more flexible

Design Design MethodMethod

Meta-designMeta-design

TestableTestablehypotheseshypotheses

Expected outcomes

Page 28: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 1 InstantiationGeneration 1 Instantiation

Page 29: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

ImplementationImplementation

• Technology– Perl, Apache– CGI scripts

• People– On-going support Webmaster other general staff – New developments: mostly 1 academic + project

students

Page 30: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

ResultsResults

• 150+ course websites, –With 100,000+ pages–Maintained by almost 100 staff

• Only 5 of 60 full-time academic staff making significant use of e-learning

• Usage dominated by system developer– 83% of content for 1 term 1998 from 1

course

Page 31: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 2: 1999-Generation 2: 1999-Give them a reason to comeGive them a reason to come

Page 32: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

WhyWhy

• Product focus and process ignorance

• Problems– Limited use– Not enough evolution

• Need to fix the process– Greater adoption– Greater evolution

Page 33: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

What - the ISDT v2.0What - the ISDT v2.0

Page 34: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Flexibility &ease of useFlexibility &ease of use

Meta-Meta-requirementsrequirements

Diffusion theory, designpatterns

Diffusion theory, designpatterns

Adopterfocus, pattern

mining

Adopterfocus, pattern

mining

Possible to construct a usable systemWill be more flexibleMore acceptable = greater adoption

Possible to construct a usable systemWill be more flexibleMore acceptable = greater adoption

Design Design MethodMethod

TestableTestablehypotheseshypotheses

KernelKernelTheoriesTheories

DE/educationliterature &experience

DE/educationliterature &experience

Hypermediatemplates, s/w wrappers

Hypermediatemplates, s/w wrappers

TemplatesTemplatesMeta-designMeta-design

Page 35: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 2 - InstantationGeneration 2 - Instantation

Page 36: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

ResourcesResources

• No significant addition of development resources

Page 37: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

What to implement?What to implement?

• Usually, driven by – Technology– Strategic planning– Pedagogy

• We adopted diffusion theory–Maximise chance of adoption, minimise

effort

Page 38: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Rate of adoptionLevel of Reinvention

Rate of adoptionLevel of Reinvention

Diffusion theoryDiffusion theoryPerceivedattributesPerceivedattributes

Innovationdecision

Innovationdecision

CommunicationChannels

CommunicationChannels

SocialSystemSocial

System

Change AgentEfforts

Change AgentEfforts

(Jones, Jamieson & Clark,2003)(Jones, Jamieson & Clark,2003)(Jones, Jamieson & Clark,2003)(Jones, Jamieson & Clark,2003)

Relative advantageCompatibilityComplexityTrialabilityObservability

Relative advantageCompatibilityComplexityTrialabilityObservability

OptionalCollectiveAuthority

OptionalCollectiveAuthority

Mass-media or interpersonalCosmopolite or localMass-media or interpersonalCosmopolite or local

Social structureCommunication structureSystem norms

Social structureCommunication structureSystem norms

Client contact: effort, timing. Rapport, empathy, credibility and homophilyClient contact: effort, timing. Rapport, empathy, credibility and homophily

Page 39: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Diffusion theoryDiffusion theory

Rate of adoptionLevel of Reinvention

Rate of adoptionLevel of Reinvention

PerceivedattributesPerceivedattributes

Innovationdecision

Innovationdecision

CommunicationChannels

CommunicationChannels

SocialSystemSocial

System

Change AgentEfforts

Change AgentEfforts

Page 40: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

"Pattern" Mining"Pattern" MiningAcademicStaff

TemplateRepository

The expected

The unexpected

Page 41: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

"Pattern" Mining"Pattern" MiningAcademicStaff

TemplateRepository

The expected

The unexpected

Page 42: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

"Pattern" Mining v2.0"Pattern" Mining v2.0AcademicStaff

TemplateRepository

The expected

The unexpected

Page 43: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

"Pattern" Mining v2.0"Pattern" Mining v2.0AcademicStaff

TemplateRepository

The expected

The unexpected

Page 44: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 2 - ResultsGeneration 2 - Results

Page 45: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

ResultsResults

• Some improvements

• But resources not appropriate

• Limited ability to adapt

• …more covered in generation 3

Page 46: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 3: 2001-Generation 3: 2001-Build what they want, fasterBuild what they want, faster

Page 47: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

WhyWhy

• Generation 2 limited by – Inadequate resources– Poor technology– Poor process

• Context changes– 2000 new Dean– 1999-2001 - 33% growth in students

Page 48: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

What - the ISDT v3.0What - the ISDT v3.0

Page 49: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Flexibility &ease of useFlexibility &ease of use

Meta-Meta-requirementsrequirements

.. Agile, emergent,ateleological

.. Agile, emergent,ateleological

..XP, emergentdevelopment

..XP, emergentdevelopment

Possible to construct a usable systemWill be more flexibleMore acceptable = greater adoption

Possible to construct a usable systemWill be more flexibleMore acceptable = greater adoption

Design Design MethodMethod

TestableTestablehypotheseshypotheses

KernelKernelTheoriesTheories

DE/educationliterature &experience

DE/educationliterature &experience

….OO designpatterns….OO designpatterns

…designpatterns…designpatternsMeta-designMeta-design

Page 50: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 3 - InstantiationGeneration 3 - Instantiation

Page 51: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Product and ProcessProduct and Process

• Development team– 1 lead developer– 3-5 other developers

• Patterns (e.g.)– Model/View/Controller web development– Greater use of factory pattern

• Extreme Programming aspects– Test driven development/Unit tests– Public User Stories– Stand-up meetings

Page 52: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 2 - ResultsGeneration 2 - Results

Page 53: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

The key to easy use of OASIS is that it is not a off the shelf product that is sooooo (sic) generic that it has lost its way as a course delivery tool.

CustomisationCustomisation

Page 54: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

..the precedent of other IT systems made available in Infocom (…) suggests that it would be extremely user friendly for people with very limited computer competence/confidence.

my positive experience with other Infocom systems gives me confidence that OASIS would be no different. The systems team have a very good track record that inspires confidence

Building trustBuilding trust

Page 55: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

I identify myself as coming from the technophobic extreme of the faculties (I have an Arts background) but even I find the systems the team have set out relatively easy to use

Ease-of-useEase-of-use

Page 56: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

OASIS - ease of use/usefulnessOASIS - ease of use/usefulness

Non-users: 19% (n=22)Users: 37% (n=99)

Page 57: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Number of Staff UsersNumber of Staff Users

Generation 1

Generation 2

Generation 3

Page 58: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 1 Generation 3

Generation 2

Number of Staff UsersNumber of Staff Users

Shadowy and feral lead topossible system shutdownShadowy and feral lead topossible system shutdown

Page 59: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Generation 1 Generation 3

Generation 2

Number of Staff UsersNumber of Staff Users

System accepted at organisational levelSystem accepted at organisational level

Page 60: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Student PortalStudent Portal

Page 61: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Staff Portal - # usersStaff Portal - # users

Page 62: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Staff Portal - # requestsStaff Portal - # requests

(* until Feb 18, 2006)

Page 63: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Online assignmentsOnline assignments

Page 64: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Implications for design Implications for design science research and ISscience research and IS

Page 65: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Usefulness of ISDTsUsefulness of ISDTs

• Applied to existing information systems– Emphasize unspoken assumptions– Open them up to question– Provide method to connect to behavioral

research - relevance– Compare different approaches

Page 66: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

Question for youQuestion for you

• What can be classed as design science research within Information Systems?

• McKay and Marshall (2005) suggest– Computer science = machine domain– Psychology = human behavior domain– IS = 'situatedness' of the machine in the social system

Page 67: The formulation of an ISDT for e-learning

http://eros.cqu.edu.au/

SummarySummary

• We built a system for e-learning

• Formulated an ISDT– It has some advantages

• The work raises questions/implications about design science research in IS