the formulation of an isdt for e-learning
DESCRIPTION
Presentation for Jones and Gregor (2006)Jones, D., & Gregor, S. (2006). The formulation of an Information Systems Design Theory for E-Learning. Paper presented at the First International Conference on Design Science Research in Information Systems and Technology, Claremont, CA.TRANSCRIPT
http://eros.cqu.edu.au/
The formulation of an ISDT The formulation of an ISDT for e-learningfor e-learning
David Jones (CQU)Shirley Gregor (ANU)
http://cq-pan.cqu.edu.au/david-jones/
http://eros.cqu.edu.au/
The QuestionThe Question
How do you design and support an information system for e-learning within
an institution of higher education?
First asked in 1996
Still an open question…..Still an open question…..
http://eros.cqu.edu.au/
It is importantIt is important
The importance of information and communication technologies and e-learning in promoting open, distance and flexible education in contemporary universities cannot be denied (deFreitas & Oliver, 2005).
Overall % of schools identifying online educationas a critical long-term strategy grew from 49%in 2003 to 56% in 2005. (Allen & Seaman, 2005)
http://eros.cqu.edu.au/
There are problemsThere are problems
Vendor identified, best-practice institutionsreport around 50% faculty adoption(Sausner, 2005)
Most HEIs are still struggling to engage a significant percentage of students and staff in e-learning (Salmon, 2005)
a relatively stable minority of Chief AcademicOfficers believe that their faculty fully accept the value and legitimacy of online education (Allen & Seaman 2005)
http://eros.cqu.edu.au/
There are problemsThere are problems
…not particularly innovative, may limit the incentive to innovate, limited in quality and may restrict the ability to integrate with other systems
(Alexander, 2001; Anonymous, 2004; Paulsen, 2002)
…Web-based learning lacks a disciplined, systematic approach to the development process.….much of its construction is carried out without a true analysis of therequirements that are proper to Web-based learning (Hamid, 2002).
http://eros.cqu.edu.au/
It is difficultIt is difficult
..complex systems incorporating a variety of organisational, administrative, instructional and technological components (Avgeriou, Papasalouros, Retalis, Skordalakis, 2003)
Absence of theory for e-Learning IS (Jones, Gregor and Lynch, 2003)
Scholars in Information Systems can offer vision on structures and processes to effectively implement technology-mediated learning initiatives (Alavi & Leidner, 2001)
http://eros.cqu.edu.au/
http://eros.cqu.edu.au/
Proposed contributionProposed contribution
• ISDT that provides 1 answer to the question– Greater adoption, customization, differentiation
• Based on 10 year action research cycle involving "real" system(1200+ staff, 22,000+ students)
• Demonstrates novel use of ISDT– To understand/improve EXISTING information
system
http://eros.cqu.edu.au/
What we didWhat we did
http://eros.cqu.edu.au/
PlanPlan
ActActReflectReflect
WatchWatch
1996-2003 3 generationsof action research
2002- Formulation of Information SystemsDesign Theory
WebfuseWebfuse24 research publications24 research publications
5 research publications5 research
publications
http://eros.cqu.edu.au/
What we'll do todayWhat we'll do today
• Laid out the context • Describe three generations–Why and what (The ISDT)– Instantiation (Webfuse) and results
• Reflect– Value of ISDT applied to existing systems–What can be IS design science research?
http://eros.cqu.edu.au/
Generation 1: 1996-1999Generation 1: 1996-1999Build it and they will comeBuild it and they will come
http://eros.cqu.edu.au/
WhyWhy
• 1996 - the web is a "good thing"
• Design– Single academic one teaching free term
• Aim to support–Web-based learning and teaching– Faculty website
http://eros.cqu.edu.au/
What - the ISDTWhat - the ISDT
http://eros.cqu.edu.au/
KernelKernelTheoriesTheories
Not explicitlystatedNot explicitlystated
DE/educationliterature &experience
DE/educationliterature &experience
Hypermediatemplates, s/w wrappers
Hypermediatemplates, s/w wrappers
Flexibility &ease of useFlexibility &ease of use
"Waterfall""Waterfall"TemplatesTemplates
Possible to construct a usable systemWill be more flexible
Design Design MethodMethod
Meta-Meta-requirementsrequirements
Meta-designMeta-design
TestableTestablehypotheseshypotheses
http://eros.cqu.edu.au/
Meta-Meta-requirementsrequirements
KernelKernelTheoriesTheories
Not explicitlystatedNot explicitlystated
DE/educationliterature &experience
DE/educationliterature &experience
Hypermediatemplates, s/w wrappers
Hypermediatemplates, s/w wrappers
"Waterfall""Waterfall"TemplatesTemplates
Possible to construct a usable systemWill be more flexiblePossible to construct a usable systemWill be more flexible
Design Design MethodMethod
Meta-designMeta-design
TestableTestablehypotheseshypotheses
What's needed?Flexibility &ease of useFlexibility &ease of use
http://eros.cqu.edu.au/
Meta-requirementsMeta-requirements• flexibility and the ability to adapt to change• platform independence• use of Internet and other widely accepted standards• not reinventing the wheel• maximising the choice and flexibility provided to
students and teachers• providing the tools not the rules• minimising online time• minimising the new skills required to use the system• being freely available
(Jones & Buchanan, 1996;McCormack & Jones, 1997)
•Support for variety and change•No specific functional requirements•No support for specific learningtheory
•Basic support for • Information distribution•Communication•Assessment•Management•Design
(McCormack & Jones, 1996; Hansen and Frick, 1997)
http://eros.cqu.edu.au/
Flexibility &ease of useFlexibility &ease of use
Meta-Meta-requirementsrequirements
KernelKernelTheoriesTheories
Not explicitlystatedNot explicitlystated
DE/educationliterature &experience
DE/educationliterature &experience
Hypermediatemplates, s/w wrappers
Hypermediatemplates, s/w wrappers
Flexibility &ease-of-use
"Waterfall""Waterfall"TemplatesTemplates
Possible to construct a usable systemWill be more flexiblePossible to construct a usable systemWill be more flexible
Design Design MethodMethod
Meta-designMeta-design
TestableTestablehypotheseshypotheses
Product
http://eros.cqu.edu.au/
Each course website starts witha default skeleton of
web pages
There can be different skeletons
http://eros.cqu.edu.au/
Content + conversion + appearance
http://eros.cqu.edu.au/
Content + conversion + appearance
http://eros.cqu.edu.au/
Edit link
http://eros.cqu.edu.au/
http://eros.cqu.edu.au/
Templates as wrappersTemplates as wrappers
Users
Discussion forum Chat room Online quizzes
http://eros.cqu.edu.au/
Templates as wrappersTemplates as wrappers
Users
Discussion forum Chat room Online quizzes
http://eros.cqu.edu.au/
Flexibility &ease of useFlexibility &ease of use
Meta-Meta-requirementsrequirements
KernelKernelTheoriesTheories
Not explicitlystatedNot explicitlystated
DE/educationliterature &experience
DE/educationliterature &experience
Hypermediatemplates, s/w wrappers
Hypermediatemplates, s/w wrappers
"Waterfall""Waterfall"TemplatesTemplates
Possible to construct a usable systemWill be more flexiblePossible to construct a usable systemWill be more flexible
Design Design MethodMethod
Meta-designMeta-design
TestableTestablehypotheseshypotheses
Process
http://eros.cqu.edu.au/
Flexibility &ease of useFlexibility &ease of use
Meta-Meta-requirementsrequirements
KernelKernelTheoriesTheories
Not explicitlystatedNot explicitlystated
DE/educationliterature &experience
DE/educationliterature &experience
Hypermediatemplates, s/w wrappers
Hypermediatemplates, s/w wrappers
"Waterfall""Waterfall"TemplatesTemplates
Possible to construct a usable systemWill be more flexiblePossible to construct a usable systemWill be more flexible
Design Design MethodMethod
Meta-designMeta-design
TestableTestablehypotheseshypotheses
Expected outcomes
http://eros.cqu.edu.au/
Generation 1 InstantiationGeneration 1 Instantiation
http://eros.cqu.edu.au/
ImplementationImplementation
• Technology– Perl, Apache– CGI scripts
• People– On-going support Webmaster other general staff – New developments: mostly 1 academic + project
students
http://eros.cqu.edu.au/
ResultsResults
• 150+ course websites, –With 100,000+ pages–Maintained by almost 100 staff
• Only 5 of 60 full-time academic staff making significant use of e-learning
• Usage dominated by system developer– 83% of content for 1 term 1998 from 1
course
http://eros.cqu.edu.au/
Generation 2: 1999-Generation 2: 1999-Give them a reason to comeGive them a reason to come
http://eros.cqu.edu.au/
WhyWhy
• Product focus and process ignorance
• Problems– Limited use– Not enough evolution
• Need to fix the process– Greater adoption– Greater evolution
http://eros.cqu.edu.au/
What - the ISDT v2.0What - the ISDT v2.0
http://eros.cqu.edu.au/
Flexibility &ease of useFlexibility &ease of use
Meta-Meta-requirementsrequirements
Diffusion theory, designpatterns
Diffusion theory, designpatterns
Adopterfocus, pattern
mining
Adopterfocus, pattern
mining
Possible to construct a usable systemWill be more flexibleMore acceptable = greater adoption
Possible to construct a usable systemWill be more flexibleMore acceptable = greater adoption
Design Design MethodMethod
TestableTestablehypotheseshypotheses
KernelKernelTheoriesTheories
DE/educationliterature &experience
DE/educationliterature &experience
Hypermediatemplates, s/w wrappers
Hypermediatemplates, s/w wrappers
TemplatesTemplatesMeta-designMeta-design
http://eros.cqu.edu.au/
Generation 2 - InstantationGeneration 2 - Instantation
http://eros.cqu.edu.au/
ResourcesResources
• No significant addition of development resources
http://eros.cqu.edu.au/
What to implement?What to implement?
• Usually, driven by – Technology– Strategic planning– Pedagogy
• We adopted diffusion theory–Maximise chance of adoption, minimise
effort
http://eros.cqu.edu.au/
Rate of adoptionLevel of Reinvention
Rate of adoptionLevel of Reinvention
Diffusion theoryDiffusion theoryPerceivedattributesPerceivedattributes
Innovationdecision
Innovationdecision
CommunicationChannels
CommunicationChannels
SocialSystemSocial
System
Change AgentEfforts
Change AgentEfforts
(Jones, Jamieson & Clark,2003)(Jones, Jamieson & Clark,2003)(Jones, Jamieson & Clark,2003)(Jones, Jamieson & Clark,2003)
Relative advantageCompatibilityComplexityTrialabilityObservability
Relative advantageCompatibilityComplexityTrialabilityObservability
OptionalCollectiveAuthority
OptionalCollectiveAuthority
Mass-media or interpersonalCosmopolite or localMass-media or interpersonalCosmopolite or local
Social structureCommunication structureSystem norms
Social structureCommunication structureSystem norms
Client contact: effort, timing. Rapport, empathy, credibility and homophilyClient contact: effort, timing. Rapport, empathy, credibility and homophily
http://eros.cqu.edu.au/
Diffusion theoryDiffusion theory
Rate of adoptionLevel of Reinvention
Rate of adoptionLevel of Reinvention
PerceivedattributesPerceivedattributes
Innovationdecision
Innovationdecision
CommunicationChannels
CommunicationChannels
SocialSystemSocial
System
Change AgentEfforts
Change AgentEfforts
http://eros.cqu.edu.au/
"Pattern" Mining"Pattern" MiningAcademicStaff
TemplateRepository
The expected
The unexpected
http://eros.cqu.edu.au/
"Pattern" Mining"Pattern" MiningAcademicStaff
TemplateRepository
The expected
The unexpected
http://eros.cqu.edu.au/
"Pattern" Mining v2.0"Pattern" Mining v2.0AcademicStaff
TemplateRepository
The expected
The unexpected
http://eros.cqu.edu.au/
"Pattern" Mining v2.0"Pattern" Mining v2.0AcademicStaff
TemplateRepository
The expected
The unexpected
http://eros.cqu.edu.au/
Generation 2 - ResultsGeneration 2 - Results
http://eros.cqu.edu.au/
ResultsResults
• Some improvements
• But resources not appropriate
• Limited ability to adapt
• …more covered in generation 3
http://eros.cqu.edu.au/
Generation 3: 2001-Generation 3: 2001-Build what they want, fasterBuild what they want, faster
http://eros.cqu.edu.au/
WhyWhy
• Generation 2 limited by – Inadequate resources– Poor technology– Poor process
• Context changes– 2000 new Dean– 1999-2001 - 33% growth in students
http://eros.cqu.edu.au/
What - the ISDT v3.0What - the ISDT v3.0
http://eros.cqu.edu.au/
Flexibility &ease of useFlexibility &ease of use
Meta-Meta-requirementsrequirements
.. Agile, emergent,ateleological
.. Agile, emergent,ateleological
..XP, emergentdevelopment
..XP, emergentdevelopment
Possible to construct a usable systemWill be more flexibleMore acceptable = greater adoption
Possible to construct a usable systemWill be more flexibleMore acceptable = greater adoption
Design Design MethodMethod
TestableTestablehypotheseshypotheses
KernelKernelTheoriesTheories
DE/educationliterature &experience
DE/educationliterature &experience
….OO designpatterns….OO designpatterns
…designpatterns…designpatternsMeta-designMeta-design
http://eros.cqu.edu.au/
Generation 3 - InstantiationGeneration 3 - Instantiation
http://eros.cqu.edu.au/
Product and ProcessProduct and Process
• Development team– 1 lead developer– 3-5 other developers
• Patterns (e.g.)– Model/View/Controller web development– Greater use of factory pattern
• Extreme Programming aspects– Test driven development/Unit tests– Public User Stories– Stand-up meetings
http://eros.cqu.edu.au/
Generation 2 - ResultsGeneration 2 - Results
http://eros.cqu.edu.au/
The key to easy use of OASIS is that it is not a off the shelf product that is sooooo (sic) generic that it has lost its way as a course delivery tool.
CustomisationCustomisation
http://eros.cqu.edu.au/
..the precedent of other IT systems made available in Infocom (…) suggests that it would be extremely user friendly for people with very limited computer competence/confidence.
my positive experience with other Infocom systems gives me confidence that OASIS would be no different. The systems team have a very good track record that inspires confidence
Building trustBuilding trust
http://eros.cqu.edu.au/
I identify myself as coming from the technophobic extreme of the faculties (I have an Arts background) but even I find the systems the team have set out relatively easy to use
Ease-of-useEase-of-use
http://eros.cqu.edu.au/
OASIS - ease of use/usefulnessOASIS - ease of use/usefulness
Non-users: 19% (n=22)Users: 37% (n=99)
http://eros.cqu.edu.au/
Number of Staff UsersNumber of Staff Users
Generation 1
Generation 2
Generation 3
http://eros.cqu.edu.au/
Generation 1 Generation 3
Generation 2
Number of Staff UsersNumber of Staff Users
Shadowy and feral lead topossible system shutdownShadowy and feral lead topossible system shutdown
http://eros.cqu.edu.au/
Generation 1 Generation 3
Generation 2
Number of Staff UsersNumber of Staff Users
System accepted at organisational levelSystem accepted at organisational level
http://eros.cqu.edu.au/
Student PortalStudent Portal
http://eros.cqu.edu.au/
Staff Portal - # usersStaff Portal - # users
http://eros.cqu.edu.au/
Staff Portal - # requestsStaff Portal - # requests
(* until Feb 18, 2006)
http://eros.cqu.edu.au/
Online assignmentsOnline assignments
http://eros.cqu.edu.au/
Implications for design Implications for design science research and ISscience research and IS
http://eros.cqu.edu.au/
Usefulness of ISDTsUsefulness of ISDTs
• Applied to existing information systems– Emphasize unspoken assumptions– Open them up to question– Provide method to connect to behavioral
research - relevance– Compare different approaches
http://eros.cqu.edu.au/
Question for youQuestion for you
• What can be classed as design science research within Information Systems?
• McKay and Marshall (2005) suggest– Computer science = machine domain– Psychology = human behavior domain– IS = 'situatedness' of the machine in the social system
http://eros.cqu.edu.au/
SummarySummary
• We built a system for e-learning
• Formulated an ISDT– It has some advantages
• The work raises questions/implications about design science research in IS