adaptive hypermedia from concepts to authoring
DESCRIPTION
Adaptive Hypermedia From Concepts to Authoring. Peter Brusilovsky School of Information Sciences University of Pittsburgh [email protected] http://www.sis.pitt.edu/~peterb/. Adaptive systems. Classic loop “user modeling - adaptation” in adaptive systems. Adaptive software systems. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/1.jpg)
Adaptive HypermediaFrom Concepts to Authoring
Peter Brusilovsky
School of Information Sciences
University of [email protected]
http://www.sis.pitt.edu/~peterb/
![Page 2: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/2.jpg)
Adaptive systems
Classic loop “user modeling - adaptation” in adaptive systems
![Page 3: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/3.jpg)
Adaptive software systems
• Intelligent Tutoring Systems– adaptive course sequencing– adaptive . . .
• Adaptive Hypermedia Systems– adaptive presentation– adaptive navigation support
• Adaptive IR systems
• Adaptive . . .
![Page 4: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/4.jpg)
Outline
• Adaptive hypermedia– Where? – Why? – What? – How? – Who?...
• Adaptive presentation
• Adaptive navigation support
![Page 5: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/5.jpg)
Adaptive hypermedia: Why?
Different people are differentIndividuals are different at different times"Lost in Hyperspace”Large variety of usersVariable characteristics of the usersLarge hyperspace
![Page 6: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/6.jpg)
Where it can be useful?
• Web-based Education– ITS, tutorials, Web courses
• On-line information systems– classic IS, information kiosks, encyclopedias
• E-commerce• Museums
– virtual museums and handheld guides
• Information retrieval systems– classic IR, filtering, recommendation, services
![Page 7: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/7.jpg)
Where it can be useful?
• Web-based educationELM-ART, AHA!, KBS-Hyperbook, MANIC
• On-line information systemsPEBA-II, AHA!, AVANTI, SWAN, ELFI, ADAPTS
• E-commerceTellim, SETA, Adaptive Catalogs
• Virtual and real museumsILEX, HYPERAUDIO, HIPS, Power, Marble Museum
• Information retrieval, filtering, recommendationSmartGuide, Syskill & Webert, IfWeb, SiteIF, FAB, AIS
![Page 8: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/8.jpg)
Adapting to what?
• Knowledge: about the system and the subject
• Goal: local and global
• Interests
• Background: profession, language, prospect, capabilities
• Navigation history
![Page 9: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/9.jpg)
Who provides adaptation?
• User
• "Administrator"
• System itself
• Adaptive vs. adaptable systems
![Page 10: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/10.jpg)
What can be adapted?
• Hypermedia = Pages + Links
• Adaptive presentation
– content adaptation
• Adaptive navigation support
– link adaptation
![Page 11: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/11.jpg)
Adaptive
hypermedia
technologies
Adaptive
presentation
Adaptive
navigation support
Direct guidance
Adaptive link
sorting
Adaptive link
hiding
Adaptive link
annotation
Adaptive link
generation
Adaptive
multimedia
presentation
Adaptive text
presentation
Adaptation of
modality
Canned text
adaptation
Natural
language
adaptation
Inserting/
removing
fragments
Altering
fragments
Stretchtext
Sorting
fragments
Dimming
fragments
Map adaptation
Hiding
Disabling
Removal
![Page 12: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/12.jpg)
Adaptive presentation: goals
• Provide the different content for users with different knowledge, goals, background
• Provide additional material for some categories of users– comparisons– extra explanations– details
• Remove or fade irrelevant piece of content• Sort fragments - most relevant first
![Page 13: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/13.jpg)
Adaptive presentation techniques
• Conditional text filteringITEM/IP, PT, AHA!
• Adaptive stretchtextMetaDoc, KN-AHS, PUSH, ADAPTS
• Frame-based adaptationHypadapter, EPIAIM, ARIANNA, SETA
• Full natural language generationILEX, PEBA-II, Ecran Total
![Page 14: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/14.jpg)
Conditional text filtering
If switch is known and user_motivation is high
Fragment 2
Fragment K
Fragment 1
• Similar to UNIX cpp• Universal technology
– Altering fragments– Extra explanation– Extra details– Comparisons
• Low level technology– Text programming
![Page 15: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/15.jpg)
Example: Stretchtext (PUSH)
![Page 16: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/16.jpg)
Example: Stretchtext (ADAPTS)
![Page 17: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/17.jpg)
Adaptive presentation: evaluation
• MetaDoc: On-line documentation system, adapting to user knowledge on the subject
• Reading comprehension time decreased
• Understanding increased for novices
• No effect for navigation time, number of nodes visited, number of operations
![Page 18: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/18.jpg)
Adaptive navigation support: goals
• Guidance: Where I can go? – Local guidance (“next best”)– Global guidance (“ultimate goal”)
• Orientation: Where am I? – Local orientation support (local area)– Global orientation support (whole hyperspace)
![Page 19: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/19.jpg)
Adaptive navigation support
• Direct guidance
• Restricting access– Removing, disabling, hiding
• Sorting
• Annotation
• Generation– Similarity-based, interest-based
• Map adaptation techniques
![Page 20: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/20.jpg)
Example: Adaptive annotation
Annotations for topic states in Manuel Excell: not seen (white lens) ; partially seen (grey lens) ; and completed (black lens)
![Page 21: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/21.jpg)
Adaptive annotation and removing
QuickTime™ and aTIFF (LZW) decompressorare needed to see this picture.
![Page 22: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/22.jpg)
Example: Adaptive annotation
1. Concept role
2. Current concept state
3. Current section state
4. Linked sections state
4
3
2
1
v
![Page 23: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/23.jpg)
Adaptive navigation support: major goals and relevant technologies
Direct
guidance
Sorting Hiding Annotation Map adaptation
Global
guidance
WebWatcher
ITEM/IP
ISIS-Tutor
SHIVA
Adaptive
HyperMan
CID
HYPERFLEX
Local
guidance
Land Use
Tutor
HyperTutor
Adaptive
HyperMan
ELM-PE
Hypadapter
HYPERFLEX
Hypadapter
PUSH
ISIS-Tutor
ELM-ART HYPERCASE
Local
orientation
support
(knowledge)
Hypadapter
ELM-PE
[Clibbon]
HyperTutor
Hypadapter
ISIS-Tutor
ELM-ART
ISIS-Tutor
ITEM/PG
Manuel Excel
Local
orientation
support
(goal)
Hynecosum
HyPLAN
ISIS-Tutor
PUSH
SYPROS
ELM-ART
ISIS-Tutor HYPERCASE
Global
orientation
support
[Clibbon]
Hynecosum
HyperTutor
ISIS-Tutor
SYPROS
ITEM/PG
ISIS-Tutor
ELM-ART
Manuel Excel
HYPERCASE
![Page 24: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/24.jpg)
What can be adapted: links
• Contextual links (“real hypertext”)
• Local non-contextual links
• Index pages
• Table of contents
• Links on local map
• Links on global map
![Page 25: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/25.jpg)
Link types and technologies
Directguidance
Sorting Hiding Annotation Mapadaptation
Contextual links OK (disabling) OK
Non-contextual links OK OK ? OK
Table of contents OK ? OK
Index OK ? OK
Local map OK OK OK OK
Global map OK OK OK OK
![Page 26: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/26.jpg)
Adaptive navigation support: evaluation
• Sorting HYPERFLEX, 1993
• Annotation (colors) and hidingISIS-Tutor, 1995
• Annotation (icons)InterBook, 1997
• HidingDe Bra’s course, 1997
![Page 27: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/27.jpg)
Evaluation of sorting
• HYPERFLEX: IR System– adaptation to user search goal– adaptation to “personal cognitive map”
• Number of visited nodes decreased (significant)
• Correctness increased (not significant)
• Goal adaptation is more effective
• No significant difference for time/topic
![Page 28: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/28.jpg)
Adaptive Hypermedia: our approach
ISIS-Tutor, MSU (1992-1994) ITEM/PG, MSU (1991-1993)
SQL-Tutor, MSU (1995-1998) ELM-ART, Trier (1994-1997)
InterBook, CMU (1996-1998) ELM-ART II, Trier (1997-1998)
ITEM/IP, MSU (1986-1994)
ADAPTS, CMU (1998-1999) COCOA, CTE (1999-2000)
![Page 29: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/29.jpg)
Annotation and hiding: ISIS-Tutor
• An adaptive tutorial for CDS/ISIS/M users• Domain knowledge: concepts and constructs• Hyperspace of learning material:
– Description of concepts and constructs– Examples and problems indexed with concepts
(could be used in an exploratory environment)
• Link annotation with colors and marks• Removing links to “not relevant” pages
![Page 30: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/30.jpg)
Concepts, examples, and problems
Example 2 Example M
Example 1
Problem 1
Problem 2 Problem K
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
Examples
Problems
Concepts
![Page 31: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/31.jpg)
Indexing and navigation
Example 2 Example M
Example 1
Problem 1
Problem 2 Problem K
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
Examples
Problems
Concepts
![Page 32: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/32.jpg)
Student modeling and adaptation
• States for concepts:– not ready (may be hidden)– ready (red)– known (green)– learned (green and ‘+’)
• State for problems/examples:– not ready (may be hidden)– ready (red)– solved (green and ‘+’)
![Page 33: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/33.jpg)
Sample index page (annotation)
![Page 34: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/34.jpg)
Sample index page (hiding)
![Page 35: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/35.jpg)
ISIS-Tutor: Evaluation
• 26 first year CS students of MSU
• 3 groups: – control (no adaptation)– adaptive annotation– adaptive annotation + hiding
• Goal: 10 concepts (of 64), 10 problems, all examples
![Page 36: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/36.jpg)
Experiment design
Educational status of the node
behind the link
A. Non-
adaptive
B. Adaptive
annotation
C. Adaptive
hiding
Outside educational goal NA NA hidden
Within educational goal NA mark "-" mark "-"
Well-learned NA mark "+" mark "+"
Known NA green color green color
Ready-to-be-learned NA red color red color
Not-ready-to-be-learned NA NA hidden
![Page 37: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/37.jpg)
Results: performance
Group Number of steps Time (sec) Concept
repetitions
"Unforced"
concept
repetitions
Task
repetitions
Non-adaptive 81.3 2196 17.3 11.2 6.2
Adaptive 65.2 1418 9.0 5.0 0.8
Restrictive 58.2 1785 8.9 4.8 0.4
Adaptive annotation makes navigation more efficient
![Page 38: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/38.jpg)
The value of adaptivity: steps
30
40
50
60
70
80
90
100
110
120
130
Units
Steps
Hiding
Annotation
Nonadaptive
![Page 39: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/39.jpg)
The value of adaptivity: repetitions
0
2
4
6
8
10
12
14
16
18
Concept repetitions Unforced repetitions
Hiding
Annotation
Nonadaptive
![Page 40: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/40.jpg)
Results: navigation
Group Concept index Concept ->
concept
Concept ->
task
Task index Task ->
concept
Non-adaptive 18.5 8.1 9.8 12.2 2.0
Adaptive 18.4 1.8 6.4 9.6 2.2
Restrictive 14.0 1.6 8.1 7.0 1.8
No effect on navigation patterns due to variety of navigation styles
![Page 41: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/41.jpg)
Results: recall
Group
Recalled concepts
correct
Recalled concepts
incorrect
Recalled links
correct
Recalled links
incorrect
Non-adaptive 7.0 0.7 8.6 5.0
Adaptive 6.2 1.2 7.5 8.5
Restrictive 6.9 0.8 5.7 5.1
No effect on recall
![Page 42: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/42.jpg)
To hide or not to hide?
Additional value of hiding is unclear. Users prefer “freedom”
Group % users visited
non-goal tasks
% users visited
not-ready tasks
% users visited
non-goal
concepts
% users visited
not-ready concepts
visited non-
goal
concepts
Non-adaptive 20 66 33 0 1.0
Adaptive 0 100 80 20 2.8
Restrictive 0 0 0 0 0.0
![Page 43: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/43.jpg)
Evaluation of hiding
• Adaptive course on Hypertext (De Bra)
• Hiding “not ready” links
• Hiding obsolete links
• Small-scale evaluation
• No significant differences
• Students are not comfortable with disappearing links
![Page 44: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/44.jpg)
InterBook: concept-indexed ET
• “Knowledge behind pages”
• Structured electronic textbook (a tree of “sections”)
• Sections indexed by domain concepts– Outcome concepts– Background concepts
• Concepts are externalized as glossary entries
• Shows educational status of concepts and pages
![Page 45: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/45.jpg)
Sections and concepts
Chapter 1
Chapter 2
Section 1.1
Section 1.2
Section 1.2.1 Section 1,2,2
Textbook
![Page 46: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/46.jpg)
Sections and concepts
Chapter 1
Chapter 2
Section 1.1
Section 1.2
Section 1.2.1 Section 1,2,2
Domain model
Concept 1
Concept 2
Concept 3
Concept 4
Concept m
Concept n
Textbook
![Page 47: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/47.jpg)
Indexing and navigation
Chapter 1
Chapter 2
Section 1.1
Section 1.2
Section 1.2.1 Section 1.2.2
Domain model
Concept 1
Concept 2
Concept 3
Concept 4
Concept m
Concept n
Textbook
![Page 48: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/48.jpg)
![Page 49: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/49.jpg)
Glossary view
![Page 50: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/50.jpg)
Navigation in InterBook
• Regular navigation– Linear (Continue/Back)– Tree navigation (Ancestors/Brothers)– Table of contents
• Concept-based navigation– Glossary (concept -> section)– Concept bar (section -> concept)– Hypertext links (section -> concept)
![Page 51: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/51.jpg)
Adaptive navigation support
• Adaptive annotations– Links to sections– Links to concepts– Pages
• Adaptive sorting– Background help
• Direct guidance (course sequencing)– Teach Me
![Page 52: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/52.jpg)
User modeling
• Overlay student model for domain concepts
• Knowledge states for each concept– unknown (never seen)– known (visited some page)– learned (passed a test)
• Information for sections– visited/not visited– time spent
• Information for tests: last answers
![Page 53: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/53.jpg)
Adaptive annotation
• Educational status for concept unknown
known
learned
• Educational status for sections not ready to be learned
ready to be learned
suggested
![Page 54: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/54.jpg)
Adaptive annotation in InterBook
1. State of concepts (unknown, known, ..., learned)
2. State of current section (ready, not ready, nothing new)
3. States of sections behind the links (as above + visited)
3
2
1
v
![Page 55: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/55.jpg)
Bookshelves and books
![Page 56: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/56.jpg)
Book view
![Page 57: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/57.jpg)
Glossary view
![Page 58: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/58.jpg)
Backward learning: “help” and “teach this”
![Page 59: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/59.jpg)
InterBook: Evaluation
• Goal: to find a value of adaptive annotation
• Electronic textbook about ClarisWorks
• 25 undergraduate teacher education students
• 2 groups: with/without adaptive annotation
• Format: exploring + testing knowledge
• Full action protocol
![Page 60: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/60.jpg)
Preferred ways of navigation
0
2
4
6
8
10
12
14
16
18
20
Cell Mean
pCONTINUE
pBACK pTEXT
pCONTENT
pINTRODUCINGpREQUIRING
no
yes
Cell Bar Char t Split By: ANS Inclusion cr iter ia: Hits > 15 from Eklund Separated.stv
![Page 61: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/61.jpg)
0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
Units
%Annotated % Sequential
no
yes
Box Plot Split By: ANS Inclusion cr iter ia: Hits > 15 from Eklund Separated.stv
.213 .172 .042 17 .043 .742 0
.282 .226 .080 8 .059 .742 0
.151 .072 .024 9 .043 .273 0
.583 .258 .063 17 .097 .913 0
.516 .306 .108 8 .097 .909 0
.643 .207 .069 9 .364 .913 0
Mean Std. Dev. Std. Er ror Count Minimum Maximum # Missing
%Annotated, Total
%Annotated, yes
%Annotated, no
% Sequential, Total
% Sequential, yes
% Sequential, no
Descr iptive Statistics Split By: ANS Inclusion cr iter ia: Hits > 15 from Eklund Separated.stv
![Page 62: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/62.jpg)
The effect of “following green”with adaptation
6
6.5
7
7.5
8
8.5
9
9.5
10
Units
Score
yes, Low-negative
yes, Low-positive
yes, High-positive
![Page 63: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/63.jpg)
Adaptation mechanisms do work!
0
20
40
60
80
100
120
140
160
Units
s1t/ s1 s2t/ s2 s3t/ s3
Box Plot Split By: Par t Inclusion cr iter ia: Hits > 15 from Eklund Separated.stv
![Page 64: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/64.jpg)
Results
• No overall difference in performance
• Sequential navigation dominates
...but ...
• Adaptive annotation encourage non-sequential navigation
• The effect of “following green”
• The adaptation mechanism works well
![Page 65: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/65.jpg)
Where is the magic?
• No magic: Knowledge behind material• Knowledge about domain (subject)• Knowledge about documents
– Simple concept indexing
• Knowledge about students– Learning goal model– Overlay student model
• Straightforward techniques of user modeling and adaptation
![Page 66: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/66.jpg)
Adaptive Hypertext: The Secret
• Adaptive hypertext has knowledge “behind” the pages
• A network of pages like a regular hypertext plus a network of concepts connected to pages
Pool of Learning ItemsDomain Model
![Page 67: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/67.jpg)
Adaptive Hypertext: Design
• Design and structure knowledge space• Design a generic user model• Design a set of learning goals• Design and structure the hyperspace of
educational material• Design connections between the
knowledge space and the hyperspace of educational material
![Page 68: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/68.jpg)
Domain model - the key
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
![Page 69: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/69.jpg)
Domain model - the key
• Knowledge about domain is decomposed into a set of fragments - domain knowledge elements (DKE)– Also called topics, knowledge items, concepts,
learning outcomes…• Most often DKE is just a name denoting a piece of
knowledge, sometimes it has internal structure• Semantic relationships between DKE can be
established
![Page 70: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/70.jpg)
Vector vs. network models
• Vector - no relationships
• Precedence (prerequisite) relationship
• is-a, part-of, analogy: (Wescourt et al, 1977)
• Genetic relationships (Goldstein, 1979)
![Page 71: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/71.jpg)
Vector (set) model
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
![Page 72: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/72.jpg)
Network model
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
![Page 73: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/73.jpg)
Overlay user model: knowledge
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
![Page 74: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/74.jpg)
Simple overlay model
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept Nyesno
no
noyes
yes
![Page 75: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/75.jpg)
Simple overlay model
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept Nyesno
no
noyes
yes
![Page 76: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/76.jpg)
Weighted overlay model
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N103
0
27
4
![Page 77: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/77.jpg)
Simple goal model
• Learning goal as a set of topics
![Page 78: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/78.jpg)
More complicated models
• Sequence, stack, tree
![Page 79: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/79.jpg)
Indexing: the key to AH
• Types of indexing– Knowledge-based hypertext (concept = node)– Indexing of nodes– Indexing of Fragments
• How to get the hyperspace indexed?– Manual indexing (closed corpus)– Computer indexing (open corpus)
![Page 80: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/80.jpg)
Knowledge-based hypertext
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
![Page 81: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/81.jpg)
Indexing of nodes
Domain model
Concept 1
Concept 2
Concept 3
Concept 4
Concept m
Concept n
Hyperspace
![Page 82: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/82.jpg)
Indexing of page fragments
Fragment 1
Fragment 2
Fragment K
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
NodeConcepts
![Page 83: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/83.jpg)
Overlay model + indexing
• Indexing nodes with concepts– InterBook, ELM-ART, ISIS-Tutor, AHA
• Indexing anchors with concepts– StrathTutor
• Indexing fragments with concepts– MetaDoc, AHA, PT
• Nodes are concepts– InterBook, ELM-ART, ISIS-Tutor
![Page 84: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/84.jpg)
Generalized overlay model
• Knowledge– overlay for set of concepts, network of concepts
• Goals– overlay for set of possible goals, tree of goals
• Stereotypes– overlay for set of stereotypes
![Page 85: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/85.jpg)
Hyperspace structuring
• Concept-based hyperspace• No imposed structure• Hierarchy• ASK approach - conversational
relationships
![Page 86: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/86.jpg)
Indexing with generalized model
• fragments are indexed with goals– PUSH
• nodes are indexed with user’s tasks– HYNECOSUM:
• nodes are indexed with stereotypes– EPIAIM, Anatom-Tutor, C-Book
• goals are nodes– HYPERFLEX
![Page 87: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/87.jpg)
What else?• More functionality with the same knowledge
Adaptive hiding (ISIS-Tutor, AHA!)Adaptive presentation (AHA!)Adaptive Testing (ELM-ART)Adaptive Recommendation (Hirashima)
• More functionality with more knowledge– Domain specific
Knowledge about programming (ELM-ART)
– Domain-neutralAdvanced indexing (ADAPTS and COCOA)
![Page 88: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/88.jpg)
ELM-ART: Lisp ITS on WWW
• Model: adaptive electronic textbook– hierarchical textbook – tests– examples– problems– programming laboratory
• Extra for Web-based teaching– messages to the teacher– chat room
![Page 89: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/89.jpg)
ELM-ART: navigation and testing
![Page 90: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/90.jpg)
Knowledge representation
• Domain knowledge– conceptual network for Lisp– problem solving plans– debugging knowledge
• Student model– overlay model for Lisp concepts– episodic model for problem-solving knowledge
![Page 91: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/91.jpg)
![Page 92: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/92.jpg)
Adaptivity in ELM-ART
• Adaptive navigation support
• Adaptive sequencing
• Adaptive testing
• Adaptive selection of relevant examples
• Adaptive similarity-based navigation
• Adaptive program diagnosis
![Page 93: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/93.jpg)
ANS + Adaptive testing
![Page 94: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/94.jpg)
Adaptive Diagnostics
![Page 95: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/95.jpg)
Similarity-Based Navigation
![Page 96: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/96.jpg)
ELM-ART: Evaluation
• No formal classroom study
• Users provided their experience
• Drop-out evaluation technology
• 33 subjects– visited more than 5 pages– have no experience with Lisp– did not finish lesson 3– 14/19 with/without programming experience
![Page 97: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/97.jpg)
ELM-ART: Value of ANS
With adaptive
annotation
Without adaptive
annotation
NEXT button 21.0 (N = 4) 25.0 (N = 3) 22.7 (N = 7)
No NEXT button 13.8 (N = 5) 9.5 (N = 2) 12.6 (N = 7)
17.0 (N = 9) 18.8 (N = 5) 17.7 (N = 14)
Mean number of pages which the users with no experience in programming languages completed with ELM-ART
![Page 98: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/98.jpg)
ELM-ART: Value of ANS
With adaptive
annotation
Without adaptive
annotation
NEXT button 23.5 (N = 6) 14.0 (N = 3) 20.3 (N = 9)
No NEXT button 22.4 (N = 5) 12.6 (N = 5) 17.5 (N = 10)
23.0 (N = 11) 13.1 (N = 8) 18.8 (N = 19)
Mean number of pages which the users with experience in at least one programming language completed with ELM-ART
![Page 99: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/99.jpg)
Adaptive annotation can:
• Reduce navigation effortsResults are not significant (variety of styles?)
• Reduce repetitive visits to learning pagesSignificant - if applied properly
• Encourage non-sequential navigation
• Increase learning outcomeFor those who is ready to follow and advice
• Make system more attractive for students
![Page 100: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/100.jpg)
What else?• More functionality with the same knowledge
Adaptive hiding (ISIS-Tutor, AHA!)Adaptive presentation (AHA!)Adaptive Testing (ELM-ART)Adaptive Recommendation (Hirashima)
• More functionality with more knowledge– Domain specific
Knowledge about programming (ELM-ART)
– Domain-neutralAdvanced indexing (ADAPTS and COCOA)
![Page 101: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/101.jpg)
ONR research project Architecture for integration of:
– Diagnostics– Technical Information– Performance-oriented Training
Technology investigation & testbed
ADAPTS
ADAPTS: What it is
IETMsTraining
Diagnostics
![Page 102: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/102.jpg)
Use
r M
od
el
Diagnostics
Content
Navigation
What task to doSystem health
What content is applicable to thistask and this user
Levels of detail
How to display this content to this user
Experience,
Preferences,
ASSESSES: DETERMINES:
Adaptive Diagnostics
Personalized Technical Support
![Page 103: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/103.jpg)
Video clips
(Training)Schematics
EngineeringData
Theory ofoperation
Blockdiagrams
Equipment
Simulations
(Training)
EquipmentPhotos
Illustrations
TroubleshootingStep
Troubleshooting step plus
hypermedia support
information, custom-
selected for a specific
technician within a specific
work context.
ADAPTS dynamically assembles custom-selected content.
What’s in adaptive content?
![Page 104: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/104.jpg)
ADAPTS - an adaptive IETM
![Page 105: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/105.jpg)
The result
Maintenance
history Preprocessed,
condition-basedinputs
Technicianand OperatorObservations
Sensor inputs(e.g., 1553 bus)
PersonalizedDisplay
IETM TrainingStretch
text OutlineLinks
Training records
Skill assessment Experience
Preferences
Content NavigationDiagnostics
How do we make decisions?
User Model
![Page 106: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/106.jpg)
Concept A
Concept A
ConceptB
ConceptB
ConceptC
ConceptC
SupportingInformation
Domain model
• Defines relationships between elements of technical information
• Indicates level of difficulty/ detail
• Shows prerequisites
![Page 107: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/107.jpg)
Domain model example
CONCEPTReeling Machine
CONCEPTSonar Data Computer
CONCEPTSonar System
RemovalInstructions
TestingInstructions
IllustratedParts
Breakdown
Principles of
Operation
Principles of
Operation
Principles of
Operation
RemovalInstructions
RemovalInstructions
TestingInstructions
TestingInstructions
IllustratedParts
Breakdown
IllustratedParts
Breakdown
![Page 108: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/108.jpg)
ReelingMachine
ReelingMachine
Sonar SystemSonar System
General Component Location
Principles of operation
Removal instructions
Principles of Operation
System Description DetailsParts List
Power Distribution
Domain model example
PART OF
Testing instructions
SUMMARY
DETAILS
TUTORIAL
![Page 109: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/109.jpg)
User model
• Characterizes user ability at each element of the domain model– Size of model is bounded by domain– Weights on different types of elements account
for learning styles and preferences– Can be time sensitive
• Constrains the diagnostic strategy
![Page 110: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/110.jpg)
User model example
Certified
CONCEPTReeling Machine
CONCEPTSonar Data Computer
CONCEPTSonar System
ROLERemoval
Instructions
ROLETesting
Instructions
ROLEIPB
ReviewedHands-on
Simulation
AT2 Smith
AD2 Jones
Preference
Reviewed
Hands-on+
Certified
Reviewed
Hands-on
Hands-on Reviewed
Reviewed
ROLETheory of Operatio
n
![Page 111: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/111.jpg)
Adaptive content selection
• Information is custom-selected for a user– Level of detail offered depends upon who the
user is (i.e., his level of expertise)– Selected at a highly granular level, e.g., for
each step within a procedure
• Performance-oriented training is presented as part of content
![Page 112: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/112.jpg)
Integrated interface
![Page 113: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/113.jpg)
Summary
• Concept-based approach to adaptive hypertext and adaptive WBS
• Concept indexing: Knowledge behind pages
• Explored– Different levels of model complexity– Different application domains– Different adaptation techniques
![Page 114: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/114.jpg)
The complexity issue
ISIS-Tutor, MSU (1992-1994)
COCOA, CTE (1998-1999)
ELM-ART, Trier (1994-1997)
InterBook, CMU (1996-1998)
ADAPTS, CMU, (1997-1998)ITEM/IP, MSU (1986-1994)
![Page 115: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/115.jpg)
Open Corpus AH
• Current AH techniques are based on manual page/fragment indexing– How to work with open Web and Digital
Libraries?
• Ways being explore– Add open corpus content, but index it manually– Use IR techniques for content-based adaptation
without manual indexing– Use social navigation approaches for adaptation
![Page 116: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/116.jpg)
Social Navigation Support
![Page 117: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/117.jpg)
AH Service: NavEx
![Page 118: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/118.jpg)
AH Service: QuizGuide
![Page 119: Adaptive Hypermedia From Concepts to Authoring](https://reader035.vdocuments.mx/reader035/viewer/2022062409/568146b4550346895db3d32d/html5/thumbnails/119.jpg)
More information...
• Adaptive Hypertext and Hypermedia Home Page: http://wwwis.win.tue.nl/ah/
• Brusilovsky, P., Kobsa, A., and Vassileva, J. (eds.) (1998), Adaptive Hypertext and Hypermedia. Dordrecht: Kluwer Academic Publishers.
• Brusilovsky, P. (2001) Adaptive hypermedia. User Modeling and User Adapted Interaction, Ten Year Anniversary Issue (Alfred Kobsa, ed.) 11 (1/2), 87-110
• Brusilovsky, P. (2003) Developing adaptive educational hypermedia systems: From design models to authoring tools. In: T. Murray, S. Blessing and S. Ainsworth (eds.): Authoring Tools for Advanced Technology Learning Environment. Dordrecht: Kluwer Academic Publishers.