2 December 2005

Open Cross-Document Linking and Browsing

based on a Visual Plug-in Architecture

Ahmed A.O. Tayeh and Beat Signer

Web & Information Systems Engineering Lab (WISE)Department of Computer ScienceVrije Universiteit Brussel

WEB & INFORMATIONSYSTEMS ENGINEERING

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 2

Isolated Digital Documents

×

HTML document PDF document

×

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014

Isolated Digital Documents …

3

×××

Word document PDF document

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 4

Isolated Digital Documents …

Limited possibilities to create hyperlinks between snippets of different document formats

document formats mainly address links to web resources (HTML)

simple embedded unidirectional links

linked documents are not aware of any links that have been defined from other source documents

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 5

Existing Support for Hyperlinks

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 6

Cross-Document Linking

Anchor (page index + ec(x,y,w,h)) Anchor (XPointer-like expression)

Anchor (start and end indices) Anchor(XPointer expression)

PDF (linear-like model) HTML (tree model)

Text (linear model) XML (tree model)

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 7

Open Hypermedia

No support for cross-document linking Limited extensibility

on the data as well ason the visual level

Microcosm

Intermedia

Sun’s link service

MADOW

AnnoteaMADCOW: A Multimedia Digital Annotation System,Paolo Bottoni et al., Proceedings of AVI 2004

Monolithic Component

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 8

Requirements for a Linking Service

No changes required to existing document formats and standards

No assumptions about the document models e.g. WYSIWYG, tree models, constrained tree models, …

Support existing as well as emerging document formats

Support advanced linking features

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 9

Open Cross-Media Architecture

Annotation/Link Service

Client Application

Annotation/Link Browser & Editor

Visual Plug-ins

Resource Plug-in Repository

Data Plug-ins

Visual Plug-ins Data Plug-insVisual Plug-ins

An Architecture for Open Cross-Media Annotation Services, Beat Signer and Moria C. Norrie, Proceeding of WISE 2009

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 10

Data Level Extensibility (RSL Approach)

As We May Link: A General Metamodel for Hypermedia Systems, Beat Signer and Moria C. Norrie, Proceeding of ER 2007

……

file://URLfile://URL…shapeXPointer(s,e)XPointer…

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 11

Visualisation Layer Extensibility

local visual plug-ins

external visual plug-ins com

mun

icat

e

Link Browser

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 12

Visualisation Layer Extensibility …

Local visual plug-ins vs. external visual plug-ins

Link Browser

communication communication

XML visual plug-in Text visual plug-in

Link visual plug-in

Link visual plug-in

Acrobat Reader Google Chrome

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 13

Visualisation Layer Extensibility …

DefaultDocument class necessary methods to visualise

any document format- getSelector()

- openDocument()

- …

event listeners and handlers

Local visual plug-ins extend DefaultDocument class might use existing visualisation libraries

Link browser instantiates DefaultDocument

DocFormat1 DocFormat3 DocFormat2

Visual Plug-ins

Visualisation

Gateway DocFormat3

DocFormat3 Application

Third-Party Applications

Vis

ualis

ation

Doc

Form

at1

Vis

ualis

ation

Doc

Form

at2

Browser

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 14

Visualisation Layer Extensibility …

External visual plug-ins provide methods to handle selections in third-party applications communicate with the link browser

via a specific gateway plug-in

Gateway plug-ins launch third-party application offer different communication

protocols- TCP sockets- WebSockets- REST API as a fallback

DocFormat1 DocFormat3 DocFormat2

Visual Plug-ins

Visualisation

Gateway DocFormat3

DocFormat3 Application

Third-Party Applications

Vis

ualis

ation

Doc

Form

at1

Vis

ualis

ation

Doc

Form

at2

Browser

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 15

Open Cross-Document Link Service Architecture

DocFormat3

Database

DocFormat2DocFormat1

DocFormat1 DocFormat3 DocFormat2

Visual Plug-ins

Visualisation

Gateway DocFormat3

RSL

Database Manager

DocFormat3 Application

Third-Party ApplicationsData Plug-ins

Core

Data Layer

Visu

alis

ation

Doc

Form

at1

Visu

alis

ation

Doc

Form

at2

Browser

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 16

Open Service Gateway initiative (OSGi)

Enhances modularisation of our linking service Use OSGi dynamic extensibility for dynamic extensibility

of the linking service OSGi versioning and dependency resolution reduces

problems in dealing with multiple libraries (“JAR hell”) linking service might provide different visual plug-ins (versions)

for a single document format

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014

Open Service Gateway initiative (OSGi) …

User Interface Manifest

Manifest-Version: 1.0

Bundle-ManifestVersion: 2

Bundle-Name: UserInterface

Bundle-SymbolicName: com.rsl.userInterface

Bundle-Version: 1.0.0.qualifier

Bundle-RequiredExecutionEnvironment: JavaSE-1.7

Export-Package: org.rsl.userInterface

Import-Package: org.rsl.core, org.rsl.service,

org.rsl.databasemanager,

org.associations.collections,

org.osgi.framework;version="1.7.0"

PDF Local Visual Plug-in Manifest

Manifest-Version: 1.0

Bundle-ManifestVersion: 2

Bundle-Name: Visual

Bundle-SymbolicName: org.rsl.pdf.visual

Bundle-Version: 1.0.0.qualifier

Bundle-RequiredExecutionEnvironment: JavaSE-1.7

Import-Package: org.rsl.core, org.rsl.userInterface,

org.rsl.pdf.data

Extension-Class: org.rsl.pdf.visual.Pdf

Extension-Type: visual

Extension-Format: application/pdf

17

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 18

Link Navigation Scenario

Browser RSL Registry DocFormat1 Gateway DocFormat3 DocFormat2 DocFormat3

getSelection()

selector

alt

getPluginType(formatName)

instantiateClass()

openDocument(resource, selectors, selector)

instantiateClass()

openDocument(resource, selectors, selector)openDocument(resource, selectors, selector)

launchApp()

[visualPlugin==local]

[else]

getLinkTarget(selector)

getDocument(resource)

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 19

Evaluation: PDF, XML, Text and HTML Plug-ins

Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.beOctober 14, 2014 20

Conclusion and Future Work

Cross-document link service prototype Extensibility on the data as well as the visualisation layer OSGi framework for extensible architecture Evaluation with PDF, XML, Text and HTML plug-ins

Dynamic plug-in extensibility Investigate extensibility in a study with developers Usability evaluation