ソフトウェア工学特論　（２）

田中　譲

Augmentation Media Architectures and Technologies; A Brief Survey

History and Evolution of Augmentation Media

(1) personal-augmentation media (2) group-augmentation media (3) organization-augmentation media (4) social-augmentation media

Pioneer: Vannevar Bush

“As We May Think,” The Atlantic Monthly, 1945 memex Personal augmentation media

Pioneer: Doug Engelbart

“Augmenting Human Intellect: A Conceptual Framework.”

AFOSR-3233 Summary Report 1962

AUGMENT/NLS group- and

organization-augmentation media

Pioneering Firsts by Engelbart

the mouse 2-dimensional display editing in-file object addressing, linking hypermedia outline processing flexible view control multiple windows cross-file editing integrated hypermedia email hypermedia publishing document version control shared-screen teleconferencing computer-aided meetings

Pioneering Firsts by Engelbart

formatting directives context-sensitive help distributed client-server architecture uniform command syntax universal "user interface" front-end module multi-tool integration grammar-driven command language interpreter protocols for virtual terminals remote procedure call protocols compileable "Command Meta Language"

Pioneer: Ted Nelson

coined the word “hypertext” ‘transclusion links’

– Any update of the original should be immediately reflected in its quotations.

Pioneer: Ted Nelson

Xanadu (early 1960s )– Anyone can publish his or her works in this reservoir,

explore this reservoir to browse some others’ works, and purchase any chunks of text from others' works for reuse in his or her own works without violating copyright.

– His vision has stimulated researchers to develop hypermedia system technologies, WWW [13], and HTML browser systems such as NCSA Mosaic [14], Netscape Navigator, and Microsoft Internet Explorer.

Nelson started from personal-augmentation media and continues to aim at social-augmentation media.

Pioneer: Alan Kay

XEROX PARC Reactive Engine in

1969 – renamed “Dynabook”

in 1972

Pioneer: Alan Kay

Alto machine– personal computer

Smalltalk– interactive object-

oriented interpretive language

Pioneer: Nicholas Negroponte

1967: Architecture Machine Group in the Architecture Department at MIT

1976: “Augmentation of Human Resources in Command and Control through Multiple Media Man-Machine Interaction.”

– research program to the U.S. Defense Advanced Research Projects Agency (DARPA).

– resulted in the Spatial Data Management System (SDMS) – Computer Corporation of America developed a practical version

of SDMS

Pioneer: Nicholas Negroponte

1970s: Aspen project 1980 to 1983, a prototype of the electronic

book 1985: MIT Media Lab

Evolution of personal-augmentation media 1986, Xerox PARC released NoteCards 1987, Bill Atkinson developed HyperCard Analyst, a commercial product developed at Xerox XSIS

Lab – Using a hypermedia framework, it integrated a wide range of

object-oriented applications, including word processing, desktop publishing, spreadsheets, forms, graphics, images, databases, and maps, with rule-based organization tools and an expert-system shell.

– It extensively used a media framework, especially a compound-document framework, for the first time to integrate varieties of applications.

– Analyst was a closed integration OIS (Office Information System).

Visual computing technologies during the last 15 window widgets

– X-Window (X11) with Xlib and Xt Intrinsics-Based Toolkit,

– OSF-Motif, OPEN LOOK Intrinsics Toolkit (OLIT)– Xview Toolkit, – Sun’s NeWS, – NextStep Application Kit, – HP NewWave, – Borland ObjectWindows class library, and – Apple’s MacApp.

The evolution of group-augmentation media

In 1984, Irene Greif and Paul Cashman introduced the term computer supported cooperative work (CSCW) as a way of describing how computing technology can help people to work together in groups.

Potential applications for CSCW

(1) Collaboration tools help groups agree on missions, allocate tasks and roles, and undertake specific group activities.

(2) Meeting room systems support face-to-face meeting activities.

(3) Desktop video conferencing enables two or more geographically separated people to work together using a common screen display, video images of other people in separate windows, and an integrated voice connection.

(4) Procedure processing or workflow technology automate paper-based forms handling and provide full summary information about status, whereabouts, and over-runs.

The evolution of organization-augmentation media For small groups, the augmentation focuses more

on tele-presence of each participant’s personality, while, for large groups, the augmentation focuses more on tele-working and coordination, rather than on individual personalities.

coordination of a large number of interrelated activities,

resource scheduling of people, products, and money, and

information management and distribution of all documents.

The evolution of social-augmentation media Xanadu, proposed by Ted Nelson WWW (World-Wide Web)

– developed by volunteers, since the original team’s main concern was supporting High Energy Physics research at CERN.

– links called URLs (Universal Resource Locators). – HyperText Markup Language (HTML)

its browser systems: NCSA Mosaic 1995, Netscape Navigator

History and Evolution of Knowledge-media Architectures Ivan Sutherland in early 1960s: Sketch Pad

– first introduction of a paper media concept into a computer system

NLS and AUGMENT by Doug Engelbart Alan Kay proposed Reactive Engine in 1969.

– a clear view of computers as media. In 1970s, Xerox PARC developed personal

workstations starting with Alto and leading to Star, and also object-oriented interactive languages such as Smalltalk for the development of a direct-manipulation graphical user interface.

Compound document architectures

began in the early 80's with the DTP – The Star workstation from Xerox – WYSIWYG (What You See Is What You Get)

Display PostScript multimedia DTPR Object-oriented system design allowed us to embed

programs in complex documents.– Trillium system by Austin Henderson used such buttons to rapidly

prototype a control panel for a photocopy machine. – Trillium also used several types of display panels that could be

pasted on the base panel. – Such systems are sometimes called glue systems

Compound document architectures

to embed display objects of various application programs into compound documents as components

how to define the application linkages? Two kinds of linkages have been proposed:

– object containment and functionally relates an embedded component object to its base object.

– object wiring the direct setting of a graphical link to interrelate two different application

objects. HP’s NewWave was the first commercial attempt for object containment and

wiring Around 1995, we saw two competitive commercial attempts to

integrate compound document architectures with object containment and wiring.

– Microsoft’s OLE (Object Linking and Embedding) [86] and – CI (Component Integration) Lab’s OpenDoc

2 styles of object-oriented programming Class-hierarchy programming (or refinement-based

programming) defines classes in their hierarchy to allow subclasses to inherit properties of their super classes.

– originated in program development Synthetic programming (or composition-based

programming) on the other hand defines a set of object instances to serve as primitive objects.

– originated both in direct manipulation of visible objects, and in prototyping techniques based on the reusability paradigm

Programmers can make a copy of any object instance to create a new independent instance with the same property as the original.

They can combine several instances of different types to define a new composite object.

Media toolkit systems

Around the mid 80s, the synthetic paradigm of object-oriented programming developed such GUI toolkit systems for window systems as Xt of MIT [34], Andrew Toolkit of CMU [88], OPEN LOOK Toolkit [36], and Motif Toolkit [35].

Around the late 80's, DBMS providers began to provide each of their DBMS products with a form construction kit.– a form interface to a database. – Studio from Ontologics– GainMomentum from Sybase.

IntelligentPad as a meme media system IntelligentPad (1989) was initially developed as a

media toolkit system, and later, around 1990, extended to serve as meme media.

The evolution of a meme pool requires that end-users can easily re-edit and redistribute meme media. For this reason, meme media need to provide not only ease of composition, but also ease of decomposition.

Meme Media and their Applications

multimedia with intelligence directly manipulable and reusable software

components GUI construction kit a framework for integrating documents and tools publication media for multimedia documents

and tools re-editable and redistributable media for

intellectual resources

An Outline of IntelligentPad

IntelligentPad as Meme Media

Component Pads anda Composition with them

Unified Treatment of Documents and Tools Various Pads

Pads to represent ...

Multimedia Documents– Multimedia components

Tools (Software Systems)– Application Systems – Utilities– Servers (Proxy Pads)– Layout Tools

Data & Rules

IntelligentPad as Meme MediaCompound Document

P2 P3

P1Containers and their Connections

P2 P3

P1

Seamless and Integrated Support of Our Intellectual Activities

think

seetry

artificial-intelligence systems

computation-intensive systems ?

IntelligentPad as Meme Media

Pads to represent Documents and Tools Compound Document Architecture Composition / Decomposition by Users Transportable across Platforms

– Decomposable at Destinations– Exchange Format – Standard APIs + Script Language

IntelligentPad Has Adopted ...

wrapper architecture frame architecture MVC architecture update dependency architecture compound document architecture

Wrapper Architecture:Objects and Linkages

(a) objects (b) linkages

(a) objects (b) linkages

Object-orientation

Media-based

Wrapper Architecture of a Pad

a pad as a

media object

intellectual

resource

display

pin plug

connection jacks(slots)

Frame Architecture

update

……………..

setProcedureForSlot1

……………..

gimmeProcedureForSlot1

setProcedureForSlotn

……………..

gimmeProcedureForSlotn

……………..

Pad Definition

MVC Architecture of a Pad

simplified from the well-known MVC

MVC

Controller ModelView message

sending

update propagationmessage

sending

Well-known MVC

M

VCController

Model

View

update propagation

message sending

Two Ways of Putting an Object in a Document

(b) the object linking(a) the object embedding

DocumentObjectDocument

copy

Object

Extended Compound Document Architecture

text as the base

object

object

padpad

pad

pad pad

2 layers multiple layers

Paste Operation andSlot Connection

Compound Document

P2 P3

P1Containers and their Connections

P2 P3

P1

Standard Message I/F between Pads

MVC

Controller ModelView message

sending

MVC

↓gimme [<slot_name>]

↓set [<slot_name>] <value>

↑update

P1

P2

A View Hierarchy and MVC

M

VC

Subview CSubview B

Controller C

Controller A

Controller B

Model A

View CView B

View A

VC VC

Subview C

Subview B

View A

(a)a hierarchy of views

and subviews

(b) View and subviews are connected through view parts

Composition of Applications by Pasting

components composition

Update Dependency

Logical state = Logical State

update dependencyThe logical state need not be identical to the state.

Event Notification

resize, copy, move

resized, copied, moved

resized, copied, moved

Overlay & Master-Slave Relationship

Pad A

Pad B

Pad C

Pad C

Pad B

Pad A

Pad D

Pad E

Pad D

Pad E

They are brothers of each other

Shared Copies of a Primitive Pad

CC VV

M

P’P

P’P

M

Shared Copies of a Composite Pad

M

independent copies

shared copies

Composite

pad P’

P P’

Composite

pad P

the model of the base pad

Extension toward 3D Representation Media: IntelligentBox