2009 02 10 beyond weiser poslad

2/11/2009

1

Beyond Weiser’s Vision of Ubiquitous Computing

Stefan PosladQueen Mary University of London

http://www.eecs.qmul.ac.uk/people/stefan/ubicom

1

Overview

• Ubiquitous Computing: achievements √• Beyond Weiser’s model: Smart DEI model of

Ubiquitous Computing• Research Examples

Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009 2

2/11/2009

2

Ubiquitous Computing (UbiCom)• The term ubiquitous, meaning appearing or

existing everywhere, combined with computing forms the term Ubiquitous Computing (UbiCom)forms the term Ubiquitous Computing (UbiCom)– Term introduced by Marc Weiser in early 1990s– Synonyms: pervasive computing, etc

• UbiCom describes a vision for computing to– Enable computer-based services to be made available

everywherey– Support intuitive human usage but yet, appear to be

invisible to the user.– Situated in physical (and human) world environments


What were ICT Environments Like? • In the late 1980s, when much of the early work on

UbiComp started– people were often unreachable if away from a fixed phone point – computing was only available at a desk computer, attached to the wired

Internet. – some early models of personal computers but no laptops were available– no pervasive wireless networks were available– little location-determination for non-military personnel was available

• Let’s look at the history of the main ICT development trends– A distinction is made between the availability of 1st prototypes (1) vs. 1st

widespread commercial uptake of an ICT product (M). – Difference between the (1) and (M) phases is about 10 years.


2/11/2009

3

Short History of ICT TechnologyWearCom1 Weiser’s

Tabs

WWW1

ClassRoom 2000

Computer11PC1

Lap-top1

PCM

PDA1

Lap-topM PDAM

WLAN1 WLANMTCP/IP Internet1

TCP/IP InternetM

WWWWWWM

Mobile Phone-DM

GPS1GPSM

1940 1970 1980 1990 1995 2000 20051985

5Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

Could also note when specific PC technologies arose, e.g., hard-disk, mouse, removal memory cards, etc

Early UbiCom Research Projects• Smart Devices: CCI

– PARC Tab, MPad & LiveBoard; Active Badge, Bat and Floor

• Smart Environments: CPI and CCI– Classroom 2000, Smart Space and Meeting Room, Interactive

Workspaces and iRoom, Cooltown, EasyLiving and SPOT, HomeLab and Ambient Intelligence

• Smart Devices: CPI– Unimate and MH-1 Robots, Smart Dust and TinyOS

• Smart Devices: iHCI– Calm Computing, Things That Think and Tangible Bits, DataTiles,

WearComp and WearCam, Cyborg

• Other UbiCom Projects


2/11/2009

4

Active Bat• Uses ultrasound, greater accuracy ~ 3 cm. • Base station asks Bat for a signal that is

Active Badge, Bat and Floor

gthen measured in multiple ceiling receivers (must add to environment), position determined using trilateration

Active Floor• Unlike, Badge & Bat, identified someone

by their type of walk or gait, not by carry an identifying token. Cons: scalability, accuracy

• Floor design requires a careful analysis to specify an appropriate spatial resolution and robustness to allow users to walk on sensors without damaging them.


Cooltown


Car is one of 5 Cooltown demos. Here car detects it is low on fuel whilst on the way to a business meeting and guides user to a convenient filling station

2/11/2009

5

iHCI: Calm Computing

• Example of calm technology was the “Dangling String” created by artist Natalie Jeremijenko, situated at PARC

• 8 foot piece of plastic spaghetti that hangs from a small electric motor mounted in the ceiling.

• Motor is electrically connected to a nearby Ethernet cable so that each bit of information that goes past causes a ti t it h f th ttiny twitch of the motor.

• Hence the degree of twitching indicates the degree of network traffic in that Ethernet segment.


Tangible Bits


MetaDesk: tangible map AmbientRoom: Light patchesBottles, Clock, Water ripples, handles

transBOARD: Networked digital interactive whiteboard Graphically view & recordDrawings, tagged pens

2/11/2009

6

DataTiles

• Allows users to manipulate data in form of tangible “tiles”• Combinations of data streams and functions make it

possible to create new applications11Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

WearComp and WearCam• Mann’s experiments with wearable computers started in late 1970s. • Main application was recording personal visual memories that could be

shared with other via the Internet. • (Photo from http://en.wikipedia.org/wiki/Wearable computing)(Photo from http://en.wikipedia.org/wiki/Wearable_computing)


2/11/2009

7

Cyborg 2.0

Electrode array surgically implanted into Warwick’s left arm and interlinked into median nerve fibres is being monitored.


Analysis of Early Projects: Distributed Access Support

• Mobile device model design for Tabs and Pads. • Supported communication & location-awareness for mobile users,

commercial mobile ICT devices, widely available wireless networks. • Late 2000s mobile devices such as phones and laptops are widely• Late 2000s, mobile devices such as phones and laptops are widely

used and wireless networks are widely available that support data communication routing to users wherever they are

• Service discovery of local network resources is weak and the discovery of other local environment resources is still virtually non-existent

• Hence, much of the vision of Cooltown is not routinely available. – smart environments not yet mature or widely available.

diversity of support needed cost and secure fixings– diversity of support needed, cost and secure fixings.


2/11/2009

8

Analysis of Early Projects: context-awareness

• Context-awareness: mainly location awareness• Early achievements based upon (local not global) location awareness• Early achievements based upon (local not global) location awareness

indoors with heavily instrumented environment.• Legacy: Global Location determinism (via Mobile phone, GPS) but

accuracy limited to 10s of M.• Integrated into some mobile devices, e.g., phones, cameras, cars. • Location-determinism tends to be supported mainly as stand-alone

devices and services that are not readily interoperable. M i l f d• Mainly for outdoor use.

• Systems for indoor use are available today based, e.g., based upon trilateration using WLAN but not ubiquitous


Analysis of Early Projects: iHCI• Electronic boards

– Allow users to collaboratively edit text and graphics – PARC prototype in early 1990s now commercial products. – “Classroom 2000” use in 1995-1998 (Abowd) now routinely used in

many educational establishments

• Wearable smart devices– several products are available but not in pervasive use.

• Robots– Heavy use in clean room manufacturing, not in open environments

t f l i i b t d b t t d tapart from cleaning, mowing robots and robot toys and pets

• iHCI– A continuing research initiative. – Very many variations – Mass market ? But Wii is popular


2/11/2009

9

Today: Living in an Increasingly Digital, Interconnected, World

• More everyday human activities heavily reliant on computers • More devices per person, building, transport vehicle, etc• Greater variety of general purpose vs. task specific computer devices• Devices smaller/ larger?, cheaper, use less energy, reliability ↑ (or ↓?)• Profusion of multi-purpose, intelligent, mobile devices can access

remote services ... and more local services• Physical & Human world strewn with embedded sensors & control

devices • More devices can interoperate in ad hoc versus planned ways• High speed wire(less) networks are pervasive & accessible by all canHigh speed wire(less) networks are pervasive & accessible by all, can

be added less disruptively into the physical environment .• ↑ Energy efficient devices , yet ↑ overall energy consumption


Everyware UbiCom Applications• Vision: ubiquitous computer systems to support people in

their daily activities in the physical world tasks to simplify these and to make these less obtrusive.

• People will live, work, and play in a seamless computer enabled environment that is interleaved into the world.

• Bushnell (1996) coined variations of term ware such as deskware, couchware, kitchenware, autoware, bedroomware and bathware to reflect the use of ubiquitous computing for routine tasks.

• Greenfeld (2006) used the term everyware to encompass the many different types of ware

• Some Everyware scenarios follow

Ubiquitous computing: smart devices, environments and interaction 18

2/11/2009

10

AV

ClockLocation Determination Display

UbiComp System

GPS Transmitter

AV player

Image Processing

Communication

AV-Capture

Projector

AV database

AV-player

Printer

Removable Memory

Automatic face detection,

recognition, etc

dd/mmyyx,y

GIS


Why is the Bus late?


2/11/2009

11

Location Determination

NO. TOA211 14:11107 14:17


Ceiling

External Gas GridAppliance,

e.g., Food heater Manual

Controller e.g., Food Heating

External Electricity Grid

Wall

Desk

WLAN

Lights

Manual Timer

Sensors Temperature

22

Grid

ICT Network

WWAN

Storage

WLAN

Mobile Phone

Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-02-2009

2/11/2009

12

CeilingExternal Physical

Appliance, e.g., Food heater

Digital, networked Controller & Timer

Sensors, food Temperature

External Electricity G id

Wall

Desk

EnvironmentSensors

Lights

23

Grid

ICT Network

WWAN

Storage

WLAN

Mobile Phone


Overview

• Ubiquitous Computing: achievements• Beyond Weiser’s model: Smart DEI

√model of Ubiquitous Computing √• Research Examples


2/11/2009

13

UbiCom: Weiser’s 3 Internal System Properties

3 main properties for UbiCom Systems were proposed by Weiser (1991)

1. Computers need to be networked, distributed and transparently accessible– In1991, little wireless computing, Internet far less pervasive

2. Computer Interaction with Humans needs to be more hidden– Because much HCI is overly intrusive

3. Computers need to be aware of environment contextp– In order to optimise their operation in their physical & human environment.


Internal System Properties: iHCI• Concept of calm or disappearing computer model:

– Device too small to be visible, embedded, – Devices visible, not noticeable, part of peripheral senses

• Implicit (iHCI) versus Explicit HCI– Much HCI is explicit, low-level, more devices -> human overload– More natural, less conscious interaction– Systems anticipate use -> explicit interaction ↓– Calm Computing– If explicit HCI ↓, -> careful balance between several factors

• Embodied Reality as opposite of VR (people in virtual world)– Devices as embodied, env. aware, service access & execute entities – Devices aware of, & bounded by, physical, not just virtual env.


2/11/2009

14

Internal System Properties: context-aware

• Context-based ubiquity rather than global ubiquity• Physical Environment Context: location, time, temperature,

rainfall, light level etc.• Human Context (or User context or person context):

interaction is usefully constrained by users’ identity; preferences; task requirements, etc.

• ICT Context or Virtual Environment Context: UbiCom system is aware of the services available that are available internally and externally, locally and remotelyy y, y y

• Active (by system) versus passive context adaptation (by user)


HumanPhysical ICT

HPI

Devices: Weiser’s 3 Internal System Properties

implicit HCI

Environments

UbiCom System

HCICPI

context-awareness

distributed ICT ICT

VirtualICT CCI

28Ubiquitous computing:beyond Weiser: University Cambridge Talk 10-

02-2009

2/11/2009

15

Devices: Extended set of Internal System Properties

3 main properties for UbiCom Systems were proposed by Weiser1. Computers need to be networked, distributed and transparently

accessible.2 Computer Interaction with Humans needs to be hidden more2. Computer Interaction with Humans needs to be hidden more

– Because much HCI is overly intrusive

3. Computers need to be aware of environment context– In order to optimise their operation in their physical & human environment.

To which two additional properties are added:4. Computers can operate autonomously, without human intervention, be

self-governedC h dl l i li i f d i i d5. Computers can handle a multiplicity of dynamic actions and interactions, governed by intelligent decision-making and intelligent organisational interaction. This entails some form of artificial intelligence.


UbiCom: Different Combinations of Core Properties versus a Single Definition

• No single, absolute definition for ubiquitous computing.• Instead propose many different kinds of UbiCom based upon

combining different sets of core propertiesHere are some examples proposed by othersHere are some examples proposed by others• Weiser (1991): distributed, iHCI, physical environment context aware• Ambient Intelligence (AmI), similar to UbiCom - intelligence

everywhere? • Arts and Marzano (2003) define 5 key features for AmI to be embedded, context-

aware, personalised, adaptive and anticipatory.

• Buxton (1995): ubiquity and transparency Endres et al (2005) distrib ted mobile intelligence a gmented realit• Endres et al. (2005): distributed mobile, intelligence, augmented reality

• Millner (2006): autonomy, IHCI etc.


2/11/2009

16

UbiCom System Model: Smart DEI• No single type of UbiCom system• Different UbiCom systems applications support:

– Internal properties (distributed, iHCI etc) to different degrees– Different types (phys. virtual, human) of external environment

interaction to different degrees– Different form-factors (six basic forms) for devices– Multiple systems can combine to form interacting systems of

systems

• 3 basic architectural design patterns for UbiCom: t D i t E i t t I t ti– smart Devices, smart Environments, smart Interaction

• ‘Smart’ means systems are:– active, digital, networked, autonomous, reconfigurable, local control

of its own resources, e.g., energy, data storage etc.


Five main properties for UbiCom require a multi-disciplinary approach to R&D

Autonomous

NetworkedTransparent accessOpenness

Sense, adapt & control environmentPhysical World Person ICT

Automated, Independent Self-governed

Context-aware Distributed

iHCI

Physical World, Person ICT infrastructure aware

IntelligentDisappearing Computer Implicit InteractionEmbodied RealityMediated Reality

Handling Non-determinism Knowledge & task sharingGoal-based, etc.Social intelligence


02-2009

2/11/2009

17

Hidden

Location

AnticipatoryUser‐aware Post‐human

Ubiquitous Computing System

Immersed iHCI

Time Context‐aware

Phenomena Person ActivityPhysical Human ICT

ResourcesSense

Distributed System (Single Virtual Computer)

EmbeddedControl

Persistent

Un‐tethered

Mediate

Reactive

OrchestrateCommunal

Knowledge Uncertainty

Auto‐nomic

Organisational Cooperate Compete

Intelligent

Autonomous

Search

Goal

Self‐Model

Env ModelIndividual

Reason Plan

LearningUtility

Distributed

y ( g p )

Shared concurrency

VirtualFault‐tolerance

Heterogeneous

Discovery OpennessMobile

Interoperable

Ad hoc vs. Fixed

Asynchronous vs. Synchronous

Wireless vs. Wired

P2P vs. Client‐server

Networked

SecureTransparent

Local vs. Global

Mobile Services

Mobile Data

Mobile Code

Mobile Devices

Mobile Communication

Mobile Context

Mobility: Dimensions

Volatile, open, dynamic distributed services

LAS Self-powered, Mobile Host

Physical DimensionsType of Mobile Host

How device is attached to host

When Mobility occurs

Skins

Pads

Animate

Inanimate

Accompanied

Surface

Manufacture to Install

Wireless Communication LAS, etc

Manual control Automatic

Pads

Tabs

Dust

Inanimate Surface-mounted

Embedded

Mobile between sessions

Mobile during sessionsWearable ImplantsMEMS, Sensors,

RFID Tags

Smart Phone

Smart Card

Laptop Computer Air or Fluid

Human

Artefacts

Remove to De-install

2/11/2009

18

Personal Social PublicEcological(Living)

Physical Environments Human EnvironmentsPhysical Phenomena

Devices: Extended set of Internal System Properties

implicit HCI

Context-AwareAutonomous Intelligent

HCI(Cooperate)

HCI(Compete)

(Survive, Adapt)

UbiComp

HCICPICPI (Sense, Adapt)

Distributed

IVirtual Environments

ICT

System ICT

CCI


Increasing Human (H) Interaction

Human interaction mediated by UbiComp

Increasing Physical (P) World

InteractionH2H

H d l f ICT

Multi-lateral HCI

H2C / eHCI

C2C

C aware of H’s context & adapts to it. C is personalised itself

Autonomous ICT system interaction

Minimum

Humans use model of ICT to explicitly interact. H personalises C

C2H / iHCI

Increasing Ubiquitous Computing (C) Interaction

C2C system interaction

0 Minimum


02-2009

2/11/2009

19

Increasing Physical (P) World Interaction

P2PPhysical interaction (No ICT mediation)

Increasing Human(H) Interaction

C Senses P

Multi-lateral CPI

C2P / CA

C Augments or Mediates P’s reality. C Adapts to P’s context

Virtual Reality

C Senses P C Aware of P’s Context

P2C/AR/MR

Increasing Ubiquitous Computing (C) Interaction

C2C /VR

Virtual Reality facilitated by C

0

Minimum

Minimum


02-2009

Spatial Dimensions

Form Factors for Smart Devices

3D Volumes

Ear Receiver

Clusters of 1D technologies

Electronic Whiteboards

2D Planar

3D Surfaces

1980s Mobile Phones

2000s Mobile Phones

Laptops

Organic UIs

Device Size

Whiteboards

Macro

NanoMicro

Centi DeciMilli Meter

1D Points

ebooks

MEMSNano-technology


02-2009

2/11/2009

20

Device Trends

Increasing capability to manufacture low power,

Increasing capability to embed devices in the

h i l i

Increasing capability for more interoperable distributed mobile devices

UbiCom System Model: Smart DEI

Use more complex, multi-functional, mobile, personalised (& private) smart devices to ease access to & embody services rather than just to

irt alise them

Use smarter environments to sense and react to events such as people, with mobile devices , entering & leaving controlled spaces

micro, more complex devices physical environment

e g walls can sense camera

Use more service access devices with simpler functions and allow them to interoperate – smarter interaction between devices

distributed mobile devices

e g camera can interconnectvirtualise theme.g., phone is also a camera, music player, is also a printer??

e.g., walls can sense camera is recording and modify lighting to improve recording

Ubiquitous Computing

e.g., camera can interconnect to phone to share recordings, direct to printer to print


02-2009

Smart DEI ModelSmart Device

1-1 Interaction 1-M, M-M Interaction

SmartEnvironment

Smart MobileDevice

Smart Interaction

Mobile, Personalised, MTOS, Remote 1-1

Fixed vs. Untethered, ASOS, sense-control, local 1 1 interaction

1-M, M-M, richer (coop, compete, semantic,

interaction local 1-1 interaction etc), P2P interaction

Design: Fat-client-proxy-server ?

Design: Autonomous Peer-to-Peer interaction?

Design Thin-client-proxy-server interaction?40

Design: Appliance Model, Service Pool, Remote Service Access Point, Service Contract, on-demand, autonomic?


2/11/2009

21

Overview

• Ubiquitous Computing: achievements• Beyond Weiser’s model: Smart DEI model of

Ubiquitous Computing• Research Examples √


CRUMPET, CReation of User-friendly Mobile Personalised for Tourism Project

• The main objectives of the CRUMPET project was to:– Implement and trial tourism- related value-added services for nomadic users:– Evaluate the use of agent technology as a suitable approach for the fast creation &

composition of such services.

• Hence based upon a Multi-Agent System ( MAS design), to – ACL to factor out common service actions to ease interoperability across services – Semantic approach to service mediation to integrate multi-service content– Content adaptation for mobile use & heterogeneous ICT, wireless network, terminals– Location-aware map service – Personalisation

• Two designs A ti i t lli t bil li t ( i t l tf ) id t t d t ti– Active intelligent mobile client (micro agent platform) aids content adaptation

– Passive mobile client (micro-browser)


2/11/2009

22

CRUMPET Project integrates 4 key technologies

Location-aware services

Personalised

Multi-agent technology

Multimedia i f ti

08.10.2001 CRUMPET Project IST-1999-20147 43

Personalised user interaction

information access

over

mobile terminals

Architecture Variation A: This deployment architecture has a larger client-side footprint and is suitable for deploying in high end PDAs and PCs


2/11/2009

23

Architecture Variation B: This deployment architecture has a very small client-side footprint and

is suitable for deploying in low end PDAs and suitably equipped mobile 'phones

TA

Fixed Network

HTTP

Service Provider

Service ProviderWireless Station

MA

SCA

MAPA

DCA CASA

GSA

Satellite

HTTP - no control over link

JAVA process -very little control

over GPS and browser

ExplorerCE

Wireless Station


Service Provider

CCA UMACA

SACRUMPET Services

CRUMPET ICT AdaptationMy IP address and port are...Map

components:• Map of the nearby “world”• Start/Edit tour• Status bar with Ok, here are your nearby Ok, here are your nearby

points of interests.

Components:• Map of the “world”• Diagnostics information• Client status (Agent and network

• Status bar with proactive “bulb”

new location.Here is my

new location.


• Client status (Agent and network status)• Points of interests

2/11/2009

24

CRUMPET System


Crumpet System: conclusions• Low resource terminals find it challenging to parse rich

semantic messages• Multi-lateral approach to adapt content:

– E.g., variable image compression, text headings vs. whole body, etc

• Context acquisition & composition design: – What is an event: user stopping? Etc?– Ordering: location -> personalisation -> terminal or different – Generic versus specific adaptation e.g., personalisation

• Agent design– Simple reactive and proactive– ACL need?


2/11/2009

25

Electronic

Nano devices roaming within blood vessels

Food

Automatic body measurement scanners

Pearson (2000) Predictions for Smart Environments

Space solarrobots outnumber

Nanotechnology garden plants

Micromechanical garden gnomes

outnumber organic pets

Reconfigurable buildings

Technology to enhance illusion

of traditionMood sensitive light bulbs

Solar reflector satellites increase sunlight

Automatic body t

packaging chips tell state of foodEmotional objects, switches in home

5 years ago to present time

solar power

stations

2010 2015 2025 20302020

robots outnumber people in some countries

Self- driving cars on smart highways

g

Anti-noise technology homes

Digital bathroom mirror

measurement scanners


Conclusions• Good opportunities for mass use of smart devices

and context-ware devices began relatively recently – these opportunities are expanding.these opportunities are expanding.

• References• Poslad S. (2009) Ubiquitous Computing: Smart Devices,

Environments and Interaction. Wiley, ISBN 9780470035603 http://www.elec.qmul.ac.uk/people/stefan/ubicom/Pearson (2000) Pearson I D (2000) Technolog Timeline• Pearson (2000) Pearson I.D. (2000) Technology Timeline -Towards Life in 2020. BT Technology J., 18(1): 19 – 31


2009 02 10 beyond weiser poslad

Documents

pervasive computing

ihci calm computing

university cambridge

fixed phone point computing

active bat

smart space

smart dust

floor smart environments