![Page 1: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/1.jpg)
HWG 1
Hans-W. Gellersen
Lancaster UniversityDepartment of ComputingUbiquitous Computing Research
Computer-Augmented Environments: Back to the Real World
![Page 2: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/2.jpg)
HWG 2
What I thought this talk would be about
“Back to the Real World”• Special Issue, CACM July 1993• Workshop ”Augmented Reality and
Ubiquitous Computing”, Feb 1993• Early work on “physical integration”
from interaction perspective• Contrasting ubicomp and the AR model
of augmented environments– ubicomp: the environment is the
interface– AR: an overlaid interface registered
with the environment
![Page 3: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/3.jpg)
HWG 3
What this talk now will be about
• Motivating a design-driven perspective on ubicomp• Some basic concepts and examples
– Wellner’s Digital Desk• Interdisciplinary ubicomp research programmes• Examples from work at Lancaster
– Pin&Play– Load Sensing Surfaces
• Design Exercise in Groups
![Page 4: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/4.jpg)
HWG 4
Motivating Scenario
Smart-Its• Prototypical Smart Device• Physical I/O
– Sensors, Actuators• Wireless networking• Processing and memory• Pervasive deployment
– Attachment to existing things/structures
SmartDevice
User
NetworkPhys.World
• No User Interface– The “host” thing/structure
is the human interface
![Page 5: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/5.jpg)
HWG 5
Technology research perspective I
“This is a distributed system”• How to organize the network ?
– Device discovery, dynamic configuration, etc• What sort of communiction protocols ?
– Communication model, abstractions etc.• How to deal with scale ?
– Number of nodes, spatial extent, density• How to deal with resource limitations ?
– Energy, processing• How to support application development ?
– Programming abstractions, middleware, e.g. service discovery
![Page 6: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/6.jpg)
HWG 6
Technology research perspective II
“This is a sensor/control system”• Sensing: which are the phenomena of interest (“context”) ?• What sensors to integrate for observation ?• Sensor control, sensor fusion etc• Perceptual computing: extracting meaning from sensor data
– Transformations, architecture• Communication models between sensor and observer• Control: what variables to control in the environment ?• Actuators, control interfaces, etc• Control protocols• Security• etc.
![Page 7: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/7.jpg)
HWG 7
Application research perspective
“this is an interesting application platform”
![Page 8: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/8.jpg)
HWG 8
Design perspective I
“These are artefacts”• Artefacts: invented things
– as much product of innovation as our new technologies• They have meaning/value in our lives
– not strictly utilitarian; aesthetics, comfort, cultural, social etc• Physical/Tangible Interaction
– Physical affordances: suggesting and guiding action– Distributed interaction: actions across artefacts
• Spatial/ambient interaction– Interaction spread through space– Meaningful spatial relationships– Spatial organisation of action/communication– Ambient interaction: “spatial attention model”
![Page 9: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/9.jpg)
HWG 9
Design perspective II
“These are computer-augmented artefacts”• Combine unique capabilities of digital technology with
properties of physical artefacts• Computers as secondary artefacts (“in the background”)• Extend foreground capabilities of the primary artefacts• Enable new relationships among artefacts• Enable new interactions across artefacts• Preserve and exploit familiarity and physical affordance
of the physical artefact
![Page 10: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/10.jpg)
HWG 10
What are affordances ?
My Cannon Cooker• Dual-fuel, dual-oven
![Page 11: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/11.jpg)
HWG 11
About affordances
Term comes from J.J. Gibson• “The ecological approach to visual perception”, 1979• Affordances as fundamental object of human perception
– e.g. perceive stairs in terms of their “climbability”– refers to attributes of both object and actor
Developed for interaction design by Bill Gaver• “Technology Affordances”, Proceedings of CHI’91
– Provision of affordances as design challengePopularized by Don Norman• “The design of everyday things”, 1988
![Page 12: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/12.jpg)
HWG 12
Examples of “hybrid designs”
• Bishop’s Marble Answering Machine• Jeremijenko’s Live Wire• Wellner’s Digital Desk
![Page 13: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/13.jpg)
HWG 13
Bishop’s Marble Answering Machine
• Physical interaction with digital information
![Page 14: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/14.jpg)
HWG 14
Examples: Jeremijenko’s Live Wire
• Giving digital interaction a physical presence• Ambient interaction
![Page 15: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/15.jpg)
HWG 15
Examples: Wellner’s Digital Desk
• interaction with paper and electronic documents• Seamless transitions: physical and digital interaction
![Page 16: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/16.jpg)
HWG 16
Interdisciplinary Ubicomp Research
The Disappearing Computer• www.disappearing-computer.net• EC Research Programme, started Jan. 2001• 17 Projects, each multi-site, multi-national, multi-disciplinary• Cross-project activities: research ateliers etc.• Bringing together
– Technology research– Design (Architecture, Products, Interaction Design)– Social Computing
![Page 17: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/17.jpg)
HWG 17
The Disappearing Computer
The Programme Vision
“the computer disappears … new artefacts appear” …
“… as a consequence human-centered notions, such as real objects and everyday settings can come to the foreground“
Objectives• Enabling smart artefacts as future versions of today’s artefacts• Architectures supporting new relationships and emergent
functionality across many artefacts• Understanding and designing user experience
![Page 18: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/18.jpg)
HWG 18
The Disappearing Computer
Technology concerns• Physical integration, software architecture, etcDesign research• Incorporating context• Managing attention• Physical space, form and affordance• New interactive stylesSocial computing • Incorporating sociological understandings
– How people interact; the role of artefacts and places• “From Human Factors to Human Actors”• Enhancing social interactions
![Page 19: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/19.jpg)
HWG 19
The Disappearing Computer
DC Jamboree• Annual Project Review Meeting• Exhibition / demonstrations of all projects• Co-located with the Ubicomp 2002 Conference• Gothenburg, Sweden, 30 Sept – 1 Oct
![Page 20: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/20.jpg)
HWG 20
More Interdisciplinary Research
The Equator Programme• www.equator.co.uk• UK “Interdisciplinary Research Challenge”, started late 2000• “Technological Innovation in Physical and Digital Life”• 8 Research Institutes across the UK• Designers, Sociologists, Psychologists, Performance Arts,
Software and Hardware Technology– Sociological field studies and cultural probes– Building technology and studying in practice
![Page 21: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/21.jpg)
HWG 21
DC/Equator Work at Lancaster
• Smart-Its• Pin&Play• Load Sensing Surfaces
![Page 22: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/22.jpg)
HWG 22
Pin&Play
Concept• “The wall as network bus for the things attached”• A new type of network to connect everyday objects
on common surfaces such as boards and walls• Use of familiar mechanism: “pinning objects to the
wall” “pinning nodes to the network”
![Page 23: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/23.jpg)
HWG 23
Pin&Play
Components• Surface: Common surface augmented with
conductive material to create network medium• Connectors: pushpin-like physical connector for
socket-less attachment of objects to the network• Objects: any type of device/object with embedded
computing and connector-interface• Network: ad hoc behaviour: “Pin&Play”
![Page 24: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/24.jpg)
HWG 24
Pin&Play
A Pin&Play Noticeboard• Fully functioning prototype for proof of concept
![Page 25: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/25.jpg)
HWG 25
Pin&Play Noticeboard
Pin&Play Surface• Corkboard augmented with two
conductive sheets• Ground layer on top, data/power
layer hidden, cork as insulator• Low cost, off-the-shelf,
deployable at large-scale
Pin&Play Connectors• Simple connector board with
pushpin for two separate connection points
![Page 26: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/26.jpg)
HWG 26
Pin&Play Noticeboard
Pin&Play Objects• 1-wire bus, Dallas MicroLAN• 16300 bits/s
Pin&Play Objects• “Smart Notification Pin”:
iButton and switchable LED• Time-in-a-can iButton: memory,
internal calendar and clock
Switch Switch
Time-in-a-Can Time-in-a-Can
![Page 27: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/27.jpg)
HWG 27
Pin&Play Pinboard Scenario
Network control• External laptop connected as
1-wire network node• runs network controller• used to pre-set pins with
priority and deadlines
User interaction• insert or remove pin• network detects change• protocol to determine
pin with highest priority
![Page 28: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/28.jpg)
HWG 28
Pin&Play
Technology Research• Network surface development• Simple and robust protocol design, zero maintenance• Scalability and density (initial target: 25 nodes/sqm)
Application Research• Augmented noticeboards and other interactive surfaces• Embedded home control buses
– Networking and free placement of controls (light switches, appliance controls etc.)
• Communication bus for wall-attached artefacts– Clocks, calendars, sensors, digital picture frames, …
![Page 29: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/29.jpg)
HWG 29
Load-Sensing Surfaces
Concept• No physical thing can escape gravitation• Use load-sensing as interface between the physical and the virtual• Augment common surfaces (floor, tables, shelves): this is where
gravitation pulls objects to
Principle• Augment surface at the corners• Force applied (e.g. by weight of an
object, or explicit pressure) is detected as load depending onposition of the pressure point
• i.e. surface detects weight/pressure and position
Force Fxat (x,y)
Force F1at (0,0)
Force F3at (xmax,ymax)
Force F4at (0,ymax)
Force F2at (xmax,0)
![Page 30: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/30.jpg)
HWG 30
Load-Sensing Surfaces
“Weight Lab”• Various augmented surfaces• Floor: 240 x 180cm, up to
800kg load• Larger table: up to 200kg• Coffee table: up to 8kg,
highly sensitive• Shelves and trays
Floor with embeddedS-load cell
![Page 31: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/31.jpg)
HWG 31
Load-Sensing Surfaces
Augmented tables
Sensor board with wireless communication
![Page 32: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/32.jpg)
HWG 32
Load-Sensing Surface
Context Acquisition• Weight of objects
– Detection depends on sensor range (i.e. small weights not detectable on heavy-load surface)
– Application: object identification (classes/ instances)• Position of objects
– cm-level accuracy– Table can be pre-loaded– Multiple objects can be positioned
if placed non-simultaneously
![Page 33: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/33.jpg)
HWG 33
Load-Sensing Surface
Context Acquistion• Beyond weight and position: events derived from
signal analysis over short time
1000
1500
2000
2500
Time
Load
E1 E2 E3 E4
knocked over
Object put down
removed New object
![Page 34: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/34.jpg)
HWG 34
Load-SensingSurfaces
Context Acquistion• Tracking of
people/objects• Prediction of
activities
![Page 35: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/35.jpg)
HWG 35
Load-Sensing Surface
Surfaces as Interaction Device
![Page 36: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/36.jpg)
HWG 36
Load-Sensing Surface
Surfaces as Interaction Device
cup book touch and move right click
moveleft click
moveleft click and
release
![Page 37: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/37.jpg)
HWG 37
Load-Sensing Surface
Surfaces as Interaction Device
A"No Interaction"
B"SurfaceTouched"
C"Tracking"
D"Clicking"
E"Object placedon the surface"
F"Object
removed fromthe surface"
X"Unknown"
![Page 38: Computer-Augmented Environments: Back to the Real World · • Perceptual computing: extracting meaning from sensor data – Transformations, architecture • Communication models](https://reader033.vdocuments.mx/reader033/viewer/2022050221/5f6680c7a0299924ff2a5aeb/html5/thumbnails/38.jpg)
HWG 38
Summary
Computer-Augmented Environments• Build on familiarity and meaning of existing artefacts / structures• Introduce digital added value in the background
Pin & Play, Load-sensing surfaces• Examples for network/tracking infrastructure integrated
with common structures and everyday uses• Low-tech, unobtrusive design