Ubiquitous Computing

What is Ubiquitous Computing?

• The names for the research vary – Pervasive

– Wearable

– Augmented

– Invisible

– Disappearing

– Calm…

• but they all share the vision of the active world.

What is Ubiquitous Computing?

• Mainframe = many people share one computer• Personal Computing = one person, one computer• Ubiquitous Computing = many computers for

each person

• Sometimes referred to as the “third wave” in computing.

Taken from a talk by Mark Weiser.http://www.ubiq.com/hypertext/weiser/NomadicInteractive/

The “father” of Ubiquitous Computing is Mark Weiser

Mark Weiser“… people live through their practices and tacit knowledge so that the most powerful things are those that are effectively invisible in use. This is a challenge that affects all of computer science.

Our preliminary approach: Activate the world. Provide hundreds of wireless computing devices per person per office, of all scales (from 1" displays to wall sized). We call our work "ubiquitous computing". It is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere.”

“For thirty years most interface design, and most computer design, has been headed down the path of the "dramatic" machine. Its highest ideal is to make a computer so exciting, so wonderful, so interesting, that we never want to be without it. A less-traveled path I call the "invisible"; its highest ideal is to make a computer so imbedded, so fitting, so natural, that we use it without even thinking about it.”

In fewer words…“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

Mark Weiser, Scientific American, September 1991

Or as simply as I can put it…

Merge the physical and

digital realms

What Ubiquitous Computing Isn't• A Mobile Computer – even if you have access to

“everything” you do it through only one access point.

• Multimedia Computing – while it may employ sound and video it should fade into the background rather than demand the focus of your attention.

• Virtual reality - where virtual reality puts people inside a computer-generated world, ubiquitous computing forces the computer to live out here in the world with people.

Why talk about Ubicomp in HCI?

• Ubicomp created a new mode of HCI.• In order to realize Weiser’s vision we must:

– Understand and support everyday practices;

– Provide heterogeneous devices (different forms of interactive experiences);

– Orchestrate devices for provide holistic experience;

Range of form factors

• Tab (inch scale) [Active Badge and Containers]• Pad (foot scale) [Digital Paper]• Board (yard scale) [LiveBoards]

• “How many tabs, pads, and board-sized writing and display surfaces are there in a typical room? Look around you: at the inch scale include wall notes, titles on book spines, labels on controls, thermostats and clocks, as well as small pieces of paper. Depending upon the room you may see more than a hundred tabs, tens of pads, and one or two boards.”

Olivetti Active Badge, circa 1990

Active Badge

• Badges emit IR signals • Sensors in the environment pick up the signals,

pass them to a central network• Doors open only to the right badge wearer, rooms

greet people by name, telephone calls can be automatically forwarded to wherever the recipient may be, receptionists actually know where people are, terminals retrieve your preferences without you logging in…

“Containers”

• A tab that serves as an extension to a computer screen.

• Touch the tab to an open window on the computer screen. The document closes and is “stored” on the tab.

• Touch the tab to a different computer screen, and the document opens.

• Organize tabs and carry them with you as you would piles of papers on your desk.

Digital Paper• Take a special pen and a special Post-it note. Write a message,

enter an e-mail address in some squares at the bottom of the note, and check a box marked "e-mail" and another marked "send."

• The pen has a pressure sensor, which activates a digital camera that records exact strokes. Bluetooth transceiver communicates the strokes to a phone or laptop nearby.

• The special pen isn't taking pictures of the pen marks -- it's recording the position of the pen on the paper. It can do this because the paper is preprinted with thousands of tiny, nearly invisible dots. They make up a kind of map on the Post-it note that the pen's camera can read. So, for instance, when you check the box marked "e-mail," it knows that that part of the map means "Send what you've captured as an e-mail message." Only the e-mail address needs to be written neatly, in designated squares for each letter, so it can be read and translated by optical character recognition software in the pen.

The Advantage of Digital Paper

• “One way to think of pads is as an antidote to windows…computer window systems are often said to be based on the desktop metaphor – but who would use a desk whose surface is 9x11? Pads, in contrast, use a real desk. Spread many pads around on the desk, just as you spread out papers.”

LiveBoards

• Whiteboard sized displays.

• Users use electronic “chalk” to manipulate the display.

• Can be used to facilitate meetings over great distances.

• Ever changing “bulletin boards”

Context Aware Computing-Placing Information in the World• GeoNotes

– Associate notes (e.g., recommendations, ratings) with locations; for personal or group use

• E-Graffiti– Campus system which associates notes with location

• comMotion– Associates to do list items with locations; individual use

GeoNotes• Permissions• Filtering

AURA

• Scans the bar code on any object with a wireless PocketPC

• Construct queries obtain product info

• Build personal histories (that can be shared, too)  

AURA

More on I/O

• Novel Input Devices [Xwand]

• Implicit Input [Sensor Rich PDA]

• Multiscaled output [Pick-and-Drop]

• Distributed output [Dangling String]

Novel Input Devices - XWand

XWand

• Lots of sensors– Accelerometer– Magnetometer– Gyroscope

• IR• FM transceiver• LEDs• Microcontroller

• Cameras in room

3D orientation

Location

What is being pointed at

What gesture is being made

Implicit Input

• An experimental PDA (Microsoft Research) to investigate how a variety of simple sensors can improve the interaction between a user and various hand-held applications;

• Technology used: Two axis linear accelerometer (tilt sensor), capacitive touch sensors, and infrared proximity range sensor;

Towards Implicit Input – example Sensor-enriched mobile device

Multi-scale output

• Output is no longer exclusively in the form of self-contained desktop or laptop visual displays that demand our attention (3 scales).

• With these new proliferation of displays, two trends have emerged:– We want to easily move information between

separate displays and coordinate interactions between multiple displays (Pick-and-Drop);

Multi-scale output

• New field of user interfaces called “multi-computer direct manipulation”.

• An extrapolation of the Drag-and-Drop.• It is more natural to allow user to manipulate a computer

object as if it were a real (physical object).

Multi-scale output

Distributed output• The Dangling String meets the challenge of how to create

calm technology.• We must learn to design for periphery so that we can most

fully command technology without being dominated by it.

• 8 foot piece of plastic spaghetti 8 foot piece of plastic spaghetti • Electrically connected to an Ethernet Electrically connected to an Ethernet cablecable• Each bit of information causes a tiny Each bit of information causes a tiny twitch of the motortwitch of the motor• A very busy network = madly whirling A very busy network = madly whirling stringstring• A quiet network = small twitch every A quiet network = small twitch every few secfew sec• Placed in an unused corner of a Placed in an unused corner of a hallway, the long string is visible and hallway, the long string is visible and audible from many offices without being audible from many offices without being obtrusive. obtrusive.

Seamless integration of physical and virtual worlds – DigitalDesk

• Ubicomp attempts to merge computational artifacts with the world of physical artifacts.

• DigitalDesk intermixes input and output.• Application: A calculator that allows people to

place documents in the desk, point to a number to enter it into the calculator.

Seamless integration of physical and virtual worlds – DigitalDesk

Cool, but…

• Can we apply a user-centered approach to create novel designs with novel technologies?

• How do we learn about user requirements?

• Let’s take an example…

Place-centered information

• Deliver information to users (on handheld / wearable devices) that is relevant to their current place (e.g., office, lab, classroom, home, grocery store, night club, coffee shop, stadium, post office, school, etc.)

What we’d like to know

• Common places a representative group of people are in over the course of a typical day

• Common transitions between types of places• Information needs relative to a given type of place• Willingness to share information relevant to a

given type of place

Hypothetical examples

• “While I’m in the grocery store, I’d like to know if any of the items I typically buy are on sale”

• “While I’m dropping my kids off at school, I realized it would be really convenient to find another parent at the school to carpool with”

• “I’d like somebody who’s passing by Prexy’s in the Union to pick me up a large diet Mountain Dew and bring it to my office”

• “I often go from my office to the WRC to the coffee shop, then back to my office”

Exercise

• Consider methods for gathering data from users– Questionnaires– Interviews

• Workshops / Focus Groups

– Observation– Self reports (e.g., time diaries)– …

• Which one(s) would work best to gather user requirements in this case? How well would any methods work? Why?

Time Diary (Rieman 1993)

Limits of self-reported data

• Basic problem – hard for people to remember and take time to enter data

• Failure of recall – esp. for unremarkable everyday events

• Bias – you remember the unusual• Bias – you remember what you expect to have

happened• Bias – you remember what you think the

experimenter wants you to remember

Another approach• Experience Sampling Method (ESM)• Subjects are periodically prompted to enter data,

typically by answering a few questions– Fixed interval; random interval; on event (up to user)

• Traditionally, prompting was via a “beeper” and questionnaire was filled out with paper and pencil

• New technologies– Call cell-phone, listen to voice prompts, press buttons or

speak responses– Blackberry pager– Handheld computer

ESM Tool Tailored for Ubicomp

• Intille et al, Ubicomp 2003• In addition to traditional prompting modes,

context-aware triggers– User is in a particular location– User’s heart rate changes– A conversation is (or is not) going on

• Ubiquitous, imaged-based sampling– Audio, video “images” captured– Users consult the images when it’s convenient and

answer the survey questions using images as recall aids