Daniel Kornev

Program Manager

Microsoft Russia

Ubiquitous Computing: building context-aware systems

• Ubiquitous Computing• Great Moments in Ubiquitous Computing

• Devices

• Concepts & Research Areas

• Context-aware Computing Systems• History

• Real-world scenarios

• Industry & Academia interest

• Q&A

Agenda

Ubiquitous Computing

• Ubiquitous Computing• Computers everywhere

• Interconnected

• Human doesn’t “see” computer but focus on solving his problem

Ubiquitous Computing

5

Great Moments in UbiComp

1991 – Mark Weiser (1952-1999) (Xerox PARC) introduces “ubiquitous computing” in Scientific American article1993 – Xerox PARC introduces PARCTAB1999 – First International Symposium on Handheld and Ubiquitous Computing (HUC)1999 – DARPA Invisible Computing Initiative funds five university projects in ubiquitous computing 2001 – HUC changed to Conference on Ubiquitous Computing (UbiComp)2001 – Intel opens four “lablets” with primary emphasis on pervasive computing2002 – First issue of IEEE Pervasive Computing2003 – Fifth UbiComp conference (Seattle)

Mark Weiser

PARCTAB

Smart Devices Entirely New

Dust – miniaturized devices without visual output displays, e.g., MEMS (Micro Electro-Mechanical Systems) – from nanometers through micrometers to millimetres

Skin – non-planar display surfaces like OLED – for clothes – networked surfaces of MEMS

Clay – 3 dimensional shapes – networked surfaces of MEMS

Tabs (wearable cm-sized devices)

Pads (handheld decimetre-sized devices)

Boards (meter sized interactive display devices)

Ubiquitous Computing: New Form Factors

7

What Is It?Ubiquitous Computing ≈ Pervasive Computing ≈ Invisible Computing≈ Sentient Computing

System is applicable in every real of daily activity, in every location and every contextSystem is aware of all activities user is participating in

UbiComp Technology Taxonomy

Context Sensing(location, activity, goals)

Mobile Devices

Ethnographic Studies

Data Availability(Web, personal server, OceanStore)

Privacy

Modeling

UI

Device Association

Applications

Automatic Behaviors

• System is applicable in every real of daily activity, in every location and every context

• System is aware of all activities user is participating in

Research Areas

• Sensor Networks

• Mobile Computing

• Context-aware Pervasive Systems

• Ambient Intelligence

• Distributed Computing

• Etc.

Ubiquitous Computing: Concepts

Research Area: History & Definitions

Context-aware Computing

1994

2001

2006

2007

2008

History

Context-aware computing system is a software that adapts according to its location of use, the collection of nearby people and objects, as well as changes to those objects over time.

in Disseminating Active Map Information to Mobile Hosts Bill N. Schilit, Marvin M. Theimer, 1994.

First definition - 1994

Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and application themselves.

(Dey, Abowd, 2001, in Providing architectural support for building context-aware applications)

Dey & Abowd’s definition - 2001

Context: Users, their goals and tasks, infrastructure (hardware, software, materials), physical and social environment in which the product is used.*

* As defined by ISO 9421-11

ISO Standard - 2006

context

–noun

the set of circumstances or facts that surround a particular event, or situation.

(Greg Parks; Brownbag on Windows Context Platform; July 2007; Microsoft Corporation)

Windows / PC3 Definition - 2007

Intel’s Definition - 2008

© Intel 2008, “Context Aware User Interface”, Intel Developer Forum

Context-aware system can be aware of:

Environmental Contexts

• Physical environment

• Devices

• Services

User context

• Identity

• Physical state

• Physiological state

• Activities

• Contacts

• Preferences

Platform Context

• Location

• Motion

• Network

• Batter Status

• Platform Features

• Running Apps

• Memory usage

Sensors – Industry & Academia Interest

Context-aware Computing

SensorsSensor is a device that measures a physical quantity and converts it into a signal which can be read by an observer

Sensors make possible for computer devices to detect elements of contexts that are hard to impossible to discover by using traditional methods.

Sensors and context-aware solutions are on the market already

Research organizations Industry Sensors Vendors

+Universities

Research Institutes

+PartnersSoftware

Developers

Real-World Scenarios

Context-aware Computing

• Location• Navigation, Location-aware Printing

• Augmented Reality• Navigation (ARDemo), Games (AR Counter Strike)

• Home Automation

• Security & Privacy• Location Privacy, Login, etc.

• Social Context

• Natural User Interfaces• UI orientation changes (Apple iPhone, ZuneHD, ARDemo), Object Recognition (Surface), Multi touch

(Surface/iPhone/Zune HD/Windows 7), Speech Recognition

• Presence

• Digital Assistant• Outlook, Sound Profiles in HTC Touch Pro

• Search

• Collaboration

• Productivity• Context-aware Tabs in Office 2007

• Context-aware Computations

Real-World Scenarios

GPS and location-based applications

Context-Aware User Experiences:

Detection of user’s location

Contextual information – POIs near user

Directions

Traffic and auto-correction of trip

Show user’s friends location on map in real mode

Apple iPhone

Natural User Interface:

Gestures

Zoom in/out

Physical inertia applied to digital objects

Context-Aware Interfaces:

UI Orientation (changes when physical orientation changes)

Light-aware UI

Screen uses presence sensor to automatically turn off while on call

Automatic switching between tasks with applying priorities (media player pauses while on call and after call resumes playing)

Contextual tabs in Office 2007

Natural User Experience:

Ribbons for editing special objects like images, charts, tables etc appear only when these objects are selected by user

Sound profiles in HTC Touch Pro

Sound Profiles:

Alarm can wake you even if you turned off system sound off

Sound profiles – normal/vibration/without sound

Special profile “Automatic” – switches app behavior to “Vibration” while on meetings (data is populated from Calendar)

Location-aware Printing in Windows 7

Location-aware Printing:

Default printer is chosen automatically based on current mobile computer’s network location

26

Security & Privacy

Device Association (HP) – use laser instead of RF broadcast to target intended device

User Login (U. Aarhus, Denmark) – augment password and smart card with proximity for login/logout

Location Privacy (IT U. of Copenhagen & Intel) – cell phone users are surprisingly willing to be tracked in return for useful location-based services

Other existing examplesAmbient Light Sensors in cars

GPS and location-aware applications

IM Presence (Skype/Messenger/ICQ)

Changing screen layout depending on device’s physical orientation (iPhone)

Contextual Tabs in Microsoft Office 2007

Sound Profiles in HTC Touch Pro

Contextual Conversations in Office Communicator

Location-aware Printing

…

• Location & AR• Campus Navigation System - ARDemo

• Natural User Interfaces• Context-aware User Interface – “Project Universe”

• Interruption Management System (in progress)

• Presence• Changing presence based on max/normal window (busy/available)

Russian Context-aware Computing Incubation Team: Scenarios

AR Demo

By Russian Context-aware Computing Incubation Team

Context-Aware Computing

Scenario

1. trip planning 2. walking

3. taking pictures

4. viewing pictures

Map follows your view direction

• Magnetic declination obtained from compass is used to align map rotation

NN

Device orientation changes UI

• Map mode if UMPC is parallel to the ground

• AR mode if UMPC is orthogonal to the ground

• Pitch angle calculated from accelerometer data is used to define change between two UI modes

Augmented reality mode

Augmented reality mode

Custom hardware boardLogitech FusionWeb camera

Freescale JM Badge Board• 3-axis accelerometer

• ambient light sensor

• capacitive touch sensor

Honeywell HMR3300digital compass

Pharos iGPS-500GPS receiver

Video

Demo

Project Universe – Demo/Motivation

Context-aware Computing

Sensed Less

Control

Loved More

automated

behavior

What Do Users Want?