Automatically Generating Interfaces for Multi-Device Environments

Jeffrey NicholsCarnegie Mellon University • Intel Research Seattle

Workshop #5: Multi-Device Interfaces for Peripheral InteractionThe Fifth International Conference on Ubiquitous ComputingSeattle, Washington USA

2

My Problem

Appliances are too complex

3

Interfaces

Control

Feedback

Specifications

My Solution

Separate the interface from the appliance!

Several ways to separate interfaces• Pre-programmed interfaces on mobile device• Downloadable pre-designed interfaces• Automatic interface generation

4

1. Single UI device controlling one or more appliances

Three Multi-Device Interactions

5

Three Multi-Device Interactions1. Single UI device controlling one or more

appliances2. Multiple users controlling an appliance

6

Three Multi-Device Interactions1. Single UI device controlling one or more

appliances2. Multiple users controlling an appliance3. One user using multiple UI devices to control an

appliance

7

Personal Universal Controller (PUC)

We have built an automatic interface generation system

• Abstract specification language

• Interface generators for multiple platforms

• PocketPC• TabletPC (Desktop)• Smartphone• Speech (with “Speech Graffiti”

platform)

• Architecture for controlling real appliances

• Windows Media Player (shown)

• Sony DV Camcorder• Axis UPnP Pan-Tilt Camera• Lutron and X10 Lighting• Many others…

8

Multi-Device UI and the PUC

PUC supports all three multi-device interactions

Third interaction could be improved however• Support for a fluid user experience

Controllers need to know what others are doing

E.g. When light controls are used on one device, other devices also display light controls

• SpecializationCurrently each controller provides access to all appliance functionality with roughly the same structure

Each controller should specialize; provide better access to features that will be accessed from its type of interface

Access to all features should be maintained

9

AcknowledgementsFunding

• National Science Foundation• Microsoft• General Motors• Pittsburgh Digital

Greenhouse

Equipment Grants• Mitsubishi (MERL)• VividLogic• Symbol Technologies• Hewlett-Packard• Lucent• Lutron• Lantronix

PUC Project Members• Brad A. Myers• Kevin Litwack• Thomas K. Harris• Roni Rosenfeld• Michael Higgins• Joseph Hughes• Rajesh Seenichamy• Mathilde Pignol• Stefanie Shriver• Pegeen Shen• Jeffrey Stylos• Suporn Pongnumkul• Peter Lucas

Thanks for listening!

For more information…http://www.cs.cmu.edu/~pebbles/puc/http://www.cs.cmu.edu/~jeffreyn/