Fasten the seat belt, here we Go

Home sweet home, where the lights along the corridor flicker

3500°F !!

The 3500 °F Incinerator for the Charcoal Grill

The Duel Between Stew and IceCube

Hey, Who Turns Off The Light?

Attention! Time for Roll Call!

Does Your Smart House Know Better?

A Safety-Oriented Programming Model for Pervasive Computing Systems

Pervasive & Mobile Computing Lab

Hen-I Yang

April 21, 2007

Characteristics of Pervasive Computing Systems

• Interaction with Physical World

• Highly Personalized/Customized

• Dynamicity

• Open

• Diversity/Heterogeneity

• Complexity and Scalability Issue

• Predication on Human Intention

• Multiple Modalities in Interface

Proposed New Model

Safety Vs. Expressiveness

Interrupt & Exception Handling

Memory access violation

Y = X/0

Divided by 0 Interrupt

Fire AlarmImpermissible Context

Out of Op Range Exception

The Four Elements

Elements in Pervasive Computing Environment

• Sensor

• Actuator

• Service

• Applications

• User

• Space

• System Support

Devices

Services

Users

Space

Element 1: Device• Sensors, actuators and other smart appliances

are the I/O devices that serve as the bridge between the physical world and the pervasive computing systems.

• Source of data and/or sink of commands

• Accurate description of devices is the key, it defines how to interact with these devices

Modeling of Device• Service oriented architecture• Can be interpreted on the fly, or created and

compiled priorType: (Sensor/Actuator/Both)

Domain of data: e.g. Temperature, Luminance, Device Internal

Data Schema: XML Document Type Declaration Including, but not limited to maximum and minimum values, resolution of data, unit of data

Interface Methods: Methods available to be invoked externally

Physical Medium Description: Description of the Interface/ Conduit/Channel that is to be used

Interaction Protocol Description: Support for Poll or Push methods, maximum frequency of data exchange

Other Support Information: e.g. Vendor and other contact/support information

Safety Measure for Device• Automatically appended exception handling

routine

• Device Safety: Avoid conflicting directives and unsafe/unacceptable operations, similar to Electric Breaker

• Regulate the incoming commands and detect abnormal command/access pattern

• Using maximum and minimum values to filter or red-flag abnormal values

Element 2: Service• The core active member of pervasive computing

system• More complicated than the usual software in

desktop because of the heterogeneity and dynamicity

• Service/Application• Service Lifecycle management is crucial

• Implemented• Aligned (Target Setup, possibly using user

preferences)• Binded (to devices or other services)• Context input as conditions• Logics/Actions/Behaviors• Change of Preference• Change of Binding• Change of Conditions• Priority Level

ActiveActive Contexts

Modeling of Services

TerminatedAligned

Binded

User Preference

ServiceBehavior 1

ServiceBehavior 2

ServiceBehavior n ….

Service Binded

Service Registry ServicesServicesServices

Programmed

ServicesServicesDevices

Active Context Change Event

Service/Device Change Event

User Preference Change Event

Impermissible Context Event

Safety Measure for Service• Service Safety: Whenever device safety, user safety or

space safety is violated, the exception handler will suspend or terminate all relevant services

• Services have to properly Binded and Aligned before become eligible for execution. The existence of Terminated state ensure each service has a Hand Break that can be activated when needed

• Throughout execution, services subscribe to relevant active contexts, and suspend or terminate when impermissible contexts become active

• Service is as safe as the safety feature of the language used to implement it, but the safety measure reduce the risk of damage caused by services

Element 3: User• In every single pervasive computing

system, even those aiming at automation without human interventions, the ultimate goal: to achieve and fulfill functionality defined by users’ desires and intentions.

• There will be no need for the existence of any system if there was nothing users want to achieve.

• Integral part of pervasive computing system, as their behaviors or mere existence inadvertently affect the system

Modeling of User• Digital representation: User Profile

• Static profile• Describes users’ relevant attributes and preferences

• Used to setup the goal of the service and applications

• Critical for customization and users’ intention prediction

• E.g. special assistance needed, or the preferred room and water temperature settings, or even users’ calendar

• Dynamic profile• Data gathered by the system about the user

• Updated by the system

• Treated as contexts in the system

• E.g. the current location of the user, the blood pressure and glucose level attained by medical sensors

Safety Measure for User• 1st and 2nd Laws of Robotics

• A robot may not injure a human being or, through inaction, allow a human being to come to harm.

• A robot must obey orders given it by human beings except where such orders would conflict with the First Law

• User Safety 1: Smart space should monitor users’ status and activate emergency measure should users appear to be in danger [using the mechanism for Space Safety]

• User Safety 2: Smart space should function to serve users’ preference and obey users’ order [using the mechanism for service safety]

Humans need to be locked up so they

cannot commit crime to harm other

humans

Element 4: Space• Most models do not consider space itself as a

critical element of pervasive computing system

• Influence the decision making (context)

• Big pictures on the effects of service and applications

• Big pictures for user apprehension and visualization

• Provide critical perspective when there are resource sharing

Modeling of Space• Independent Context Manager and Context

Graph• Contexts defined using standardized ontology• Categorization of

preferred, transitional and impermissible contexts

• Visualization in a summary-in-a-glance view on overall context of the space

Hot Indoor

Warm Indoor

Cold Indoor

Overcooked Steak

Cooked Steak

Raw Steak

Murky air

Clean air

Context

Potential Transitions

Preferred Context

Impermissible Context

Blown fuse

Medium power draw

Low power draw

Safety Measure for Space

• Space Safety Monitor: Whenever impermissible context is detected, the space safety monitor will invoke exception handling routine, similar to interrupt/signal capture

• In the worst case where there is no exception handling available, a prompt would be send to the user or the emergency contact person, so human actions can take place to handle the situation

Recap• Enforce Safety at Each h/w and s/w Entity in

the intelligent environment• Device Safety Enforcer• Service Signal Handling Vector

• Respect Users’ Wishes, and Constantly Look After Them

• Look at the Big Picture, and make sure the interactions among users and entities are not harmful

Prior Endeavors

ATLAS Middleware

IDE: Atlas Plug-in for Eclipse

What does Atlas plug-in do?• Browse Available Entities and Services

• Provide Information on OSGi bundles

• Design by Selection

• Semi-automatic Environment Setup

• Integrate seamlessly with all the nice features and assistance provided by Eclipse

• Deploy New Bundles Back to Smart Space

Context-Driven Model

• Reactive Programming Model

• Ontology Based context graph

• Categorize contexts into desirable, transitional and impermissible context

• Find the actions required to take us from currently active contexts to preferred contexts, and activate special measures when impermissible contexts are active

Hybrid Model

Epilogue

Past, Present and Future• Past

• Establish safety-oriented programming model• Implement IDE to support basic OSGi bundle

authoring

• Present• Implementation of the new safety framework with

safety enforcer into both the ATLAS Middleware and the IDE support

• Future• Evaluating the safety feature of the intelligent

environment created based on this framework

Conclusion• Pervasive Computing Systems are Intimate

• The failure and unreliability can result in strong feeling of betrayal and frustration

• Intelligent Environment Affects Numerous Aspects of the Daily Life

• Ensure Safety of the System by Ensuring Safety of All Entities and the Environment as a Whole

• Safety Enforcement is accomplished by the collaboration at both implementation/ compilation time as well as runtime.

Thank You

Design Philosophy• Safety Feature:

• Each individual device has a device breaker to monitor abnormal access pattern and invalid values

• The smart space as a whole guarded by context manager

• Orthogonal Elements:• Simplify the design, implementation and maintenance

of the heterogeneous and dynamic system• Dual text/object approach:

• Text based implementation allows common, configurable services to be created without coding and maximum portability;

• Object embodies more complicated or less-common logic to be created for specific services

The Path Forward

• Analysis of Pervasive Computing Systems and elements of them

• Design programming model and necessary abstractions to simplify design and implementation

• Focuses on safety features to protect users, devices and the smart space as a whole

Modeling of Services

Programmed Service

Aligned Service

Binded Service

User Preference

Service Registry

ServicesServicesServices

DevicesDevicesDevices

Active Contexts

ServiceBehavior 1

ServiceBehavior 2

ServiceBehavior n ….

Active Context Change Event

Service/Device Change Event

User Preference Change Event

Service

Binded

Pervasive Computing• “The most profound technologies are those that disappear.

They weave themselves into the fabric of everyday life until they are indistinguishable from it.” -- Mark Weiser

• “The creation of environments saturated with computing and communication yet gracefully integrated with human users”

-- Pervasive Magazine, Inauguration Issue

• “Embedding computation into the environment and everyday objects to enable people to interact with information-processing devices more naturally and casually than they currently do, and in whatever location or circumstance they find themselves.”

Other alternative terms: Ubiquitous computing, calm technology, things that think, everyware -- Wikipedia

Goals of Proposed Programming Model

• Safe

• Service Oriented Architecture (SOA)

• Open

• Configurable/Customizable

• Orthogonal Description

• Easily Programmable

Background

ATLAS Architecture

Applications

Intelligent Environment(e.g. Smart Home)

ATLAS Sensor/Actuator Technology

Programming Tool

ATLAS Sensor Platform• The basic building block for programmable pervasive

spaces. • Atlas provides

• physical nodes for connecting various heterogeneous devices,

• a system for translating those devices into software services, • a system for maintaining a library of device services and

their interfaces, and • a runtime environment for

accessing services and

composing applications.

Middleware• Atlas Developer API• Context Manager

• Originally designed and implemented for Context-Driven Model

• Implemented using ontology software and inference engine

• Parts of implementation concerning impermissible contexts and their emergency handling routines are retained and modified

• Visualization will be added to provide quick snapshot view

Method Class/Interface Description

receivedData AtlasClient Data handler called by service bundle when data arrives

addProperty AtlasService Add a property pair (key, value) to be associated with this service

removeProperty

AtlasService Remove a property associated with this service

getProperties

AtlasService Get all properties associated with this service

sendCommand

AtlasService Send control commands (used by service associated with an actuator)

getData AtlasService Pull data from sensor associated with this service

Subscribe AtlasService Request data stream from sensor

Unsubscribe AtlasService Halt data stream from sensor

isSubscriber AtlasService Check if application is receiving data stream from particular sensor

dataHandler AtlasService Data handler called by middleware when data arrives for particular service. Must be implemented by device service developer.