Building Blocks for a Participatory Web of Things: Devices, Infrastructures, and Programming Frameworks
Doctoral Defense – 26 August 2011 Vlad Trifa
Vlad TrifaDoctoral Defense, 26.8.2011
Context
2
■ Networked embedded devices (NEDs) omnipresent■ Increasingly powerful CPUs, various sensors/actuators■ Mobile Internet cheap and ubiquitous
■ Tremendous benefits when integrating their data and services ■ Social: energy-efficient building, smarts buildings and cities■ Business: real-time enterprise, optimized logistics
Vlad TrifaDoctoral Defense, 26.8.2011
Distributed Sensing Applications (DSA) Today
■ Tedious process that requires many resources (skills, time, $$$)■ Various functionalities, sensors, requirements■ Incompatible protocols, standards, programming models, APIs, etc.■ “Wheel reinvention” is common (hard-wired applications)
3
Web Gateway
low-power radioprotocols
(ZigBee, etc.)
base-stationconnected via
serial linestorage
analysis&
processingWeb page
Vlad TrifaDoctoral Defense, 26.8.2011
Problem Statement
4
How to create an infrastructure for facilitating the development of asynchronous DSAs on top of heterogeneous, mobile NEDs?
Vlad TrifaDoctoral Defense, 26.8.2011
The Web of Things
5
■ Leverage Web architecture, standards and techniques■ REST, HTTP, HTML, JSON, MIME, caching, authentication, etc.■ TCP/IP & Web granted, Wi-Fi routers ubiquitous
■ HTTP/REST: many advantages for larger DSAs■ Flexible, loosely coupled, scalable, lightweight
■ Smooth integration with existing Web infrastructure■ Blend real-world services and devices with the Web■ Development of simple Web apps: cheaper & faster
Vlad TrifaDoctoral Defense, 26.8.2011
Research Questions
6
Is the Web as is sufficient to support DSAs? What is required for most DSA?
How to build a framework for developing applications that are well integrated into the Web?
Part 1
Web-enabled Devices
Vlad TrifaDoctoral Defense, 26.8.2011
Web-enabled Devices
■ Devices run Web servers ■ services RESTful resources
■ REST core architecture of Web■ HTTP implements REST style
■ URI for each component■ Uniform Interface■ HTTP verbs & codes
■ Representations■ HTML: humans■ JSON/XML/CSV: machines
8
D. Guinard, V. Trifa, F. Mattern, and E. Wilde. Architecting the Internet of Things, Chapter From the Internet of Things to the Web of Things: Resource Oriented Architecture and Best Practices. Number 5. Springer, May 2011.
GET fire/alerts.xml
PUTtv/channel/4
GET fridge/food.html
WebHTTP
proprietary
Bluetooth
X10
IEEE802.15.4
DLNA
HTTP
HTTP
Google APIs
Flickr API
GatewayAPI
Vlad TrifaDoctoral Defense, 26.8.2011
Gateways
9
V. Trifa, S. Wieland, D. Guinard, and T. M. Bohnert. Design and implementation of a gateway for web-based interaction and management of embedded devices. In Proc. of the 2nd International Workshop on Sensor Network Engineering (IWSNE’09), Marina del Rey, CA, USA, 2009.
Vlad TrifaDoctoral Defense, 26.8.2011
Gateway Internal Architecture
10
Sun
SPOT
TMoteDT1
DT2
device
services
device layer
control layer
presentation layer
REST
engine
HTTP
server
proxy application
user
services
application
services
templating
engine
devices
DB
device
drivers
device thread pool
../devices/1
../devices/2
direct
cached
0 100 200 300 4000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Response time [ms]
Cum
ulat
ive
Den
sity
Fun
ctio
n
DirectCached
Vlad TrifaDoctoral Defense, 26.8.2011
Response time for Consecutive HTTP GET Requests
11
D CMin. 159 2Max. 3075 36Avg. 203 3.9q99% 282 19
RTT [ms] of 10’000 local read requests for direct and cached approaches
Part 2
Distributed Infrastructures
Vlad TrifaDoctoral Defense, 26.8.2011
Distributed, Location-aware Infrastructures for WoT
13
V. Trifa, D. Guinard, and S. Mayer. Leveraging the Web for a Distributed Location-aware Infrastructure for the Real World. Chapter 17 in REST: From Research to Practice, Edited by Wilde and Pautasso, Springer-Verlag, 2011.
/ethz
/ethz/CNB
/ethz/CNB/FloorD
../southWing/D48.1
/ethz/CNB/FloorEvirt
ual g
atew
ays
(can
run
anyw
here
)ph
ysic
al g
atew
ays
(mus
t be
embo
died
)
../southWing/D48.2
../floorD/southWing
Vlad TrifaDoctoral Defense, 26.8.2011
Service 1: HTTP-based Devices and Resources Discovery
14
Vlad TrifaDoctoral Defense, 26.8.2011
Service 1: HTTP-based Devices and Resources Discovery
14
Device with machine-readable
embedded metadata (description,
API, etc.)
1LAN
1. Device connects to router and
gets an IP address via DHCP
RouterLAN (Ethernet)
Vlad TrifaDoctoral Defense, 26.8.2011
Service 1: HTTP-based Devices and Resources Discovery
14
Device with machine-readable
embedded metadata (description,
API, etc.)
1LAN
1. Device connects to router and
gets an IP address via DHCP
RouterLAN (Ethernet) LAN
2. Gateway polls the DHCP
allocation table on the router Gateway
2
Vlad TrifaDoctoral Defense, 26.8.2011
Service 1: HTTP-based Devices and Resources Discovery
14
Device with machine-readable
embedded metadata (description,
API, etc.)
1LAN
1. Device connects to router and
gets an IP address via DHCP
RouterLAN (Ethernet) LAN
2. Gateway polls the DHCP
allocation table on the router Gateway
2
3
3. For each new device it gets the root
page, parses metadata and adds it to
registry
Vlad TrifaDoctoral Defense, 26.8.2011
Service 2: Infrastructure-aided Location-aware Search
15
/zurich/ethz
/zurich/ethz/CAB
/zurich/ethz/CAB
"GET all meters at
/zurich/ethz/CNB/"/zurich
../room103 ../room102
"Here you are:
../room102/meter,
../room103/meter"
../meter ../meter ../spot
query scope
Part 3
Web-based Messaging
Client Server Application
Event
Event
Event
ResponseRequest
ResponseRequest
ResponseRequest
Vlad TrifaDoctoral Defense, 26.8.2011
Real-time Web
■ HTTP designed as a request-response protocol (scalability)■ “Eventing” done via polling or federation (feeds: RSS/ATOM)■ Growing community around Real-time Web techniques■ HTTP callbacks, Comet, XMPP, Web Sockets, etc.
17
Client Server Application
Request
Event
Event
Event
Response
Response
Response
Comet, Web socketsAJAX (Web polling)
Vlad TrifaDoctoral Defense, 26.8.2011
Web Messaging System (WMS)
■ Gateways & devices need a minimal Web push mechanism■ Not a protocol (Google’s pubsubhubbub), only Web patterns
■ Notification implemented as HTTP push callbacks [1]■ RESTful, Simple, light (less options) to run on NEDs■ Generic pub/sub support (queues & subscriptions)
18
Subscribe to a channel:POST SUB http://.../channels/ethz/CNB/H/subscribers
Publish to a channel:POST MSG http://.../channels/ethz/CNB/H/
Receive notifications:foreach SUB in http://.../channels/ethz/CNB/H/subscribers POST MSG SUB
[1] V. Trifa, D. Guinard, V. Davidovski, A. Kamilaris, and I. Delchev. Web messaging for open and scalable distributed sensing applications. In Proc. of the 10th International Conference on Web Engineering (ICWE 2010), Vienna, Austria, June 2010.
Vlad TrifaDoctoral Defense, 26.8.2011
Web Sensor Streams
■ Sensor data represented as sequence of messages ■ More complex constructs on top of minimal pub/sub implementaiton■ Not bound to transport protocol (HTTP callbacks, WebSockets, etc.)
■ Parameterized■ devices■ sensors■ sampling frequency■ filter data
■ Messaging: bi-directional glue between components to support asynchronous communication
19
t4 t3 t2
Devices
t1 Client
D1
D2
D3
Web Sensor Streamdata=light,temperature&devices=D1,D2,D3
{"time":123223, "data": [{"D1":{"light":4, "temp":22}}, {"D2":{"light":21, "temp":22}}, {"D3":{"light":200, "temp":33}} ]}
Part 4
End-to-end Event-driven Applications
Vlad TrifaDoctoral Defense, 26.8.2011
WISSPR - Web Infrastructure for Sensor Stream PRocessing
21
MCDCvarious data
acquisition protocols
1. raw
3. raw data
Client SC
QC
3'. processed
5. raw & processed
Client
2. raw/filtered 4. processed
Sensors 1. Device Connector
2. Messaging Connector
3. Query Connector
4. Storage Connector
Vlad TrifaDoctoral Defense, 26.8.2011 22
MC
Same-machine brokerlow latency
low scalability
high coupling
Same-LAN brokermedium scalability
medium latency
lower coupling
Cloud-based broker
Highest scalability and flexibility
lower latency
lowest coupling
Wisspr main nodehttp://wisspr.net
Uniform messaging
interface (Web API)
JAVA
AMQP [1] REST
REST
[1] Advanced message queuing
protocol amqp.org
Vlad TrifaDoctoral Defense, 26.8.2011
Complete HTTP Request (& Application)
23
POST /streams/ HTTP/1.1Host: wisspr.netContent-Type: application/x-www-form-urlencoded
devices=D1,D2,D3&data=temperature,light&frequency=2Hz&filter=light<200&&temperature>19&wms.cbk=http://store.wisspr.net/stores/31
Note: content should be %-encoded.
Vlad TrifaDoctoral Defense, 26.8.2011 24
0 500 1000 1500 200010
0
101
102
103
104
Load [msg/s]
Dela
y [m
s]
processingfetching
End-to-end Notification Delay
20 Devices simulated in DC■ Varied sampling frequency (SF)
■ Average notification delay [ms](DC-MC-Clients) delay:
■ Scalability limited by clients
SF Msg/s Proc. Fetch. Total25 500 2.5 17.3 2050 1000 3.4 98 101.575 1500 4.2 250.8 255.1
100 2000 5.2 1177.7 1183
Vlad TrifaDoctoral Defense, 26.8.2011
Main Contributions
1. Set of reusable patterns & best practices for physical Web■ Eventing, streaming, querying constructs■ Pervasive middleware services (discovery, search)■ Location concepts directly integrated
2. Proof of concept implementations and evaluations■ Entirely Web-based infrastructure for heterogeneous DSAs■ Reasonable scalability for devices, users, and real-time data
3. End-to-end 1-line programming framework■ Collection, processing, sharing, and storage■ Seamless Web integration
25
Thanks for your attention!
Questions?
M. [email protected]. vladtrifa.comT. @vladounetB. webofthings.com
Part 5
Backup slides
fears?
Vlad TrifaDoctoral Defense, 26.8.2011 29
Local network (Ethernet LAN)
Main
broker
Gateway(broker)
WMS
Subscriber
Subscriber
Subscriber
ATOM Pull
Comet
Web Hooks
POST
POST
Gateway(broker)
WMSPOST
POST
Private sensor network (radio, Wi-Fi, etc.)
Public network(Internet)
Vlad TrifaDoctoral Defense, 26.8.2011 30
MCDCOSGi Framework
OSGi Framework
3. Streamoutput
2. Raw Data
QC
Client
1. Register query [Query, CBK-URI]
3'. Stream output
HTTP/AMQPWisspr main node
http://wisspr.net
External query processor http://query.wisspr.net
JAVA
OSGi
HTTP
HTTP
real-time (raw) data streams
this is you
Web API
LIVE Singapore!
http://www.faroutof.it
http://senseable.mit.edu/livesingapore
t4 t3 t2
Devices
t1 Client
D1
D2
D3
Data Streamdata=light,temperature&devices=D1,D2,D3
{"time":123223, "data": [{"D1":{"light":4, "temp":22}}, {"D2":{"light":21, "temp":22}}, {"D3":{"light":200, "temp":33}} ]}
Vlad TrifaDoctoral Defense, 26.8.2011
Web Stream details
32
Vlad TrifaDoctoral Defense, 26.8.2011
Web stream with frequency and filtering
33
L=25 L=18 L=28 L=22
Client
Raw Data Stream
L=25L=22
samplingperiod
L=22L=25
YES YESNO
Resulting Stream
data=light,temperature&filter=light>20&frequency=2
or this?http://liip.to/niwea
devices and services ecosystem
Vlad TrifaDoctoral Defense, 26.8.2011 36
20 Background and Motivation
extensions that add many features from secure communication to discovery. Most applications
and programming languages support XML and HTTP, and one could easily build stubs to
interact with specific services that do not need to be upgraded as long as the WSDL file does
not change. Unfortunately, even though tools that help to automate the development of such
services became widely spread and used, SOAP-based Web services remain quite difficult to
understand and use, as in practice different implementations of the same WS-* standards are
rarely entirely compatible with each other.
Another problem persists with SOAP-based Web services: they are a legacy of RPC style
protocols, therefore are not flexible enough for highly dynamic environments where new, un-
known devices continuously appear and disappear. Besides, WS-* consider HTTP only as a
transport protocol to perform remote procedure calls instead of being directly integrated into
the Web [244], in which case there would be no need for any additional API or descriptions of
resource/function. This situation gave birth to numerous debates about ROA versus SOA5.
Attribute OOA SOA ROA
Granularity objects services data (resources)Main focus marshaling creation of payload addressing (URI)Addressing object instance unique endpoint individual resources
Replies cacheable No No YesAPI language-specific application-specific generic
Lightweightness +++ + ++Flexibility + ++ +++Scalability +++ ++ +++
Loose-coupling + ++ +++Simplicity +(+) + +++Standard + ++ +++
Tools available +++ +++ +Security +++ ++ ++Verbosity + +++ ++Ambiguity + ++ ++
Table 2.1: Overall feature comparison of different architectural styles (+ means low, ++ means
medium, +++ means high).
In Table 2.1, we summarize the differences and properties of the various styles we presented.
Based on the requirements we defined in Section 2.1.4, we suggest the ROA style is the most
suitable architectural style to implement a scalable participatory infrastructure for NED ap-
plications. In particular, as long as applications do not require custom protocols to meet
particular “hard” requirements (such as reliability, security or throughput), the loose coupling
of ROA architectures offers many advantages and features useful for NED applications. Also,
5An insightful discussion about the roots of this debate can be found here:http://www.prescod.net/rest/rest_vs_soap_overview/