Jie Liu

Principal Researcher Microsoft Research Redmond, WA 98052

Towards the Wikipedia of World Wide Sensors

With thanks to Yan Xu, Suman Nath, Aman Kansal, Heitor Ramos, and Qiang Wang

Paradigm Shifts in Computing

Consumer Computing

Community Computing

Cloud Computing

Computing in the Real World

Energy

Climate Change

Environment

Homeland Security

Disaster Response

Critical Infrastructure

Transportation

Asset Management

Healthcare

Assisted Living

...

Experimental, theoretical, and computational, and data-driven science.

4th Paradigm of Scientific Discovery

Microsoft Sensing Research

With applications in

• Environmental monitoring

• Data center operation and energy management

• Mobile computing

Collaboration Collection Extraction

S

N

Atlantic Rainforest Micrometeorology Sensor Network in Brazil (University of São Paulo, Microsoft Research, Johns Hopkins University)

(Images courtesy of Humberto Rocha, Rob Fatland, and Andreas Terzis)

Towers

50meters

Serra do Mar

SwissEx

Put all data together for better understanding Share data with other scientists

Snow Soil Streams Temperature Humidity

• Sensor networking

• Energy management

• Data yield improvements

• Deployment strategy

• Data management

• Data interoperability

• Data archival

• Sensor tasking

• Data visualization

• Temporal-spatial indexing

• Online aggregation and representation

Key Technical Challenges

Participatory Environmental Monitoring Toolkit

Objectives

• Facilitate socially inclusive environmental observation

o Time & GPS location

o Temperature & Humidity

o CO2

o H2S

• Leverage existing Microsoft technologies and user communities

• Deliver a HW+SW toolkit in open source form

Key technologies

• Microsoft Research low energy GPS location sensing and mobile data collection services

• OData

• World Wide Telescope(WWT)

• Windows Azure

* In development

Sense Web: The Wikipedia of sensors

Real-time indexing, aggregation, and tasking.

Cypress: Data Stream Compression

• Compress data to reduce storage and I/O cost

• Answer queries directly on compressed data

• 100X compression typical sensor data streams

Columns

• Take advantage of data types.

• Trim data based on sufficient precision.

Trickles

• Spectrum analysis – store data based on frequency bands

• Store anomalies separately

• Use sketches to compress “noise” – preserve data correlation.

GAMPS:

• Find correlations among data streams.

• Store data as differences or ratios to reference streams.

Open Data Sharing

*Cyberinfrastructure for the waters networks: a Survey of AEESP and CUSHAI Members, K.A. Lawrence et al, May 2006,

Popular Software Packages* Factors Influencing Technology Adoption*

SQL

The Lowest Common Denominator OData • Easy of use • Additional value • Professional technical support

WWT and Geo-Data Visualization

WorldWide Telescope (WWT)

• A visualization software environment

• Enables a computer to function as a virtual telescope

• (astronomers call it “the best VO (virtual observatory) implementation”)

• Visualizes geo-data in 4D (space + time)

• Integrated with Excel

• Allows data sharing with controlled access – WWT Community

• Empowers high-quality, intuitive, and interactive visual presentation via “WWT tour”

• Datasets under consideration

• Seismic event distribution against sbuductionslab slab models (USGS NEIC)

• Standardized-format datasets (OGC, WxS, NetCDF, Shapefile, CSV, HDF, …)

• Dataset and model output concept: plugging data generators directly into WWT

• Draped raster, e.g. MODIS ocean, land and atmospheric products

• Alternatice topgraphy, e.g. ice sheet thickness and bathymetry

• Climate change thematic datasets, e.g. monthly sea ice extent from NSIDC

• Free for research and education use

* In development

WWT and Dust Storm Simulation

• A mutually beneficial case study

• Mind-swap, e.g. at Open Data for Open Science Developers Training

• Improve science modeling

• Improve computer engineering

* In development

Wikipedia of Environmental Sensing

• Open platform, open data

• Free participation

• Discoverable, searchable, interoperable

• Visualized, annotated, built on top of each other

Microsoft Environmental Informatics Since 2010 • Vision: facilitate seamless access to environmental data and information

• Focus: data discoverability, accessibility, and consumability

• Objectives:

• advance the technology use in environmental research

• create design wins using Microsoft technologies to

• Foster innovations in computational environmental research

• Advance interoperability of data and information sharing

• Facilitate citizen science for environmental research

• Build a community among multiple disciplines and stakeholders