o ak r idge n ational l aboratory u. s. d epartment of e nergy making usable data available and...

OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY

Making Usable Data Available and Available Data Usable: The Application of Geospatial Standards in Near Real-Time

Sensor Networks

Daniel GetmanAaron Myers

Geographic Information Science and Technology Group

Oak Ridge National LaboratoryApril 20, 2007


What is SensorNet

Much more than what is presented here Stand Alone Sensor Networks

• Mobile Sensor Systems• Weigh Station Systems• Port Systems• Military Base Systems

Distributed Sensor Networks• Connecting stand alone systems in a larger

network Overarching Concepts

• Interoperability/Standards• Real Time Data Distribution and Alerting• Ubiquitous Data Access• Development of Global Scale Sensor Networks


Describing the Stand Alone Systems

Multiple Sensors Chemical – Radiation – Biological – Weather Video with real time analysis capabilities

Multiple Viewers ArcMap Google Earth Browser based viewers

Integration with Models HPAC

Data Distribution Capabilities Mobile systems communicating with

operations centers and other systems


Defining The Goals

Minimum System Goal Provision of near real-time data within the stand

alone system while also providing access to data to external users including Operation Centers and other emergency response personnel

Maximum System Goal Develop an open standards based system within

which a variety of sensors, viewers, and complete stand alone systems can plug-and-play can use and share near real-time sensor data

Stretch Goal Make the openness of the system transparent to

all users


Standards Used in the Stand Alone Systems

Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE)

• Includes Sensor Observation Service (SOS), Sensor Alert Services (SAS), and Sensor Planning Service (SPS)

Web Feature Service (WFS) Geography Markup Language (GML)

U.S. Department of Homeland Security (DHS) ANSI N42.42 (Radiation Data Representation)

Organization for the Advancement of Structured Information Standards (OASIS) Common Alerting Protocol (CAP)

National Institute of Standards and Technology (NIST) IEEE 1451 (Sensor Plug-n-play)

Courtesy: David Resseguie, ORNL


Why Standards are Great

Standards are great once they are adopted Data sharing is not only easier but can be

easily automated Everything is less expensive…eventually Creativity is stimulated in application

development


Why Standards Can Be Difficult to Use

Querying differences Data Size Differences Data Structure Differences

DB

User Viewer

SQL Query

Response Containing Desired Data

Traditional Database Query

WFS Database Query

User Viewer

WFS Query

Response Containing More Than Desired Data

DBWFS Request to SQL Conversion Process

SQL Query

Raw Data to XML Conversion Process

GML, or other standard, response


WFS and SQL Queries

Traditional SQL QuerySELECT Col1, Att1, Val1 FROM TABLE1 WHERE Val2 = “SomeValue”

WFS Query<wfs:GetFeature service="WFS" version="1.0.0" xmlns:gml="http://www.opengis.net/gml" xmlns:ogc="http://www.opengis.net/ogc" xmlns:snet="http://www.sensornet.gov/snet" xmlns:wfs="http://www.opengis.net/wfs"> <wfs:Query typeName=“snet:Table1"> <ogc:Filter xmlns:ogc="http://www.opengis.net/ogc"> <ogc:PropertyIsLike> <ogc:PropertyName>//snet:Val2</ogc:PropertyName> <ogc:Literal>SomeValue</Literal> <ogc:PropertyIsLike> </ogc:Filter> </wfs:Query></wfs:GetFeature>


Simple WFS Performance TestsXhive and Oracle bassed WFS and MySQL DB

Processing Time For n Concurrent Queries While Processing Ten Inserters at Ten Features Per Insert

0

5000

10000

15000

20000

25000

1 5 10 15 20 25 30 35 40 45 50 100

Concurrent Requesters

Ave

rag

e R

esp

on

se T

ime

(ms)

XHive based WFS

Oracle based WFS

MySQL DB


Practical Difficulties

Clients can have conflicting expectations Things that are difficult with Standards Based Systems

• Legacy Software and Data– Few software packages will natively access

standards based data sources• Familiar/Robust/Highly Functional Interface

– It has to be as good as the proprietary system• Near Real-Time Dynamic Data

– Lots of queries, some very specific (alerting)• Database Design Limitations

– Designed for support of standard rather than support of user activities

– Large Data Capacity can be problematic» Keeping all of the data without impeding

system speed• Data Integration

– Leveraging of existing proprietary data


Idealistic SensorNet System

Standards Based Viewer

Sensor

Standard Output FormatStandard Query Format

Standard Output Format

Everyone Else

WFS


Where does this leave the ISS?


Sensor

Standard Query Format

Standard Output Format

Sensor

Sensor

Proprietary Output Format 1

Standard OutputFormat

Proprietary OutputFormat 2

ProprietaryViewer 1

ProprietaryViewer 2

??

? ?

Everyone Else

??WFS


The Duct Tape Approach

Implement standards where the technology and client requirements allow

Implement component based solutions that handle the places in the system where standards either do not exist or are not fully implemented

Plan to remove the duct tape when the standards are implemented and adopted


Slightly More Realistic SensorNet System


Sensor

Sensor

Sensor




ProprietaryViewer 1

ProprietaryData Source

ProprietaryViewer 2

Everyone Else

? ?Custom

ConversionUtility

Custom Conversion

Utility

WFS


Tale of Two Deployments

JFHQ SNAPS in Washington DC Mobile SensorNet System (Actually in a trailer) JOC System designed to receive data from

Mobile and Base systems GPS enabled Radiation, Chemical, Weather,

Video HPAC Integration

SRRPP at the Port of Charleston Land and Marine based SensorNet Systems Vehicle and Officer GPS Tracking GPS enabled Radiation, Video


JFHQ SNAPS Deployment


SRRPP Deployment


Lessons Learned with the SMRSS

Intrinsic problems include Data is duplicated Two conversion processes to maintain

Performance Issues Both WFS and ArcSDE are bottlenecks compared to direct

database interaction. Together, they can really hinder performance

The conversion to ArcSDE caused a large number of reads on the WFS. This had a negative impact on the overall performance of the system. More reads per second means less writes per second

Inserting data into ArcSDE is significantly slower than inserting data directly into a database• We found that ArcObjects inserts were faster than SDE API

inserts• Querying against the data tables directly in Oracle to gather

data used in the conversion process, rather than going through SDE, is a good shortcut

Conversion process can be used for other tasking that would otherwise require an additional WFS call• Alerting and interfacing with other systems (cameras for

example)



Sensor

Sensor

Sensor




ProprietaryViewer 1


ProprietaryViewer 2

Everyone Else

Custom Conversion

Utility

WFS

Realistic Idealistic SensorNet System

No pounding the WFS with SDE Inserts

Single Conversion Utility

Everyone Gets What They Want


Evolution of the RISS


Sensor

Sensor

Sensor




ProprietaryViewer 1


ProprietaryViewer 2

Everyone Else

Custom Conversion

Utility

WFS

Times They Are A-Changin'


Evolution of the RISS


Sensor

Sensor

Sensor




ProprietaryViewer 1

ProprietaryViewer 2

Everyone Else

Custom Conversion

Utility

Many, ManyData Services

DB


New & Improved Idealistic SensorNet System


Sensor

Sensor

Sensor


ProprietaryViewer 1

ProprietaryViewer 2

Everyone Else



Many, ManyData Services

DB


- Questions and Comments -

Daniel [email protected]

865-241-1745

o ak r idge n ational l aboratory u. s. d epartment of e nergy making usable data available and...

Documents