swiss experiment epfl-lsir report hoyoung jeung swissex annual meeting, zurich 15 th june
Post on 19-Dec-2015
218 views
TRANSCRIPT
![Page 1: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/1.jpg)
Swiss Experiment EPFL-LSIR Report
Hoyoung Jeung
SwissEx Annual Meeting, Zurich 15th June
![Page 2: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/2.jpg)
Deliverable Overview
![Page 3: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/3.jpg)
Framework Overview
capturedcaptured
queriedqueried
processedprocessed
![Page 4: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/4.jpg)
Data Integration
Requires close collaboration with end-users– BIG efforts to find appropriate solutions to very specific needs– Continuous migrations of data along with hardware evolution and
addition of new types of sensors.
in 2010– SwissEx general (hosted by SLF) – http://montblanc.slf.ch– LTE – http://ltepc.epfl.ch– APUNCH – http://ifu-apunch.ethz.ch:22001/– PermaSense – http://data.permasense.ch
Continuing in 2011– CCES general portal - http://montblanc.slf.ch:22001– CCES public data - http://montblanc.slf.ch:22002/– Radar images and data from APUNCH: http:// ifu-
apunch.ethz.ch:22001– Disdrometer and radar data from Environmental Remote Sensing
Laboratory: http://ltepc3.epfl.ch:22001
![Page 5: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/5.jpg)
Data Repository/Mirroring Services
In addition to the aforementioned instances of GSN, mirror some of the data using additional machines at EPFL.– Near real-time data duplication.– Multipurpose: backup, query load distribution.– Accessible only for authorized users
LSIR repository– Mainly data from SLF, backup purpose
Lafouly repository– Data from a catchment used by the EU project Hydrosys in
Lafouly region. Mainly used for GEOtop model simulations.
PlanetData repository– Mirroring Wannengrat public data at SLF for query load distribution.
![Page 6: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/6.jpg)
Spatial Data Support
Extending GSN for handling geographical data
Support spatially-enabled processing – Data types: point, polyline, polygon,…– Spatial queries: e.g., average wind speed in Lausanne yesterday?
Two implementations– Java topology suite (JTS) integration: used for any DB– PostGIS integration
• Only for PostgreSQL DB• Support spatial index
Output in GML format
![Page 7: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/7.jpg)
Sensor Data Search
Processing in a federated network– Wiki: central point, easy to use GUI based on Google Maps interface– GSN instances: distributed, very fast data visualization
Ontology-based– “temp=temperature=air temp”– W3C SSN + NASA SWEET– On-going work
Plan to scale up– Search beyond swissex data
![Page 8: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/8.jpg)
User-Interactive Data Cleaning
Model-based anomaly detection– User can select model/parameters, parametric/non-parametric
original data stream↓
approximation using user-selected models↓
detecting anomalies↓
user confirmation: anomaly is an actual error?
![Page 9: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/9.jpg)
New Wrapper Developments
Grid wrapper– Reads ESRI grids (e.g., digital elevation models) from files and stores as
binary objects in DB, to be queried using a new web method (/griddata)– Returns grids (ESRI ASCII format or rendered images) for a given time
range or a stream of values for a single cell within a grid (as a time series).
JDBC wrapper– Connects to remote DB directly, without going through standard GSN communica
tion channels– More efficient processing, used in PermaSense
Sensorscope wrapper– The previous Sensorscope wrapper was extended to handle the new
packet format and new sensor types (e.g. Decagon 10HS and Apogee SP-212).
Tweeter wrapper– Enable data fusion between sensor data and Tweeter data in real time
![Page 10: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/10.jpg)
Various Technical Support
Data access via web services– A standard web service was developed for accessing data in GSN
from external tools like Geotop and Alpine3D. – A C++ client for data access using user’s existing programs.
Continuation of the access control component in GSN– GSN discretionary access control for each virtual sensor. – Access can be made through web interface (using interactive login)
or through passing of parameters using REST web calls or standard web services.
– Direct passing of parameters without going through the interactive web sessions was needed for automatic tools such as Mathlab or R.
Install, configuration, patches
Continuous development of SwissEx wiki
![Page 11: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/11.jpg)
Planned Work
Linking Swissex data – e.g., Linked Open Data (LOD) data– PlanetData (FP7 NoE) consortium– OpenIoT (FP7 strep)– Scalable solution necessary
Completion of ontology-based sensor data search
Cloud-based sensor data management– Mirroring the current data with public/private cloud services
![Page 12: Swiss Experiment EPFL-LSIR Report Hoyoung Jeung SwissEx Annual Meeting, Zurich 15 th June](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d405503460f94a1a15a/html5/thumbnails/12.jpg)
Questions ?