towards a model-based approach for context-aware ... · training operation real-time situation...
TRANSCRIPT
Department für Informatik Juniorprofessur Datenbank- und
Internettechnologien Carl von Ossietzky Unversität Oldenburg
Nils Koppaetzky and Daniela Nicklas
Towards a model-based approach for context-aware assistance systems in
offshore operations
CoMoRea 2013
Overview
Why?
What?
How?
What‘s next?
Diskussion
2
Towards a model-based approach for
context-aware assistance systems in
offshore operations
Building and maintaining off-shore wind farms: complex marine operations
Off-shore operations are complex & risky! • Harsh environment • Long supply chains • ... Safety Bankability
Source: Bard Offshore
Application Domain: Safe Offshore Operations
Planning
Risk analysis
Simulation, Training
Operation
Real-Time Situation Analysis
Scenarios
Evaluation
Offline / On-shore Online / Off-shore
Planning
Risk analysis
Simulation, Training
Scenarios
Source: Bard Offshore
Goal: Improve the safety of offshore operation for people, machines, and processes (4 year project, more @ http://soop.offis.de)
SOOP tools and products
For planning and training:
- Mission editor
- Tools for risk analysis
- Safety-driven Simulator for Off-shore Operations
For operation:
- Maritime Sensor Network
- Situational-aware assistant system for offshore operations
Ontology defined in
scenario analysis
Simulation
Risk Analysis
Pe
op
le
En
viro
nm
ent
Ma
ch
ine
s
Infr
astr
uctu
re
Mission
Editor
Mission
model
Visualization
Training
Offshore Operation
Assistant System
Real-time Situation
Analysis
Warnings
Aggregations
Pe
op
le
En
viro
nm
ent
Ma
ch
ine
s
Infr
astr
uctu
re
Situation
models
Sensor Network
planning and training
(on-shore)
operation
(off-shore)
?
Planning
Risk analysis
Simulation, Training
Operation
Real-Time Situation Analysis
Scenarios Planning
Risk analysis
Simulation, Training
Scenarios
Real-Time Situation Analysis
?
Process models
M3
M2
M1
Situation Analysis on Data Streams
Situation models
Deploy-
ment
Environment
Sensor network
Off-shore Operation Assistant System
Observe
Recognize
Predict
Warn
Data mangement systems
Separate the concerns of data management (modeling, representation, storage, retrieval) from application
Provide application-independent support for these concerns - sometimes tailored to certain application domains
Examples - classical relational database system (e.g., DB2, PostGres)
- object –relational mappers (e.g., Hibernate, ..)
- main-memory databases (e.g., SAP Hana)
- map reduce frameworks (e.g., Hadoop and Hbase)
- complex event processing and data stream management systems (e.g., Esper, Infosphere Streams, Odysseus)
data base
management
systems
data stream
management
systems
(and complex event
processing)
? ?
?
Architecture of a Data Stream Management System (Odysseus)
9
Sensors
DSMS
Applications
Exec
uti
on
En
viro
nm
ent
Monitor Op
Plan Manager
Scheduler
Repository
Query Interface
SELECT FROM WHERE
Op
Op
Op
Op
PATTERN WHERE RETURN
S(P(a,b))
User Management
Op
Op
Op
Source Source
Op Op
Op
Op
Op
Op
Source
Alarm Dashboard
Controller
Security QoS/SLA
Translate Rewrite Transform
http://odysseus.offis.uni-oldenburg.de
What do reuse in a data stream management system
Standard (Object-)relational operators
Operators for sensor data fusion - Kalman filters, association,
classification
Context model management - Object hypothesis management - … leads to cylcles in query plans
Reasoning operators / techniques - Specification-based
• E.g., formal logic, spatio-temporal logic, ontologies, fuzzy logic
- Learning-based • E.g., Bayesian networks, pattern
mining, neural networks, SVN
10
19.03
.2013
Sensor description
Transformation
Association
Prediction
Filtering
Context
model
Preprocessing
Sensor data
Situation Analysis
recursive
query plan
Typical object tracking query plan
How to set up the Situation Analysis
Ontology
Sensor gateway(s From Risk Analysis:
Hazard List
From Planning Phase:
Operation Model
(Actors, Activities, Resources, Processes)
What we got …
(with some
built-in sensors)
What we need …
A ship
Data stream
management
system
Off-shore Operation
Assistant System
Stream
Processing
Plans
Visualize Warn
A bunch of
sensors
Observe
Recognize
Predict
Warn
For each sensor
configuration
For each
mission
Dynamic Context Model
How to set up the Situation Analysis
Use generic system architecture from context-aware applications!
- Data: typed data from sensors - Context: relevant observable parameters from the
enrivonment • E.g.: crew.position, sea.level, crane.status, … • In SOOP: defined by the context model
- Situation: relevant combination of context parameters - Adaptation/Application: warn crew
In SOOP: - Non-critical situations: current activity (from process
model), general status of ressources - Critical situations:
• Hazards („crew member overboard“) • Trends towards hazards („sea rising“)
How to implement in the DSMS?
How do we get there …
Data
Context
Situation
Adaptation
Application
Situation models
Situation definition: - Semi-formal description of the situation
- Developed by domain experts
- Uses system model (context variables)
Situation model: - Executable query plan(s) based on context
variables
Context definition: - Semi-formal description of parameters
in the system model
Context model: - Set of executable query plans to determine
context parameters based on sensor data
DSMS
Query Plans
for Context
Model
Query Plans
for Situation
Models
Sensor gateway(s
Offshore Operation Assistant
System
higher level „view“ on sensor data / context model
Models and approach
ASSISTANCE SYSTEM
Da
ta S
tre
am
Ma
na
gem
en
t S
ys
tem
Runtime Context Model
SENSORS
Re
cogn
itio
n /
Inte
rpre
tati
on
De
sign
ONTOLOGY CONTEXT MODEL
MODEL OF SENSORS
SITUATION MODELS
MODEL OF CONTEXT RECOGNIZERS
CONTEXT VARIABLES
MODEL OF APPLICATION / INTERFACE
Deploy
CONTEXT RECOGNIZERS
SITUATION MODELS
APPLICATION / INTERFACE
CONTEXT VARIABLES
Model-based approach
Source models - Offshore ontology from planning and risk assessment (UML)
- Operation models (UML/ BPMN)
- Hazards / Situation models (LTL, linear temporal logic)
Transformation models - RelDSS: model for relational data stream schemata (stream definitions)
- RelDSQ: model for relational data stream queries
Deployment „models“ - Data stream processing plans for context recognition and situations
• CQL queries – SQL like continuous query language
• PQL queries – Procedural querys („boxes and arrows“)
Model transformations
Offshore ontology meta model
Operations planning and Risk analysis Offshore ontology instance
LTLquery instance RelDSS intance
RelDSQ-Instance
// Register Query
SELECT Sensornode_1.timestamp, Sensornode_2.Distance, Sensornode_2.Distance
FROM Sensornode_1, Sensornode_2
WHERE Sensornode_1.Distance > 2.0 AND Sensornode_2.Distance > 2.0
// Create 2nd Stream
CREATE STREAM SensorNode 2 (
timestamp STARTTIMESTAMP,
Distance DOUBLE,
GPS STRING,
) CHANNEL localhost : 10108
Modelling
Manual
Transformation
Transformation
thanks to Jan-Benno Meyer zu Holte
Proof of concept: Field Test Installing the assistance system on a ship
Measure the ships dimensions
Install the anchor sensor knots of the sensor network
Measure the positions of the anchor knots
Install sensor gateway
Connect sensor gateway and computer with situation recognition software
Adapt situations / query plans to measured geometry
Install situations /queries to system
Sensor gateway
Critical Area
Measurements - Sensors UWB and Smartphones
were attached to fake cargo and person taking the role of lift supervisor
- Person and cargo move on deck - Positions and 3D acceleration data
were tracked and recorded
First insights - Live recognition of person in
critical area was possible - Localization on the sensor nodes
didn’t always work • install more context
recognizers to exploit also partial data?
cargo
quarter turn
lift supervisor
moving pattern
Proof of concept: Field Test Installing the assistance system on a ship
Conclusion
Flexible safety monitoring applications are a further example for context aware application
We need abstractions to reduce development time
Context detection and reasoning can be realized with a data stream management system
- Context model acts as „view“
Future work: - implement more situations
- include training from learning approaches
- evaluate model transformations in other application domains
Thank‘s for the fish!
20 [email protected] 20 [email protected]
… also to the Odysseus team and Jan-Benno Meyer zu Holte.