Implementing WMS interface on Imagery Database (BDI)
Frédéric GUILLAUD
DSI/DEV
EGOWS Meeting
KNMI, June 2009
2
Overview
Context– Towards SOA for production and workstations (Marie-Françoise talk)
– DWG Meteorology and WMS IE
Soprano imagery database (BDI) WMS implementation Issues Still to do Conclusion
3
Context
IT SOA pilot project Goals
– Improve our skill with SOA architecture and tools (ESB)
– Improve our skill with OGC standards
– Early identify issues
– Make technical decisions
DWG meteorology IE on WMS– Calendars compatibles (Over 6 month by the end of 2009)
– WMS BDI may be a service involved in IE experiments
– To do this, WMS should be deployed in DMZ
– This lead to security issues
4
Soprano Imagery database (BDI) : Before
RDBMS Oracle 3 days real-time data online (“blob” data type). Data is not archived. About 250 datasets Legacy catalogue content not always consistent … Native formats
– BUFR, TIFF-MeteoFrance (TIFF with - open - specific MF tags…)
Native extent depends on dataset origin (MSG, METOP, other geostationary platforms, radar data ..)
Native CRS– “Space View” (One-Point Perspective Projection ?) and “Polar Stereographic”
Data access services– Low level (application binary executable)– No API available– No remote access possible
5
Soprano Imagery database : product overview
Basic radar reflectivity measurements (for each radar) Mosaics
– Precipitation flux (through reflectivity conversion)
– Precipitation accumulation (5’, 15’, …)
– Forecasted intensities (2PIR algorithm)
6
Soprano Imagery database : product overview
Satellite data• From MSG, METOP, GOES platforms …• Composite CMS products, …
7
Soprano Imagery database : products overview
Composite products– Radar precipitation intensity calibrated with ground measurements
(ANTILOPE)
– Satellite-Based fog diagnostics (CARIBOU)
8
WMS BDI features : Now
About 250 Layers GetMap returns only one LAYER at a time LAYER name is an internal identifier (tokens delimited by “_”, in
order to make easier mapping WMS – RDBMS data access Pre-defined STYLE : Each LAYER has a default colour map, which
is the official colour map on operational workstations at Meteo-France.
User defined styles are not yet supported (SLD=) If TIME not specified, GetMap returns the last image Outputs formats : image/gif; image/png Output CRS : only EPSG:4326 and Polar Stereographic at the
moment No support for metadata layers (e.g. pixel timestamp, pixel
quality, ...) but may be WCS scope ?
9
WMS Implementation : Reuse of legacy components
Data access component : “RDImage”– Application Binary Interface (executable)
– Performs data extraction from BDI on criteria :• Data type (radar, satellite …)• Production type (observation, forecast …)• Data origin (sensor platform, system, algorithm …)• Native domain covered by the dataset • Parameter (rain intensity, accumulation, radiance/wave length …)
Image processing component : “TransImage”– Application Binary Interface (executable)
– Performs :• Sampling a sub-domain• Format translation• Coordinate reference system operations (re-projection)
10
WMS Implementation
Meteo-France WMS framework– Implements WMS interface in PHP
– Check on GetMap – GetCapabilities consistency
– WMS 1.1.1 or 1.3.0
WMS BDI driver– Links GetMap parameters to RdImage & TransImage ones
– Depending on GetMap parameters, invoke TransImage if necessary
– Generate GetCapabilities response (XML)
11
WMS implementation overview
BDI (Oracle)
OCI
RDImage (.exe C/C++)
TransImage (.exe C/C++)
Capabilities (C )
Meteo-France WMS Framework (PHP)
GetMap GetCapabilities
BDI Driver (PHP)
WMS
12
Issue : GetCapabilities layering
GetCapabilities response – Mainly product metadata
– Describe product types (characteristics common to product instances)
GetCapabilities tree : Observations & Measurements approach– 3 Levels of layers :
• Process– Product type, Platform, Algorithm e.g Satellite MSG
• Feature Of Interest– Domain e.g Europe-Atlantic …
• Observed property– Parameter e.g IR 108
– Leaf layer is a named product type (queryable)
13
Issue : GetCapabilities layering
Layer granularity – Balance LAYER / Multi-dimensional LAYER (LAYER plus DIM)– E.g :
LAYER=SA__OBS__MSGC__V_EURATL__WV-62Or
LAYER= SA__OBS__MSGC__V_EURATL DIM_Wavelength=62 Or
LAYER=SA_OBS DIM_Platform=MSGC DIM_NativeDomain=V_EURATL DIM_Wavelength=62
Parameter names, units, … for DIMENSION usage– We need controlled vocabularies
E.g : DIM_Wavelength=62, 73, 108 , DIM_AccumulationTime=5, 15, 60 …DIM_BaseTime , DIM_Quality, DIM_PixelTimeStamp
The granularity of the layers will impact : – The efficiency of catalogue searching – The size of GetCapabilities response and performances
14
Issue : Keywords & Titles
Issue : Finding the layer name to perform GetMap request …– Layer names are quite complicated (internal identifiers), but unique :
• Ex : “RD_CPO_P__OBS__RD_ETRANGER__V_PIEMONT__REFLECTIVITE”
– About 250 layers in GetCapabilities– At the moment “Title” == “Name” (that is bad !)
• difficult to be interpreted by a human… Title should be explicit
– No abstracts, no keywords To help catalogue searching, we are looking for keywords, “Controlled
vocabularies”, “Ontology” …– E.g “METEOSAT” “MSG”, “METEOSAT-8” “METEOSAT-9”, …
Keywords should be hierarchic– E.g “TEMPERATURE -- CLOUD -- TOP”
Catalogue searching engine tested : GeoNetwork Possible keywords references :
– Global Change Master Directory (GCMD)– Common WMO code tables (Table C13, …)– AMS Glossary of Meteorology (GMET)– EUMETSAT , HMA (for satellite data) ?– Others ? To be defined ?
15
GeoNetwork evaluation as BDI catalogue
16
ISO 19128 – ISO 19115 relationship
ISO 19128 (WMS) Metadata element
property ISO 19115
Title CI_Citation.title
Abstract MD_DataIdentification.abstract
Keyword element in Keyword list MD_TopicCategoryCode
MD_Keywords
EX_GeographicBoundingBox EX_GeographicBoundingBox
BoundingBox EX_BoundingPolygon
DataURL MD_metadata.dataSetURI
Attribution CI_ResponsibleParty
Identifier and AuthorityURL CI_Citation.identifier
17
Issue : ISO 19128 – ISO 19115 relationship
How to keep consistency between WMS GetCapabilities (ISO 19128) and ISO19115 Metadata common fields ?
Where is the metadata reference or entry point ?– SOPRANO ISO 19115 metadata database
• To be created …• Partially harvested from VGISC metadata (probably not sufficient for BDI
because focussed on GTS products)
– BDImage (or other databases) descriptive schema content • Incomplete (keywords, abstracts are missing …)• Not always consistent (because of the database history…)• Difficult to change (because of backward compatibility)
18
Issue : Time
Time information– It would be useful to include product instance availability information
(ISO 19108) in GetCapabilities response :• Potential availability (time range plus step)• Real time availability (real time list : may be complex and CPU time –
consuming
– There is a TIME parameter in GetMap, but no dedicated tags for TIME information in GetCapabilities response WMS 1.1.1 and 1.3
– GetMap : no support for multi-dimensional time (forecasted imagery)e.g. : Forecasted precipitation intensity = f ( Base time, Validity Time )
– TIME semantics may be ambiguouse.g : Sampling Time, Validity Time, Result Time, Base Time …
Possible approach :– Using “Dimension” / “ExtendedCapabilities” ?
19
Issue : CRS identifiers
There are three methods to identify and describe CRS Namespaces & codes
– We need CRS identifiers not yet define in EPSG database and / or AUTO namespace, others …
• EPSG (not parametric)
– Lot of codes to be defined ( all CRS in use at Meteo-France ? )
• AUTO2 (parametric) :
– Universal Transverse Mercator AUTO2:42001
– Transverse Mercator AUTO2:42002
– Ortographic AUTO2:40003
– Equirectangular AUTO2:40004
– Mollweide AUTO2:40005
• Is it possible to define other codes ?
– Ex Polar Stereographic for each prim meridian, “Space View”, .. ?
Reference (URL) to ISO 19111 documents Proj4 “input parameters – like” ?
20
Issue : Performances
It takes about 1 seconds to process full disk MSGC image “on the fly”– Reprojection appears to be very expensive
– ”On the fly” processing probably not possible if many concurrent requests.
We probably need pre-defined CRS– This would allow pre-calculation of the projection matrix for each CRS
transformation (in TransImage)
– WMTS implementation ? But we’ll have to pre-compute tiles for each layer, CRS, Zoom Level, Format, Time …
– WMTS implementation is obviously a challenge for real-time Met data.
21
Left to do …
Lot of work on Metadata– Keywords, abstracts, ….– Link GetCapabilities response – ISO 19115 metadata
New output CRS New output formats (does GeoTiff make sense in a WMS ?) Support for multi dimensional times Support for metadata layers (pixel time stamp, pixel quality, … ?) Support for client style (SLD=) Improve performances
Limited set of CRS, projection matrix in TransImage component , WMTS Proj4 – GDAL migration of TransImage ( GEDAL is already in use on
SYNERGIE) TransImage as service Security topics to be investigated
22
Conclusion : BDI WMS works but …
To get really interoperable WMS, we need to agree on :
Metadata implementation rules (at least keywords) Colour Maps / Units of Measure
– E.g : Radar Data : Reflectivity (DBZ) – Intensity ( Kg m-2 / h)
CRS policies (EPSG, AUTO2, …) How to manage multiple time dimensions Layer granularity for each product type ( balance LAYER – layer
DIMENSION ) Parameter names, units for DIMENSION usage
Thank you for your attentionquestions ?