19/04/2023 1

TELEIOSVirtual Observatory Infrastructure for Earth Observation Data

2nd User Community WorkshopDarmstadt, 10 May 2012

Presenter: Manolis Koubarakis (NKUA)

1

19/04/2023 2

Outline

• Why TELEIOS?

• State of the art in Earth Observation data centers

• Going beyond the state of the art: developing a Virtual Earth Observatory using TELEIOS technologies

• Technical work highlights for M1-M20

• Impact

2

19/04/2023 3

Motivation

3

19/04/2023 4

Motivation (cont’d)

~20 PB

4

Big Data!

19/04/2023 5

Motivation (cont’d)

Damage AssessmentEarthquake Haiti, 2010

Estimated directly affected inhabitants Japan, Honshu 2011

5

19/04/2023 6

TELEIOS: Virtual Observatory Infrastructure for Earth Observation Data

• A STREP project funded under the 5th call of FP7/ICT under the Strategic Objective ICT 2009.4.3 “Intelligent Information Management”.

• Emphasis of this objective was on “scalability” and “systematic empirical testing” of systems for “very large amounts of data” (i.e., big data).

6

19/04/2023 7

Consortium

1. National and Kapodistrian University of Athens

2. Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e. V.

3. Deutsches Zentrum für Luft- und Raumfahrt e.V.

4. Centrum Wiskunde & Informatica

5. National Observatory of Athens

6. Advanced Computer Systems

With support from

7

19/04/2023 8

Catalogue

Processing Chains

PDGS

Archive

State of the Art in EO Data Centers

Raw Data

Users

8

19/04/2023 9

Example

• Can I pose the following query using EOWEB?

Find images taken by the MSG2 satellite on August 25, 2007 which contain fire hotspots in areas which have been classified as forests according to Corine Land Cover, and are located within 2km from an archaeological site in the Peloponnese.

9

19/04/2023 10

Example (cont’d)

10

19/04/2023 11

Example (cont’d)

• Well, only partially. Find images taken by the MSG2

satellite on August 25, 2007 which contain fire hotspots in areas which have been classified as forests according to Corine Land Cover, and are located within 2km from an archaeological site in the Peloponnese.

11

19/04/2023 12

Example (cont’d)

• But why?

• All this information is available in the satellite images and other auxiliary data sources of EO data centers or on the Web.

• However, EO data centers today do not allow:• the mining of satellite image content

and • its integration with other relevant

data sources so the previous query can be answered.

12

19/04/2023 13

Linked Geospatial

Data

Semantic Annotations Ontologies

Knowledge Discovery and Data Mining

Raw Data Ingestion

Archiving

Cataloguing

ProcessingContent

Extraction

GIS Data Derived Products Metadata

WebPortals

RapidMapping

Sci

enti

fic

Dat

abas

e an

d

Sem

anti

c W

eb T

ech

no

log

ies

The TELEIOS Earth Observatory: Concept View

DATA

KNOWL-EDGE

Features

19/04/2023 14

Highlights of M1-M20

• Community forming• User requirements capture• Theory (data models, query languages,

knowledge discovery algorithms, software architecture)

• Development of individual software components

• First version of the TELEIOS infrastructure

14

19/04/2023 15

Community Forming

• Organized the first TELEIOS user community workshop on October 13, 2010 in Frascati (ESA/ESRIN).

• Attended by:−Users−Data providers−Scientists−Technology providers

• Disseminated TELEIOS work program

15

19/04/2023 16

Capturing User Requirements

• We collected detailed requirements for the two TELEIOS use cases:−DLR Use Case:

• A Virtual Observatory for TerraSAR-X data

−NOA Use Case: • Real-time fire monitoring based on continuous

acquisitions of EO images and geospatial data.

16

19/04/2023 17

Community Forming (cont’d)

• We are today in the 2nd TELEIOS user community workshop.

• Being attended by 25 participants (12 outside TELEIOS)

• Goals: Present our achievements for M1-M20. Get your feedback!

17

19/04/2023 18

Technical Contributions

• EO data as multidimensional arrays: the SciQL query language.

• Data vaults.• Implementation in the column store MonetDB.

18

19/04/2023 19

What is SciQL?

• SciQL is an SQL-based query language for scientific applications with arrays as first-class citizens.

• SciQL advantages:• Express low level image

processing and image content analysis in a high-level declarative language.

• Uniform access to image data (arrays) and metadata (tables) for knowledge discovery.

19

19/04/2023 20

Why SciQL?

void firesClassify(double *t039, double *t108, int rows, int columns, int *fire) { int win, win_2, r, c, i; double mean039, std039, mean108, std108; double thr[] = {0.0,310.0,2.5,8.0,2.0,310.0,4.0,10.0,2.0};   win=3; win_2=win>>1;  for(i=0;i<rows*columns;i++) fire[i]=0;  for(r=win_2;r<rows-win_2;r++) { for(c=win_2;c<columns-win_2;c++) { temperatureMeanAndStd(t039, columns, win, r, c, &mean039, &std039); temperatureMeanAndStd(t108, columns, win, r, c, &mean108, &std108);  /* Fire Test */ if (t039[r*columns+c]>thr[5] && std039>thr[6] && t039[r*columns+c]-t108[r*columns+c]>thr[7] && std108<thr[8]) fire[r*columns+c]=2;  /* Potential Fire Test */ if (fire[r*columns+c]==0 && t039[r*columns+c]>thr[1] && std039>thr[2] && t039[r*columns+c]-t108[r*columns+c]>thr[3] && std108<thr[4]) fire[r*columns+c]=1; } }}

Fire Classification in C

19/04/2023 21

Why SciQL? (cont’d)

void temperatureMeanAndStd(double *T, int columns, int windowSize, int r, int c, double *mean, double *std) { double s, ss,d; int i,j; int win_2, win2;  win_2=windowSize>>1; win2=windowSize*windowSize;   s=ss=0.0; for (i=0;i<win;i++) { for (j=0;j<win;j++) { d=T[(r-win_2+i)*columns+(c-win_2+j)]; s+=d; ss+=d*d; } }  *mean=s/win2; *std=sqrt(ss/win2-(*mean)*(*mean));}

Fire Classification in C

19/04/2023 22

Why SciQL? (cont’d)

CREATE ARRAY hrit_T039_image_array

(x INTEGER DIMENSION, y INTEGER DIMENSION, v FLOAT);

CREATE ARRAY hrit_T108_image_array

(x INTEGER DIMENSION, y INTEGER DIMENSION, v FLOAT);

22

19/04/2023 23

Why SciQL? (cont’d)

SELECT [x], [y],

CASE

WHEN v039 > 310 AND v039 - v108 > 10 AND v039_std_dev > 4 AND v108_std_dev < 2

THEN 2

WHEN v039 > 310 AND v039 - v108 > 8 AND v039_std_dev > 2.5 AND v108_std_dev < 2

THEN 1

ELSE 0

END AS confidence

FROM ( SELECT [x], [y], v039, v108,

SQRT( v039_sqr_mean - v039_mean * v039_mean ) AS v039_std_dev,

SQRT( v108_sqr_mean - v108_mean * v108_mean ) AS v108_std_dev

FROM ( SELECT [x], [y], v039, v108, AVG( v039 ) AS v039_mean,

AVG( v039 * v039 ) AS v039_sqr_mean, AVG( v108 ) AS v018_mean,

AVG( v108 * v108 ) AS v108_sqr_mean

FROM ( SELECT [T039.x], [T039.y], T039.v AS v039, T108.v AS v108

FROM hrit_T039_image_array AS T039 JOIN hrit_T108_image_array AS T108

ON T039.x = T108.x AND T039.y = T108.y

) AS image_array

GROUP BY image_array[x-1:x+2][y-1:y+2]

) AS tmp1;

) AS tmp2

23

19/04/2023 24

What is a data vault?

• The data vault framework addresses the problem of ingesting scientific data of various formats into tables or arrays.

• Data vault advantages:• Make the DBMS “aware” of

external file formats.• Files are loaded into tables/arrays

only when needed.

24

19/04/2023 25

Technical Contributions (cont’d)

• A framework for knowledge discovery from EO data, metadata and relevant geospatial data sets.

25

19/04/2023 26

Search Engines for TerraSAR-X data

• Implemented TerraSAR-X image search engines based on the developed knowledge discovery framework:−Based on semantic annotations representing the

classes that characterize the image content, and semantics-based queries being used to access this characterizations.

−Content-based image retrieval using innovative image similarity measures

26

19/04/2023 27

Technical Contributions (cont’d)

• Semantic annotations capturing the knowledge discovered: the model stRDF, the query language stSPARQL and their implementation in the system Strabon.

27

19/04/2023 28

What are stRDF and stSPARQL?

• stRDF is an extension of the W3C standard RDF for the representation of geospatial data that may change over time.

• stSPARQL is the query language for stRDF.

28

19/04/2023 29

Why stSPARQL? (cont’d)

29

SELECT ?IM

WHERE

{ ?H rdf:type noa:Hotspot . ?H strdf:hasGeometry ?HGEO .

?H noa:isDerivedFromFile ?SHP . ?SHP rdf:type noa:Shapefile .

?SHP noa:isDerivedFromFile ?RD . ?RD rdf:type noa:RawData .

?RD noa:hasAcquisitionDateTime ?T . ?RD noa:hasFilename ?IM .

?RD noa:isDerivedFromSatellite "METEOSAT9" .

?R rdf:type clc:Region . ?R clc:hasLandUse ?F .

?F rdfs:subClassOf clc:Forests . ?R strdf:hasGeometry ?RGEO .

dbpedia:Peloponnese strdf:hasGeometry ?PGEO .

?SITE rdf:type dbpedia:Archaeological_sites .

?SITE strdf:hasGeometry ?SGEO .

FILTER( strdf:anyInteract(?RGEO,?HGEO) &&

strdf:distance(?RGEO,?SGEO) < 2 &&

strdf:contains(?PGEO,?SGEO) &&

"2007-08-25T00:00:00" <= str(?T) &&

str(?T) <= "2007-08-25T23:59:59") }

19/04/2023 30

Technical Contributions (cont’d)

• Helping users to formulate stSPARQL queries: a visual query builder.

30

19/04/2023 31

The TELEIOS Architecture and Infrastructure

• The TELEIOS software architecture

• Integrated system for the NOA use case featuring the NOA fire monitoring service implemented using MonetDB and Strabon.

• Individual components for DLR use case including the image search engines.

31

19/04/2023 32

Impact

• Important societal and industrial impact

• ICT, Space, Environment and Security research

• European programs in Earth Observation

− GMES (EO data management, Emergency Response Core Service)

− ESA (KIM, KEO, KLAUS, EOLib etc.)

• Scientific Data Management

• Geospatial Semantic Web and Linked Geospatial Data

• Open Source Intelligence

32

19/04/2023 33

Welcome again and thank you for your attention!

Questions?

33