kes: knowledge enabled services for better eo information use
TRANSCRIPT
KES: Knowledge Enabled Services for better EO Information Use
Andrea ColapicchioniAdvanced Computer Systems
Space [email protected]
IGARSS 2004 - Image Information Mining
The problem
n During the last decades, the satellite image catalogues have stored huge quantity of data
n State of the art catalogues permit only to specify location, time of interest, metadata like platform, sensor, acquisition mode…
IGARSS 2004 - Image Information Mining
The interpretation task
n The interpretation of EO images requiresn Fusion of data/information for better
understanding of structuresn Aggregation with existing knowledge
specific to the application fields (at higher level)
IGARSS 2004 - Image Information Mining
A little bit of history..
n 1996-2000 IIM (Image Information Mining) (http://isis.dlr.de/mining)
n 2001: KIM (Knowledge Information Mining) (http://www.acsys.it:8080/kim)
n 2002: KES (Knowledge Enabled Services)n 2003: KIMV (KIM Validation) n 2004: KEO (Knowledge-centred Earth Observation)
1996 2006
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006
1996 - 2000IIM (DLR, ETH)
2001 - 2002KIM (ACS, DLR, ETH)
2002 - 2004KES (DLR, ACS)
2004 - 2006KEO (ACS, DLR, GTD, CNES)
2003 - 2004KIMV (ACS, DLR)
IGARSS 2004 - Image Information Mining
KIM: Knowledge Driven Information Mining
Images
PrimitiveFeatureExtraction(at pixel level)
010010101101110010
010010101101110010
PrimitiveFeatures:- Texture- Colour- Shape
IGARSS 2004 - Image Information Mining
More in detail
IGARSS 2004 - Image Information Mining
KIM Interactive learning
IGARSS 2004 - Image Information Mining
From data to semantic
Images PrimitiveFeatureExtraction
010010101101110010
010010101101110010
PrimitiveFeatures:- Texture- Colour- Shape
InteractiveLearning
River
Forest
Cloud
Features
DATA SEMANTIC
IGARSS 2004 - Image Information Mining
From KIM to KES (Knowledge Enabled Services)
n Image interpretation is not a simple task. Each user needs a set of accessory data, as for example GIS layers or texts obtained through Internet.
n Yet, the amount of available information makes searches a demanding and expensive task.
n An environment where images are at the focal point, and where each user can navigate through a taxonomically structured knowledge, could be of extreme value.
IGARSS 2004 - Image Information Mining
Other semantic
ImagesPrimitiveFeature
extraction
PrimitiveFeatures
Image Features
Text
GISLayers
InteractiveLearning
Interaction
Interaction
Text Features
GIS Features
DATA SEMANTIC
IGARSS 2004 - Image Information Mining
KES: New interface
IGARSS 2004 - Image Information Mining
Semantic Grouping
n In KIM it is simple to define a feature by identifying a river through positive and negative examples.
n However, the “river" might become part of a wider concept (e.g.: water). This should be implemented without retraining the system for all possible water types.
IGARSS 2004 - Image Information Mining
Aggregated Featuresn In KES a new kind of grouping, called
“aggregated features”, has been introduced.n In a similar way as in defining positive and
negative examples on the image, it will be possible to define the concept “water” as:
n Positive examples: n sea + river + lake + water reservoir
n Negative examples: n streets + houses + mountains
IGARSS 2004 - Image Information Mining
Semantic Grouping
ImagesPrimitiveFeature
extraction
PrimitiveFeatures
Image Features
Text
GISLayers
Interaction
Text Features
GIS Features
DATA SEMANTIC
SemanticGrouping
AggregatedImage Features
InteractiveLearning
Interaction
ObjectDescriptor
Descriptive
GroupingsDescriptive
Groupings
DescriptiveGroupings
DescriptiveGroupings
IGARSS 2004 - Image Information Mining
Ontology
n Ontology is the specification of a conceptualisation. It can be related to a system (System Ontology, which can be domain independent and reused for different domains) or to a domain (Domain Ontology, specific for that domain).
IGARSS 2004 - Image Information Mining
Domain Ontology
n Domain ontology is the set of definitions and concepts pertaining and belonging to a specific domain (and shared by concerned people).
n Different domains have generally different ontologies. As an example, a climatology expert could have a different vision of (and terms to describe) water compared to that of an oceanography expert.
IGARSS 2004 - Image Information Mining
TaxonomyDomain 1 Domain n
…
Sub domain Sub domain Sub domain
AggregatedImage Feature
ImageFeature
Image Feature
AggregatedImage Feature
Image Feature
Image Image Image Image
TextFeature
GISFeature
Text TextGISLayer
GISLayer
IGARSS 2004 - Image Information Mining
DescriptiveGroupings
Data / Semantic / Ontology
Images PrimitiveFeatures
Image Features
Text
GISLayers Interaction
Text Features
GIS Features
DATA SEMANTIC
SemanticGrouping
AggregatedImage
Features
Interaction
ObjectDescriptor
Descriptive
Groupings
DescriptiveGroupings
DescriptiveGroupings
PrimitiveFeature
extraction
InteractiveLearning
Domains
DOMAINONTOLOGY
ONTOLOGY
IGARSS 2004 - Image Information Mining
Implicit and explicit knowledge transfer
n The KIM prototype permits to associate weighted combinations of primitive features to image features. While defining features, the user explicitly transfers knowledge to the system, which is stored and made available to the same or other users.
n In addition there is a further type of knowledge (implicit) that could be discovered: if the user domain of interest is known, by observing the user interactions with the system during search and browsing, it is possible to infer the data of likely user interest and link it with the pertinence domain.
IGARSS 2004 - Image Information Mining
Knowledge transfer
Lake
Sea
Roads
ERSImage
LandsatImage Meris
Image AlgaeBlooming
ClimateChanges
Cities
Temperature
CensusMap
IKONOSImage Roads
UrbanAreas
Factories
PoliticalFacts
IGARSS 2004 - Image Information Mining
Knowledge Graph
www.a.com
MERISImage
www.a.com
Spectral(feature)
www.a.comIkonosImage
WaterGIS
Water
River
RiverGIS
www.b.com
Hydrology
IGARSS 2004 - Image Information Mining
Knowledge Discovery
Exploring this graph means discovering the user's knowledge
KIMV (KIM Validation)
A practical case study:Automatic cloud classification in
MERIS images
IGARSS 2004 - Image Information Mining
Cloud Cover Characterization
MERIS Level 1 – ReducedResolution
“cloud”
KIMV System
IGARSS 2004 - Image Information Mining
Cloud cover characterization
Map ofCloudobject
IGARSS 2004 - Image Information Mining
MAP Object Extraction
Binarized, closedMAP of “cloud” label
IGARSS 2004 - Image Information Mining
Tiling
(x1, y1)-(x2,y2)…2012345
(x1,y1)-(x2,y2)…. 6012345
ShapeVoteIMG
IGARSS 2004 - Image Information Mining
MASS
IGARSS 2004 - Image Information Mining
Linux Cluster
n Cluster nodes: 10 Dell 1750, dual Xeon 3 GHz
n Database and Application Server: Dell 6600, 4 Xeon 2.8 GHz
RACK 01
Power Vault(14x146 GB)
PowerEdge 1750 (x 5)
PowerEdge 2650
6 6 0 F
1
2
3
4
5
6
1
2
3
4
5
6
PowerVault
3 6 G B
1 0 K rpm F C
36 GB10 K rp m FC 36 GB10 K rp m FC 3 6GB1 0K rp m FC 3 6GB1 0K rp m FC 3 6 GB1 0K rp m F C 3 6 GB1 0K r pm F C 3 6 GB10 K r pm F C 36 GB10 K r pm F C 36 GB10 K rpm F C 36 GB10 K rp m FC 3 6GB1 0K rp m FC 3 6GB1 0K rp m FC 3 6GB1 0K rp m FC
7
121
6
19 241 3 18
3C17203 Super Stack 3
1 2 3 4 5 6 7 8 9 1 0 11 12
1 31 4 15 16 17 18 19 20 2 12 2 23 24
1 2 3 4 5 6 7 8 9 1 0 11 12
1 31 4 15 16 17 18 19 20 2 12 2 23 24
Status
P a c k e t
Status
P a c k e t
Packet - Green = Full Duplex
Green = 100MbpsYellow = 10MbpsStatus -
Yellow = Half Duplex
on = enabled, l ink okflashing = disabled
42
3
1
8
6
7
5Module 1 Module 2
P o w e r / S e l f T e s t
Switch 4400
Giga Switch
17 50P o we rE dg e
X EO N
17 50P ow e rE dg e
XE ON
17 50P ow e rE dg e
XE ON
17 50P ow e rE dg e
XE ON
KVM Switch
Rackable Monitor & Keyboard (1U)
IGARSS 2004 - Image Information Mining
System Context
KIMSYSTEM
Internet
ParallelIngestion Chain
Expert userDirect Connection
MASSTOOLBOX
RDBMS
KIM Server
Internet
Ingestion chain monitor
Direct ConnectionJDBC
WEB SERVICE
Ingestiondaemon
WorkflowEngine
User Client
Internet
MERISARCHIVE FACILITY
MASSPORTAL
IGARSS 2004 - Image Information Mining
KIM prototype
n http://www.acsys.it:8080/kim
n Andrea Colapicchionin [email protected]
Visit the ACS stand for a demo