geobia 08 baatz - university of calgary in...

12 August, 2008

1

Ten Years Object-oriented Image Analysis for Geospatial Applications:

Evolution and Outlook

Dr. Martin BaatzVice President Solutions & Services

Trends in Earth Observation

More, better & cheaper imagery

Increased awareness of GIS community for Earth Observation

Emerging directives and standards

data costs

data availability

1998 2008

Earth Observation Market Evolution

Strong increase in numbers of remote sensing satellites

Earth observation goes commercial -Increase in privately held operations

More and more countries developing own satellite capabilities

New Remote Sensing Satellites 1999 to 2007

0

1

2

3

4

5

6

7

8

2000 2001 2002 2003 2006 2007

Sar Lupe 3

WorldView 1Sar Lupe 1

Sar Lupe 2Kompsat 2IRS P6

Terrasar XResurs DK 1Bilsat 1

COSMO-Skymed 1CloudsatBnscsat 1EOS PM-1

Egyptsat 1Eros B1Orbview 2Spot 5Bird

IRS 2AAlosIcesatEnvisat 1Quickbird 2Eros A1

200720062003200220012000

Technological Evolution

Earth Observation technology has evolved

Increase in spatial resolution – from 30 meters to 60 cm

Increase in temporal resolution – from 14 days to 3 days

More accurate data processing

Higher workflow automation

Better processing power / storage / distribution

Landsat

Ikonos

Data/Details/Value/Complexity

Resources/Productivity

More data and details

continuously or with high update rate

over large areas

on multiple scales

from multiple sources

Need for detailed, up-to-date information as a basis for planning

and decision making in industry, administration and security

Business Drivers in Earth Observation Market Trend: Out of the niche into the GIS mainstream

"NASA's Earth Science Program sensors bringing down terabytes ofinformation every day, but there's no way to ingest and integrate that data on the ground," explained Dangermond. "I want to bring that rich Earth science data to the GIS community," he said.

Jack Dangermond, President of ESRI

Trends:

Out of the niche into the GIS mainstream

Google Earth:

Increasing awareness for Earth Observation into the general public

12 August, 2008

2

Challenges for Remote Sensing Data Analysis

Varying forms of appearance of specific landcover features

Varying imaging conditions and data quality

Complex and varying tasks

Sensor and application specific knowledge required

Semi-automated workflow to be considered

Fully automated workflows to be supported

Government Business L H

H

Productivity

Existing Approaches Do not Solve the Problem

unserved cases

feasible today

Sem

anti

csC

om

ple

xity

Human Mind

Computers

Existing Technology Approach

Existing pixel-based technology cannot deliver necessary sophistication and automation of analysis. (Context, reliability, etc.)

Existing business processes revolve around

low throughput single user

departmental systems with human-intensive focus time-consuming, high cost and not scalable, subjective & inconsistent results

Silos of data which cannot be meaningfully analysed and shared for enterprise/organisational wide purposes.

Why Object Based Image Analysis

Landsat:

Pixel == Object

High Resolution:

Pixel too small to represent meaningful objects

30 meters 30 meters

Context counts

Input image

Context counts

Initial classificationbased on gray values

12 August, 2008

3

Context counts (on the proper Scale)

Shadow

Rectangular gray area

Context counts (on the proper Scale)

Bright concrete betweenbuilding and road

Bright concretebehind building

Bright concrete linear shape

Object Oriented Image Analysis at Definiens:

a brief History

The Start 1995 - 96

1995

think tank founded by Gerd Binnig, Nobel Laureate for Physics

objective: develop software-based methods for handling complex data

the fractal-hierarchical approachmulti-scale systems analysis and representation

1996

task: spatially related simulations

high resoluted airborne data: needed the ‘objects’ for simulations

no way to get the object out of the images

nobody in the team with image analysis or remote sensing background (however cognitive science)

Gerd forbid to read respective technical literature

image analysis is a ‘fractal-hierarchical’problem, lets do it

ourselves

Segmentation & Classification Methods

Segmentation methods

quadtree segmentation

clustering

histogram-based methods

edge detection

region growing

level set methods

graph partitioning methods

watershed transformation

model based segmentation

multi-scale segmentation

Semi-automatic segmentation

neural networks segmentation

…

Classification methods

thresholding

neural networks

maximum likelihood

Bayesian classifier

fuzzy classification

decision tree

clustering

nearest neighbor

discriminant analysis

support vector analysis

adaptive thresholding

kernel methods

…

Overview of OBIA / OOIA Approaches

segmentationprocedure

pixel cluster

Classical Segmentation Techniques

imagepixels

classificationprocedure

classifiedpixels

Pixel & filter based Classification

imagepixels

polygonsof pixel clusters

data & area of limited landcover

classification

segments fromclassified

pixels

12 August, 2008

4

What we were looking for

generic

multi scale / hierarchical

object-oriented, i.e. first objects, then classification, then objects ..

segmentation

adaptable: multi resolution

high quality

classification

transparent

retraceable

trainable / interactive

Publications around Segmentation with Classification

not: Classification with subsequent Segmentation !

1971 Echo Classifier

1976 Pavlidis "Picture segmentation by a tree traversal algorithm – Horovwitz, T

1991 Spatial and spectral classification fo remote-sensing imagery – Franklin, Wilson –

1994 Knowledge-based classification method for crop inventory using high resolution satellite data. M.Sc. thesis – Abkar

1996 On Unsupervised Segmentation of Remotely Sensed Imagery Using Nonlinear Regression – Acton -

1996 Multi-spectral quadtree based image segmentation. Int'l – Gorte -

1996 Supervised segmentation of remotely sensed imagery –Lat -

1998 An Introduction to the Theory of Spatial Object Modelling for GIS – Molenaar -

1999 Definiens eCognition (commercially available)

2007 ENVI / ERDAS (commercially available)

The Start 1995 - 96

1995

Think tank founded by Gerd Binnig, Nobel Laureate for Physics

Objective: develop software-based methods for handling complex data

The fractal-hierarchical approach multi-scale systems analysis and representation

1996

Task: spatially related simulations

high resoluted airborne data: needed the ‘objects’ for simulations

no way to get the object out of the images

image analysis is a ‘fractal-hierarchical’ problem, lets do it ourselves

nobody in the team with image analysis or remote sensing background (however cognitive science)

Gerd forbid to read respective technical literature

First multi scale segmentation approach connected to a fuzzy classification system with a set of attributes.

First version of a generic object-based image analysis image analysis workflow

The extraction of objects of interest is a challenging problem.

Far too complex to be addressable by one button-press

solution that serves all cases

Overview of Image Analysis Approaches


pixel cluster


imagepixels

classifiedpixel cluster


pixel cluster

Object based Image Analysis

imagepixels


data & area of improved

landcoverclassification


classifiedpixels


imagepixels



classification

1997 - 99

1997 Getting better Improving the approach

hierarchical and topological network of objects

context attributes: embedding, contact, structure

fuzzy nearest neighbor classifier

underlying concept: Cognition Network Technology

Many different experimental studies:

simulated annealing

neural networks / Bayesian networks

optimisation strategies

first object-oriented image analysis techniques

First external presentations of technology: DLR, Ministry for Environmental Affairs

Significant improvement of Multiresolution Segmentation into its final form

Hierarchical Network of Objects

pixels

pixel cluster (objects) on different scales

The object oriented approach

represents & processes image information on different scales simultaneously

extends pixel-based methods by systematically processing pixel cluster (objects)

replaces the implicit topology of the pixel raster by an object network

12 August, 2008

5

Multiresolution Segmentation

Deforestation Monitoring from ASTER data. Rondonia, Brazil

Object generation in high quality and in arbitrary resolution to adress structure on different scales

Object-oriented Image Analysis at Definiens is a derivate of the generic

Cognition Network Technology

Getting better 1997 - 99

1997

Improving the approach

hierarchical and topological network of objects

context attributes: embedding, contact, structure

fuzzy nearest neighbor classifier

underlying concept: Cognition Network Technology

Many different experimental studies:

simulated annealing

neural networks / Bayesian networks

optimisation strategies

first object-oriented image analysis techniques

First external presentations of technology: DLR, Ministry for Environmental Affairs

Significant improvement of Multiresolution Segmentation into its final form

1998 -1999

Optimisation of data structures, functionality, performance and usability in many details,rewriting the software

first beta users

many different application studies

first project in the life sciences

efficient data structurefor handling and

processing the dynamic object network is very

challenging but it is a keycomponent

We are on the right path but we are still far away from extracting objects of interest operationally

2000: Going to Market

eCognition

Generic Platform for Object-based Image Analysis

for geospatial applications

Build product: software, documentation, marketing

Community

Enlarge the network of users

First Center of Excellence: ZGIS University Salzburg

further Center of Excellences

Going to market is a lot of work..

2002 Essential Technology Breakthrough

Cognition Network Language (CNL)

Developer Environment

‘the processes’, graphical editor for CNL: modules, loops, conditions

sub-domains, parent-process objects

variables ..

The Object Domain Concept

each single procedure is targeted to a specific sub-set of objects in the network

work locally specifically

supports modularisation, essential simplification of rule set development

sub-domains

Real Object-oriented Image Analysis supported

automatic extraction of objects of interest

Large nuin Life Sciences applications:

Since 2003 marketed for Life Sciences applications

Definiens Developer

Definiens Developer

‘the processes’

develop rule sets

develop applications

combine, modify and calibrate rule sets

process data

execute and monitor analysis

review and edit results

12 August, 2008

6

(local) analysis and evaluation of the network

Classification

Object-oriented Image Analysis: the Generic Procedure

network of objects ofinterest

imagepixels

correct labeling

correct shape

network ofobject

primitives

Segmentation(local) modification

of the network

First Applications in the Life Sciences

The first time we can extract complex objects of interest fully automatically in high

throughput

2004 Scalable IT architecture

Distributed Client-Server Architecture

from stand alone installation to production environment

different functional layers in software architecture

supports a number of different workflows

high throughput through parallel processsing

First internal application of ‘the processes’ to Geospatial Applications

large number of projects and feasability studies

Viewer Analyst Architect Developer

EII Client Services

EII Application Services

EII Production Services

Image Acquisition Data Management

EII Connectivity Services

Informatics

Server 1 - n Server 1 - n Server 1 - n Server 1 - n

Client Services

Application Services

Production Services

Connectivity Services

Life Applications Earth Applications Partner Applications

Scalable Image Analysis Platform

Workflow In New Client – Server Architecture

Parallel ProcessingParallel Processing

Definiens eCognition Servers

Lab 2Lab 2Lab 1Lab 1

ArchitectDeveloper

Processing Storage

Rule set and application development

Expert with CNL*/eCognition knowledge

Adjust classification + manual editing

Image Analyst, no specific eCognition know-how necessary

Segmentation(Local) Modification

of the network

(Local) Analysis and Evaluation of the network

Classification

Input

Image data

Result

Network of Structures of Interest

proper shapeproper classification

Object-oriented Image Analysis of Remote Sensing Data

Image data courtesy Lockheed Martin

12 August, 2008

7

Lockheed Martin: Map Generator for Mobility Data Base

Multiple Data Sources

Landuse based on IKONOSSlope Analysis based on SRTMSoil type using USGS dataWeather layer using METARS

Automated Road Detection & Centerline Generation

automated off-road and on-road navigation

Overview of OBIA / OOIA Approaches

objects ofinterests

(proper shape & classification)


pixel clusterclassification

segmentation

Object oriented Image Analysis

imagepixels

classifiedpixel cluster


pixel cluster

Object based Image Analysis

imagepixels



pixel cluster


imagepixels

data & area of improved

landcoverclassification

data & polygonsof landcover

units of interest


classifiedpixels


imagepixels



classification

2005

Openess

integratability, for usage in third party workflows

software development kit (SDK) and APIs:

add any algorithms and classifiers

data imput / output

Open Framework and Research Tool

providing underlying data structure and functionality

for extensively exploring & applying OBIA methods

Assisted Object Recognition – Ship Detection

Assisted Object Recognition – Ship Detection Objects of Interest

12 August, 2008

8

Definiens Architect

Combine, modify and calibrate applications

‘Developer Light’

Execute and monitor analysis

Review and edit results

Machine learning

Usibility for End User

Architect: easy adaptiblity of librarymodules

Complete workflow support

2006 / 2007

Definiens eCognition Server

Complete Workflow Support

Definiens Architect

Definiens Developer

Develop rule sets Tune & Configure Execute & Review

Submit Sub

mit

Definiensextension to ArcGIS

Definiens Datamanagement for ArcServer ( ArcSDE)

store

Definiens eCognition Product Timeline

eCognition v1

eCognition v4

Definiens Professional v5

eCognition v3

eCognition v2

Definiens Developer v4



Definiens Developer v7 Definiens eCognition Server v7

Definiens eCognition Server v6



1999

2004

2007

2008

eCognition server

eCognition Server – Tiling and Stitching

Tiling

Parallel processing of tiles

Stitching of tiles

eCognition server

eCognition server

eCognition server

Definiens Analyst/Architect

Definiens eCognitionServer

Definiens Analyst/Architect

Original full scene

Result on full scene

The bandwith of details to be handled and the variability of

landcover features set a considerable challenge

2008

Performance

object-based object processing complimented by

pixel- and filter-based object processing

object domain concept applied to pixels and filter procedures

significant performance gain

xD Multidimensional image

support for 2D timelaps, 3D, 3D timelaps data

Feasability studies on

LIDAR pointclouds

seismographic data

12 August, 2008

9

CT Oncology Liver DetectionCT Oncology Liver Detection

Detection and quantification of liver tissue and liver lesionsTask

Conclusion & Outlook

Operational use of OBIA today

Numerous customers rely on Definiens to run operational image analysis

Impervious surface analysis: Australia, Europe, US

Large Scale land cover mapping (5-10m): South Africa, UK, US

Change detection high resolution: Saudi Arabia,

Definiens is fully integratable in 3rd party workflows

Automation API

Software SDK

ESRI Integration (ArcCatalog / ArcSDE / ArcGIS Image Server)

NaturalResource

Infrastructure

Defense Civil

Markets

BasicMap

Landcover

Change

Object

Complexity

Lidar Data Provider

Security

Woolpert

Woolpert

Imagery Programs:

Ohio Statewide 105,000km2

Indiana Statewide 96,000km2

Florida Statewide 150,000km2

30cm & 15cm Resolution

True / False Color IR

LiDAR

Products:

“Statewide” value added datasets

Impervious/Pervious features

Agricultural Use Analysis

NaturalResource

Infrastructure

Defense Civil

Markets

BasicMap

Landcover

Change

Object

Complexity

Satellite Data Provider

Security

CSIRSatellite Application

Center

South African Earth Observation Strategy (SAEOS)

Objectives:

information exchange amongst government departments.

warehouse promoting accessibility of spatial information by all tiers of government

coordinating the collection, assimilation and dissemination of Earth Observation products.

Customers:

Department of Water Affairs and Forestry

Department of Agriculture

Department of Defense

South African Police Service

NaturalResource

Infrastructure

Defense Civil

Markets

BasicMap

Landcover

Change

Object

Complexity


Security

Skog + Landskap

Norsk Institut for Skog og Landskap, Norway

Objectives

Nation wide CORINE classification

Semi-automated land cover classification based on CORINE nomenclature

Replacement of time consuming manual delineation and classification

Optimization of existing workflow

Results

reduction of production time of 60 %

“… delineate and classify a whole Landsat scene manually is about 10 to 12 weeks. With our new production line using Developer and Definiens eCognition Earth Server it takes 4 week. “

12 August, 2008

10

NaturalResource

Infrastructure

Defense Civil

Markets

BasicMap

Landcover

Change

Object

Complexity


Security

Countryside Council for Wales

Countryside Council for Wales

Objectives

Country wide habitat mapping

Replacement of time consuming manual delineation and classification

Optimization of existing workflow

Results

Definiens incorporated as key automation component

„..the project would not have been possible without Definiens...“

Definiens Application Partners

Dendron

Single Tree Inventory Program

Lagen Spatial

Imperious surface maps

Intermap

Basic landcover classification

From Specialist to End User

Google / Microsoft

Opens up access to Earth Observation data to end users

Highlights Earth Observation capabilities and creates new demands

Creates new applications as well as expectations

Image Servers

Standardization of input data

Easier distribution of image data

From Research towards Operation

Operational Earth Observation image analysis

Government Programs

North America NASA / NOAA / USGS / USDA / NGA

Europe GMES

Global Monitoring for Environment and Security joint data providers & user effort to establish operational services

South Africa SAEOS

South African Earth Observation Strategy – centralized data platform for government agencies

Earth Observation Data Providers & Value Adders

Digital Globe / GeoEye / Infoterra / SPOT

AAM Hatch / Woolpert / Pixxures

The Professional Market will grow

Expertise and Knowhow are required

they need to be considered in the academic curricula

Definiens is intensively interacting with the academic community:

Numerous publications

Academic programs

Internships

Center of Excellences

PartnerAcademia

Berkeley

From Academia towards Industry User Community

User Forum

Steady increase in posting activity

Average of 10 new members / month

Rule Set Exchange

Started 2008

Knowledge exchange

New concepts & ideas

http://forum.definiens.com/index.php

0

10

20

30

40

50

2004 2005 2006 2007 2008

12 August, 2008

11

Take-aways

OBIA / OOIA Methods are Generic

There is no inherent or ontological difference in OBIA/OOIA methodology between

geospatial applications,

biomedical applications

applications in material sciences

Even issues in

ground truth

validation

generalization

change detection

are very similar.

thresholding

neural networks

maximum likelihood

Bayesian classifier

fuzzy classification

decision tree

clustering

…

Classifier

procedures for information processing

Increasing

bandwidth of

attributes,

specificity

and depth of

information

Key Components of Image Analysis Procedures

Pixel-based, object-based or network-based methods can be essentially distinguished by the information carrying topological data structures

region growing

region merging

wave let transformation

multiresolutionsegmentation

edge detection

level set methods

graph partitioning

…

Segmentation

procedures for creating & modifying data structure

pixels

pixel cluster ~ objects

network of objects and relations

hierarchical network of objects and relations

variform network of objects and relations

…

Information Carrier

topological data structure

Object-oriented Image Analysis: the Generic Procedure


Don’t discuss only individual segmentation or classification techniques,

discuss the process as a whole.

imagepixels

classification

segmentation

correct label

correct shape

network ofobject

primitives

What is a good segmentation or classification technique ?

Segmentation: an object primitive is as good as it fulfills these two purposes:

a good information carrier for subsequent classification steps

an appropriate building block for further modification/optimization of object shapes

A classification result is as good as it fulfills these two purposes:

being a good reference for subsequent classification steps (domain, relations)

being a good starting point for further modification/optimization of object shapes

The Object Domain

allows to modularize problems along the way

takes away a lot of burden of individual segmentation or classification steps

Landcover Classification

network of objects

primitives

imagepixels

classification

segmentation

correct labeling

objectprimitives 1

objectprimitives 2

objectprimitives 3

correct shape

not relevant

12 August, 2008

12

Key Aspects of OBIA / OOIA


imagepixels

classification

segmentation

Increasing usage of expert knowledge and semantics

Increasing degree of abstraction

knowledge based

context driven

pixels objects networks

geobia 08 baatz - university of calgary in...

Documents