scandinavian edition - dataflow.center · more than 50 raster formats to choose from...

44
Scandinavian Edition

Upload: vuhanh

Post on 15-Jul-2019

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Scandinavian Edition

Page 2: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial
Page 3: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial
Page 4: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial
Page 5: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial
Page 6: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Goals for These Two Days

Foster dialogue and feedback

Build FME skills

Share best practices

Page 7: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Interact with Us on Twitter

Tweet about these days using “#fmeodyssey”

@SafeSoftware

@FMEDoctors

@FMEEvangelist

@FMEBetaBuilder

Page 8: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Top FME Recipes: Demos & Best Practices from the Safe Kitchen

Dale Lutz

VP Development

Page 9: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Recipe #1Infuse your GIS with Essence of CAD

Page 10: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

CAD & GIS

What’s important to CAD users?

Visual clarity / Editing tools Symbology

Labels & white space

Dimensions

What drives the GIS?

Data structure / Data model Consistency

Attribution / enumerated values

Location

Connectivity

Analysis

Example: GIS as-built CAD Design GIS Proposed

What CAD Formats?

• AutoCAD DWG

• AutoCAD Map 3D

• AutoCAD Civil 3D

• MicroStation

• PLS-CADD (XML)

• …

Page 11: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

GIS to CAD

GIS CAD key points

Create AutoCAD blocks / MicroStation Cells

Preserve attributes as Map 3D object data

Translate the Smallworld annotation

Schema Transformation (i.e. National CAD Standard, etc.)

Page 12: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

CAD to GIS

CAD GIS

Preserve attribution

Data Improvement

Create connectivity (snapping)

Recover attribution from text or blocks

Data Quality

Layer / Level consistency

Valid attributes

Page 13: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

CAD to GIS

Pair Labels with Lines based on:

Angle

Distance

Page 14: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Recipe #2: Cookin’ Up Raster With FME

Page 15: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Taste Test #1Translate

More than 50 raster formats to choose from

ArcSDE&GeoDatabase Raster

CADRG

ECW

Geodatabase Raster

GIF / PNG

JPEG / JPEG2000

Oracle Spatial GeoRaster

Page 16: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Taste Test #1 FME Raster Formats List

ADRG

ASRP

Aircom ENTERPRISE

BMP

Bathymetric Attributed Grid (BAG)

CADRG

CDED

DMDF

DTED

ECW

ENVI .hdr RAW Raster

ER Mapper ERS

ESRI .hdr RAW Raster

ESRI ASCII Grid

ESRI ArcGIS Binary Grid

ESRI ArcGIS Image Server

ESRI ArcSDE Raster Catalog

ESRI ArcSDE Raster Map

ESRI Geodatabase Raster

GIF

GeoTIFF

Golden Software Surfer Binary Grid

Hierarchical Data Format 4 (HDF4)

JPEG

JPEG 2000

Landmark Z-MAP

Leica Geosystems ERDAS IMAGINE

Leica Geosystems ERDAS RAW

Maptech BSB Nautical Chart

Marconi PlaNet

MrSID

NETCDF

NITF

Oracle Spatial GeoRaster

PCI Geomatics Database File

PNG

R Statistical Raster (RDATA)

RADARSAT-2 XML

Shuttle Radar Topography Mission Height (SRTM HGT)

TIFF

USGS DEM

Vertical Mapper Grid (NGrid)

Web Map Service

X11 Pixmap (XPM)

Page 17: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Taste Test #2Reproject

Page 18: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Taste Test #3 - Mosaic

Page 19: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Taste Test #4 - Clip

Page 20: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Taste Test #5 - Tile

Page 21: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Taste Test #5a – Web Tiling

Page 22: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Taste Test #6 - Drape

Page 23: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Resources

Dmitri’s Raster Tutorial

http://bit.ly/masterraster

Raster from the basics on up…

Page 24: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Resources

Dmitri’s Raster Studio

http://bit.ly/dmitri_raster

20+ examples, including

Non-rectangular raster clipping

Point clouds

Raster to 3D translation

RasterExpressionEvaluator

Rasterization for Web Mapping Platforms

Raster Palette Manipulations

Avoiding black edges while reprojectiong rasters

Raster Mask for KML output

And more!

Page 25: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Recipe #3Cook’n up some XML

Page 26: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Why is XML so difficult?

XML is really a language

The language describes the data structures as well as the data content

XML folks seem to revel in complex data structures

Hierarchical data structures

GIS systems are usually based on relational data models

FME & XML

You need to tell FME how to interpret the XML data structures.

Page 27: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Reading XML

Options for Reading XML with FME

GML 3.2 & XSD

Most GML can be interpreted if FME can use the XSD

This will work if the OGC spec. is followed and internal feature constructs are not overly complex.

Xfmap

Proprietary script for mapping XML structure to FME features

Xquery

W3C compliant Xquery

Map XML tags to features or attributes using XQueryExploder & XQueryExtractor

27

Page 28: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Writing XML

Options for Writing XML with FME

GML standard formats

FME supports a wide range of GML profiles (GML SF0, OS MasterMap, etc.)

GML 3.2 & XSLT

Apply an XSLT (Extensible Stylesheet LanguageTransformations) to the output

Xquery

W3C compliant Xquery

Map FME features to XML templates using XMLTemplater transformer

28

Page 29: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

XML Transformers

Reading

XQueryExploder

Split XML documents into manageable “features”

XQueryExtractor

Extract attributes and other structures

Writing

XMLTemplator

Convert FME attributes to XML

Transformation

XSLTProcessor

29

Page 30: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Example: XML Object hierarchy

Creating an electricsubstation XML:

Coordinates

and

Attributes

and contains

Feeder(s) contain

Attributes

XML Document

Substation

Feeder A

Feeder B

Page 31: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Example

Sample XML: Source of XML

templates

Add Xquery: replace attribute values with Xquery statements

Sample XML <substation objectID="Vancouver Substation">

<mapLocation>

<coord>

<X>669170.169</X>

<Y>1000199.534</Y>

</coord>

</mapLocation>

<facilityID>5E67B377</facilityID>

<sectionID>RPG123</sectionID>

<phaseCode>ABC</phaseCode>

<feederList>

<feederObject objectID="8-64">

<feederName>10-24</feederName>

<feederNo>8763</feederNo>

<eaLoc>9D2D</eaLoc>

</feederObject>

<feederObject objectID="8-65">

<feederName>10-25</feederName>

<feederNo>8766</feederNo>

<eaLoc>9D3F</eaLoc>

</feederObject>

</feederList>

<name>Vancouver Central</name>

</substation>

<feederObject objectID="{fme:get-attribute("objectID")}">

<feederName>{fme:get-attribute("feederName")}</feederName>

<feederNo>{fme:get-attribute("feederNo")}</feederNo>

<eaLoc>{fme:get-attribute("eaLoc")}</eaLoc>

</feederObject>

Page 32: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Recipe #4Add a Dash of Update to Your Database

Page 33: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Dump and Reload Tables (Or Whole Databases)

or

Update Individual Rows

Database Update Options

Page 34: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Use fme_db_operation format attribute

Feature Level Updates

Page 35: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

FME 2010:

Joiner

SQLExecutor

ArcSDEQuerier

OracleQuerier

FME 2011 adds:

FeatureReader

SQLQuerier

SQLCreator

Database Transformers

Page 36: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Database Update Example

Update Address Points for selected Zip Code boundaries

Read the Zip Codes

Spatial query for the Address Point updates (based on the zip code boundary)

Create the fme_db_operation attribute

Write to database with insert | update | delete

Page 37: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Recipe #5Schema Mapper!

Page 38: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Schema Mapper

Move the intelligence of attribute and feature type mapping out of the workspace

Very powerful in combination with Dynamic Workspaces

Makes workspaces easier to maintain!

Page 39: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Schema Mapper (Input)

nam,add,pho

Ulf,Staffanstorp,2743-028 244

Peter,Malmo,0792-234 098

Eddie,Abyggeby,0775-700 500

Page 40: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Schema Mapper (Mappings)

old,new,dataType,tableNameAttr,outputTable,geometry

nam,Name,fme_char(40),fme_feature_type,Addresses,fme_no_geom

add,Address,fme_char(80),fme_feature_type,Addresses,fme_no_geom

pho,Phone,fme_char(150),fme_feature_type,Addresses,fme_no_geom

Page 41: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Schema Mapper (Output)

Name,Address,Phone

Ulf,Staffanstorp,2743-028 244

Peter,Malmo,0792-234 098

Eddie,Abyggeby,0775-700 500

Page 42: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Recipe #6HAVE FUN!!!!

Page 43: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Questions???

Please visit the FME Support Pantry & Dr’s Office

Page 44: Scandinavian Edition - dataflow.center · More than 50 raster formats to choose from ArcSDE&GeoDatabase Raster CADRG ECW Geodatabase Raster GIF / PNG JPEG / JPEG2000 Oracle Spatial

Thank You!

Don’t hesitate to drop me suggestions, comments, ideas, opportunities, etc:

Email: [email protected]

@DaleAtSafe

http://fmepedia.com

http://www.safe.com

http://blog.safe.com