CONNECT. TRANSFORM. AUTOMATE.

Production of a midscale road reference network with FME Bruno De Lat GIM Belgium GIS Project Manager

2

Who am I? Bruno De Lat GIM FME Certified Professionel FME Certified Trainer

GIS Project Manager

VAR partner (Gold Value) Belgium 40 employees

GIS GEO-ICT Earth Observation Geomarketing

3

Presentation

1. MRB Road project 1. Goal 2.  2 data sources 3.  Simplified Data Specs

2.  Solutions 1.  FME production 2.  FME validation 3. manual corrections

4

Presentation

1.  MRB Road project 1.  Goal 2.  2 data sources 3.  Simplified Data Specs

2.  Solutions 1.  FME production 2.  FME validation 3. manual corrections

5

Goal MRB roads

!  Create new authentic road network !  Flanders + Brussels + Buffer 50 km

6

Presentation

1.  MRB Road project 1. Goal 2.  2 data sources 3.  Simplified Data Specs

2.  Solutions 1.  FME production 2.  FME validation 3. manual corrections

7

2 data sources GRB (red) ITGI (yellow)

Owner AGIV NGI

Available layers Wegverbinding (wvb) Wegknopen (wkn)

RO_RoadSegmentWithStreetname RO_PathSegment RO_DirtRoadSegment

Coverage Flanders Belgium

Content Street axis Streets (cars) Bike paths Walking paths

Attributes Streetcode CRAB Paved/unpaved Road type

Road Width Paved/unpaved Number of lanes Road Type 2-lane road Street name

8

Presentation

1.  MRB Road project 1. Goal 2.  2 data sources 3.  Simplified Data Specs

2.  Solutions 1.  FME production 2.  FME validation 3. manual corrections

9

Simplified Data specs

2 data sources:

ITGI

GRB

Geometry General rule: priority for yellow

10

Simplified Data specs

!  Geometry !  Exceptions

!  Keep red if yellow is not present (bikes, walking paths)

!  Keep red if red is 2-lane-structure

11

Simplified Data specs

!  Attributes !  Road type: yellow or red !  Street code: yellow (or red) !  Number of lanes: red !  Paved/unpaved: yellow or red !  Road width: red !  Original id’s: red and yellow

12

Dynamic segmentation

!  Some attributes are stored with from to positions.

© AGIV

13

Simplified dataspecs

!  Brunnels !  Bridges and tunnels !  Lower segment/upper segment

14

Presentation

1. MRB Road project 1. Goal 2.  2 data sources 3.  Simplified Data Specs

2.   Solutions 1.  FME production 2.  FME validation 3. manual corrections

FME production steps

15

yellow

red

(1)  Preprocessing: Buffers and Brunnels

Buffers

(2) Recuperation

Geometry

yellow_keep

Red_keep

yellow_remove

Red_remove

(4) Create recuperation table

(3) Geometry manipulations (Add, snap, overshoots

nodes…)

Road-nodes

Original Brunnels

Road segments

Recuperationtable

(5) Calculate Attributes

MRB-roads Brunnels

16

(1) preprocessing

yellow

red

(1)  Preprocessing: Buffers and Brunnels

Buffers (2) Recuperation

Geometry

yellow_keep

Red_keep

yellow_remove

Red_remove

(4) Create recuperation table

(3) Geometry manipulations (Add, snap, overshoots

nodes…)

Road-nodes

Original Brunnels

Road segments

Recuperationtable

(5) Calculate Attributes

MRB-roads Brunnels

17

!  Buffers: !  1m, 5m, 10m, !  VORONOI-principle !  Around yellow segments

!  Use: !  Red is located inside X

number of yellow buffers

!  Y% of red is inside a yellow buffer

(1) Preprocessing Buffers en Brunnels

18

!  Detect original brunnels:

(1) Preprocessing Buffers en Brunnels

19

(2) Recuperation Geometry

yellow

red

(1)  Preprocessing: Buffers and Brunnels

Buffers (2) Recuperation

Geometry

yellow_keep

Red_keep

yellow_remove

Red_remove

(4) Create recuperation table

(3) Geometry manipulations (Add, snap, overshoots

nodes…)

Road-nodes

Original Brunnels

Road segments

Recuperationtable

(5) Calculate Attributes

MRB-roads Brunnels

20

(2) Recuperation Geometry

!  LINK yellow to red if they represent the same road

!  No corrections, just linking original geometries !  Importance:

!  Determine which geometry will be kept/removed. !  Determine from which original segment the

resulting geometry needs to get attribute-information.

21

(2) Recuperation Geometry 2 data sources:

GRB

ITGI

2 data sources:

GRB

ITGI

2 data sources:

ITGI

GRB

22

(2) Recuperation Geometry 2 data sources:

GRB

ITGI

2 data sources:

GRB

ITGI

Difficulties:

(1)  Different digitalisations of crossings.

(2)  1-0, 0-1, 1-1, 1-n, n-1, n-n

0-1

1-0

1-1

2-1

1-2

23

(2) Recuperation Geometry 2 data sources:

GRB

ITGI

2 data sources:

GRB

ITGI

Solution: Waterfall matching

Start with segments which are easy to match and put them away. Continue with the unmatched records. Match again but with less restrictions… continue..

24

(2) Recuperation Geometry 2 data sources:

GRB

ITGI

2 data sources:

GRB

ITGI

Example: Step 2: 0-1

red segments which are completely outside a 10 m buffer of yellow.

25

(2) Recuperation Geometry 2 data sources:

GRB

ITGI

2 data sources:

GRB

ITGI

Example: Step 4: Parameters 1-1

-  At least 70% red needs to be inside 1 5-m-buffer of yellow

-  Length red is 80%-120% of length yellow

-  Maximum average azimuth difference between yellow and red is 15°

-  Street name must match.

-  Distance between both centroids is maximum 20m

26

(2) Recuperation Geometry 2 data sources:

GRB

ITGI

2 data sources:

GRB

ITGI

Example: Step 6: 1 - n

All red segments completely inside 1m GRB buffer

Sum of segment lengths red is 90%-110% of length yellow

27

(2) Recuperation Geometry 36 steps later…

28

(2) Recuperation Geometry

!  Re-use submodels by creating custom transformers with parameters

29

Waterfall matching approach

!  Advantages: !  Full control !  Re-usable custom transformers

!  Disadvantages: !  Weakest spot of the solution !  Parameters can be successful for some situations,

but can introduce bad matches on others.. (Emperical values)

30

FME production steps

yellow

red

(1)  Preprocessing: Buffers and Brunnels

Buffers (2) Recuperation

Geometry

yellow_keep

Red_keep

yellow_remove

Red_remove

(4) Create recuperation table

(3) Geometry manipulations (Add, snap, overshoots

nodes…)

Road-nodes

Original Brunnels

Road segments

Recuperationtable

(5) Calculate Attributes

MRB-roads Brunnels

31

(3) Geometry manipulations

!  Automated geometrical operations !  Merge segments !  Split segments !  Advanced snapping !  Clean arrow-structures !  Clean zigzag-structures !  Clean Triangle Structures !  Determine node types

32

Clean Arrow Structures

!  Integration 2 lane roads

!  Find sharp angles (<82°) crossed by line with dead endpoint (or beginpoint)

33

Result Geometry

34

FME production steps

yellow

red

(1)  Preprocessing: Buffers and Brunnels

Buffers (2) Recuperation

Geometry

yellow_keep

Red_keep

yellow_remove

Red_remove

(4) Create recuperation table

(3) Geometry manipulations (Add, snap, overshoots

nodes…)

Road-nodes

Original Brunnels

Road segments

Recuperation table

(5) Calculate Attributes

MRB-roads Brunnels

35

!  Recuperation tables: Keep link between original ID’s and final road segments.

!  2 tables !  1 contains result of matching (1-0,0-1,1-1,1-n,n-1,n-n) !  1 contains geometrical corrections

!  1-1 !  n-1 (merge) !  1-n (split)

!  FFS-format: Lists!

(4) Recuperation table

36

FME production steps

yellow

red

(1)  Preprocessing: Buffers and Brunnels

Buffers (2) Recuperation

Geometry

yellow_keep

Red_keep

yellow_remove

Red_remove

(4) Create recuperation table

(3) Geometry manipulations (Add, snap, overshoots

nodes…)

Road-nodes

Original Brunnels

Road segments

Recuperationtable

(5) Calculate Attributes

MRB-roads Brunnels

37

(5) Calculate attributes

!  Example project attributes (INPUT)

38

(5) Calculate Attributes

!  Example project attributes (Result)

39

!  For each final segment

!  Loop over (1 or more) Geometry_id !  Loop over (1 or more) original red segments

!  Calculate from-pos and to-pos (“Project” original geometry on road segment)

!  Use necessary attributes !  Loop over (1 or more) original GRB-segments

!  Calculate from-pos and to-pos !  Use necessary attributes

!  After calculation a cleaning operation is necessary if values are equal.

(5) Calculate attributes

40

(5) Calculate Attributes

!  First solution: Algorithm with a loop inside a loop: !  Works fine when 1 records enters, but can work

incorrectly from the moment more than 1 records come in.

!  Must be executed with workspacerunners to avoid interaction in second loop.

!  Workspace runners slows down the solution (FME needs to start each time)

!  Looking for an alternative…

41

The alternative!

!  Cloner! !  Clone each original segment X

times, with X = number of times it is used by new segment

!  Give ID and proces frompos topos algortihm with group by-options.

!  Performance benefit?

Area size Double loop Cloner

5km x 5km 1h30 1m30

20km x 50 km 3 days 10 minutes

42

Presentation

1. MRB Road project 1. Goal 2.  2 data sources 3.  Simplified Data Specs

2.  Solutions 1.  FME production 2.  FME validation 3. manual corrections

43

Automated Validation

!  51 Validation Rules !  Buffer-shapefiles around potentional errors and

situations !  Manual corrections possible

!  Statistics

Annecy - 20-02-2013 44

Automated Validation

45

Presentation

1. MRB Road project 1. Goal 2.  2 data sources 3.  Simplified Data Specs

2.  Solutions 1.  FME production 2.  FME validation 3.  manual corrections

46

Manual Corrections

!  Not all situations can be corrected automatically

47

FME Conclusions

!  Keep pre-processing steps (buffers + brunnels) separated from different models

!  Avoid a double list, use cloner-solution if possible

Thank You!

!  Questions?

!  For more information: !  Bruno De Lat !  GIM (Belgium)

!  www.gim.be !  bruno.delat@gim.be

CONNECT. TRANSFORM. AUTOMATE.