overcoming poor data quality amidst a crisis
Post on 16-Nov-2014
581 Views
Preview:
DESCRIPTION
TRANSCRIPT
March 2008
Overcoming Poor Data Quality Amidst a Crisisv
Ulf MånssonSWECO
Combined expertise in consulting engineering, environmental technology and architecture
So where does ULF constantlyinform (nag) about FME?
SWECO
4
The Nordic region’s leading provider of consulting services in the fields of engineering, environmental technology and architecture
4,900 employees in 10 countries
3.9 billion SEK- annual net sales
24,000 assignments each year
7,000 clients
Present Establishments
Sweden:2.560
Norway: 600
Denmark: 40
Finland:1.000
Estonia: 225
Lithuania: 175
Employees 20074.900
Russia: 110
Bulgaria
Czech rep: 270Slovakia
Assignments Worldwide
Assignments in more than 100 countries since 1903
Increased international presence
Ongoing assignments in 75 countries
20% –
15% –
15% –
14% –
12% –
9% –
8% –
5% –
2% –
Industrial Engineering
Water and Environment
Transport and Civil Eng.
Energy Systems
Building Service Systems
Architecture
Structural Engineering
Project Management
Geographic Positioning
Ulf Månsson
Modern organizations depend more on data quality than ever before.
Only a few spare parts in stock
Large centralized districs, no local offices
No extra staff
Modern Organizations
Ulf Månsson
Modern Organizations
Organizations rely on systems and their data. Geographic data are crucial:
No local offices Poor knowledge of local geography
Where are my supplies/spare parts?
Where are my people? How can they find me?
Ulf Månsson
C R I S I S
The importance of data quality becomes strikingly clear in a crisis management process.
FME is an important tool for managing ”ad-hoc” situations at a major crisis.
The hurricane Gudrun is the best Swedish example.
”Spontaneous ETL”
Instantly connect data from many systems to improve quality and derive the needed information output.
Hurricane Gudrun 2005
Wind speed 10-46 m/s 20 000 km electrical cables damaged 2 500 people in workforce Out of supplies, gas, equipment,etc.
The – first map illustrating the situation. (2D-Pointreplacer)
Power outages lasted for weeks We used FME for weeks
Services Provided for Our Client
Information to the crisis management teams Rescue workers and municipalities Daily delivery of maps and lists of persons/addresses
Data quality
Prognoses for re-connections
Providing maps for field-work
Maps for reconstruction
”Spontaneous ETL”
Large power company – many IT systems Outage management systems Network schema systems Network Information Systems (NIS) Customer Information Systems (CIS)
Oracle Text files Excel Shape files Etc…
FME El Dorado !!!
Challenge: Making a MapShowing Areas with Outages
Connecting the Parts
OutageManagement
System (Oracle)
Customer IDOutage data
NetworkInformation
System(Oracle Spatial)
Customer IDX,Y
Customer Data(SAP->Txt files)
Customer IDAddresses, Zip-
Codes etc.
XLSLists of addresses
Bufferedareas with
outage data
Shp filesGIF files
FMEWorkbenches
Key FME Features
The diversity of readers and writers GIF, XLS, Shape, etc… (today we would have used pdf)
Joiner-functionality
Buffer and dissolver
Scripting Today, we would have used the fanout-functionality
It all worked gloriously (in theory) but bad data…
The mercyless ConvexHullAccumulator exposes bad quality.
Defining municipality by zip-code from CIS…
Solution Use municipality-
borders to clip data
ConvexHullAccumulator - Good Data
Ugly Data
”You should at least have used the
2DPointReplacer before deployment!”
Connecting the Parts
OutageManagement
System (Oracle)
Customer IDOutage data
NetworkInformation
System(Oracle Spatial)
Customer IDX,Y
Customer Data(SAP->Txt files)
Customer IDAddresses, Zip-
Codes etc.
XLSLists of addresses
Bufferedareas with
outage data
Shp filesGIF files
FMEWorkbenches
”To know what you not know”
Polygons representing reconnection forecasts A simple geometry manageable by each municipality The buffer radius correspond to the data quality Polygons marked as ”We simply don’t know…”
Conclusions
Keep it simple GIF and PDF files are easy to distribute and print.
Spontaneous ETL Know your FME and it’s a great problem-solver in
unexpected situations.
FME Prototyping A great way to define and describe processes. These can
also be implemented in other systems.
SWECO Vision
To be Europe’s most respected knowledge company in consulting engineering, environmental technology and architecture
Recent artwork(or ”unexpected
results”)
More artwork
25
QUESTIONS?
More information:www.sweco.seUlf.Mansson@sweco.se
Thank You!
top related