building enterprise applications using open source
TRANSCRIPT
![Page 1: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/1.jpg)
Building enterprise applicationswith open source
Peter Batty Ubisense
@pmbatty
![Page 2: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/2.jpg)
• Brief background on our customers
• Our software stack and how we got there
• Business models
• Simplicity
• Field applications, online and offline
OVERVIEW
![Page 3: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/3.jpg)
Photo by whalt - http://flic.kr/p/7D7rd
Physical Network Inventory 4.1 Product Description
© 2007 General Electric Company All rights reserved Page 15 of 65
groupings built by the system administrators. This ensures that end users only see the sets of equipment relevant to their work, both speeding up data entry and ensuring that the correct equipment is used for network designs.
The population of this catalogue is the first task that must be completed before a Physical Network Inventory installation can be rolled out to end users. To assist in populating the catalogue individual and/or groups of specifications can be imported and exported.
3.3 Strand And Structure
Physical Network Inventory provides support for both overhead and underground strand and structure. Data model entities and tools are provided to allow for the documentation, design and management of a wide variety of ‘holes and poles’.
Favourites can be used to very quickly and accurately design new network. This maybe using a standard combination of entities, for example a combination of trenches and manholes used for a particular customer connection, or individual entities. The relevant network equipment can also be inserted during the same operation greatly increasing end user productivity.
All network entities that are related to a structure, for example splices housed inside underground structures such as manholes or pole-mounted splices are related to the structure and can be accessed from the ‘owning’ structure, making it simple to interrogate the network from the supporting structures.
3.3.1 Underground structures
The following entities can be represented in the underground network:
Underground routes represent the location of the trench, bore or plough which contains (or will contain) the ducting and\or cables of the network. Information about the dimensions of the trench, i.e. depth and width of reinstatement layers, the type of backfill material used and surface material can all be recorded if desired.
Conduits model the structures within the trench. Conduits are used to protect the cables within the trench and to make it easier to add new cables without the need to dig. Conduits are not needed to associate cable to an underground route - as in a plough route where cable is directly buried for example. Conduits may be ‘nested’ to model sub-conduit network constructions. There are no limitations on the number of levels of nesting that can be modelled. Conduits are also used to model micro-tubes as used in blown fibre deployments. Below is a picture of a trench as conduits are being installed and on the right is the cross-sectional view of this trench within the application.
Figure 11: Conduits in an Underground Route or trench
Underground utility boxes are the structures that provide access to the underground network of ducts and conduits, examples of underground utility boxes include manholes, vaults or hand holes. Standard configurations of underground utility boxes can be created and saved for re-use. A typical manhole can be seen below and to its right the butterfly diagram managed in the application. For more details see Figure 19.
For Internal GE Use Only
Customers: utilities and telcos
Photo by ☺ Lee J Haywood - http://flic.kr/p/7iX7f5
![Page 4: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/4.jpg)
Sample cable data
Detailed asset maps from Smallworld
![Page 5: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/5.jpg)
Sample cable data
Detailed asset maps from Smallworld, with live customer status overlaid
![Page 6: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/6.jpg)
Sample cable data
Detailed asset maps from Smallworld, with live customer status overlaid
![Page 7: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/7.jpg)
Sample cable data
Detailed asset maps from Smallworld, with live customer status overlaid
![Page 8: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/8.jpg)
SAMPLE CUSTOMER• Large communications company with 6 million subscribers across
~20 states
• 23 geographic regions created for offline use – from one common master database
• Offline databases vary from 136MB to 22GB
• Incremental updates applied to all offline databases nightly
• 5000 online users, ~400 offline users active so far, growing to 2000
• Weekly features updated ~2.3m total, ~250,000 largest region
![Page 9: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/9.jpg)
Smallworld
![Page 10: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/10.jpg)
flic.kr/p/7rnNAD
flic.kr/p/7NEJzF
Pervasive and SimpleLocation is now
in consumer applications
![Page 11: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/11.jpg)
![Page 12: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/12.jpg)
Software stackVersion 1
![Page 13: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/13.jpg)
Software stackVersion 2
![Page 14: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/14.jpg)
Software stackVersion 3
![Page 15: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/15.jpg)
Functionality Cost Support
Terms PredictabilityFlexibility
![Page 16: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/16.jpg)
BUSINESS MODELS
• Sell support, services
• e.g. Boundless
• Sell hosting
• e.g. CartoDB, MapBox
• Hybrid closed and open source
• Google, Safe, Ubisense
![Page 17: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/17.jpg)
![Page 18: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/18.jpg)
SIMPLICITY
![Page 19: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/19.jpg)
We are building applications for
the 95%of people in an organization who are not GIS users
![Page 20: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/20.jpg)
20
“Deciding what not to do is as important as deciding what to do”
![Page 21: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/21.jpg)
TOPICS• Familiarity
• Panning and zooming
• Search (including queries)
• Minimal layers
• Focused applications
• Network links
• Rather than “trace”
![Page 22: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/22.jpg)
FIELD APPLICATIONSONLINE AND OFFLINE
![Page 23: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/23.jpg)
DEMO OVERVIEWThis presentation features screen shots that step through a demo of Ubisense
myWorld for damage assessment. The demo features two separate users:
Operations center user, with overview of situation, including data from all field users
Damage assessor with laptop in the field, capturing damage assessment data
![Page 24: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/24.jpg)
Operations Center
Operations center overview map – currently all is quiet, no damage shown
![Page 25: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/25.jpg)
Damage Assessor in Field
Currently online (wireless)
WiFi is on
Damage assessment user in the field, currently in online mode (with wireless connection)
![Page 26: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/26.jpg)
Damage Assessor in Field
Create damage pointDamage assessor creates damage assessment point in the field
![Page 27: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/27.jpg)
Damage Assessor in Field
Damage point is created in local database and displayed on map on laptop
![Page 28: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/28.jpg)
Operations Center
Syncs to operations center within ~1 minute, damage point displayed and overview map updated
![Page 29: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/29.jpg)
Damage Assessor in Field
Offline
WiFi is off
Back in the field, we are now in offline mode (wireless connection is lost). Google Map has disappeared here.
![Page 30: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/30.jpg)
Damage Assessor in Field
Now we are displaying OpenStreetMap as an offline map instead of Google (which can’t be used offline)
![Page 31: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/31.jpg)
Damage Assessor in Field
In offline mode, we can continue to display maps, select objects, and search for assets
![Page 32: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/32.jpg)
Damage Assessor in Field
We can also continue to create damage points while in offline mode (which are stored locally)
![Page 33: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/33.jpg)
Damage Assessor in Field
We have now created two more damage points offline
![Page 34: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/34.jpg)
Operations Center
Back in the operations center, we still just see the single damage point at the moment
![Page 35: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/35.jpg)
Damage Assessor in Field
Online
WiFi is on
Now the field laptop has a network connection again and is back online, so the damage data can sync
![Page 36: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/36.jpg)
Operations Center
The data has synced and so we now see three damage points back in the operations center.
![Page 37: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/37.jpg)
Operations Center
We can report on the damage points and get aggregate information
![Page 38: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/38.jpg)
Field users can also use tablets like iPad or Android, which support taking photos.
![Page 39: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/39.jpg)
Take a photo on the iPad
![Page 40: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/40.jpg)
Operations Center
Operations Center also gets immediate visibility of photo (assuming user online)Photos can also be emailed in from any smart phone
![Page 41: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/41.jpg)
OFFLINE REQUIREMENTS
• Cross platform
• Android, iOS, Windows 7 onwards (and web)
• Need ability to sync large datasets (gigabytes) for some situations
• In other situations may be able to use smaller datasets (for single job, like walking an electrical circuit)
![Page 42: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/42.jpg)
DATABASE SYNC
• Sync of bulk GIS data
• Initially read only for short term applications
• Typically want to do nightly over non-cellular connection
• Time critical data like damage assessment
• Want to sync as soon as possible (two way)
• Needs to connect / disconnect / sync transparently
![Page 43: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/43.jpg)
APPROACHES TO MOBILE DEVELOPMENT
![Page 44: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/44.jpg)
THREE ARCHITECTURES TRIED
PostGISMapfish
Offlinestorage
SpatiaLiteJavaScript
Replication Caching
![Page 45: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/45.jpg)
ARCHITECTURE 1 LAPTOP REPLICA
PostGISMapfish
Windows (or Linux or Mac) only
Use identical software stack to main server
Need custom replication - haven’t found PostgreSQL replication for sometimes disconnected model
Pros: Simplest to implementCons: Laptop only, heavyweight
![Page 46: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/46.jpg)
REPLICA CONCEPT
Master database
Local database
Potentially multiple gigabytes (raster tiles)
Incremental update is important
Full or partial (spatial) replica
![Page 47: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/47.jpg)
ARCHITECTURE 2CORDOVA REPLICA• Have our web application running offline
on Android and iOS
• Replaced PostGIS with SQLite - for geometry and attributes, and also tiles
• Wrote JavaScript wrappers to replace (basic) MapFish services offline
• Refactored code so small number of classes contain differences between online and offline - tile and database access
![Page 48: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/48.jpg)
ARCHITECTURE
PostGIS
MapFish (REST API)
SQLite (mbTiles)
Tile server
myWorld clientLeaflet
myWorld server (Python)
SpatiaLite
JavaScript wrappers
SQLite (mbTiles)
Tile server
myWorld clientLeaflet
myWorld server (JavaScript)
ONLINE OFFLINE
![Page 49: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/49.jpg)
![Page 50: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/50.jpg)
CAUTION WITH IOS
• Apple terms and conditions regarding approval and software distribution can be a major pain
• All code, and all changes, need to be reviewed by Apple - even for testing (this has tightened up recently, since they bought TestFlight)
• If you need custom apps for individual customers there is a separate Enterprise Developer Program or Volume Purchase Program, which each have their own complications
![Page 51: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/51.jpg)
“ADVANCED HYBRID” APPROACH
Higher level HTML & JavaScript code Includes myWorld core, plus any applications and customizations
Sync common code package to each device as needed
Use Cordova to create single executable for each platform (Electron on Windows 7)
Low level code Handles data storage and access using SQLite, and Web View which is a container for HTML & JavaScript code
![Page 52: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/52.jpg)
ARCHITECTURE 3HTML5 OFFLINE
• Cache smaller amounts of data usingHTML5 offline capabilities
• More dynamic and easier to deploythan bulk replication
![Page 53: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/53.jpg)
HTML5 SUMMARY
• Tried various approaches including caching individual tiles, caching mbTiles files and reading then with sql.js, caching geoJSON
• LargeLocalStorage was a useful library
• Worked reasonably well for small data volumes
• But a number of issues including limitations on data size, inconsistency across browsers, general flakiness, and inability to manage offline data directly within the application
![Page 54: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/54.jpg)
OFFLINE SUMMARY• It will be great when we have universal wireless coverage!
• Today, offline is still harder than you would hope
• Large scale robust sync for enterprises is not rocket science, but a lot of detailed work to do well
• HTML5 caching has a lot of promise, is workable in some scenarios today, but still not quite fully baked
• We have gone with Cordova which gives you much more control over local storage, and most of the benefits of HTML5
![Page 55: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/55.jpg)
OVERALL SUMMARY
• We have built a very robust enterprise web and mobile geospatial product based on open source foundations including PostGIS, GeoServer, Leaflet, SpatiaLite and Cordova
• We have completed large implementations in several top 10 utility and telecommunications companies, in conservative IT environments
• Our experience with all the open source technology we’ve used has been great
![Page 56: Building enterprise applications using open source](https://reader031.vdocuments.mx/reader031/viewer/2022021813/5870fa231a28ab5f528b5723/html5/thumbnails/56.jpg)
?