arcgis geoevent server: applying real-time analytics · 2018. 4. 6. · web device desktop live...

Real-Time GIS: Applying Real-Time Analytics

RJ Sunderman

Ken Gorton

Agenda

Performing Analysis in Real-Time

Use Case 1: Identifying Conditions

Use Case 2: Finding Patterns in Data

Use Case 3: Workforce Tracking

Summary & Resources

1

2

3

4

5

Performing Analysis in Real Time1

DesktopWeb Device

live & historic

aggregates & features

map & feature service

• Ingest high velocity real-time &

Internet of Things (IoT) data into

ArcGIS.

• Perform continuous analytics on

events as they are received.

• Store observations in a

spatiotemporal big data store.

• Visualize high velocity &

volume data:

- as an aggregation

- or as discrete features.

• Notify about patterns of

interest.

• Adjust behavior of things in our

environment through actuation.

stream service

live features

ArcGIS

Enterprise

GeoEvent

Server

ingestion

analytics

ArcGIS Enterprisewith real-time & big data capabilities

GeoAnalytics

Server

spatiotemporal

big data store

storage analytics

visualization

actuation

Applying real-time analyticsGeoEvent Services

• A GeoEvent Service configures the flow of GeoEvents

- The Filtering and GeoEvent Processing steps performed

- The input(s) data comes from and the output(s) to which results are sent

Applying real-time analyticsprocessors and filters

• Perform continuous analytics on GeoEvents as they are received

using processors and filters.

GeoEvent Extension

Inp

uts

Ou

tpu

ts

GeoEvent Services

Buffer Creator

Convex Hull Creator

Difference Creator

Envelope Creator

Field Calculator

Field Enricher

Field Mapper

Geotagger

Incident Detector

Intersector

Projector

Simplifier

Symmetric Difference

Track Gap Detector

Field Reducer Union Creator

Ou

t o

f th

e B

ox

Add XYZ

Esri

Gallery

Bearing

Ellipse

Event Volume Control

Extent Enricher

Field Grouper

GeoNames Lookup

Range Fan

Reverse Geocoder

Service Area Creator

Symbol Lookup

Track Idle Detector

Unit Converter

Visibility

Motion Calculator Query Report

You can create

your own processors.

Fleet Asset Tracking Pseudo Servicereal-time analytics design

Processors

• Some work only on geometry

• Some work on geometry and/or attributes

• Some respond to spatial and/or attribute conditions and generate new messages

• Processors typically transform the event record being processed in some way

Filters

• Allow event records to pass only if a

conditional expression evaluates TRUE

• The expression can use attributes or

geometries you have established as geofences

Filters

• You can use Boolean logic (AND, OR, NOT) to

combine expressions

Spatial Operators

• Don’t confuse “ANY” and “ALL” with the regular expression pattern

used to select a set of geofences

geofence selection vs. spatial operator scope

GEOMETRY INTERSECTS ALL .* /.*

GEOMETRY DISJOINT ANY .* /.*

Spatial Operatorsoverlapping geofences

• Intersects All geofences• Intersects Any geofence

Spatial Operators

• Spatial operators:

- Inside

- Outside

- Enter

- Exit

- Intersect

- Disjoint

- Touches

- Contains

- Crosses

- Equals

- Overlaps

- Within

GeoFences

• Import from a feature service

- Reads once (good for static geofences)

• Synchronize with a feature service

- Periodically refreshes to update geofences

feature service synchronization

Feature Service

Event FilteredEvent

Import Synchronize

GeoFences

• Category (e.g. “RestrictedZone”)

o SecurityPavilion

o LoadingDock

o ObservationHanger

GeoFences

• GeoFences are constantly updated

- Allows geofences to become dynamic

• Requires active management and

purging of geofences as they expire

stream service synchronization

Stream Service

Event FilteredEvent

Synchronize

GeoFences

• Category (e.g. “ActiveTracks”)

o A12345

o G65190

o K91263

Dynamic GeoFences

33.322336

-117.416323

Dynamic GeoFences

33.322125

-117.398225

Identifying Conditions (demo)2

Identifying conditions with real-time datareal-time analytics design

Identifying conditions with real-time datareal-time analytics design

Field Mapper processor

• Use a Field Mapper when you need to:

- Change the schema or structure of an event record

- Translate from one GeoEvent Definition to another

- Specify how attribute values map from an inbound to an outbound event record

Field Mapper processor

TranslatedEvent

IncomingEvent

Mapping

TrackID J7890

Date 1405176845553

Geometry -117.123…, 36.064…

Category AnkleBraceletGPS

TrackID J7890

Date 1405176845553

Sensor 2

BatteryLevel Medium

Latitude 36.064

Longitude -117.123

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

CourseDeg 250.0

Geometry -117.123…, 36.064…

Category AnkleBraceletGPS

Identifying conditions with real-time datareal-time analytics design

Field Enricher processor

• Use a Field Enricher when you need to:

- Enrich an event record with new attribute values from an secondary source

An attribute join is used to retrieve values from the secondary table.

The processor retrieves specified data values and then enriches an

event record by either appending new fields or writing the data to new fields.

Field Enricher processor

TrackID NoContact NoEntry

K90123 Temecula gangland

V10987 F65432 Pass Christian School

… … …

Event

EnrichedEvent

Feature Service

TrackID V10987

Date 1405176845553

BatteryLevel Low

Distance 105.6

Speed 1.2

Course 186.4

geometry -117.123…, 36.064…

TrackID V10987

Date 1405176845553

BatteryLevel Low

Distance 105.6

Speed 1.2

Course 186.4

geometry -117.123…, 36.064…

NoContact F65432

NoEntry Pass Christian School

Identifying conditions with real-time datareal-time analytics design

Buffer Creator processor

• Use a Buffer Creator when you need to:

- Construct a polygon around an event’s point, polyline, or polygon geometry

- Enrich an event record by adding a new geometry field

- Replace an event record’s geometry with a derivative

Buffer Creator processor

TrackID S90909

Date 1405176845553

BatteryLevel High

Distance 0.2

geometry -117.123…, 36.064…

TrackID S90909

Date 1405176845553

BatteryLevel High

Distance 0.2

geometry -117.123…, 36.064…

buffer rings" : [ [ [ -116.3175, 33.6703],[-

116.3175, 33.6703]…]]

Event

EnrichedEvent

Buffer Creator

Identifying conditions with real-time datareal-time analytics design

Intersector processor

• Use an Intersector processor when you need to:

- Generate a geometry representing the intersection between

a event record’s geometry and a set of specified geofences

Geometry processors can either enrich an event record by adding a new geometry

field or replace an event record’s geometry with a derivative geometry.

Intersector processor

TrackID S90909

Date 1405176845553

BatteryLevel High

Distance 0.2

geometry rings" : [ [ [ -114.3175,

33.6703],[-114.3175,

33.6703]…]]

TrackID S90909

Date 1405176845553

BatteryLevel High

Distance 0.2

geometry -117.123…, 36.064…

intersection rings" : [ [ [ -116.3175, 33.6703],[-

116.3175, 33.6703]…]]

Event

EnrichedEvent

Intersector

Finding Patterns in Data (demo)3

Finding patterns in real-time datareal-time analytics design

Finding patterns in real-time datareal-time analytics design

GeoTagger processor

• Use a GeoTagger when you need to:

- Enrich an event record with the name of a geofence with which

the event has a spatial relationship

The processor uses a spatial expression to identify related geometries.

The “name” or unique identifier of the related geofences is appended to the event record.

A GeoTagger is essentially a spatial join.

GeoTagger processor

Event

EnrichedEventExpression

TrackID J7890

Date 1405176845553

Sensor 2

BatteryLevel Medium

Latitude 36.064

Longitude -117.123

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

CourseDeg 250.0

Geometry -117.123…, 36.064…

Category AnkleBraceletGPS

TrackID J7890

Date 1405176845553

Sensor 2

BatteryLevel Medium

Latitude 36.064

Longitude -117.123

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

CourseDeg 250.0

Geometry -117.123…, 36.064…

Category AnkleBraceletGPS

IsInside Temecula ganglandTemecula

gangland

Finding patterns in real-time datareal-time analytics design

19382,21472,39820,…CrimeIDs

Finding patterns in real-time datareal-time analytics design

Field Calculator processor

• Use a Field Calculator when you need to:

- Calculate new values using data from a received event record

- An expression is evaluated and used to calculate the new values

- Results can be written to a new field or used to update an existing attribute

Expressions can be mathematical, string operations, or function invocations

which use regular expressions.

Field Calculator processor

TrackID V10987

Date 1405176845553

BatteryLevel Low

Distance 0.02

Speed 1.2

Course 186.4

geometry -117.123…, 36.064…

TrackID V10987

Date 1405176845553

BatteryLevel Low

Distance 105.6

Speed 1.2

Course 186.4

geometry -117.123…, 36.064…

Event

EnrichedEventExpression

Convert fromFeet to MilesExpression:

Distance / 5280

Finding patterns in real-time datareal-time analytics design

Finding patterns in real-time datareal-time analytics design

Workforce Tracking (demo)4

Workforce Trackingreal-time analytics design

Workforce Trackingreal-time analytics design

Field Splitter processor

• Use a Field Splitter when you need to:

- Split a list of delimited into multiple event records

Lists are normally a single delimited string (e.g. comma delimited values).

A multicardinal version of this processor can handle more complex lists.

Field Splitter processor

TrackID V10987

Date 1405176845553

BatteryLevel Low

msgCode 1

freeText “stopped for fuel and lunch”

Addresses “5712940342,

admin@inspections.com"

geometry -117.123…, 36.064…

Event

EnrichedEventExpression

TrackID V10987

Date 1405176845553

BatteryLevel Low

msgCode 1

freeText “stopped for fuel and lunch”

Addresses “5712940342”

geometry -117.123…, 36.064…

TrackID V10987

Date 1405176845553

BatteryLevel Low

msgCode 1

freeText “stopped for fuel and lunch”

Addresses “admin@inspections.com”

geometry -117.123…, 36.064…

Workforce Trackingreal-time analytics design

Track Gap Detector

• Use a Track Gap Detector when you need to:

- Detect the absence of event record reporting

- Alert or notify someone that expected data was not received

Track Gap Detector

trackId V10987

gap true

lastReceived 1405176855553

geometry -116.93…, 33.93…

trackId V10987

gap true

lastReceived 1405176855553

geometry -116.93…, 33.93…

GapClosed

GapDetected

TrackID V10987

Date 1405176845553

Geometry -116.93…, 33.93…

TrackID V10987

Date 1405176855553

geometry -116.93…, 33.93…

trackId V10987

gap true

lastReceived 1405176855553

geometry -116.93…, 33.93…

trackId V10987

gap true

lastReceived 1405176855553

Geometry -116.93…, 33.93…

trackId V10987

gap false

lastReceived 1405176915553

geometry -117.123…, 36.064…

TrackID V10987

Date 1405176915553

geometry -116.93…, 33.93…

TrackID V10987

Date 1405176925553

geometry -116.93…, 33.93…

TrackID V10987

Date 1405176935553

geometry -116.93…, 33.93…

TrackID V10987

Date 1405176945553

Geometry -117.123…, 36.064…

Workforce Trackingreal-time analytics design

Incident Detector

• Use an Incident Detector when you need to:

- Detect that a condition has occurred and monitor its duration

Uses a filter expression to specify an opening and (optional) closing condition.

Maintains state for the duration of the incident, closing the incident when a

specific closing condition is observed.

Incidents may also close (expire) when no further events are received for the

TRACK_ID associated with the incident.

Incident Detector

IncidentEnded

IncidentStarted

GEOMETRY INSIDE

AnkleBraceletGPS-Geofences/Temecula Gangland

AND TrackID = J7890

gggggggggg

TrackID J7890

Date 1405176845553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176855553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176865553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176875553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176885553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176895553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176905553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176915553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176925553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

TrackID J7890

Date 1405176935553

BatteryLevel Medium

Distance 0.01

DurationMin 1.03

SpeedMPH 0.62

Geometry -117.1..…, 36.0..…

Incident Detector

id c982db54-…-3bbb61211eb6

name Geofence violation

type Cumulative

status Started

alertType Warning

openCondition (INSIDE(AnkleBraceletGPS/Tem

ecula gangland) AND (TrackID =

J7890))

closeCondition

description Started at 7/12/14 10:54 AM

timestamp 1405176865553

definitionName incident

definitionOwner com.esri.ges.processor/Incident

Detector/10.3.0

trackId J7890

geometry -117.123…, 36.064…

duration 0

dismissed false

assignedTo

note

id c982db54-…-3bbb61211eb6

name Geofence violation

type Cumulative

status Ongoing

alertType Warning

openCondition (INSIDE(AnkleBraceletGPS/Tem

ecula gangland) AND (TrackID =

J7890))

closeCondition

description Ongoing for last 10 seconds.

timestamp 1405176875553

definitionName incident

definitionOwner com.esri.ges.processor/Incident

Detector/10.3.0

trackId J7890

geometry -117.123…, 36.064…

duration 10000

dismissed false

assignedTo

note

id c982db54-…-3bbb61211eb6

name Geofence violation

type Cumulative

status Ongoing

alertType Warning

openCondition (INSIDE(AnkleBraceletGPS/Tem

ecula gangland) AND (TrackID =

J7890))

closeCondition

description Ongoing for last 20 seconds.

timestamp 1405176885553

definitionName incident

definitionOwner com.esri.ges.processor/Incident

Detector/10.3.0

trackId J7890

geometry -117.123…, 36.064…

duration 20000

dismissed false

assignedTo

note

id c982db54-…-3bbb61211eb6

name Geofence violation

type Cumulative

status Ongoing

alertType Warning

openCondition (INSIDE(AnkleBraceletGPS/Tem

ecula gangland) AND (TrackID =

J7890))

closeCondition

description Ongoing for last 30 seconds.

timestamp 1405176895553

definitionName incident

definitionOwner com.esri.ges.processor/Incident

Detector/10.3.0

trackId J7890

geometry -117.123…, 36.064…

duration 30000

dismissed false

assignedTo

note

id c982db54-…-3bbb61211eb6

name Geofence violation

type Cumulative

status Ended

alertType Warning

openCondition (INSIDE(AnkleBraceletGPS/Tem

ecula gangland) AND (TrackID =

J7890))

closeCondition

description Ended at 7/12/14 10:54 AM and

lasted for 40 seconds

timestamp 1405176905553

definitionName incident

definitionOwner com.esri.ges.processor/Incident

Detector/10.3.0

trackId J7890

geometry -117.123…, 36.064…

duration 40000

dismissed false

assignedTo

noteTemecula

gangland

Workforce Trackingreal-time analytics design

Reverse Geocoder processor

• A Reverse Geocoder is an example of a custom processor

- Enriches an event record with the street address nearest the event’s location

Uses a point geometry from a received event to enrich the event record with new attribute

field(s) representing a matched address.

Reverse Geocoder processor

Event

EnrichedEvent

Reverse Geocoder Service

REST

Reverse GeocoderTrackID J7890

Date 1405176845553

Sensor 2

BatteryLevel Medium

SpeedMPH 1.79

Geometry -116.97…, 33.98…

TrackID J7890

Date 1405176845553

Sensor 2

BatteryLevel Medium

SpeedMPH 0.01

Geometry -116.97…, 33.98…

Address 39583 Avenida Sonrisa

Neighbor

City Beaumont

Subregion

Region California

Postal 92223

PostalExt

CountryCode USA

Match_addr 39583 Avenida Sonrisa,

Beaumont, California, 92223

Loc_name USA.PointAddress

Summary & Resources5

Summary

• ArcGIS is a dynamic platform that enables continuous analytics

and real-time visualization for better understanding of our world

• The ArcGIS GeoEvent Server allows you to:

- know what is happening, as it happens

- react and make smarter decisions faster

- be notified when interesting events occur

GeoEvent Server – real-time analytics for your ArcGIS Enterprise

Summaryself-paced training and resources

• Step-by-Step Tutorials, free to download

- Introduction

- Stream services

- Spatiotemporal Big Data Store

- Notifications

• Blogs and discussions on the forum

- http://links.esri.com/geoevent-forum

• Video recordings of technical workshops

- http://www.esri.com/videos

Real-Time & Big Data Sessions!

• Real-Time GIS: Best Practices Tue, 2:10- 3:10 pm, 152 B

Wed, 2:45 - 3:45 pm, 152 B

• ArcGIS GeoEvent Server: Configuring Real-Time Web Apps Tue, 3:20 - 4:20 pm, 152 B

Wed, 5:15 - 6:15 pm, 152 B

• ArcGIS GeoEvent Server: An Introduction Wed, 8:15 - 9:15 am, 152 B

• Real-Time & Big Data GIS: The Road Ahead Wed, 1:30 - 2:30 pm, 152 B

• ArcGIS GeoEvent Server: Applying Real-Time Analytics Wed, 4:00 – 5:00 pm, 152 B

• Real-Time and Big Data: Tue, 4:30pm - 5:30 pm, 152 B

Leveraging the Spatiotemporal Big Data Store Wed, 11:00 am -12:00 pm, 152 B

Print Your Certificate of AttendancePrint stations located in the 140 Concourse

Tuesday Wednesday12:30 pm – 6:30 pm GIS Solutions Expo Hall B

5:00 pm – 6:30 pm GIS Solutions Expo SocialHall B

10:30 am – 5:15 pm GIS Solutions Expo Hall B

6:30 pm – 9:00 pm Networking ReceptionSmithsonian National Portrait Gallery

Download the Esri Events

app and find your eventSelect the session

you attended

Scroll down to find the

feedback section

Complete answers

and select “Submit”

Please Take Our Survey in the Esri Events App