tibco spotfire analytics for energy · business mix operations optimization profiling fraccs well...

© Copyright 2000-2012 TIBCO Software Inc. All rights reserved.

TIBCO Spotfire Analytics for Energy

MICHAEL O’CONNELL, PhD

Sr Director, Analytics

TIBCO

© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.

• Introductions & Spotfire® Basics

• Production demonstration

• Predictive Analytics

• Upstream exploratory – multiple use cases

• ESRI Integration

• Portfolio optimization

• Forecasting (time series)

• Sales & marketing

• Health & safety

• Social and mobile

• Tibbr® and collaborative computing

• iPad

• Wrap up and questions

• Deployment approaches

Outline


Automation Cloud Event

Processing Analytics Social

• Messaging

• SOA

• BPM

• MDM

• Events/CEP

• Rules

• In-memory

• Visual Analysis

• Data Discovery

• Statistics

• Private/hybrid

Cloud Platform

• Apps

• Deployment

Option

• Enterprise

Social Media

• Collaboration

Key Components of 21st Century Platform


Dimension-Free

Data Exploration

Data Mashup

Predictive &

Event Driven

Contextual

Collaboration

Enterprise-

Class

• Visual • Interactive • No Constraints

• Combine Data Sources • No Scripting • IT-Free

• Data at Rest & Data In Motion

• Open Source & 3rd Party

• “Two Second Advantage”

• Bookmarks • Guided Apps • Portals & Social Platforms

• Unmatched Performance

• Massive Scalability

• 24x7 Expertise

Spotfire Areas of Focus


Telco, Media, Networks • Customer life-cycle management: social churn • Mobile marketing & advertising • Fraud detection

Finance / Insurance • Customer insight • Risk management & Trading analytics • Fraud and Compliance

Consumer Goods & Retail • Category/brand – value/share analysis & forecasting • Market – product / price analytics, segmentation • Execution – promotion effectiveness

Predictive Analytics Data

TIBCO Industry Analytics

Energy • Exploration, drilling analytics; asset recovery prediction • Trading and financial analytics • Portfolio optimization

Attributes [Customer]

Context / Facts

Behavior [Real-time]

Analytic Solutions

Life Sciences • Research chemistry, biology • Clinical development • Post-marketing / Healthcare


• Supervised Learning

• Regression

• Linear / non-linear models - continuous response (Gaussian underlying errors)

• Generalized linear models – discrete response (e.g. Poisson, binomial errors)

• Trees, Nets, SVM, Additive Models, Discriminant Functions, Nearest Neighbors, …

• Ensembles – Forests

• Fit a bunch of models and average (continuous) or vote (categorical)

• Data Mining – Unsupervised Learning

• Hierarchical and K-means clustering

• Identify segments in the space of X variables

• Association Rules

• Identify associations e.g. between products, documents

• Simulation and Optimization

• Monte Carlo methods and (linear) optimization (mixed integer)

• Unstructured Data Mining

• String matching e.g. bipartite graph algorithm

• Network analytics

Modeling and Data Mining


EUR Predn; Decline/Type

P10/P50/P90 Estimation

Oil and Gas Reserves Analysis

Business Mix Operations Optimization

Profiling Fraccs

Well Economics Unconventional

Methods Optimization

Predictive Analytics in Energy

Analytics Components

• Non-linear modeling (exponential mixtures for recovery estimation)

• Distribution and frequency function estimation (eCDF): P10, P50, P90

• Constrained resource optimization based on well characteristics

• EUR models: effects of extraction parameters and spatial regression

• Optimizing unconventional plays

Real-Time Decisions

Portfolio

Reserves

Dashboards

A&D Analytics

Operations/ BPM

Geospatial Regression

Operational Health & Safety Safety

Portfolio Mgt Portfolio

Operating Wells

Environmental Effects

Land/Lease Potential


• Decline / Type curves

• Algorithms: Power law loss ratio, modified hyperbolic; sophisticated pre-processing / automation / over-rides

• P10, P50, P90

• Distributions: Log-normal, Weibull etc; Bootstrap (non-parametric)

• Geospatial regression

• Kriging, loess/splines; completions analysis / optimization

• ESRI Integration

• Shape file import/export

• Capital planning / portfolio optimization

• Portfolio

• Well / pad optimization

• Sales and marketing

• Health and safety analytics

• Text analysis

• Corpus analysis / networks

• Social and mobile applications

• Collaborative computing / iPad deployment and applications

Analytics Use Cases


Arkoma Basin Data

Extract from IHS Enerdeq

Production on ~1300 wells

Production data 2005-2010

Completion information by frac


Shale Gas


#1. Decline/Type Curves: EUR Estimation • Response

• Oil/gas production data across time

• Analysis

• Fit production decline and predict ultimate production

• Preprocess to remove outliers eg zero production days

• Backtest, report to SEC

Model

Arp’s equation: hyperbolic

qt = qi (1 + bDi t)^(-1/b)

b = reservoir decline exponent

Hyperbolic – exponential

Transition to exponential decline at user=specified time

Power law loss-ratio decline

qt = qi exp(-Dy t - Di t^n)

Dy = decline rate at infinite time

EUR via economic limit e.g. 1 mcf/d (0.03 10^3m^3/d) or time limit

References: Ilk et al, SPE 114947, SPE 116731; Texas A&M

New methods readily added via S+/R scripts


Decline and Type Curves


Decline and Type Curves • Pre-processing

• Response

• Oil/gas production data across time

• Analysis

• Fit monotone smoothing spline (2 df)

• Assess median absolute deviation from smoothing

spline and remove outliers from rest of pack at low

p-value eg 0.0001

Process / Application

• Works well when large number of wells and/or

noisy data (eg zero production days) - most of

time!!

• Manual over-ride is built-in i.e. if user doesn’t like

automated time point removal, can manually set

the points to be included in the decline curve fit


Decline and Type Curves


#2. EUR P10, P50, P90 Estimation


Oil and Gas Reserves: EUR P90, P50, P10


#3. Geospatial Regression • Response

• Production, Environmental variables (groundwater)

• Explanatories

• Spatial locations

Model

• y = f (x, b) + e

• f : spatial regression, kriging, splines, loess

• packages : S+spatial, R-fields

• Thin plate splines:

• yk = f (x, k) + ek min ||y- f||2 + l int f ‘‘(x)2 dx

• Kriging:

• yk = P(x, k) + Z(x, k) + ek


Geospatial Regression


#4. Integration with ESRI ArcMap

• Two way interactivity between TIBCO Spotfire and ArcMap

• Load data from an ArcMap project into TIBCO Spotfire

• Push a Map image from ArcMap into TIBCO Spotfire

• Interactivity between multiple concurrent TIBCO Spotfire and ArcMap sessions

• Re-establishment of the connectivity between previously integrated TIBCO Spotfire and ESRI ArcMap sessions with minimal user input

• Perform ad hoc analysis on spatial and tabular project data with immediate corresponding display in ArcMap


Spotfire Connection

ESRI-Spotfire Integration

• Data associated with “layers”

• Point (e.g. Cities, Wells)

• Polygon (e.g. Countries, Field Reservoirs)

• Attribute table associated with each layer


After a connection has been set up, marking records in Spotfire will select the corresponding features in ArcMap

Spotfire & ArcMap Interactivity


Correspondingly, selecting features in ArcMap will mark the matching records in Spotfire

Spotfire & ArcMap Interactivity


The extension also allows a MapChart plot to be added to the Spotfire session with background taken from the ArcMap session

The MapChart background can be exported either as a shape file for the selected layer or as a raster image based on what is currently displayed in ArcMap

Push a Map image from ArcMap to Spotfire


• Publish

• OpenSpirit Scan Utility

• Publish all G&G spatial data to GIS repository (and keep it synchronized)

• Interact with Desktop

• OpenSpirit ArcGIS Extension

• Use ArcMap as a data selector and viewer to interact with G&G data and other OpenSpirit enabled apps

• Interact with Web

• OpenSpirit Web Services

• “Publish” and interact with G&G spatial data through GIS Web portal using OpenSpirit web services

Spotfire – OpenSpirit Integration


#5. Resource (Compression) Optimization


S+ NuOpt • S-PLUS module: Premier Toolkit for numerical optimization

Large-scale optimization problem solving, including:

Linear programming

Mixed integer programming

Quadratic programming

Unconstrained nonlinear optimization

Multi-objective programming

NuOpt comprises a collection of optimization methods:

primal-dual interior point method with higher order

correction for Linear Programming (LP) models. [higher]

simplex method for Linear Programming (LP) and mixed

integer programming (MIP) models. [simplex]

primal-dual interior point method based on line search for

general Convex Programming (CP) models. [line]

primal-dual interior point method based on trust region

method for general Non-Linear Programming (NLP) models.

[trust]

primal-dual interior point method based on quasi-Newton

method for general Non-Linear Programming (NLP) models.

[BFGS]

NuOpt for S+ is a combination of the S+ programming environment, the NuOpt optimizer and the SIMPLE modelling language.


#6. Portfolio Management


#7. Sales and Marketing


Sales Forecasting


S+ FinMetrics

• S-PLUS module: 500+ functions for time series analysis, econometric modeling and prediction

• Many other related functions in base S-PLUS e.g. Fourier analysis

Vector Autoregressive models (VAR)

Cointegration and Vector Error Correction Model (VECM)

Univariate and multivariate GARCH Volatility Modelling

State Space Modelling

Factor Models for Portfolio Allocation

Fixed Income Analytics (rate conversion, yield curve fitting …)

Extreme Value Theory and Copula Analysis\

Time Series Manipulation and Visualisation

Rolling Statistics and Technical Indicators

Rolling Estimation and Backtesting Strategies

Time Series regression (static, dynamic regression)

Systems of equations estimation (SUR)

Classical Time Series Analysis (ARIMA, unit root ...)

Long-memory modelling


Forecasting: Parameter Selection


#8. Forecasting Commodity Prices


#9. Health and Safety

• Near real-time safety data: gives organization as much time as possible to take action and reduce incidents

– Had been doing monthly/quarterly reviews

– Web deployment

• Move from lagging indicators to leading

– Track incident pre-cursors, prevent injuries before they happen

• Easy to use for adoption and speed

– Increase participation through higher data interactivity

• Depth and Detail

– Make it clear how the organization should respond by providing high level trends and incident level details

– Transparency allows easy spotting of

errors


#10. Social and Mobile

• Social

• Collaborative computing

• Tibbr®

• Mobile

• iPad application in

Spotfire 4.5


• tibbr® and Apache • Solution for geographically distributed

teams on drilling projects • share articles, meeting notes, operating

procedures, documentation on tools

• ideas on ways to improve technology, processes and team objectives.

• Drillers can upload pictures from the field and crowdsource feedback to be shared in the community

• improves the quality of responses to issues and reduces trouble costs

• tibbr streams can be integrated in SharePoint - context of relevant projects

Social and Mobile


Social and Mobile

Drilling Site #4


Social and Mobile

Drilling Site 4 Drilling Site 5


Social : Spotfire-to-Tibbr®


Social : Tibbr®


#11. Spotfire® Deployment Kit for Apple iPad

• Deploy white-labeled analytic apps to mobile devices

– Without mobile development team

– Configure & deploy in under an hour, no coding

• Spotfire’s mobile engineering team monitors & adapts to Apple’s rapidly changing APIs and policies


Display a custom URL on startup • Spotfire app: • Custom example:


Customize the Analysis Control in

Deployment Kit

Control in .dxp / Web

Player

Author Once,

Deploy Anywhere


SE

RV

ER

S

CLI

ENTS

Zero install web client Visual, Analytic Hybrid In-Memory Engines

Spotfire

Web Player

Spotfire

Enterprise Player

Spotfire

Professional

DA

TA

SO

UR

CE

S

XML RDBMS

Spotfire

Developer

IT / DBA

Development,

Solutions,

Integration Flat

Files

Web

Services Spread-

sheets

Spotfire S+ (R)

Managers, Consumers,

Executives

Analysts

Independents

Configurators

Analysts

Informaticians

Statisticians

SAP R/3

Managers, Consumers,

Executives

Spotfire Miner

In-Memory Engine

Spotfire

Web Player

Server

Administration & Integration

Spotfire Server

App Data Connectivity

Spotfire

Advanced Data

Services Real-Time Connectivity

Complex Event

Processing

Services

Computation Engine

Spotfire Statistics

Services

Informaticians

Statisticians

Oracle

E-Business Siebel

eBusiness

EMS, FTL

Batch Analysis

Spotfire

Automation

Services

SDKs

Product (Spotfire) Stack


Product (Spotfire) Stack


Statisticians

Develop analytic

Matlab/SAS Spotfire Miner R/S+

Integrating Statistics into Spotfire


Statisticians

Develop analytic

Spotfire Statistics Services

Deploy and register analytic

Spotfire Server

Make analytic available to

application authors



Statistician

Develop analytic Analysts and Application Authors

Configure Application to use analytic No coding required


Spotfire Server




Statistician

Develop analytic Analysts, Application Authors

Configure Application to use analytic

Managers, Consumers, Executives

One-click deployment of web applications

Centrally-managed application with governance managed by statistics /

analytics team



Spotfire Server



Fastest to Actionable Insight Instantly turn insight into action by enabling anyone to rapidly discover

hidden insights and quickly collaborate

in context

Universal Adaptability Leverage a single analytics and data discovery platform to empower

anyone, anywhere to make insightful decisions

Self-Service Discovery Freely explore data to any level of detail, radically accelerating decision

making, while dramatically reducing dependence on IT

Visibility Into the Unknown Discover unexpected insights hidden in Big Data and Real-Time Events to

immediately identify strategic business opportunities or threats.

Spotfire Value Drivers


Michael O’Connell, PhD

Sr. Director Analytics

TIBCO

[email protected]

Contact Info

David Meade

Sr. Solution Consultant

TIBCO

[email protected]

Peter Shaw, PhD

Sr. Consultant

TIBCO

Tim Crocker

Sr. Solution Consultant

TIBCO

mailto:[email protected]

mailto:[email protected]

tibco spotfire analytics for energy · business mix operations optimization profiling fraccs well...

Documents