emea users conference • berlin, germanycdn.osisoft.com/osi/presentations/2016-users... · title:...


Presented by

Bridge IT and OT with a

process data warehouse

Franco Camba, OSIsoft

Matt Ziegler, OSIsoft

Frank Ruland, SAP


Audience Poll

How many of you are running or considering a Big Data Project?

Have you invested or are you looking into Business Intelligence tools ?

What do you believe is the hardest part when it comes to use operational data within IT Systems?

a) Data Preparation

b) Lack of context/metadata

c) Business Case not well defined

d) Performance

e) Ease of access of Operational Data

Of these tools/technologies, which are the ones you are looking to integrate Operational data into?

a) PowerBI

b) Qlik

c) Tableau

d) Tibco Spotfire

e) SAP

f) Hadoop

g) Other


What you will hear during this talk:

4

– Why IT-OT convergence?

– PI Integrator for Business Analytics: Product details

– SAP HANA IoT Integrator by OSIsoft – Frank Ruland

– Streaming pattern

– Roadmap

– Q&A

Empowering Business in Real-Time. © Copyright 2016 OSIsoft, LLC

OT IT

StrategyArchitecture

SecurityGovernanceHardware

ERP PLM

Esri

SCADA MES

HMI

Use

r In

terf

aces

User In

terfaces

OperationalInsights

Business Insights

ERP

Big Data

Unstr. DataD

ata

/ A

sset

Data / A

sset

SCADAPLC

LIMS

PI Data Arch.

Typical information landscape.

OT – Operation Technology

IT – Information Technology

The Convergence of Information and Operational Technology


Problem Complexity Drives the Need for Integration

Real-time visibility

6

Monitoring

Process Optimization

Benchmarking

System Optimization

Disparate assets or interacting one-by-one Interacting with common assets as a fleet

Real-time & historical view across any plant

asset

Fleet-wide performance comparison

Large scale multi-variate analysis

Co

mp

lexi

ty

Business Intelligence

Big Data Analytics

Machine Learning


How can I do this?

7

Predict Outages

Estimate RUL

Material Management

Compare asset

performance

Production Forecasting

ShiftAnalysis

Predictive Maintenance

Fleetwide BI reports Root-Cause

Analysis


Is it a smooth Journey?

8

o Knowledge

o Time

o Support

o Technology first

o Scope

o Flexibility

More than 50 % of Big Data projects are unsuccessful !


Getting process data analysis-ready

Enhance

Calculate

Correlate

Apply Algorithms

Enriching the raw data by calculating KPIs, aggregations and different analysis

Using statistics and machine learning to find insights in data across multiple variables

Wrapping a layer of context around the data, assets and events

Collecting high-fidelity high-frequency data from a variety of sources and systems

Finding patterns and relationships between variables across multiple datasets and datatypes

Collect

OT

IT

Data Preparation … feels like


Data Scientist is the sexiest job of 21st century, but…

10

Source: http://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says/#5481f6037f75

© Copyright 2016 OSIsoft, LLCEMEA USERS CONFERENCE • BERLIN, GERMANY 11

Time series Data is….complex!T

urb

ine 1

Speed

Torque

Bearing Temp

Oil Temp

Manufacturer

Last Service

Turb

ine 2

Time

Vestas

Siemens

June 20, 2013

Comm Failure

Additional Measure

Uneven SpacingSpike / Out of Range

Bad Sensor

Different Archive Start Times

Speed

Torque

Bearing Temp

Oil Temp

Wear Factor

Manufacturer

Last Service


What do we need to approach this problem?

Enhance

Calculate

Transport

Correlate

Apply Algorithms

Enriching the raw data by calculating KPIs, aggregations and different analysis

Using statistics and machine learning to find insights in data across multiple variables

Wrapping a layer of context around the data

Collecting high-fidelity high-frequency data from a variety of sources and systems

Transform the time-series data into row-column format and push it right to the desired tools

Finding patterns and relationships in data sets that aren’t revealed in one data set

Augment Increase the information content by adding statistics and summary calculations

Collect

OT

IT

Align & Cleanse Synchronize multiple data sources so they are comparable and purify the raw data using filters

Shape Building a data model to respond a specific business question or analysis


PI Integrator for Business Analytics/SAP/Azure

13

CLEANSE

AUGMENT SHAPE

TRANSMIT

PULL

PUSH

Quality / Validate

Model your data structure

Send the information

directly to the tool

Increase information content


Situation: We are tracking compressor process data and we are able to track downtimes as they happen

Problem: We want to compare different compressors in terms of downtimes and understand which ones are offsetting from the baseline. We want to predict potential new outages to maximize asset availability

Specific Capabilities: Prepare and craft data model for Business Intelligence on downtimes, answer a set of questions in terms of asset performance.

Prepare and craft data model for Machine Learning analysis, bring the predictions back to PI and analyze the predictions to identify potential downtimes.

Demo: how are our compressors performing?


Demo: how are our compressors performing?

15


Demo Recap

16

I was able to:

• Build a data model to answer a specific question

• Provide Large amount of information in context

• Quickly consume the data in BI/Machine Learning and get results

Key Benefits:

• CAST

• Self-Service

• Performance

• Supports for multiple targets


Operational Reporting & Analysis Architecture

17

17

… All BI tools that support ODBC

Tableau

Spotfire

SAS

MSFT BI

PI Server

PI Integrator for Business Analytics – Business Intelligence Edition

System of Record

Visualization & Analytics

Data Preparation and Integration

Layer

I want to analyze operations data stored in the PI System using

modern BI tools

PI Integrator for BA: Business Intelligence Edition


Enterprise Data Warehouse Architecture

18

18

PI Server

PI Integrator for Business

Analytics

Tableau Spotfire

System of Record

Visualization & Analytics

Data Preparation and Integration

Layer

Oracle DW, SQL Server, TeradataEnterprise Data

Warehouse / Data Mart / Data Lake

Custom Applications

SASMSFT

BI

Custom or 3rd Party Data Management and ETL

EAMSalesCRM

ERP HR …

Hadoop

I need to fit operational data into my existing company

IT information architecture

PI Integrator for BA: Data Warehouse Edition


Integration with SAP HANA: High level Architecture

19

EAMSalesAggregation PI Server

SAP HANA IoTIntegrator by OSIsoft

Systems of Record

Visualization, Analytics, & Business Process Applications

Data Preparation and Integration Layer

SAP HANAEnterprise Analytics and Applications Platform

SAP HANA Enterprise Information Management and Data Provisioning Agent

Partner Applications

PI Integrator Framework

CRM

ERP HR …

SAP BW

SAP LOB Solutions & Applications

Lumira GISBOBJ

SAP HANA IoT Integrator by OSIsoft


SAP HANA IoT Integrator by OSIsoft

20

Frank Ruland, SAP


Solution architecture - PullPI data into SAP HANA via SAP HANA Smart Data Access

SAP HANA

HANA SDI

Aggregation PI Server

PI Asset Framework (AF)

SAP DP Agent

HANA Studio

PI Integrator FrameworkWeb UI Shape Designer

Windows

Windows

Linux

Microsoft SQL

Component sold by SAP

Component sold by OSIsoft

Legend:Included with SAP HANA IoT Integrator by OSIsoft (no fee). Provisioned by OSIsoft

User creates PI View in Web UI Shape Designer via PI Integrator Framework

PI View definition is stored in PI System (AF). PI View data is stored in optimized format in AF-managed SQL Server

SAP HANA user configures virtual tables in SAP HANA Studio using SAP HANA SDI and SAP HANA IoT Integrator by OSIsoft

PI SQL DAS controls access to PI Views

Windows (All Java)SAP HANA IoT Integrator by OSIsoft

PI JDBC

PI View

SAP HANA IoT Integrator by OSIsoft retrieves data from PI View located in SQL Server via PI JDBC driver

1

1

2

3

4

5

2

3

4

5

PI Data Archive

PI SQL DAS

Note:- This is the heart of the Integrator. Prepare the time series data via CAST into a row/column

format for consumption in a relational dbase environment


Solution architecture - PushPI data into SAP HANA via SAP HANA Client

Aggregation PI Server

SAP HANA

PI Data ArchivePI Asset

Framework

HANA ClientHANA Client

PI Integrator FrameworkWeb UI Shape Designer

Windows

Windows

Linux

Microsoft SQL

Windows (ODBC)

1

2

User creates PI View in Web UI Shape Designer via PI Integrator Framework1

2 PI Integrator Framework pushes data to HANA via HANA Client (ODBC)PI

View

Component sold by SAP

Component sold by OSIsoft

Legend:Included with SAP HANA IoT Integrator by OSIsoft (no fee). Provisioned by OSIsoft

Note:- This is the heart of the Integrator. Prepare the time series data via CAST

into a row/column format for consumption in a relational dbase

environment


In-memory Processing Engines

Streaming

Engine

Graph

Engine

Spatial

Engine

Text

Engine

Calculation Engine

PAL APL AFL R Scripts

SAP HANAReal-time in-memory predictive analytics platform**

Predictive Analysis

Library (PAL)

Accelerated predictive

analysis and scoring with

native in-database

algorithms for both data-at-

rest and for streaming data

R integration

Execution of R scripts via

high- performing parallelized

vector based connection;

R scripts embedded as part

of overall query plan

Application Function

Library (AFL)

Application Function Library

(AFL) framework allows SAP,

partner, and customers to

develop, deploy, load, and

leverage their own advanced

analytic custom functions in

SAP HANA

Automated Predictive

Library (APL)

The predictive analysis

capabilities of SAP

Predictive Analytics

automated analytics

engine (formerly KXEN

/ II) in SAP HANA

SAP Custom Open Source

Streaming Algorithms*

Adaptive Hoeffding Tree

Denstream

Data-at-rest Algorithms

Association Analysis

Cluster Analysis

Classification Analysis

Time Series Analysis

+60 Native Algorithms

R Engine

Application

Function Modeler

SAP HANA

Studio

SAP HANAIn-memory In-database Predictive Analytics

* Predictive Algorithms for Streaming come with Smart Data Streaming License


SAP HANA In-Memory Predictive Analytics

– Predictive Analysis Library (PAL) Native predictive algorithms In-database processing for powerful

and fast results Quicker implementations Support for clustering, classification,

association, time series etc…

– R Integration for SAP HANA Enables the use of the R open source

environment (> 3,500 packages) in the context of the HANA in-memory database

R integration enabled via high performing parallelized connection

R script is embedded within SAP HANA SQL Script

Combine the depth and power of in-memory analytics within SAP HANA with the

breadth of R to support a variety of advanced analytic and predictive scenarios


Compressor Demo video


Product Roadmap

26

Matt Ziegler, OSIsoft


Problem Complexity Drives the Need for Integrators

Real-time visibility

27

Monitoring

Process Optimization

Benchmarking

System Optimization

• BI App (i.e. Tableau, Spotfire, Lumira)

• PI Integrator for Business Analytics

• PI Integrator for SAP HANA

• Machine Learning (Azure ML, R)

• PI Integrator for Business Analytics

• PI Integrator for SAP HANA

• HMI

Disparate assets or interacting one-by-one Interacting with common assets as a fleet

Real-time & historical view across any plant

asset

Fleet-wide performance comparison

Large scale multi-variate analysis

Co

mp

lexi

ty

• PI ProcessBook• PI Coresight• PI Datalink


More integration options, more systems

Business

Intelligence &

Data Warehouses

Available Today

Scalable BI for the PI System

v1.0

• Fleet Asset Reporting

• Reduce Reporting Time

• Integrate w/ Data Warehouse

Available Today

Expanded Systems and Events

v1.1

• + Oracle

• + Hadoop (HIVE & HDFS)

• Event Frames

Streaming

Systems

Research

Streaming Pattern

• Enabling computations in real-

time with an external compute

engine

Planned (1H 2017)

Stream Systems

• Azure Event & IoT Hub

• Kafka

• Custom json

2015 1H-2016 Future

Planned (2H 2016)

Cloud Platforms

• Microsoft Azure

• Azure SQL, SQL DW

• Azure Data Lake

• SAP HANA Cloud Platform

Partner Platform

Research

Enable partners and customers

to build applications and interact

programmatically using PI

Integrator Framework


Tables vs Streams

29

• Business Intelligence

• Human readable

• Batch / Bulk Process

• Normalized data

• Regularly scheduled

• Large data, few messages

• In-line (Streaming) Analytics

• Computer readable

• Specific Data / Targeted Process

• Raw or “Packages” of data

• Triggered

• Small data, many messages


Integration Patterns

30

Tables

Files

Databases

Streams Other Patterns

Metadata

Programming

On-Demand

Workflow & Transactions

Files

Queues

Messaging

External Analytics Engines


2016 Technical Roadmap - Specific Planned

Enhancements SAP HANA IoT Integrator by OSIsoft

Version 1 (Dec 15) Version 1.5 (May 16) Version 2 (Q4 16)

Leverage event frames

(batches) of PI System

data published into SAP

HANA for ad hoc projects

and analysis in memory

Publish PI System data to

SAP HANA or HANA

Cloud Platform using SDI.

Retrieve live information

from PI Views using SDI.

Batch-cleanse, filter and

aggregate PI System data

into federated tables

within SAP HANA using

SDA

Protect your technology investment!

© Copyright 2016 OSIsoft, LLCEMEA USERS CONFERENCE • BERLIN, GERMANY 32

SAP HANA / HANA Cloud

Platform

SAP HANA IoT

Integrator 2015

(Smart Data Access)

Lumira

S/4 HANA

PdMS

AnalyticsPull Data

Access

Publish

Data (Push)

SAP HANA IoT

Integrator 2015 SP1

2016 (SDI)

Query Data

(API)SAP IoT Integrator

2016

Planned 2017

Stream Data

(Stream

Events-SDS)

Receive Data

(Predictions)

Receive

Metadata

(Assets / PM)

Road Map Databases & Applications

Planned 2017

Research

Virtual Tables in HANA

Smart Data Streaming

S4 (PM) AIN

Example Roadmap - SAP HANA IoT Integrator by OSIsoftS

AP

HA

NA

Ecosyste

m


Contact Information

Franco [email protected]

Systems Engineer

OSIsoft ltd UK

3333

Matt [email protected]

Product Manager

OSIsoft, LLC

Frank [email protected] of Industry Ecosystem for Energy and Natural Resources

SAP SE

mailto:[email protected]




Questions

Please wait for the

microphone before asking

your questions

Please remember to…

Complete the Online Survey

for this session

State your

name & company

34

http://ddut.ch/osisoft


Thank You

emea users conference • berlin, germanycdn.osisoft.com/osi/presentations/2016-users... · title:...

Documents