tibco data science team april 2019 tibco analytics meetup · spark apis ingest streams, transform...

TIBCO Analytics Meetup

April 2019

TIBCO Data Science Team

The following information is confidential information of TIBCO Software Inc. Use, duplication, transmission, or republication for any purpose without the prior written consent of TIBCO is expressly prohibited.

CONFIDENTIALITY

© Copyright 2000-2019 TIBCO Software Inc.

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. This document is provided for informational purposes only and its contents are subject to change without notice. TIBCO makes no warranties, express or implied, in or relating to this document or any information in it, including, without limitation, that this document, or any information in it, is error-free or meets any conditions of merchantability or fitness for a particular purpose. This document may not be reproduced or transmitted in any form or by any means without our prior written permission.

The material provided is for informational purposes only, and should not be relied on in making a purchasing decision. The information is not a commitment, promise or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion.

During the course of this presentation TIBCO or its representatives may make forward-looking statements regarding future events, TIBCO’s future results or our future financial performance. These statements are based on management’s current expectations. Although we believe that the expectations reflected in the forward-looking statements contained in this presentation are reasonable, these expectations or any of the forward-looking statements could prove to be incorrect and actual results or financial performance could differ materially from those stated herein. TIBCO does not undertake to update any forward-looking statement that may be made from time to time or on its behalf.

DISCLAIMER


4

AgendaTIBCO Analytics – Michael O’Connell

• Spotfire X, TIBCO Data Science

• New* Snappy Data Acquisition, New* AI on Demand initiative

GeoAnalytics – Peter Shaw• Voronoi Tessellation – New* Spotfire Data Function

High Tech Manufacturing – Steven Hillion

• Digital Twin, Anomaly Detection, Wide Data

• New* AI on Demand Solution Starter

TIBCO Community Update – Heleen Snelting

• AI on Demand, Anomaly Detection, Tips and Tricks, ..


Please submit your questions at any time via Q&A optionWe will answer at the end or will get back to you via email

5

TIBCO Analytics Meetup – opportunities to learn and network

Meetup group keeps growing and now has 725+ members!

Join this TIBCO Analytics Meetup group and receive automatic invites to future TIBCO Analytics Meetups

http://www.meetup.com/TIBCOSpotfireOnlineusergroup/

https://bit.ly/2Jm4iOn


https://www.meetup.com/TIBCOSpotfireOnlineusergroup/

http://www.meetup.com/TIBCOSpotfireOnlineusergroup/

https://bit.ly/2Jm4iOn

6

Analytics & Data Science

TIBCO Update


Michael O’ConnellChief Analytics Officer

Michael O’ConnellChief Analytics Officer

7

Spotfire A(X) Analytics Experience

AugmentedAI-Powered Insights

Start in seconds, instant insights

AutomatedAutomagical Dataflows

Author & audit with automatically recorded dataflow steps

AgileReimagined User Interface

Agile exploration made even easier

AcceleratedReal-time Insights

Real-time awareness and action


TIBCO Data Science

In-DB ETL

In-DB ML

MLETL

TIBCO Spotfire X

AI Recommendations

Live Data Streams

Data Science + Visual Analytics


MLETL

Data Science + Visual Analytics: in Operations


Deep Scale

High Volume

MPP DB

Real-time designLow latency, HA, concurrency

Batch design, high throughput, Rich API,

Eco-system

Single Unified HA ClusterOLTP + OLAP + Streaming for Real-time Analytics

AQPColumnar

Snappy Data = TIBCO Compute DB

Spark fused with Hybrid In-memory DB

Any Modern Data Source

EnterpriseApps

In-memory

Virtual Tables

Synopses

Stream process

Apply updates as they happen

Acquire data, schema❖ Connect to most/all modern data stores❖ Discover schema, register virtual tables

Snappy Engine

Spark APIs

Ingest streams, transform❖ Model streams as tables❖ Ingest, join with anything❖ Score ML/DL models

Use Synopses for big data❖ Use far fewer resources❖ Fast response time

SQL/Spark/REST as API❖ ASK ANYTHING: complex or

pin-pointed❖ Update in real time

Hybrid Core engine❖ Super fast aggregations❖ Index driven point lookups, writes❖ MVCC transactions …..

Snappy Data = TIBCO Compute DB

Analytics Data Fabric

BI & Analytics MQData Science MQ

Gartner Analytics MQs 2019


14

AI Success Stories

FINANCE:Fraud Detection

MANUFACTURING:Anomaly Detection

HEALTHCARE:Patient Diagnosis

TRANSPORTATION:Demand Forecasting

E-COMMERCE:Recommender Systems

ENERGY:Production Surveillance


15

AI on Demand Solution Starters

• Digital Twin, Anomaly Detection (Wide Data) – Manufacturing

• Customer Engagement – Propensity to Buy

• Risk Management – Fraud, AML, ..

• Production Surveillance – Energy

• Dynamic Pricing – Willingness to Pay

AI on Demand — Solution Starters


https://community.tibco.com/wiki/ai-demand-data-science-operations


Energy - Production Surveillance

Spotfire : 60 sec appGeoAnalytics

Data VirtualizationData Science

StreamingEBX Data Mgt

IHS – Live Demo: Spotfire X

Oxy – Live Demo: Production Surveillance

Hunt Oil – Real-Time Drilling

TIBCO – Data on Fire Demonstration

Energy Day – Houston, Feb 2019

17

TIBCO, Hi Tech Manufacturing, AI on DemandGlobal Hi Tech Manufacturing Ecosystem Customers

• Most Top 20 Semiconductor Leaders

• System manufacturers & software suppliers

Participation in Hi Tech Manufacturing Industry groups• SEMI Smart Manufacturing groups

• IRDS Factory Integration Chapter – Proactive Yield Management

Event presentations on Big Data, AI & Blockchain

• SEMICON US July 9-11, Korea May 7-9

• ASMC May 6-9

< AI on Demand Action >

• Data Science Central April 30


18

Peter ShawData Scientist Staff

Voronoi Polygons Spotfire Data Function — Demo


Swap Logoif neededIn master slide

19

Voronoi Polygons Data Function


https://community.tibco.com/modules/voronoi-polygons-tibco-spotfire

https://community.tibco.com/wiki/tibco-energy-day-houston-february-2019

https://community.tibco.com/wiki/tibco-energy-day-houston-february-2019

20

Digital Twin for Yield in Manufacturing — Demo


Steven HillionSenior Director Data Science

Use unsupervised techniques to identify anomalies, and define clusters of interest

Build supervised models to relate Product Yield Failures (Yi ) with Process Parameters (P

i )

StartProcess Step 1

Process Step i

Measurement Step 1

Process Step i+2

Process Step n

・・・・・・

Yi = f(P1 , P

2, P

3, … P

n)

Digital TwinYield Models

P1

Y = f(Y1, Y

2, Y

3, … Y

n )

Pi

Pi+2

Pn

Equipment Sensor Data

Pi+1

Physical Measurement Data

Yield,Yield Components

or ClustersProcess OptimizationProcess Monitoring

Process Control

~ 1K Steps, > > 1M Process Parameters

Run continuously to capture new relationships as they emerge

Equipment Names

Product Test:Pass / Fail, Failure Modes

Creating the Digital Twin


22

• Not just Big Data – many rows: lots, wafers, die• Also Wide data – many columns: > 1M Process Parameters

• Sensor traces – time series for every sensor on each tool • Data explodes to hundreds of points per sensor

• Measurement data• Equipment attributes – very high cardinality

Extreme Challenge of Big & Wide Data• Not just Big Data – many rows: lots, wafers, die• Also Wide data – many columns: process parameterso Sensor traces – each sensor on every tool, amount of

data “exploding”o Measurement data & Equipment attributes – very high

cardinality

Extreme Challenge of Big & Wide Data• Not just Big Data – many rows: lots, wafers, die• Also Wide data – many columns: process parameterso Sensor traces – each sensor on every tool, amount of

data “exploding”o Measurement data & Equipment attributes – very high

cardinality

The Extreme Challenge of Big & Wide Data


23

Solution Architecture

• TIBCO® Data Science – Team Studio• In-database parallelized computing• Leverages Hadoop, Apache Spark™

• TIBCO® Data Science – Statistica®

• In-memory dedicated fast server• TIBCO Spotfire®

• Interactive in-memory visualization environment

Data Prep, feature engineering & selection

Further feature selection & model building

Visualization of Results


24

TIBCO Community Update


Heleen SneltingDirector Data Science


26

AI on Demand: Data Science in Operations


AI on Demand● New set of featured analytic applications

demonstrating AI and continuous, closed-loop learning

● Global live Meetup Tour - sign-up links on the Community page

● Deep Dive workshops at the various TIBCO NOW events this year

● Links to blogs, whitepapers, demo videos and more!


27

TIBCO NOW now.tibco.com


http://now.tibco.com

http://now.tibco.com


28

1. Guidance on getting access to your products and setting them up.

2. Recommendations and best practices for starting usage.

3. Awareness of the many additional resources available to help you and your

organization achieve your goals.

Spotfire Customer Orientation

https://community.tibco.com/wiki/tibco-spotfire-customer-orientation




29

Dr. Spotfire - Hand-on with Analytics


● Free 30-minute webinar● For all Spotfire users ● Includes Feature Demonstration and User Q&A● Hosted by our top Data Scientists and Spotfire

Expertshttps://community.tibco.com/wiki/doctor-spotfire-office-hours

1st Tuesday of every month at 9AM PT

30

Spotfire X, Spotfire Data Streams - learn moreWhat’s New in Spotfire TIBCO Community Page And Spotfire X Webinar Series

https://community.tibco.com/wiki/whats-new-tibco-spotfire


https://www.tibco.com/events/series/spotfire-x-webinar-series

Data Prep with the all new Data Canvas!

Video: https://youtu.be/B5W8u5X1fuc

Real-Time Dashboards with Spotfire Data Streams!Blog: https://bit.ly/2R6ajRTInteractive Demo: https://bit.ly/2Wac2cj (public)TIBCO Community how-to: https://bit.ly/2Wac2cj

AI Recommendations for Fast Insights

Video: https://youtu.be/FQjcTdzHj7E

https://community.tibco.com/wiki/whats-new-tibco-spotfire

https://www.tibco.com/events/series/spotfire-x-webinar-series

https://youtu.be/B5W8u5X1fuc

https://bit.ly/2R6ajRT

https://bit.ly/2Wac2cj

https://bit.ly/2FLyvqy

https://youtu.be/FQjcTdzHj7E


31

Spotfire Book - to be released April 30!

TIBCO Spotfire: A Comprehensive Primer - Second Edition: Building enterprise-class data analysis and visualization solutions

Primary Author: Andrew Berridge, TIBCO Data Science

Key Features● Learn how to use TIBCO Spotfire to create enterprise-class

analysis and visualization solutions● And how to use different data and visualization techniques

to build insightful and interactive analyses with Spotfire● Lots of practical examples● Up-to-date with the latest versions of Spotfire X

TIBCO Community page with details and updates on the book: https://bit.ly/2IJP4ph

You can (pre-)order your book here https://bit.ly/2Pwp5SK (Packt) and here https://amzn.to/2W7GAuY (Amazon)


https://bit.ly/2IJP4ph

https://bit.ly/2Pwp5SK

https://bit.ly/2Pwp5SK

https://amzn.to/2W7GAuY

32

Analytics Bulletin - Keeps you informed

Objective: Inform you in one-go of the relevant Analytics webinars, demos, and product updates available to you

If you registered for this TIBCO Analytics Meetup you’ll receive the next monthly Analytics Bulletin (recent initiative) via e-mail

Soon available via the TIBCO Community as well!

33

Anomaly Detection and Autoencoder ML

https://bit.ly/2Walf4G


https://bit.ly/2Walf4G

34

What’s new in… for example TIBCO Data Virtualization

https://community.tibco.com/wiki/whats-new-tibco-data-virtualization

Customer Orientation and Customer Success Center - ideal for on-boarding and staying up to date

https://community.tibco.com/wiki/tibco-spotfire-customer-orientation https://community.tibco.com/wiki/tibco-spotfire-customer-success-center

AI on Demand: Data Science in Operations - TIBCO Data Science latest work and Meetup Tour 2019


Dr Spotfire

https://community.tibco.com/wiki/doctor-spotfire-office-hours

TIBCO Geo-Analytics capabilities

https://community.tibco.com/wiki/tibco-spotfire-location-analytics-mapping-geoanalytics-and-spatial-statistics

TIBCO Analytics Meetup pages with recordings and presentations

https://community.tibco.com/wiki/tibco-analytics-meetup

Data Literacy

https://community.tibco.com/wiki/data-literacy

Top TIBCO Community links to bookmark

© Copyright 2000-2019 TIBCO Software Inc. © Copyright 2000-2019 TIBCO Software Inc.

https://community.tibco.com/wiki/whats-new-tibco-data-virtualization


https://community.tibco.com/wiki/tibco-spotfire-customer-success-center


https://community.tibco.com/wiki/doctor-spotfire-office-hours

https://community.tibco.com/wiki/tibco-spotfire-location-analytics-mapping-geoanalytics-and-spatial-statistics

https://community.tibco.com/wiki/tibco-analytics-meetup

https://community.tibco.com/wiki/data-literacy

35

Questions & Contact

Thank you!Michael O’[email protected]

@MichOConnellH

Heleen Snelting [email protected]@HeleenSnelting

TIBCO Communitycommunity.tibco.com

TIBCO Exchangecommunity.tibco.com/exchange

Spotfire Trialspotfire.tibco.com/trial


mailto:[email protected]

mailto:[email protected]

https://community.tibco.com/wiki/tibco-spotfire-community-wiki

http://community.tibco.com/exchange

https://spotfire.tibco.com/trial

tibco data science team april 2019 tibco analytics meetup · spark apis ingest streams, transform...

Documents