denodo datafest 2016: comparing and contrasting data virtualization with data prep, data blending,...

O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A

#DenodoDataFest

RAPID, AGILE DATA STRATEGIESFor Accelerating Analytics, Cloud, and Big Data Initiatives.

Comparing and Contrasting Data Virtualization with Data Prep, Data Blending, Data Catalog and Other Technologies

Paul Moxon

Head of Product Management, Denodo

Agenda1.Business Intelligence ‘Swim Lanes’

2.Data Prep – What is it and how does it work?

3.All you want to know about Data Blending

4.Data Catalogs – What, When, and How

5.Mapping to the Swim Lanes

6.Where Does Data Virtualization Fit?

7.Q&A

3

Business Intelligence ‘Swim Lanes’

4

• Task focused

• Productivity

• Self-service

• Quick and easy access

to data

• Automation (or

simplification) of data

gathering

• Tactical

• Team/Departmental

• Drives business

operations

• Shared data

• Process oriented

• Strategic

• Executive and KPI

dashboards

• Drives strategic

decisions

• Managed, governed

data

• Consistent data

Data Prep

What is it and how does it work?

5

Data preparation is the process of gathering,

combining, structuring and organizing data so it

can be analyzed as part of business intelligence or

analytics process.

Leading Data Prep Vendors

• Trifacta

• Paxata

• Alteryx

• Datameer

• Talend Data Preparation Desktop

• Informatica Rev

• SAS Data Loader

• IBM Watson

7

How Does It Work?

Interactive Data Prep process:

1. First data is ingested from data sources (or just a sample of data)

2. The user can define transformations to prepare the data

a. De-duplication, cleansing, combining data, pivoting, splitting rows/columns,

etc.

3. Run the transformation and export the data

a. Local file (typically CSV) or into Hadoop (Hive table or CSV file)

b. Alternatively export to BI Tool (e.g. Tableau Data Extract file)

Operationalize:

1. Schedule data prep transformations to generate new data files (à la ETL)

2. Publish results to collaboration environment

8

Data Prep Tools

9

Pros:

• Ease of use

• Iterative data transformation

• Very good with delimited files

• Sampling makes tools responsive

• Data profiling help detect ‘suspect’

data

Cons:

• Ad-hoc rather than operational

• Reuse is limited to collaborative data

sets

• Performance

• Consistency and governance – data

chaos?

Pros and Cons of Data Prep

10

Data Prep is great for ad-hoc discovery and analytics

• “I need to combine this with that and run it through my analytics application…”

Not so good for consistent, repeatable integration

• (Think: BI swim lanes)

But…

• Data Prep provides valuable knowledge that can be used in systematic data integration

Data Prep and Systematic Data Integration

11

Data Prep and Virtual Sandboxes

12

Data Blending

13

All you want to know about…

Data blending is about working with multiple sources of

data by preparing them and joining them together for a

specific use case at a specific time. It’s different from data

integration, because data blending is about solving a

specific use case, whereas data integration typically gives

you a single source of truth…

Leading Data Blending ‘Vendors’

• Tableau

• Microstrategy

• SAP Business Objects

• IBM Cognos

• Qlik View

• etc.

15

How Does it Work?

Defining the data blending ‘model’:

1. Connect to data sources

a. Databases, Data Warehouse (via ODBC or JDBC), Files (Excel, CSV, etc.),

Hadoop, NoSQL, etc.

2. Select data you want to use – a sample is usually loaded

3. Build model using graphical tool to create Joins, Unions, etc.

4. Run the model for the full data set

5. Build your report or dashboard

Operationalize:

1. Model can be saved and expose as a ‘data source’ (usually in a ‘server’)

2. Accessed by other users

16

Data Blending

17

Pros:

• Built into BI/visualization tools

• Graphical query designer

• Provides semantic layer on top of

data sources

• Quick time from ‘data to analysis’ i.e.

removes wait for IT to provision a

data mart or similar

Cons:

• Ad-hoc rather than operational

• Specific to each BI/visualization tool

• Performance

• Consistency and governance

Pros and Cons of Data Blending

18

Francois Ajenstat, Chief Product Officer, Tableau

There are two flows; the ad-hoc and the operational…where we are

coming from is…I just want to integrate these two sources. It's not

formalized, per se, it's not a project. I just want to connect this and

this and I want to analyze it. How do we go from data to analysis as

quickly as possible? And when you want to formalize it, operationalize

it, make it repeatable, then [you use other tools].

19

Data Catalogs

What, when, and how?

20

Data Catalogs provide capabilities that enable any user –

from analysts to data scientists to developers – to discover,

understand, and consume data sources. Data Catalogs

typically include a crowdsourcing model of metadata and

annotations, and allow all users to contribute their

knowledge to build a community and culture of data.

Leading Data Catalog Vendors

• Alation/Teradata

• Cambridge Semantics Anzo Platform

• Informatica Enterprise Information Catalog

• Microsoft Azure Data Catalog

• Waterline Data

22

How Does it Work?

Building catalog:

1. Connect to data sources and consumers

a. Extract and analyze ‘technical’ metadata

b. Sample data and build data profile

2. Use NLP and ML for ‘auto-titling’ – based on defined business glossary

3. Use expert sourcing to validate catalog entries

4. Use crowd sourcing to build veracity profile

Accessing catalog:

1. Search tools for ‘natural language’ searches

2. APIs for tool integration

23

Data Catalog

24

Pros:

• Great for analyzing data source and

inferring meaning from technical

metadata

• Gather ‘tribal knowledge’ about data

within organization

• Allow curation of metadata

• Provide single tool to find – and

understand - data

Cons:

• Do not address ‘data provisioning’ –

you need another tool for this

• File-based data?

Pros and Cons of Data Blending

25

Summary

Back to the swim lanes…

26

Business Intelligence ‘Swim Lanes’

27

Data Blending

Data Catalog

Data Prep

Data Virtualization

Q&A

Thank you!

© Copyright Denodo Technologies. All rights reservedUnless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.

O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A

#DenodoDataFest