introduction to oracle big data discovery

Oracle Big Data DiscoveryCON9101 - Introduction to Oracle Big Data Discovery

Enterprise Use of Hadoop Continues to Mature

3%

21%

More Hadoop

0%

20%

40%

60%

80%

Hadoop Workloads

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

76%

Less Same More

2

0%

ETL Data Science Business

Analytics

Today Future

Only 3% say they will do less with Hadoop over

the next 12 months

While ETL rules Hadoop’s first phase, Business Analytics will be the focus of

Hadoop’s second phase

2015 Hadoop Maturity Survey, conducted by AtScale Inc.

Not Easy to Get Analytic Value at Fast Enough Pace

80% effort typically

spent on evaluating

and preparing data

Data Uncertainty

• Not familiar and overwhelming

• Potential value not obvious

• Requires significant manipulation

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 3

Tool Complexity

• Early Hadoop tools only for experts

• Existing BI tools not designed for Hadoop

• Emerging solutions lack broad capabilities

Overly dependent on

scarce and highly

skilled resources

Requires a Fundamentally New Approach

A single intuitive and visual user interface, to...


quickly transform

and enrich it to make

it better

unlock big data for

anyone to discover

and share new value

find and explore big

data to understand its

potential

find explore transform discover

Oracle Big Data Discovery. The Visual Face of Hadoop


find explore transform discover

Find

• Access a rich, interactive catalog of all data in Hadoop

• Familiar search and guided navigation for ease of use


ease of use

• See data set summaries, user annotation and recommendations

• Provision personal and enterprise data to Hadoop via self-service

Explore

• Visualize all attributes by type

• Sort attributes by information potential


• Assess attribute statistics, data quality and outliers

• Use scratch pad to uncover correlations between attributes

• Intuitive, user driven data wrangling

• Extensive library of powerful data transformations and

Transform


transformations and enrichments

• Preview results, undo, commit and replay transforms

• Test on sample data then apply to full data set in Hadoop

• Join and blend data for deeper perspectives

• Compose project pages via drag and drop

Discover


drop

• Use powerful search and guided navigation to ask questions

• See new patterns in rich, interactive data visualizations

Oracle Big Data Discovery. Technical Innovation on Hadoop

Oracle Big Data Discovery Workloads

Hadoop Cluster(BDA or Commodity Hardware)

BDD node

name node

In-Memory Discovery Indexes• For keyword search, faceted navigation and analytics

• Indexes data and organizes as key value pairs

Web Studio• Catalog, explore, transform and discover UI’s• Self service data provisioning to Hadoop• Metadata configuration and administration

Hadoop 2.x

Other Hadoop Workloads

MapReduce

Spark

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 10

data node

data node

data node

data node

name node

BDD Data Processing (Spark on YARN)• Profiling: catalog entry creation, data type &

language detection, schema configuration

• Sampling: dgraph (index) file creation

• Transforms: >150 data wrangling functions

shipped out-of-the-box

• Enrichments: location (geo), text (cleanup,

sentiment, entity, key-phrase, whitelist tagging)

Self-Service Provisioning & Data Transfer• Personal Data: Import from JDBC, CSV and XLS to HDFS

Indexes data and organizes as key value pairs

• In-memory, columnar, multi-core architectureHadoop 2.x

Filesystem(HDFS)

Workload Mgmt(YARN)

Metadata(HCatalog) Hive

Pig

Oracle Big Data SQL (BDA only)

Oracle Big Data Discovery. The Visual Face of HadoopOracle Big Data Discovery. The Visual Face of Hadoop

www.oracle.com/bigdatadiscovery

introduction to oracle big data discovery

Technology