obteniendo valor de toda la información: big data

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Jordi TrillCore Tech Business Development Manager

Oracle IbéricaVitoria, 17 Mayo 2015

Obteniendo valor de toda la información: Big Data

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | - Oracle Big Data Architecture

EnterpriseProductivity

DisruptiveTechnology


“[Facebook] started in the Hadoop world. We are now bringing in relational to enhance that. We're kind of going [in] the other direction ... We've been there, and [we] realized that using the wrong technology for certain kinds of problems can be difficult.”

Ken RudinDirector of AnalyticsFacebook

http://tdwi.org/Articles/2013/05/06/Facebooks-Relational-Platform.aspx?Page=1


Execution

Innovation

- Oracle Big Data Architecture

4th Generation Data Architecture for Big Data

WarehouseData FactoryReservoir

Data Streaming

Data Platform

Discovery Lab

Analytics

APIs

Enterprise Data

Other Data Sources

Data Streams

BusinessData

Social/LogData

Model FirstAnalytics

• Reporting-oriented• Often enterprise wide

in scope, cross LoB• “you know the

questions to ask”

Reports & Dashboards

Data FirstAnalytics

• Data Exploration• Highly visual and/or

interactive• “you don’t know the

questions to ask”

Discovery

• Telematics• Industry Services• Internet of Things• Sentiment

DataServices


Execution

Innovation


Comprehensive Oracle Solution for Big Data

WarehouseFactoryReservoir

Data Streaming

Data Platform

Discovery Lab

Analytics

APIs

Enterprise Data

Other Data Sources

Data Streams

BusinessData

Social/LogData




questions to ask”


Data FirstAnalytics



questions to ask”

Discovery


DataServices

ApacheOracleNoSQL

OracleCAF & OEP

Oracle Data Integration & Governance

Oracle Database& Big Data SQL

OracleR

OracleBig Data

Discovery

OracleBusiness

Intelligence

OracleBig Data

Discovery

Apache


Execution

Innovation


Integrated Oracle Engineered Systems for Big Data

Data Streaming

Data Platform

Discovery Lab

Analytics

APIs

Enterprise Data

Other Data Sources

Data Streams

BusinessData

Social/LogData




questions to ask”


Data FirstAnalytics



questions to ask”

Discovery


DataServices

APIs

Analytics


Execution

Innovation


Visionary Oracle Cloud for Big Data

Data Platform

Discovery Lab

Analytics

APIs

Enterprise Data

Other Data Sources

Data Streams

BusinessData

Social/LogData




questions to ask”


Data FirstAnalytics



questions to ask”

Discovery


DataServices

Data Streaming


Oracle Brings Business Value to Big Data

Enterprise-Grade Capabilities

Discover and Predict – Fast

Govern and Secure All Data

Simplify Access to All Data

Performance Integration Availability Scalability Manageability


Oracle Big Data


Discover and Predict, Fast



Build On Your FoundationHow can your team help ?

11

Data Warehouses

Surveys &Reviews

Social Media

3rd party data

Machine Data

Big Data

Enterprise Applications

Websites

Spreadsheets

Skills

Process

Infrastructure

Tools


Driving Business Value from Technology Innovation

Use the Right Tool for the Job and benefit from the Power of “AND”

RelationalHadoop NoSQL Graph

Run the Business Integrate existing

systems

Mission-critical

tasks

Use existing

investments

Ensure skills

relevance

Change the Business

Disrupt competitors

Disintermediate

supply chains

Leverage new

paradigms

Exploit new analyses

Scale the Business

Serve data

faster

Persist data streams

Meet mobile and

device challenges

Scale-out

economically

Link the Business

Associate complex

business entities

Link to Open Data

Share data sets via

Ontology

Evolve data and

schema together


Proven, Cost Effective Solution

“Oracle Big Data Appliance is an excellent choice for customers looking to work with the full suite of Cloudera’s leading Hadoop-based technology. It’s more cost-effective and quicker to deploy than a DIY cluster.”

⁻ Mike Olson, Cloudera founder, Chief Strategy Officer, and Chairman of the Board

Source:ESG White Paper

21%Cost Savings

33%Faster Time

to Value

http://www.oracle.com/us/corporate/analystreports/corporate/esg-bigdata-intel-2193858.pdf


SocialMedia

Expert System

Decision Engine

Oracle Event

Processing

Ejemplo Arquitectura Oracle Big Data

SmartSelling


SocialMedia

Expert System

Decision Engine

Oracle Event

Processing


SmartSelling

Fast Data Solution:

Oracle Event Procesing

Oracle Real Time Decisions


Oracle Event ProcessingCompleta dinámicamente el evento y procesa el dato

Una solución completa para construir aplicaciones para filtrar,

correlacionar y procesar eventos en tiempo real

Asegura que las aplicaciones a continuación son de inteligencia en

tiempo real

Ayuda a eliminar, consolidar, correlacionar y filtrar datos antes

que molesten al data warehouse

Analiza Streams de datos masivos en tiempo real

Base para el “Internet of Things” (en dispositivos)

Sólo envia al DataCenter información relevante


Arquitectura Real-Time DecisionsSistema Experto de Recomendaciones en tiempo Real

Eventos de Negocio candidatos

Acciones de Negocio propuestas Multicanal y Multi-propósito

Motor de Recomendaciones masivas en Tiempo Real

(“RECOMENDADOR”)

SocialSocial

Objetivos de Negocio

Marcados por Analistas

Reglas y autoaprendizaje, para mejor predicción del comportamiento

del cliente

Base Datos Clientes 360 º


Dell

Improve revenues and operations through personalized predictive analytics

$132M in net new revenue FY2012

20% increase profit margin per call

“The insights are amazing. You can really see customers' buying patterns and interests, how they change over time, and we can take action on that.”

Mark Surcrese, Marketing Director

Powered by Oracle Fast Data Solution


SocialMedia

Expert System

Decision Engine

Oracle Event

Processing


SmartSelling

Oracle Data Integrator

Data Quality

Golden Gate


Oracle Data Integration

Oracle GoldenGate

Captura cambios en los datos y eventos en

tiempo real

Sin impacto en el rendimiento de los sistemas origen de los datos


Aplica filtrados y transformaciones a los datos

de los puntos de captura

Transforma y Carga datos estructurados o no-

estructurados para el análisis

Non-invasive real-time feeds

Small batch windows

Aproximación No-Invasiva de Alto Rendimiento


Business Value of ODI: Low Cost and High Dev Efficiency

21

No ETL engine is required

Separation of Logical and

Physical design

Physical exec on SQL, Hive, Pig, or

Spark

Runtime exec in Oozie or via ODI

Java Agent

Rich set of pre-built operators

User defined functions


ETL Tool


• Offload long running ETL jobs to Hadoop

• New Sources via Hadoop

• Leave existing ETL in place

22

Staging

Temp

Files

Files

Detail

Fast load

Data Warehouse Sources

MR

MR

SQL

OracleGoldenGate



Cap

ture

Trai

l

Ro

ute

Del

iver

Pu

mp


Oracle GoldenGate for Big Data

GoldenGate

Real-timeData Delivery


SocialMedia

Expert System

Decision Engine

Oracle Event

Processing


SmartSelling

Big Data

HDFS

NoSQL

Conectores


Oracle NoSQL Database Escalable, Alta Disponibilidad, Base de Datos Key-Value

Application

Storage NodesDatacenter B

Storage NodesDatacenter A

Application

NoSQL DB Driver

Application

NoSQL DB Driver

Application

• Datos Key-value, JSON y RDF

• Large Object API

• Transacciones ACID

• Load Balancing Transparente

• Software y Soporte Empresarial

• Online Rolling Upgrade

• Online Cluster Management

• Construida con Berkeley DB

• Integrada con Event Processing

• Latencia predecible

Características


Oracle Big Data Connectors

Data LoadOracle Loader for Hadoop

Data AccessOracle SQL Connector for

HDFS

R AnalyticsOracle R Advanced Analytics

on Hadoop

Oracle Data Integrator Knowledge Modules

XML/XQueryOracle XQuery on Hadoop

XQueryR Client

Optimized for Hadoop

Maximum parallelism

Fast performance

Analyze all your data in-place


SocialMedia

Expert System

Decision Engine

Oracle Event

Processing


SmartSelling

DWH Oracle DB

Olap, Spatial,

Avd. Analytics,

Data Mining & R

InMemory Opt.

BigData SQL


BOTH row and column

in-memory formats for

same table

Simultaneously active and transactionally

consistent

Analytics & reporting use

New Column format

Billions of rows/sec scan rate per CPU

core

OLTP uses row format

In-Memory Option: Dual Format In-Memory Database

Column

Format

Memory

Row

Format

Memory

AnalyticsOLTPSales Sales


Oracle In-Memory Columnar Technology

Memory

Pure Columnar

Pure in-memory format with no logging (

no writes to disk )

Near zero overhead on changes – even

for OLTP

New memory-optimized compression

format – 2x to 10x

Data loaded in-memory for active tables or

partitions

On startup or first access


Oracle Advanced Analytics

In-database data mining algorithms and open source R algorithms

– Statistical Functions

– Data Mining & Predictive Analytics

– Text Search & Text Mining

– Spatial & Graph Analysis

– In-Database MapReduce

SQL, PL/SQL, R languages

Scalable, parallel in-database execution

Workflow GUI and IDEs

Enables enterprise analytical applications

El más rápido despliegue de Análisis Predictivo y Data Mining


R Statistical Programming Language

Open source language and environment

Used for statistical computing and graphics

Strength in easily producing publication-quality graphs

Highly extensible


Oracle Big Data SQL

One fast query, on all your data.

Oracle SQL on Hadoop, NoSQL and beyond, with a Smart Scan service inspired by Exadata and the security and certainty of Oracle Database

Use Existing Skills, Process and Tools to Deliver Big Data Apps and Analytics


Data Lifecycle Management & Query OffloadMore data on-line and available at a lower cost

Move Partition to BDA

Oracle Big Data SQL

Rolling 13 months

Month 14-n

Big Data Rolling Windows

• Process

• Copy older partition to BDA

• Update views

• Drop older Exadata partition

• Offloaded data can be accessed via Oracle & Hadoop

• No Application changes required

OracleData

PCI FLASH

DRAM

Warm Data

Hottest Data

Active Data

Hadoop Data

Deep Data


SocialMedia

Expert System

Decision Engine

Oracle Event

Processing


SmartSelling

Oracle BI

Foundation &

BigData

Discovery


Oracle Big Data





Not Easy to Get Analytic Value at Fast Enough Pace

37

Tool Complexity

80% evaluating and preparing data

Data Uncertainty

Dependent on highly skilled resources


BigData Discoverybenefits

• Get Started Fast

• Simplify BigData analysis

• Justify project investment and get results

• Spend 80% of your effort in Analytics

• Leverage more of your analytics talent

find explore transform discover share


Exadata

Oracle Business Intelligence Foundation Suite

BI Server - Modelo Común de Información Corporativa

• Metadatos Comunes sobre todas las fuentes• Servicios Comunes de Seguridad, Control de Acceso, Autorización, Auditoría• Servicios Comunes de Acceso Optimizado a Datos y Generación de Consultas• Servicios Comunes de Clustering, Gestión y Despliegue• Sistemas y Gestión del Ciclo de Vida Comunes

Dashboard –

Cuadros Mando

Publisher –

ReportingAnswers –

Análisis

Delivers –

Alertas

Office Plug-In

IntegraciónScorecard Mobile SimulaciónEssbase

Endeca Information Discovery

Endeca

Server

Bigdata Appliance

Adaptive In-

Memory Tools

In-memory

Software

In-Memory

Hardware


Toda la Funcionalidad con Sist. Ingeniería ConjuntaSólo Oracle provee Sistemas de Ingeniería completos

Oracle Big Data Appliance

Oracle Exadata

InfiniBand

Acquire Organize Analyze & VisualizeStream

Oracle Exalytics

InfiniBand


Oracle Big Data





Data Governance is Not Easy, Hadoop is No Silver Bullet!

Data Governance

Metadata Management

Business Glossary

Data Profiling

Data Cleansing

Data Archiving

Data Privacy

PEOPLE

PROCESS TECHNOLOGY

…people and process first, …tools and capabilities next, …and, there is no magic!

“…the overall impact of poor-quality data on the whole dataset remains the same. In addition, much of the data that organizations use in a big data context comes from outside, or is of unknown structure and origin. This means that the likelihood of data quality issues is even higher than before. So data quality is actually more important in the world of big data."

- Ted Friedman, Gartnerhttp://www.gartner.com/newsroom/id/2854917


Comprehensive Governance

FastLoad

Speed Layer

Batch Layer

DataGovernanceFoundation

Enterprise Data Quality(Profile & Cleanse)

Enterprise Metadata Management & Business Glossary(Business Glossary, Data Lineage, Impact Analysis and Data Provenance)

Veridata(Verify)

Data Enrichment(Prepare)

Data Governance– Prepare unstructured data– Profile data with sampling– Clean data in real time or batch– Verify data for consistency– Trace lineage of all data– Define glossary of business terms


Oracle Enterprise Metadata ManagementTrust your data

Executives Dashboards / Reports

!=

!!!???

•Dashboard does not match report?

•Where did this data come from?

•Can I trust my other dashboards and reports?


Value of Enterprise Metadata Management

47

ETL

BIDashboards

App

ETL

ETL

How was sales figure calculated?

What will happen if I change this

table?

What reports use the mainframe

data? Sys Admin

Executive

BI Developer

Where did this data

come from?

Application User

Which reports use this

customer data?

CDC

Data Reservoir

Data Steward

Can I trust the sources of this

customer data?

ETL

Developer

Solves a significant pain point for a wide variety of business consumers and technical staff

I want to design an experiment to measure the

success of a signup page. What data do I have?

Data Scientist

GG

http://hadoop.apache.org/

http://hadoop.apache.org/


Govern & Secure All your DataHadoop, NoSQL & Relational

• Existing Capability– Authentication through Kerberos

– Authorization through Apache Sentry

– Auditing through Oracle Audit Vault

– Encryption for Data-at-Rest

– Network Encryption

• Big Data SQL adds– Advanced Security on Hadoop & NoSQL• Masking and Redaction

– Virtual Private Database• Fine-grain Access Control

49


Oracle: Big Data para la Empresa

• Solución Completa– Incluye BigData, FastData y Analytics para todos los datos

• Optimizada para el Extreme Analytics–Data Discovery y Análisis Predictivo con acceso a todos los datos

• Engineered to Work Together–Elimina el riesgo del despliegue y del soporte

• Enterprise Ready–Rendimiento y Escalabilidad Extremos


Gracias!!

obteniendo valor de toda la información: big data

Data & Analytics