aufbau eines semantic layers zwischen db und hadoop? .oracle certified professional exadata...

Download Aufbau eines Semantic Layers zwischen DB und Hadoop? .Oracle Certified Professional Exadata Certified

Post on 28-Aug-2018

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Aufbau eines Semantic Layers zwischen DB und Hadoop?

    DOAG Konferenz Nrnberg, 18.11.2015 Matthias Fuchs

    Public Company Confidential Customer Confidential Sensitive

  • ber mich

    10+ Jahre Erfahrung mit Oracle

    Oracle Certified Professional

    Exadata Certified

    Oracle Engineered Systems

    Exadata

    Exalytics

    Big Data

    Exalogic

    DWH, Hadoop, Monitoring, Audit

    Senior Solution Architect matthias.fuchs@capgemini.com

    Twitter: @hias222

  • Copyright Capgemini 2015. All Rights Reserved

    3 Hadoop_semantic_layer_15_11.pptx

    Agenda

    ber Capgemini

    Semantic Layer

    Big Data - Daten und Tools

    Datenbank - Hadoop Kombination

    Schlussfolgerung

    Ausblick Analytic Views

    Demo Big Data SQL

  • Copyright Capgemini 2015. All Rights Reserved

    4 Hadoop_semantic_layer_15_11.pptx

    Agenda

    ber Capgemini

    Semantic Layer

    Big Data - Daten und Tools

    Datenbank - Hadoop Kombination

    Schlussfolgerung

    Ausblick Analytic Views

    Demo Big Data SQL

  • Capgemini eine starke Gruppe

    Umsatz nach Branchen* Umsatz nach Geschftsbereichen*

    Telecom, Media

    & Entertainment

    Other Managed

    Services

    Local

    Professional

    Services

    Consulting Services

    Application

    Services

    Energy, Utilities

    & Chemicals

    Others

    Public Sector

    Manufacturing,

    Automotive &

    Life Sciences

    14%

    4%

    7% 19%

    16%

    23%

    4%

    58% 23%

    15%

    Cap Gemini S.A. ist im CAC 40 gelistet;

    Paris, ISIN code: FR0000125338

    Unsere Marke ist Capgemini, an der Pariser Brse sind

    wir unter Cap Gemini S.A. gelistet.

    Financial

    Services

    Copyright Capgemini 2015. All Rights Reserved

    5 Hadoop_semantic_layer_15_11.pptx

    17%

    Customer Products,

    Retail, Distribution &

    Transportation

    Operative Marge : 970 Mio.

    Operativer Gewinn : 853 Mio.

    Jahresgewinn : 580 Mio.

    Netto-Barmittel und bargleiche Mittel : 1.22 Mrd.

    Umsatz 2014: 10,57 Mrd.

    * Stand: 1. Halbjahr 2015 * Stand: 1. Halbjahr 2015

  • In ber 40 Lndern engagieren sich 180.000 Mitarbeiter fr unsere Kunden (Stand Juli 2015)

    Kanada

    USA

    Mexico

    Brasilien

    Argentinien

    Europa

    Marokko

    Australien

    China

    Indien

    Chile

    Guatemala

    Singapur

    Philippinen

    Taiwan

    Vereinigte Arabische

    Emirate

    Mitarbeiter Offshore 96.000

    Malaysia

    Neuseeland

    Japan

    Sdafrika

    Kolumbien

    Vietnam

    Copyright Capgemini 2015. All Rights Reserved

    6 Hadoop_semantic_layer_15_11.pptx

  • Capgemini kombiniert seine hohe fachliche Kompetenz mit fundiertem Branchen-Know-how

    Ausgewhlte Referenzkunden

    Copyright Capgemini 2015. All Rights Reserved

    7 Hadoop_semantic_layer_15_11.pptx

    Automotive Public Sector

    Telecom, Media & Entertainment

    Manufacturing, Retail &

    Distribution

    Financial Services

    Energy, Utilities & Chemicals

    http://www.de.capgemini.com/insights/referenzen/bundesagentur-arbeit-it-beratung-arbeitslosengeld/http://www.de.capgemini.com/insights/referenzen/bayerische-versorgungskammer-altersvorsorge/http://www.de.capgemini.com/insights/referenzen/aok-aufbau-softwarehaus/http://www.de.capgemini.com/insights/referenzen/deutsche-telekom-t-online/http://www.de.capgemini.com/insights/referenzen/vodafone-netherlands-customer-management/http://www.de.capgemini.com/insights/referenzen/telefonica-o2-go-to-market-geo-marketing/http://www.de.capgemini.com/insights/referenzen/bmw-data-warehouse/http://www.de.capgemini.com/insights/referenzen/daimler-global-ordering/http://www.de.capgemini.com/insights/referenzen/volkswagen-gruppe-service42/http://www.de.capgemini.com/insights/referenzen/johnson-controls-operational-forecasting/http://www.de.capgemini.com/insights/referenzen/allianz-insurance-customer-service/http://www.de.capgemini.com/insights/referenzen/thyssen-krupp-anlagenplanung/http://www.de.capgemini.com/insights/referenzen/tchibo-internationale-handelsplanung-platin/http://www.de.capgemini.com/insights/referenzen/schenker-transportmanagement-luft-see/http://www.de.capgemini.com/insights/referenzen/dhl-business-mobile-communications-devices/http://www.de.capgemini.com/insights/referenzen/syngenta-b2b-portal-sap-crm-sap-bw-sap-erp/http://www.de.capgemini.com/insights/referenzen/lanxess-globales-reporting-sap/http://www.de.capgemini.com/insights/referenzen/deutsche-post-elektronisches-auftragsmanagement/http://www.de.capgemini.com/insights/referenzen/raiffeisenbank-post-merger-integration/

  • Copyright Capgemini 2015. All Rights Reserved

    8 Hadoop_semantic_layer_15_11.pptx

    Agenda

    ber Capgemini

    Semantic Layer

    Big Data - Daten und Tools

    Datenbank - Hadoop Kombination

    Schlussfolgerung

    Ausblick Analytic Views

    Demo Big Data SQL

  • Definition Semantic Layer

    Copyright Capgemini 2015. All Rights Reserved

    9 Hadoop_semantic_layer_15_11.pptx

    A semantic layer is a business representation of corporate data that helps end users access data autonomously using common

    business terms. Developed and patented by Business Objects, it maps complex data into familiar business terms such as product, customer, or revenue to offer a unified, consolidated view of data

    across the organization.

    Unified Semantic View of Information: The Oracle BI Foundation Suite allows an organization to model the complex information sources of their business as a

    simple, semantically unified, logical business model. It provides facilities to map complex physical data structures including tables, derived measures, and OLAP

    cubes into business terms - abstracting how a business user expresses calculations. It translates familiar, easy to understand business concepts into the

    technical details required to access the information. The Oracle BI Foundation Suite is unique in the market because it defines an enterprise semantic layer that

    spans across the unified enterprise view of information.

    Quelle: http://www.oracle.com/us/obiee-11g-technical-overview-078853.pdf

  • Semantic Layer

    Business Intelligence Semantic Layer Produkte

    Copyright Capgemini 2015. All Rights Reserved

    10 Hadoop_semantic_layer_15_11.pptx

    Quelle: http://www.datasciencecentral.com/profiles/blogs/its-time-to-unleash-the-semantic-layer

  • Semantic Layer Beispiel OBIEE

    Copyright Capgemini 2015. All Rights Reserved

    11 Hadoop_semantic_layer_15_11.pptx

  • Copyright Capgemini 2015. All Rights Reserved

    12 Hadoop_semantic_layer_15_11.pptx

    Agenda

    ber Capgemini

    Semantic Layer

    Big Data - Daten und Tools

    Datenbank - Hadoop Kombination

    Schlussfolgerung

    Ausblick Analytic Views

    Demo Big Data SQL

  • Hadoop Polyglot Persistence

    Copyright Capgemini 2015. All Rights Reserved

    13 Hadoop_semantic_layer_15_11.pptx

    Hadoop

    SQL

    Not only SQL

    Document DB

    Key, Value

    Hadoop

    File System

    Bilder, Texte

    ...

    RDBMS

    Relational

    SQL

    Resource Description Framework

    Web Semantic

    Tripel

  • Big Data - Tools

    Copyright Capgemini 2015. All Rights Reserved

    14 Hadoop_semantic_layer_15_11.pptx

    Quelle: https://www.linkedin.com/pulse/100-open-source-big-data-architecture-papers-anil-madan

    Metadata HCatalog

    Engines Interactive, Batch

  • HCatalog

    Copyright Capgemini 2015. All Rights Reserved

    15 Hadoop_semantic_layer_15_11.pptx

    Entstanden im Rahmen von Hive

    Erstellt einen View/Beschreibung auf die Daten im Hadoop Filesystem

    Stellt die Daten anderen Applikation zur Verfgung

    HCatalog supports

    RCFile, Parquet, OCR

    CSV

    JSON

    SequenceFile (Key, Value)

    Avro (Binary)

    Hcatalog Erweiterung

    InputFormat, OutputFormat

    SerDe (Serializer Deserializer)

    Z.B. Datapump Files

  • Process Engines

    Copyright Capgemini 2015. All Rights Reserved

    16 Hadoop_semantic_layer_15_11.pptx

    General Purpose Processing Frameworks, Apache Projekte

    MapReduce Erstes Process Framework auf Hadoop, Batchverarbeitung

    Tez schneller als MapReduce, interaktive Datenverarbeitung, in Memory Verarbeitung, Integration in YARN

    Spark Performance hnlich Tez, auch Standalone mglich

    Weitere wie z.B. Flink Humboldt Uni Berlin

    Auf Basis der Engines laufen viele SQL Frameworks, das wichtigste Hive

    Es gibt auch SQL Frameworks mit eigener Process Engine wie z.B. Impala

  • Copyright Capgemini 2015. All Rights Reserved

    17 Hadoop_semantic_layer_15_11.pptx

    Agenda

    ber Capgemini

    Semantic Layer

    Big Data - Daten und Tools

    Datenbank - Hadoop Kombination

    Schlussfolgerung

    Ausblick Analytic Views

    Demo Big Data SQL

  • SQL Zugriff aus Sicht von Hadoop

    Copyright Capgemini 2015. All Rights Reserved

    18 Hadoop_semantic_layer_15_11.pptx

    Hive

    MapReduce Spark* Tez

    Hadoop Storage HDFS HBase, Kudu **

    HA

    WQ

    Imp

    ala

    Drill

    Processing Layer

    SQL Queries

Recommended

View more >