sap hana data warehousing foundationa248.g.akamai.net/n/248/420835/cfaf8220e1d4cc817c... · sap...
TRANSCRIPT
This is the current state of planning and may be changed by SAP at any time.
SAP HANA Data Warehousing Foundation Data Distribution Optimizer / Data Life Cycle Manager – DWF SP03
February, 2016
This is the current state of planning and may be changed by SAP at any time.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 2Public
Disclaimer
This presentation outlines our general product direction and should not be relied on in making a
purchase decision. This presentation is not subject to your license agreement or any other agreement
with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to
develop or release any functionality mentioned in this presentation. This presentation and SAP's
strategy and possible future developments are subject to change and may be changed by SAP at any
time for any reason without notice. This document is provided without a warranty of any kind, either
express or implied, including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this
document, except if such damages were caused by SAP intentionally or grossly negligent.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 3Public
SAP HANA Data Warehousing Foundation Tools in detail
SAP HANA Data Warehousing Foundation 1.0
delivers specialized SAP HANA XS based
applications
Data Distribution Optimizer to plan, adjust and
analyze landscape reorganizations for SAP HANA
scale out systems
Data Lifecycle Manager to deliver a possibility to
archive / displace data from a SAP HANA persistency
to Dynamic Tiering, SAP IQ or Hadoop
Data Warehouse Monitor to provide a
comprehensive overview about current and past
activities in the data warehouse
Data Warehouse Scheduler to maintain
dependencies between single processes with the
focus to provision data warehouse models
SAP HANA
Data Warehousing
Foundation
Data Lifecycle Manager
Data Warehouse Monitor **
Data Warehouse Scheduler **
• DWF 1.0 SP00: DDO >= HANA Rev. 83
• DWF 1.0 SP01: DDO+DLM >= HANA Rev. 96
• DWF 1.0 SP02: DDO+DLM >= HANA Rev. 101
• DWF 1.0 SP03; DDO+DLM >= HANA Rev. 110 ** Outlook
Data Distribution Optimizer
This is the current state of planning and may be changed by SAP at any time.
Data Distribution Optimizer
(DDO)
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 5Public
Functionalities
SAP HANA Scale Out Landscape Overview with respect to system
configuration and data distribution
Specify different configurations for reorganization on different
levels of granularity (e.g. DB-Schema, Table, Table Group, Table
Subtype, Location Group)
Create, adjust and simulate different reorganization plans to
achieve an optimal data distribution in an interactive fashion.
Repeat the steps until a sufficient reorganization plan has been
created.
Graphical visualization of reorganization plans by comparing the
simulated with the actual system landscape
Export and import reorganization plans across system boundaries
(-> DDO remote mode deployment)
Execute & Schedule SAP HANA reorganization runs
Analyze reorganization plan steps and logs of planned and
executed SAP HANA landscape reorganization runs
Join Path Analysis to identify associated database tables in the
system and create a proposal for table grouping (SP03)
SAP Data Warehousing Foundation Data Distribution Optimizer
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 6Public
SAP Data Warehousing Foundation DDO SPS03 - Join Path Analysis (JPA)
• Identify associated database tables and make a proposal which are frequently used together are going to be placed
together (via Table Grouping)
• Use the proposal to assign the grouping information to the
database tables to significantly improve the placement of
tables, so that table joins are executed with an adequate level
of speed.
• Based on the data available in the statement cache
(HOST_SQL_PLAN_CACHE view of the _SYS_STATISTICS
schema) JPA uses currently the execution count of a
statement for prioritizing joins
• Only executed SQL statements that access tables or views will be
analyzed.
• SQL statements that haven't been executed yet or view definitions are
not considered for join path analysis. This means that depending on the
time of execution the results of the join path analysis may differ.
Table 1
Table 2 Table 7
Table 3
Table 8
Table Group 1 Table Group 2
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 7Public
SAP Data Warehousing Foundation DDO SPS03 - Join Path Analysis (JPA) - (Tabular Display)
Leverage the Grouping Granularity
Slider:
When increasing the granularity, the
groups get smaller (less tables within
a group) and the number of groups
increases.
When decreasing the granularity, the
groups get larger (more tables within a
group) and the number of groups
decreases.
The goal is to find a trade-off that has
enough groups so that the system can
be balanced well-enough, while at the
same time the number of cross-node-
joins is minimized.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 8Public
SAP Data Warehousing Foundation DDO SPS03 - Join Path Analysis (JPA) - (Chart Visualization)
Adjustment to the Grouping
Granularity Slider would
apply for both Tabs:
The related Group(s) are
highlighted when selecting a
Group (here: Group 131)
representing the JPA
derived dependencies.
Further drill-down
capabilities are supported
within the Bubble-Chart.
The Tables + Table Size
listed in the “Tables in
Group xxx” box are derived
based on the selected
Group in the Bubble-Chart
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 10Public
Planned Innovations Future DirectionToday
SAP Data Warehousing Foundation Product Roadmap Overview – Data Distribution Optimizer
Future innovations
• Further enhancements of DDO
• Support and optimize
table partitioning in native
SAP HANA cases
• Flexible management of hosts
• Deeper insights and simulations
for data distributions• Additional KPIs
Q4 / 2015
SAP DWH Foundation 1.0 SP03
Data Distribution Optimizer (DDO)
• Deeper insights and simulations
for data distributions
• Expected duration
• Modification of proposals
• Improved Plan Generation runtime
• Workload analysis (e.g. plan cache
analysis)
• Analysis of SQL plan cache / participating
tables
• Improvements to the simulation of
landscape reorganization runs
Q2 / 2016
Upcoming planned release
Data Distribution Optimizer (DDO)
• Parametrization of JPA (e.g.
SCHEMA)
• Integration of JPA with Plan
Generation
• Optimal Group Granularity Detection
• Maintenance of Table Group
assignments for native HANA
Datamarts
This is the current state of planning and may be changed by SAP at any time.
Data Lifecycle Manager
(DLM)
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 12Public
SAP Data Warehousing Foundation DLM SPS03 - Graphical Exploration Module (Drill Down, Top 10)
• If the utilization forecast on the Home screen indicates that the utilization of one or more nodes could become
critical over time, you can use the exploration function to establish which tables are appropriate candidates for
data relocation.
Context
With a two-step approach, you can first
perform graph-based exploration on
different levels. With a sunburst chart, you
can drill down to information on different
levels, host, schema, or table level for
example. You can thus identify the ten
largest tables of all the tables in the
system, or the ten largest tables in a
given schema on a given host for
example. In a second step, you can use
form-based exploration to see how data
of a selected SAP HANA table is spread
according to a selected column filter. With
this insight, you can derive meaningful
business rules for relocating the data.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 13Public
SAP Data Warehousing Foundation DLM SPS03 - Home Screen: KPIs
• Additional KPI: Forecast „HANA: Disk“
• Forecast „HANA: Disk“ in 100 days vs.
current utilization in addition to
Forecast „HANA: Memory“ in 100
days vs. current utilization
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 14Public
SAP Data Warehousing Foundation DLM SPS03 - Manage Storage Destinations
A storage destination specifies an instance of the storage stack which data can be relocated to with Data Lifecycle Manager.
Supported storage destination types are SAP HANA Dynamic Tiering, Spark SQL (DESTINATION) or SAP IQ for example. For
deletion of data, Data Lifecycle Manager offers storage destination type Deletion Bin.
A storage destination is assigned to a lifecycle profile. It is used there to define to which destination persistence the data will be
relocated to.
The Manage Storage Destinations screen of the Data Lifecycle Manager is split up into two main areas:
On the left side of the Storage Destinations area, you see a list of all available storage destinations. Here, you can search for
storage destinations by storage destination name or the user who last changed the destination. You can add new destinations or
delete destinations that are not needed anymore.
Note:
The instances of the storage stack have
to be installed and configured properly
before you can use them for defining
storage destinations.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 15Public
SAP Data Warehousing Foundation DLM SPS03 - Supported Storage Destination Types
Storage Destination Type Restrictions
SAP HANA Dynamic Tiering Local The following factors prevent lifecycle profile from being activated:
• Source tables with keys of type
VARCHAR/CHAR/VARBINARY/BINARY and length greater than 255
• Source tables with keys of type NVARCHAR/NCHAR and length
greater than 85
• Source tables of type row
• Source tables with columns with data type BLOB
• Source tables without a primary key
SAP IQ over SDA
Spark SQL (DESTINATION) The following factors prevent the lifecycle profile from being activated:
• Source tables with columns with data type TEXT, SHORTTEXT,
BINTEXT, BLOB, CLOB, TIME, VARBINARY, NCLOB, or ALPHANUM
Deletion Bin Destination The following factors prevent the lifecycle profile from being activated:
• Source tables without a primary key
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 16Public
SAP Data Warehousing Foundation DLM SPS03 - Hadoop Storage Destination
Depending on the type of stores you would like to use for relocating data:
• The SAP HANA Spark Controller has been installed and configured, and added as Remote Source
to SAP HANA. The SAP HANA system has been configured for Spark connectivity.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 17Public
SAP Data Warehousing Foundation DLM SPS03 - Hadoop Storage Destination (cont.)
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 18Public
SAP Data Warehousing Foundation DLM SPS03 - Manage Modeled Persistence Objects
• Modeled persistence objects help you treat multiple source persistence objects, like tables, jointly in matters of
data relocation. You can define a source persistence object from multiple SAP HANA tables. Such a modeled
persistence object can be used in a lifecycle profile. The relocation rule in this case can be defined for common
columns, which have been selected during definition of the modeled persistence object.
Note:
Modeled Persistence Objects are supported for storage destination types
SAP IQ over SDA and SAP HANA Dynamic Tiering.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 19Public
SAP Data Warehousing Foundation DLM SPS03 - Destination Persistence Modeled Persistence Objects
Manage Lifecycle Profile –
Destination Persistency:
Visibility of the Data Flow(s) of the
defined Modeled Persistence
Objects SAP HANA Table Group
(here: YAM_ORDER_HDR +
YAM_ORDER_ITM)
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 20Public
SAP Data Warehousing Foundation DLM SPS03 - Rule Editor: Manage Modeled Persistence Objects
When you create and execute a lifecycle profile
for a modeled persistence object, the following
applies for all tables of the modeled object:
• A common relocation rule will be defined and
executed.
• A common clash strategy and packet size applies.
• A single relocation procedure will be generated.
• For all tables corresponding target tables will be
generated.
An example for a modeled persistence object is a
table group. A table group is a set of SAP HANA
tables with common columns. Table groups are
supported for storage destination types SAP IQ
over SDA and SAP HANA Dynamic Tiering Local.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 21Public
SAP Data Warehousing Foundation DLM SPS03 - Import / Export of DLM Entities
• Data Lifecycle Manager entities like storage destinations, modeled persistence objects, or lifecycle
profiles can be described in JSON format. For transport or backup purposes, you can export and
import the entities in JSON format.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 22Public
SAP Data Warehousing Foundation DLM SPS03 - Lifecycle profile: Nominal key
• For a source persistence type SAP HANA table, you specify a schema and table name for example.
Additionally, for tables without a primary key, you can specify a nominal key, which is treated like a
primary key by Data Lifecycle Manager. Using the nominal key, you can activate a life cycle profile
for a table without primary key.
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 23Public
Planned Innovations Future DirectionToday
SAP Data Warehousing Foundation Product Roadmap Overview – Data Lifecycle Manager
Future innovations
• Integration with SAP HANA Vora
• Pre-delivered Application DLM
templates
• Support for multi-tiering
• Integrated lifecycle management
• Exploration module based on SQL
Query Access Cache
• Parallelization of relocation packet
processing
• Support for HANA CDS Entities as
source persistence object
• Automated substitution of data
foundation in Calculation Views
(table replaced by generated view)
Q4 / 2015
SAP DWH Foundation 1.0 SP03
• Support for Table Groups
• Generate DLM profiles for a predefined
set of tables (table group)
• Graphical Exploration module for HANA
Table
• SPARK (SQL): HADOOP as storage
destination
Q2 / 2016
Upcoming planned release
• Table Hierarchy support (Foreign
Key Relationship)
• Rule Editor enhancements
(Semantic checks / Customer exit)
• Enhanced support for Table
Hierarchies and Table Groups for
certain storage destinations
• Feature completeness for HADOOP
as storage destination (packet size,
clash strategy)
• Adapt Relocation Agent module for
Table Groups & Table Hierarchies
for supported persistent storage
destinations
This is the current state of planning and may be changed by SAP at any time.
Appendix
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 25Public
1. Data Warehousing Foundation (DWF) SAP Help Page : http://help.sap.com/hana_options_dwf
2. SAP HANA DWF in SCN: http://scn.sap.com/docs/DOC-62482
3. Product Availability Matrix (PAM): https://support.sap.com/release-upgrade-maintenance/pam.htm
4. SWDC: https://support.sap.com/software/installations
1. -> A – Z Index -> SAP HANA Data Warehousing Foundation
5. Support / Incident Components
1. Component HAN-APP-DWS - for all SAP HANA Data Warehousing Foundation Option issues.
2. Component HAN-APP-DWS-DDO – for DDO related issues
3. Component HAN-APP-DWS-DLM – for DLM related issues
6. SAP HANA Academy Channel - Data Warehousing Foundation (DWF)
https://www.youtube.com/playlist?list=PLkzo92owKnVxZcLRGmfZS-bdlIFYRQ39R
7. DDO System Meta-Data Extraction: SAP Note 2138117
8. SAP DWF 1.0 Release SAP Note 2092669
9. SAP DWF 1.0 SP03 DDO Patch 1: 2259749
10.SAP DWF 1.0 SP03 DLM Patch 1: 2259304
SAP Data Warehousing Foundation Additional Information on DWF
© 2015 SAP SE or an SAP affiliate company. All rights reserved.
Thank you
© 2015 SAP SE or an SAP affiliate company. All rights reserved. 27Public
© 2015 SAP SE or an SAP affiliate company. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate
company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.
National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its
affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and
services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as
constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop
or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future
developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time
for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-
looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place
undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.