optimal deployment of sap hana on ibm power platform ... · pdf filefor selected customers in...

© Copyright IBM Corporation 2015. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.

Optimal Deployment of SAP HANA on IBM Power Platform:

Early Customer Experiences

Damir Rubic

IBM / SAP Power Systems Technical Sales

[email protected]

SAP TechEd in Las Vegas, October 19–23, 2015

mailto:[email protected]

1IBM Systems Technical University, October 5-9 | Hilton Orlando

© Copyright IBM Corporation 2015. Technical University/Symposia materials

may not be reproduced in whole or in part without the prior written permission of

IBM.

Agenda:

• IBM Systems Solutions for SAP HANA

• HANA – A Brief Architectural Overview

• IBM & SAP HANA Stack Overview

• TDI Deployment Model

• Power 7+ & Power 8 Performance

• Landscape & Sizing considerations

• HA & DR Solution(s) for SAP HANA

• Backup / Restore Solutions





IBM.

The SAP HANA platform - More than just another database


Source: SAP, 2014




IBM.

Source: SAP, 2014





IBM.

Column-Oriented Storage

• Data in columnar model is kept in columns

• Since data in a single column is almost always homogeneous it's frequently compressed which often provides

for dramatic reduction in memory consumption.

• Aggregate functions are very fast on columnar data model since the entire column can be fetched very quickly

and effectively indexed.

• Inserts, updates and row functions, however, are significantly slower than their row-based counterparts as a

trade-off of columnar approach (inserting a row leads to multiple columns inserts)

• Assistance provided by Delta Merge Process





IBM.

History - Development activities

August 2013 SAP initiates Product Development

Program for HANA on IBM Power technology

December 2013 HANA code optimized compiled/linked (HANA SPS08, SLES11SP2)

February 2014 All HANA Servers running, IBM code checked in, full function

testing commencing, Power 8 optimization agreed

June 2014 SaphireNow/Bernd Leukert announces “Test and Evaluation Program”

for selected customers in 3Q 2014

October 2014 SAP TechEd && d-code/Bjoern Goerke announces

“HANA on POWER SAP Ramp-Up Program” beginning 1Q2015

November, 2014 BW 7.31+ TEA code ships to customers

March, 2015: SAP Ramp-up opened for customer applications

July 2015: SAP Ramp-up succesfully closed

August, 2015: GA of SAP HANA on IBM POWER





IBM.

IBM - Broad Flexibility for HANA on POWER Stack

• Protect existing customer investments

• Unique PowerVM advantages: Virtualization out of the Box

• TDI like approach re-use existing IT assets and operational patterns

• More granular and flexible memory increments possible

• Leverage POWER performance and scalability for SAP Business Suite

• Significant Power SMT throughput advantages versus Intel x86

• in-box co-existence with established SAP landscape using AIX, IBM i

• Intended for mission critical 7X24

Enterprise customer operations

• Not an Appliance, running on

traditional Power7+ and Power8

servers

• Best Reliable, Available,

Serviceable (RAS) in the market





IBM.

Deployment Model - TDI (Tailored Datacenter Integration)

• Solution validation done by SAP and partner

• Preconfigured hardware set-up• Preinstalled software

• Installation needs to be done by customer

• Customer aligns with the hardware partner on individual support mode

Fast Implementation

Support fully provided by SAP

More Flexibility

Save IT budget and existing investment





IBM.

HWCCT – SAP Hardware configuration check tool

• Determine if system meets KPI requirements

• Landscape test

• OS config validity

• Consistency of landscape based on reference architecture

• File system test (throughput/latency)

• Network test (throughput for multi-node configurations)

• 9.5 GBits for single stream

• 9.0 GBits for duplex stream





IBM.

HANA DB Stack on POWER as of September 2015

Any TDI compliant Storage Sybsystem or

IBM Storage subsystem which can fulfill

the SAP Storage KPIs

POWER7+ and POWER8

PowerVM only

Dedicated(-donating) partitions for HANA, rest can be in a shared pool

Fibre Channel

Linux Operating System SLES 11 SP3 with patches + SP4

XFS and NFS (hana shared) Filesystem or

future:GPFS (FPO out of scope)

Multipath I/O for SAN attached disks / NPIV

HANA SPS9 and above (HANA DB, BW)

Ethernet

SVCIBM Elastic

Storage Server

(GPFS)

Infiniband

Internal

Disks

Dual VIOS for ETH and FC Dual VIOS for ETH and FC Direct

Direct BW

Application

Server





IBM.

Sample HANA PowerVM Topology





IBM.

SAN / LAN Connectivity Pattern for Scale-Up HoP Instance

• If dedicated I/O is used, plan for redundant I/O adapters per server

• Dual-VIO requires access to a HMC. IVM is not supported

Minimum per

VIO

single-host multi-host Function

LAN

Ethernet - 1* 10 Gb port

- 1* 1 Gb port

- 2* 10 Gb port

- 1* 10 Gb port

- 1* 1 Gb port

- 1* 10 Gb port

- HANA inter-node

- HANA to App-Server

- Admin network

- GPFS network

SAN

Fiber Channel

- 1* FC 8Gb dual-port - 1* FC 8Gb dual-port - Attach to storage or SVC

• For redundancy with dual-VIO setup plan these ports per VIO:





IBM.

deg

ree

of

flexib

ilit

y

Think Landscapes, Not Isolated HANA DB-Server!

DEV/SBX/EDU/Test HANA DB up to 2xGB/core ratio

(min 2 Cores)

Pre-Prod/QA HANA DB temp. 1x PRODGB/core ratio

TDI aligned, Linux,

dedicated donating

“Free”,

Shared Pool

BW HANA PROD 32GB/core, min. 4cores

Data

HANA SoH PROD (SoD)

BW App-Server

ERP App-Server

HANA BW Non-RROD

HANA ECC Non-PROD

HANA Analytics PROD

VIO

S

VIO

S

Share

d p

ool

ERP HANA PROD 96 GB/core, min. 4cores

Data

Data

Data

Traditional Sizing,

Shared Pool

others

Converge with S/4

BW App PROD per SAP QS

ERP App PROD per SAP QS





IBM.

• High Level Summary

• Operating System

• SUSE Linux Enterprise Server 11 SP3 for IBM Power (w/ SMT8/CAS16 patch)

• SUSE Linux Enterprise Server 11 SP4 for IBM Power (w/ full Power8 support)

• PowerVM virtualization support (for production use Dedicated – donating LPARs)

• Hardware

• Minimum IBM Power Server with POWER8 processor technology (for production env.)

• Minimum IBM Power Server with POWER7+ processor technology (for non-production

env.)

• Core to Memory Ratio for BW on HANA on Power

• The initial core to memory ratio for SAP HANA on POWER is 32 GB per core

• Initial support 96 cores / 3TB

• If the planned system size exceeds either 3TB or does not fit into the ratio please contact

SAP

• Use Cases

• At the first release, only SAP Business Warehouse (BW) on SAP HANA is supported,

scale-up only

• SAP NetWeaver BW version 7.31 or higher

Supported Use Cases and Scope





IBM.

• SAP HANA, version for IBM Power Systems architecture – Scope Description

• 2133369 - SAP HANA on IBM Power Systems: Central Release Note

• 2055470 - HANA on POWER Planning and Installation Specifics - Central Note

• Documentation

• During the ramp-up phase the document SAP HANA, Version for IBM Power

Systems Architecture – Documentation, attached to SAP Note 2130682 - SAP

HANA on IBM Power Systems: Documentation for Ramp-Up, provides the

documentation for the product SAP HANA, version for IBM Power Systems

architecture.

• SAP HANA on IBM Power Systems and IBM System Storage - Guides

• Technical details about how to plan and deploy SAP HANA on IBM Power

Systems SAP external IBM Planning Guide on IBM Techdocs:

• http://www-03.ibm.com/support/techdocs/atsmastr.nsf/Web/Techdocs

• http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102347

Supported Use Cases and Scope


http://service.sap.com/sap/support/notes/2133369



http://www-03.ibm.com/support/techdocs/atsmastr.nsf/Web/Techdocs

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102347




IBM.

Built-in parallelism in SAP HANA exploits both:

Scale-up and Scale-out

• Parallel processing in HANA

• SAP HANA was designed to perform its basic calculations, such as analytic

joins, scans and aggregations in parallel.

• Often it uses hundreds of cores at the same time, fully utilizing the

available computing resources of distributed systems.

• Column-based storage also allows execution of operations in parallel using

multiple processor cores / SMT Threads. In a column store, data is already

vertically partitioned. This means that operations on different columns can

easily be processed in parallel.

• In addition, operations on one column can be parallelized by partitioning

the column into multiple sections that can be processed by different

processor cores / SMT Threads.

• Job Scheduler

• Central place in HANA to execute plan-operator jobs

• Re-adjust dynamically the concurrency level avoiding “over-parallelization”

• Can consider additional .ini configuration data to limit the concurrency level

Source: SAP, 2014




IBM.

Power 8 Multi-Core / SMT Parallelization

SMT Thread

SMT Thread

SMT Thread

SMT Thread





IBM.

SAP HANA Delta-Merge Process

• A column store table is comprised of two index types for each column, a Main index and a Delta index

• The Delta storage is optimized for write operations and the Main storage is optimized in terms of read performance and memory consumption

• The use of the Delta tables addresses the performance issues of loading directly to compressed columns.

• Main store, Delta stores and the Merge Process are all located in RAM.

• Merge process takes the data out of the Delta memory structures and puts it into the Main memory structure. All changes are logged in the file system (data and log files).

• This is a very CPU/Memory intensive task (!!!)





IBM.

192750

172450

137010126980

4818

2395 2283 2116

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

Nav

igatin

g s

tep

s/H

ou

r/Co

re

Ad

-Ho

c N

av

igati

on

Ste

ps/H

ou

r

New World record set by IBM Power E870 on SAP BW Enhanced Mixed Load

Standard Application Benchmark with 2 Billion records

2Xper core

(1) IBM Power Enterprise System E870 on the SAP BW Extended mixed load standard application benchmark running SAP Netweaver 7.31 application; 4 processors / 40 cores / 320 threads, POWER8; 4.19GHz, 1024 GB memory, 192.750 adhoc navigation steps per hours on SuSE Linux Enterprise Server 11 and SAP Hana 1.0, Certification #: 2015024 Result valid as of June 1, 2015. Source: http://www.sap.com/benchmark.(2) Dell PowerEdge R930, on the SAP BW Extended mixed load standard application benchmark running SAP Netweaver 7.31 application; 4 processors / 72 cores / 144 threads, , Intel Xeon Processor E-7 8890 v3 ,2.5 GHz; 1536 GB memory, 172.450 adhoc navigation steps per hours on SuSE Linux Enterprise Server 11 and SAP Hana 1.0, Certification #: 2015014 (3) Dell PowerEdge R920, on the SAP BW Extended mixed load standard application benchmark running SAP Netweaver 7.31 application; 4 processors / 60 cores / 120 threads, , Intel Xeon Processor E-7 4890 v2 ,2.8 GHz; 1024 GB memory, 137,010 adhoc navigation steps per hours on SuSE Linux Enterprise Server 11 and SAP Hana 1.0, Certification #: 2014044 (4) HP DL580 Gen8, on the SAP BW Extended mixed load standard application benchmark running SAP Netweaver 7.30 application; 4 processors / 60 cores / 120 threads, , Intel Xeon Processor E-7 4880 v2 ,2.5 GHz; 1024 GB memory, 126,980 adhoc navigation steps per hours on SuSE Linux Enterprise Server 11 and SAP Hana 1.0, Certification #: 2014009

SAP and all SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. All other product and service names mentioned are the trademarks of their respective companies.

IBM

E870

POWER8

4p/40c/320t

Dell

PowerEdge

R930

E7-8890 v3

4p/72c/144t

HP

DL580 Gen8

E7-4880 v2

4p/60c/120t

Dell

PowerEdge R920

E7-4890 v2,

4p/60c/120t

12% faster


http://www.sap.com/benchmark




IBM.

Power Systems RAS versus x86

RAS Feature POWER x86

Application/Partition RAS

Live Partition Mobility Yes Yes

Live Application Mobility Yes Yes, support issues

Partition Availability priority Yes No

System RAS

OS independent First Failure Data Capture Yes EX – MCA Recovery

Memory Keys (including OS exploitation) Yes No

Processor RAS

Processor Instruction Retry Yes No

Alternate Processor Recovery Yes No

Dynamic Processor Deallocation Yes No

Dynamic Processor Sparing Yes No

Memory RAS

Chipkill™ Yes Yes, some vendors

Survives Double Memory Failures Yes Yes, optional

Selective Memory Mirroring Yes No

Redundant Memory Yes Yes

I/O RAS

Extended Error Handling Yes No

I/O Adapter Isolation (PI-Bus and TCEs) Yes No





IBM.

MemoryBuffer

DRAMChips

DDR Interfaces

POWER8

Link

Scheduler &

Management

16MB

Memory

Cache

Intelligence Moved into Memory• Scheduling logic, caching structures

• Energy Mgmt, RAS decision point

– Formerly on Processor

– Moved to Memory Buffer

Processor Interface• 9.6 GB/s high speed interface

• More robust RAS

•“ On-the-fly” lane isolation/repair

Performance Value• End-to-end fastpath and data retry (latency)

• Cache latency/bandwidth, partial updates

• Cache write scheduling, prefetch, energy

POWER8 Memory

10 chips per rank for double chipkill





IBM.

SAP HANA In-Memory Database Sizing

• SAP HANA sizing consists of:

• Memory sizing for static data

• Memory requirements for static data is derived from the database footprint of the corresponding tables of the source

database system. Database specific scripts and more details on how to determine the database footprint can be

found in note 1514966.

• Memory sizing for objects created during runtime (data load and query execution)

• Additional memory required for objects that are created dynamically when loading new data and when executing queries

(RAMtotal = RAMdynamic + RAMstatic)

• Disk Sizing

• Disk size for persistence layer = 1.2 * RAM

• Disk size for log files = 0.5 * RAM

• Note that this only covers disk requirements for the database files. As with any database system, additional space

must be reserved for backup, exports & executables

• We recommend reserving approximately another 2-3 times the RAM value for these purposes.

• CPU Sizing

• Sizing approach similar to user based CPU sizing of BW and BWA

• Important OSS Notes:

• 1514966 - SAP HANA 1.0: Sizing SAP In-Memory Database

• 1637145 - SAP BW on HANA: Sizing SAP In-Memory Database

• 1793345 - Sizing for SAP Suite on HANA








IBM.

Storage Layout Considerations





IBM.

Migration to HANA on POWER

• SAP heterogeneous system copy procedures apply

• Standard migrations using Software Provisioning Manager (SWPM) and

customer R3load exports

• Software Upgrade Manager with Database Migration Option (SUM/DMO)

• can be used to combine a release upgrade and the heterogeneous database

migration to a new target system





IBM.

LPAR #3 (Linux)

SAP HANA

System (primary)

High Availability and Disaster Recovery Set-Up

SAP Primary

Application Server

HANA DB Client

SAP (A)SCS(ABAP) Central Services

LPAR #1

data

logs

LPAR #4 (Linux)

SAP HANA

System

data

logs

SAP HANA

System Replication

Full Sync

Sync

Syncmem

(A)SCS / PAS

Virt. IP

Virt. hostname

SAP HANA

Virt. IP/ hostname

SAP Additional

Application Server

HANA DB Client

LPAR #2

Cluster takeoverSAP ERS

(Enqueue Replication Server)

Cluster takeover

SAP HANA

System Replication

LPAR #6 (Linux)

SAP HANA

System

data

logs

AIX LPAR #5

Async

SAP Primary

Application Server

HANA DB Client

SAP (A)SCS(ABAP) Central Services

Tier 1 Tier 2 Tier 3





IBM.

IBM Spectrum Protect (for SAP HANA) Announcement

IBM Spectrum Protect

(formerly Tivoli Storage Manager – TSM )

Spectrum Protect is IBM’s proven Data Protection solution for SAP

Added SAP HANA support for IBM Power Systems !

General Availability – as part of Spectrum Protect 7.1.3 on

September 11th, 2015

SAP Certification in progress

IBM Contacts

Gerd Munz – IBM Spectrum Protect Development

[email protected]

Cyrus Niltchian – IBM Spectrum Protect Product Management

[email protected]

Protect your SAP HANA data with IBM Spectrum Protect

New

“The IBM Spectrum

Protect enhancement

supporting SAP HANA

on Power provides a

reliable and scalable

data protection solution

for SAP HANA

customers.”

Bharti Patel, Director of IBM

Spectrum Protect and Control







IBM.

IBM Spectrum Protect & SAP HANA Backint data stream





IBM.

Scale-up versus Scale-out from an SAP HANA

perspective

Scale-up Scale-out

P

R

O

S

• Better performance

• Easier to set-up and to

maintain

• Larger database can remain in-memory

• Higher performance and throughput boundaries

with respect to big data and number of users

C

O

N

S

• Database size is limited to

maximum physical

memory available in one

sigle cabinet

• Inter-node communication overhead (update,

distribution of data, etc.)

• More complex to set-up and to admistrate the

database

• Controlled support of SoH use-cases





IBM.

Scale-up Scale-out

P

R

O

S

• Better performance

• Better RAS features (high-end systems

offer higher RAS)

• Reduced infrastructure complexity

• Administration costs are lower (Less OS

and database images, less network

infrastructure, etc.)

• HA easier to implement by n+1 concepts

• Lower price S8xxL-type of nodes

possible

C

O

N

S

• Maximum physical memory determines

amount of data „in-memory“

• HA resources more expensive

• More network infrastructure needed

• Higher administration costs

• Usually less reliable

• Extensions of number of nodes might be

difficult after a few years (because of

technology changes)

Scale-up versus Scale-out from an SAP HANA

perspective





IBM.

Comparing both SAP HANA virtualized environments (1)

Noteworthy properties of x86 setup Noteworthy properties of Power setup

Not officially supported by SAP, thus we are on our

own in handling any potential issues

As we come from a POWER setup running mostly

AIX LPARs, SLES is a new stack for us, for which

we neither have lots of know-how nor any tools

which would allow for e.g. faster deployment

x86 virtualization, at least in our scenarios, proves

to be much more unstable compared to our

POWER LPARs (frequent performance issues,

crashing HANA servers, etc.)

Management options offered by x86 systems are

not as refined as for our POWER systems (e.g. no

HMC support)

Officially supported by SAP and IBM in case of

software or hardware issues

As SLES 11 SP4 (PPC) is a prerequisite for running

SAP HANA on Power, we are not able to leverage our

previous experiences, tools and self-developed script

as we would have been with AIX

SAP HANA on POWER LPARs proved to be very

stable and worked like a charm throughout our

various setups (BW on HANA, BW on HANA w/ high

availability, SAP ERP on HANA, SAP ERP with

ERPsim game as well as standalone development

HANAs)

Even the configurations are not supported yet like

MDC or scale-out did not give us any trouble

Technische Universität München

Source: TUM, 2015





IBM.

Comparing both SAP HANA virtualized environments (2)

Noteworthy properties of x86 setup Noteworthy properties of Power setup

x86 systems require a lot more ramp up effort as

they do not come with a built-in hypervisor

(unlike POWER systems which sport PowerVM).

Building an own, strapped-down hypervisor

Linux distro would be possible but very time-

consuming (also: additional maintenance effort)

The host OS and hypervisor require a healthy

amount of RAM and CPU, which in turn is not

available for any guest OS

We chose this setup as we could not afford a

dedicated appliance, which would have occupied

space and resources in our data center but did

not provide any scalability in terms of computing

resources or usage type (it would have only be

able to run HANA and could not share any

resources)

Management options offered by the previous

POWER generation are also available for POWER8

(HMC, VIOS, AME, and so forth)

Initial setup of a server is rather easy compared to

x86 systems due to the built-in hypervisor

As the PowerVM hypervisor has a very small

footprint and is efficiently implemented, more

system ressources can be assigned to LPARs

instead of being occupied by the hypervisor and

host OS

Ressource utilization is much more efficient due to

the whole server employing virtualization

technologies whereas a dedicated appliance could

only run a HANA workload

Technische Universität München

Source: TUM, 2015





IBM.

SAP HANA on POWER

Upcoming Pre-GA programs for SAP Business Suite

Phase 1: Test and Evaluation Program (TE) for SAP Business

Suite

Five to ten customers to participate

Open for modules: ERP, CRM, SRM, PBC, S-FIN

Period: starting in August until October

Phase 2: Test and Evaluation Program (TE) for SAP Business

Suite

Additional customers to participate

Open for modules: SCM, SEM, CAR, Bank Analyzer

Period: starting in November until January/February (or earlier)





IBM.

SAP on Power Team Power/SAP Lab Environment

• Located in San Jose, CA

• Provides customer demo’s briefings,

education, webinars, PoC’s (small

scale)

Available demo’s SAP HANA on IBM Power Linux

SAP & IBM DB2 BLU

IBM Flash System & SAP

IBM PowerVM virtualization & SAP

IBM Storage virtualization & SAP

PowerHA & SAP

Tivoli based Backup / Restore solutions

The Team:

Manager – Donald Whitt

([email protected])

Andrew Castillo

Michael Dang (Canada)

Mark Gordon

Lou Lamprinakos

Chau Nguyen

Nelson Orengo

Damir Rubic

For SAP Lab related activities, please

contact:

• Donald Whitt

• Damir Rubic ([email protected])





IBM.


optimal deployment of sap hana on ibm power platform ... · pdf filefor selected customers in...

Documents