sap hana on lenovo x6 systems - suse.com · 3.1.8 sap hana lifecycle management ... 5.1.6 x3850 x6...

Front cover

In-memory Computing with SAP HANA on Lenovo X6 Systems

Introduces System x solution for SAP HANA

Explores SAP HANA features and use cases

Discusses operational aspects for SAP HANA

Describes SAP HANA high availablity and disaster recovery scenarios

Martin Bachmaier

Ilya Krutov

https://lenovopress.com/updatecheck/SG248086/b63d3a49059fb6c1f4836d40f455441a

SAP HANA on Lenovo X6 Systems

January 2017

SG24-8086-03

© Copyright Lenovo 2015. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract

Last update on January 2017

This edition applies to Lenovo X6 rack servers and Flex System X6 compute nodes

Note: Before using this information and the product it supports, read the information in “Notices” on page vii.

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiTrademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixAuthors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Chapter 1. Basic concepts of in-memory computing . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Keeping data in-memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.1 Using main memory as the data store . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.1.2 Data persistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Minimizing data movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.1 Compression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.2 Columnar storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2.3 Pushing application logic to the database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3 Dividing and conquering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3.1 Parallelization on multi-core systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.2 Data partitioning and scale-out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Chapter 2. SAP HANA and SAP S/4HANA overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1 SAP HANA overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.1 SAP HANA architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.1.2 SAP HANA appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2 SAP S/4HANA overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3 SAP HANA and SAP S/4HANA delivery model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3.1 SAP HANA as an appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.3.2 SAP HANA tailored data center integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.3.3 SAP HANA in the cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.4 Sizing SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.4.1 Memory per core ratio for SAP HANA appliances . . . . . . . . . . . . . . . . . . . . . . . . 162.4.2 Sizing approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.5 SAP HANA software licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Chapter 3. Software components and data replication methods . . . . . . . . . . . . . . . . . 193.1 SAP HANA software components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.1.1 SAP HANA database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.1.2 SAP HANA client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.1.3 SAP HANA studio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.1.4 SAP HANA studio repository. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.1.5 SAP HANA landscape management structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.1.6 SAP host agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.1.7 SAP HANA Database Lifecycle Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.1.8 SAP HANA Lifecycle Management tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.1.9 Solution Manager Diagnostics agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.2 Data replication methods for SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2.1 Trigger-based replication with SAP Landscape Transformation . . . . . . . . . . . . . . 303.2.2 ETL-based replication with SAP BusinessObjects Data Services . . . . . . . . . . . . 313.2.3 Extractor-based replication with Direct Extractor Connection . . . . . . . . . . . . . . . . 313.2.4 Log-based replication with Sybase Replication Server . . . . . . . . . . . . . . . . . . . . . 323.2.5 Comparing the replication methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

© Copyright Lenovo 2013, 2014, 2015, 2016. All rights reserved. iii

Chapter 4. SAP HANA integration scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.1 Basic use case scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.2 SAP HANA as a technology platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2.1 SAP HANA data acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.2.2 SAP HANA as a source for other applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.3 SAP HANA for operational reporting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404.4 SAP HANA as an accelerator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.5 SAP products that are running on SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.5.1 SAP NetWeaver Business Warehouse that is powered by SAP HANA . . . . . . . . 454.5.2 Migrating SAP NetWeaver Business Warehouse to SAP HANA . . . . . . . . . . . . . 494.5.3 SAP Business Suite that is powered by SAP HANA. . . . . . . . . . . . . . . . . . . . . . . 534.5.4 SAP S/4HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.6 Programming techniques that use SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Chapter 5. Lenovo System x solutions for SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . 575.1 Lenovo X6 systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5.1.1 Intel Xeon processor E7 v2, v3, and v4 family . . . . . . . . . . . . . . . . . . . . . . . . . . . 605.1.2 Memory subsystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.1.3 Flash technology storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.1.4 Lenovo Storage D1224 Drive Enclosure for SAP HANA . . . . . . . . . . . . . . . . . . . 705.1.5 Lenovo Storage D1024 Expansion Solution for SAP HANA. . . . . . . . . . . . . . . . . 715.1.6 x3850 X6 Workload Optimized Solution for SAP HANA . . . . . . . . . . . . . . . . . . . . 725.1.7 x3950 X6 Workload Optimized Solution for SAP HANA . . . . . . . . . . . . . . . . . . . . 775.1.8 All-flash X6 solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825.1.9 X6 Partitioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

5.2 Storage subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 845.2.1 XFS technology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855.2.2 IBM Spectrum Scale technology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855.2.3 Single node SAP HANA using XFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 875.2.4 Single-node SAP HANA using Spectrum Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 885.2.5 Scaling-out SAP HANA using Spectrum Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

5.3 Networking options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 945.3.1 Lenovo RackSwitch G8296. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 955.3.2 Lenovo RackSwitch G8272. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 965.3.3 Lenovo RackSwitch G8264. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 985.3.4 Lenovo RackSwitch G8124E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 995.3.5 Lenovo RackSwitch G8052. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Chapter 6. SAP HANA IT landscapes with X6 solutions . . . . . . . . . . . . . . . . . . . . . . . 1036.1 eX5 based environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1046.2 X6 based environments that use Intel Xeon E7 v2 processors. . . . . . . . . . . . . . . . . . 105

6.2.1 Single-node X6 solution for Business Warehouse . . . . . . . . . . . . . . . . . . . . . . . 1086.2.2 Single-node X6 solution for Business Suite on HANA . . . . . . . . . . . . . . . . . . . . 1116.2.3 Scale-out X6 solution for Business Warehouse . . . . . . . . . . . . . . . . . . . . . . . . . 114

6.3 Lenovo X6 based environments that use Intel Xeon E7 v3 processors . . . . . . . . . . . 1176.3.1 Single-node X6 solution for Business Warehouse . . . . . . . . . . . . . . . . . . . . . . . 1226.3.2 Single-node X6 solution for Business Suite on HANA . . . . . . . . . . . . . . . . . . . . 1256.3.3 Scale-out X6 solution for Business Warehouse . . . . . . . . . . . . . . . . . . . . . . . . . 128

6.4 Upgrading X6 from Intel Xeon E7 v2 to E7 v3 CPUs . . . . . . . . . . . . . . . . . . . . . . . . . 1316.4.1 Upgrading a single node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1326.4.2 Converting a single node into a scale-out environment . . . . . . . . . . . . . . . . . . . 1326.4.3 Upgrading a scale-out environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

6.5 Lenovo X6 based environments that use Intel Xeon E7 v4 processors . . . . . . . . . . . 1336.5.1 Single-node X6 solution for Business Warehouse . . . . . . . . . . . . . . . . . . . . . . . 137

iv SAP HANA on Lenovo X6 Systems

6.5.2 Single-node X6 solution for Business Suite on HANA . . . . . . . . . . . . . . . . . . . . 1416.5.3 Scale-out X6 solution for Business Warehouse . . . . . . . . . . . . . . . . . . . . . . . . . 144

6.6 Upgrading X6 to Intel Xeon E7 v4 CPUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1476.6.1 Upgrading a single node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1486.6.2 Converting a single node into a scale-out environment . . . . . . . . . . . . . . . . . . . 1486.6.3 Upgrading a scale-out environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

6.7 All-Flash X6 solutions for SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1496.7.1 All-flash X6 solution for single-node BW. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1496.7.2 All-flash X6 solution for Business Suite on HANA. . . . . . . . . . . . . . . . . . . . . . . . 151

6.8 Partitioning the x3950 X6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1536.8.1 Single-node X6 solutions for SAP BW supporting Partitioning . . . . . . . . . . . . . . 1546.8.2 Single-node X6 solutions for Business Suite supporting X6 Partitioning . . . . . . 1556.8.3 Scale-out X6 solutions for SAP BW supporting Partitioning . . . . . . . . . . . . . . . . 156

6.9 Migrating from eX5 to X6 servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1566.9.1 Disruptive migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1566.9.2 Hybrid SAP HANA cluster with eX5 and X6 nodes (using Intel Xeon E7 v2

processors) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1576.10 Consolidation of SAP HANA instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

6.10.1 Multitenancy built into SAP HANA (MDC). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1596.10.2 Consolidation options of SAP HANA (pre SPS09 release) . . . . . . . . . . . . . . . . 1596.10.3 SAP HANA on VMware vSphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

6.11 Security and encryption of an SAP HANA system . . . . . . . . . . . . . . . . . . . . . . . . . . 1656.11.1 Encrypting SAP HANA data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1656.11.2 Securing SAP HANA communication channels . . . . . . . . . . . . . . . . . . . . . . . . 1676.11.3 Securing SAP HANA access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

Chapter 7. Business continuity and resiliency for SAP HANA . . . . . . . . . . . . . . . . . . 1697.1 Overview of business continuity options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

7.1.1 Spectrum Scale based storage replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1727.1.2 SAP HANA System Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1737.1.3 Special considerations for DR and long-distance HA setups . . . . . . . . . . . . . . . 176

7.2 HA and DR for single-node SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1777.2.1 High availability (by using Spectrum Scale) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1787.2.2 Stretched high availability (by using Spectrum Scale) . . . . . . . . . . . . . . . . . . . . 1807.2.3 Disaster recovery (by using Spectrum Scale) . . . . . . . . . . . . . . . . . . . . . . . . . . . 1837.2.4 Disaster recovery (by using SAP HANA System Replication) . . . . . . . . . . . . . . 1877.2.5 HA plus DR (by using Spectrum Scale) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1897.2.6 HA (by using Spectrum Scale) plus DR (by using SSR) . . . . . . . . . . . . . . . . . . . 191

7.3 HA and DR for scale-out SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1947.3.1 HA by using Spectrum Scale storage replication . . . . . . . . . . . . . . . . . . . . . . . . 1957.3.2 DR by using Spectrum Scale storage replication . . . . . . . . . . . . . . . . . . . . . . . . 1957.3.3 HA by using Spectrum Scale replication plus DR by using SAP HANA Replication.

2047.4 Backup and restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

7.4.1 Basic operating system backup and recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . 2057.4.2 Basic database backup and recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2067.4.3 File-based backup tool integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2077.4.4 Database backups by using Spectrum Scale snapshots . . . . . . . . . . . . . . . . . . 2087.4.5 Backup tool integration with Backint for SAP HANA. . . . . . . . . . . . . . . . . . . . . . 2087.4.6 Backup and restore as a DR strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Chapter 8. SAP HANA operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2118.1 Installation services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

Contents v

8.2 Lenovo SAP HANA Operations Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2128.3 Interoperability with other platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2138.4 Monitoring SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2148.5 Installing more agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2148.6 Software and firmware levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2158.7 Support process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

8.7.1 Lenovo and SAP integrated support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2168.7.2 Lenovo SAP Center of Competence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Appendix A. Additional topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219A.1 Spectrum Scale license information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220A.2 File-based backup with IBM Tivoli Storage Manager for ERP . . . . . . . . . . . . . . . . . . 221

A.2.1 Setting up Data Protection for SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221A.2.2 Backing up the SAP HANA database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222A.2.3 Restoring the SAP HANA database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229Lenovo Press publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229Help from Lenovo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

vi SAP HANA on Lenovo X6 Systems

Notices

Lenovo may not offer the products, services, or features discussed in this document in all countries. Consulty our local Lenovo representative for information on the products and services currently available in yourarea. Any reference to a Lenovo product, program, or service is not intended to state or imply that only that Lenovo product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any Lenovo intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any other product, program, or service.

Lenovo may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to:

Lenovo (United States), Inc.1009 Think Place - Building OneMorrisville, NC 27560U.S.A.Attention: Lenovo Director of Licensing

LENOVO PROVIDES THIS PUBLICATION “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. Lenovo may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

The products described in this document are not intended for use in implantation or other life support applications where malfunction may result in injury or death to persons. The information contained in this document does not affect or change Lenovo product specifications or warranties. Nothing in this document shall operate as an express or implied license or indemnity under the intellectual property rights of Lenovo or third parties. All information contained in this document was obtained in specific environments and is presented as an illustration. The result obtained in other operating environments may vary.

Lenovo may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Any references in this publication to non-Lenovo Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this Lenovo product, and use of those Web sites is at your own risk.

Any performance data contained herein was determined in a controlled environment. Therefore, the result obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment.

© Copyright Lenovo 2016. All rights reserved. vii

Trademarks

Lenovo, the Lenovo logo, and For Those Who Do are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. These and other Lenovo trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by Lenovo at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of Lenovo trademarks is available on the Web at http://www.lenovo.com/legal/copytrade.html.

The following terms are trademarks of Lenovo in the United States, other countries, or both:

eX5™eXFlash™Lenovo®RackSwitch™

Lenovo(logo)®ServeRAID™System x®TruDDR4™

VMready®X-Architecture®X5™

The following terms are trademarks of other companies:

Basis, Intel, Intel Core, Itanium, Xeon, and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Excel, Hyper-V, Microsoft, Windows, Windows Server, Windows Vista, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

viii SAP HANA on Lenovo X6 Systems

http://www.lenovo.com/legal/copytrade.html

Preface

The forth edition of this Lenovo® Press publication describes in-memory computing appliances from Lenovo and SAP that are based on Lenovo X6 flagship systems and SAP HANA. It covers the basic principles of in-memory computing, describes the Lenovo X6 hardware offerings, and explains the corresponding SAP HANA IT landscapes that use these offerings.

This book also describes the architecture and components of the Lenovo System x® solution for SAP HANA. The following SAP HANA operational disciplines are explained: Scalability options; high availability and disaster recovery; backup and restore; and virtualization possibilities for SAP HANA appliances.

The following topics also are covered:

� Basic principles of in-memory computing

� SAP HANA overview

� Software components and replication methods

� SAP HANA use cases and integration scenarios

� The System x solution for SAP HANA

� SAP IT landscapes that use the System x solution for SAP HANA

� Options for business continuity (high availability, disaster recovery, and backup and restore)

� SAP HANA operations

This book is intended for SAP administrators and technical solution architects. It is also for Lenovo Business Partners and Lenovo employees who want to know more about the SAP HANA offering and other available Lenovo solutions for SAP clients.

Authors

This book was produced by Lenovo Press in collaboration with a team of subject matter experts from around the world.

Martin Bachmaier is an IT Versatilist in the Lenovo Solutions development lab at Stuttgart, Germany. He is the architect of the Lenovo Solution for SAP HANA and leads the innovation area. Martin has an extensive background in designing, implementing, and managing scale-out data centers, HPC clusters, cloud environments, and high performance storage systems. In 2013, he implemented one of Africa's most powerful computing hubs in IBM's newly opened Research Lab in Nairobi, Kenya. Martin frequently presents university lectures and at conferences, and holds credentials in CCNA, CCNA Security, and VMware Certified Professional. He has authored over a dozen books and papers.

© Copyright Lenovo 2013, 2014, 2015, 2016. All rights reserved. ix

Thanks to the authors of the previous edition of this book:

� Gereon Vey� Tomas Krojzl

Thanks to the following people for their contributions to this project:

From Lenovo:

� Irene Hopf� Dr. Oliver Rettig� Tag Robertson� David Watts� Mark T. Chapman

Ilya Krutov is a Project Leader at Lenovo Press in the Lenovo Enterprise Business Group. He manages and produces presale and post-sale technical publications on various IT topics, including x86 rack and blade servers, server operating systems and software, virtualization and cloud, and data center storage and networking. Ilya has more than 15 years of experience in the IT industry, and he performed various roles, including Team Leader, Portfolio Manager, Brand Manager, IT Specialist, and Certified Instructor. He has written more than 200 books, papers, and other technical documents. He has a Specialist's degree with honors in Computer Engineering from the Moscow State Engineering and Physics Institute (Technical University).

x SAP HANA on Lenovo X6 Systems

Chapter 1. Basic concepts of in-memory computing

In-memory computing is a technology that allows the processing of massive quantities of data in main memory to provide extremely fast results from analysis and transactions. The data that is processed is ideally real-time data (that is, data that is available for processing or analysis immediately after it is created).

To achieve the preferred performance, in-memory computing adheres to the following basic concepts:

� Keep data in main memory to speed up data access

� Minimize data movement by using the columnar storage concept, compression, and performing calculations at the database level

� Divide and conquer. Use the multicore architecture of modern processors and multiprocessor servers (or even scale out into a distributed landscape) to grow beyond what can be supplied by a single server.

This chapter describes these basic concepts and provides some examples. It does not describe the full set of technologies that are used with in-memory databases, such as SAP HANA, but it does provide an overview of how in-memory computing is different from traditional concepts.

This chapter includes the following topics:

� Keeping data in-memory� Minimizing data movement� Dividing and conquering

1

© Copyright Lenovo 2013, 2014, 2015, 2016. All rights reserved. 1

1.1 Keeping data in-memory

Today, a single enterprise class server can hold several terabytes of main memory. At the same time, prices for server main memory dramatically dropped over the last few decades. This increase in capacity and reduction in cost makes it a viable approach to keep huge amounts of business data in memory. This section describes the benefits and challenges.

1.1.1 Using main memory as the data store

The most obvious reason to use main memory (RAM) as the data store for a database is that accessing data in main memory is much faster than accessing data on disk. The compared access times for data in several locations are shown in Figure 1-1.

Figure 1-1 Data access times of various storage types relative to RAM (logarithmic scale)

The main memory is the fastest storage type that can hold a significant amount of data. Although CPU registers and CPU cache are faster to access, their usage is limited to the actual data processing. Data in main memory can be accessed more than a hundred thousand times faster than data on a spinning hard disk drive (HDD), and even flash technology storage is approximately a thousand times slower than main memory. Main memory is connected directly to the processors through a high-speed bus, and hard disks are connected through a chain of buses (QPI, PCIe, and SAN) and controllers (I/O hub, RAID controller or SAN adapter, and storage controller).

Compared with keeping data on disk, keeping the data in main memory can improve database performance through the advantage in access time.

Volatile Non-volatile

1,000,000

100,000

10,000

1,000

100

10

1

0,1

0,01

0,001CPU register CPU Cache RAM SSD/Flash Hard disk

12x

17x

2,000x

150x

2 SAP HANA on Lenovo X6 Systems

1.1.2 Data persistence

Keeping data in main memory brings up the issue of what happens if there is a loss of power.

In database technology, atomicity, consistency, isolation, and durability (ACID) is the following set of requirements that ensures that database transactions are processed reliably:

� A transaction must be atomic. If part of a transaction fails, the entire transaction must fail and leave the database state unchanged.

� The consistency of a database must be preserved by the transactions that it performs.

� Isolation ensures that no transaction interferes with another transaction.

� Durability means that after a transaction is committed, it remains committed.

Although the first three requirements are not affected by the in-memory concept, durability is a requirement that cannot be met by storing data in main memory alone. Main memory is volatile storage. It loses its content when it is out of electrical power. To make data persistent, it must be on non-volatile storage, such as HDDs, solid-state drives (SSDs), or flash devices.

The storage that is used by a database to store data (in this case, main memory) is divided into pages. When a transaction changes data, the corresponding pages are marked and written to non-volatile storage in regular intervals. In addition, a database log captures all changes that are made by transactions. Each committed transaction generates a log entry that is written to non-volatile storage, which ensures that all transactions are permanent.

Figure 1-2 shows this setup by using the example of SAP HANA. SAP HANA stores changed pages in savepoints, which are asynchronously written to persistent storage in regular intervals (by default, every 5 minutes). The log is written synchronously. A transaction does not return before the corresponding log entry is written to persistent storage to meet the durability requirement.

Figure 1-2 Savepoints and logs in SAP HANA

After a power failure, the database can be restarted much like a disk-based database. The database pages are restored from the savepoints and then the database logs are applied (rolled forward) to restore the changes that were not captured in the savepoints. This action ensures that the database can be restored in memory to the same state as before the power failure.

1.2 Minimizing data movement

The second key to improving data processing performance is to minimize the movement of data that is within the database and between the database and the application. This section describes measures to achieve this state.

Time

Data savepointto persistent

storage

Log writtento persistent storage

(committed transactions) Power failure

Chapter 1. Basic concepts of in-memory computing 3

1.2.1 Compression

Although today’s memory capacities allow keeping enormous amounts of data in-memory, compressing the data in-memory is still preferable. The goal is to compress data in a way that does not use up the performance that is gained while still minimizing data movement from RAM to the processor.

By working with dictionaries to represent text as integer numbers, the database can compress data significantly and thus reduce data movement while not imposing more CPU load for decompression; in fact, it can add to the performance, as shown in Figure 1-5 on page 6. This situation with a simplified example is shown in Figure 1-3.

Figure 1-3 Dictionary compression

The original table is shown on the left side of Figure 1-3, and it contains text attributes (that is, material and customer name) in their original representation. The text attribute values are stored in a dictionary (upper right), and an integer value is assigned to each distinct attribute value. In the table, the text is replaced by the corresponding integer value as defined in the dictionary. The date and time attribute also are converted to an integer representation. The use of dictionaries for text attributes reduces the size of the table because each distinct attribute value must be stored only once in the dictionary; therefore, each additional occurrence in the table must be referred to by the corresponding integer value.

The compression factor that is achieved by this method highly depends on data being compressed. Attributes with few distinct values compress well, but attributes with many distinct values do not benefit as much.

There are other, more effective compression methods that can be used with in-memory computing. However, for these methods to be useful, they must have the correct balance between compression effectiveness, which gives you more data in your memory or less data movement (that is, higher performance), resources that are needed for decompression, and data accessibility (that is, how much unrelated data must be decompressed to get to the data that you need). Dictionary compression combines good compression effectiveness with low decompression resources and high data access flexibility.

Row ID

Date/ Time Material

Customer Name Quantity

1 14:05 Radio Dubois 1

2 14:11 Laptop Di Dio 2

3 14:32 Stove Miller 1

4 14:38 MP3 Player Newman 2

5 14:48 Radio Dubois 3

6 14:55 Refrigerator Miller 1

7 15:01 Stove Chevrier 1

# Customers

1 Chevrier

2 Di Dio

3 Dubois

4 Miller

5 Newman

# Material

1 MP3 Player

2 Radio

3 Refrigerator

4 Stove

5 Laptop

Row ID

Date/Time Material

Customer Name Quantity

1 845 2 3 1

2 851 5 2 2

3 872 4 4 1

4 878 1 5 2

5 888 2 3 3

6 895 3 4 1

7 901 4 1 1


1.2.2 Columnar storage

Relational databases organize data in tables that contain the data records. The difference between row-based and columnar (or column-based) storage is how the table is stored.

Row-based storage stores a table in a sequence of rows. Column-based storage stores a table in a sequence of columns.

The row-based and column-based models are shown in Figure 1-4.

Figure 1-4 Row-based and column-based storage models

Both storage models have benefits and drawbacks, which are listed in Table 1-1.

Table 1-1 Benefits and drawbacks of row-based and column-based storage

Row ID

Date/Time Material

CustomerName Quantity

1 845 2 3 1

2 851 5 2 2

3 872 4 4 1

4 878 1 5 2

5 888 2 3 3

6 895 3 4 1

7 901 4 1 1

Row ID

Date/Time Material

CustomerName Quantity

1 845 2 3 1

2 851 5 2 2

3 872 4 4 1

4 878 1 5 2

5 888 2 3 3

6 895 3 4 1

7 901 4 1 1

Row-based Column-based

Row-based store

Column-based store

1 845 2 3 1 2 851 5 2 2 3 872 4 4 1 4 878 1 5 2

1 2 3 4 845 851 872 878 2 5 4 1 3 2 4 5

Row-based storage Column-based storage

Benefits � Record data is stored together.� Easy to insert/update.

� Only affected columns must be read during the selection process of a query.

� Efficient projections.a

� Any column can serve as an index.

a. Projection refers to the view on the table with a subset of columns.

Drawbacks All data must be read during selection, even if only a few columns are involved in the selection process.

� After selection, selected rows must be reconstructed from columns.

� No easy insert/update.


The drawbacks of column-based storage are not as grave as they seem. In most cases, not all attributes (that is, column values) of a row are needed for processing, especially in analytic queries. Also, inserts or updates to the data are less frequent in an analytical environment.1 SAP HANA implements a row-based storage and a column-based storage; however, its performance originates in the usage of column-based storage in memory. The following sections describe how column-based storage is beneficial to query performance and how SAP HANA handles the drawbacks of column-based storage.

Efficient query executionTo show the benefits of dictionary compression that is combined with columnar storage, Figure 1-5 shows an example of how a query is run. (Figure 1-5 refers to the table that is shown in Figure 1-3 on page 4.)

Figure 1-5 Example of a query that is run on a table in columnar storage

The query asks to get all records with Miller as the customer name and Refrigerator as the material.

First, the strings in the query condition are looked up in the dictionary. Miller is represented by the number 4 in the customer name column. Refrigerator is represented by the number 3 in the material column. This lookup must be done only once. Subsequent comparisons with the values in the table are based on integer comparisons, which are less resource-intensive than string comparisons.

In a second step, the columns are read that are part of the query condition (that is, the Customer and Material columns). The other columns of the table are not needed for the selection process. The columns are then scanned for values that match the query condition. That is, in the Customer column, all occurrences of 4 are marked as selected, and in the Material column, all occurrences of 3 are marked.

1 An exception is bulk loads (for example, when replicating data in the in-memory database, which can be handled differently).

Resultset 1 2 3 4 5 6 7

Customer 3 2 4 5 3 4 1

Get all records with Customer Name Miller and Material Refrigerator

# Customers

1 Chevrier

2 Di Dio

3 Dubois

4 Miller

5 Newman

# Material

1 MP3 Player

2 Radio

3 Refrigerator

4 Stove

5 Laptop

Material 2 5 4 1 2 3 4

Combine0 0 1 0 0 1 0 0 0 0 0 0 1 0

bit-wise AND

0 0 0 0 0 1 0

Integer comparison operations

Only those columns are read which are part of the query condition

Dictionary lookup of the strings�Strings are only compared once!

The resulting records can be assembled from the column stores fast, because positions are known(here: 6th position in every column)


These selection marks can be represented as bitmaps, which are data structures that allow efficient Boolean operations on them that are used to combine the bitmaps of the individual columns to a bitmap that represents the selection or records that match the entire query condition. In this example, the record number 6 is the only matching record. Depending on the columns that are selected for the result, the extra columns must be read to compile the entire record to return it. However, because the position within the column is known (record number 6), only the parts of the columns that contain the data for this record must be read.

This example shows how compression can limit not only the amount of data that must be read for the selection process, but can simplify the selection while the columnar storage model further reduces the amount of data that is needed for the selection process. Although the example is simplified, it shows the benefits of dictionary compression and columnar storage.

Delta-merge and bulk insertsTo overcome the drawback of inserts or updates that affect the performance of the column-based storage, SAP plans to implement a lifecycle management for database records.2

The lifecycle management for database records in the column-store is shown in Figure 1-6.

Figure 1-6 Lifetime management of a data record in the SAP HANA column-store

The following types of storage for a table are available:

� L1 Delta Storage is optimized for fast write operations. The update is performed by inserting a new entry into the delta storage. The data is stored in records, as with a traditional row-based approach. This action ensures high performance for write, update, and delete operations on records that are stored in the L1 Delta Storage.

� L2 Delta Storage is an intermediate step. Although organized in columns, the dictionary is not as optimized as in the main storage because it appends new dictionary entries to the end of the dictionary. This action results in easier inserts, but has drawbacks regarding search operations on the dictionary because it is not sorted.

� Main Storage contains the compressed data for fast read with a search optimized dictionary.

2 Efficient Transaction Processing in SAP HANA Database - The End of a Column Store Myth, which is available at this website:http://dl.acm.org/citation.cfm?id=2213946.

L1 Delta L2 Delta Main storeMerge

Unified Table

Update / Insert / Delete Bulk Insert

Read

Merge


http://dl.acm.org/citation.cfm?id=2213946

All write operations on a table work on the L1 Delta storage. Bulk inserts bypass L1 Delta storage and write directly into L2 Delta storage. Read operations on a table always read from all storages for that table. The result set is merged to provide a unified view of all data records in the table.

During the lifecycle of a record, it is moved from L1 Delta storage to L2 Delta storage and finally to the Main Storage. The process of moving changes to a table from one storage to the next one is known as Delta Merge, which is an asynchronous process. During the merge operations, the columnar table is still available for read and write operations.

Moving records from L1 Delta storage to L2 Delta storage involves reorganizing the record in a columnar fashion and compressing it, as shown in Figure 1-3 on page 4. If a value is not yet in the dictionary, a new entry is appended to the dictionary. Appending to the dictionary is faster than inserting, but results in an unsorted dictionary, which affects the data retrieval performance.

Eventually, the data in the L2 Delta storage must be moved to the Main Storage. To accomplish that task, the L2 Delta storage must be locked and a new L2 Delta storage must be opened to accept further additions. Then, a new Main Storage is created from the old Main Storage and the locked L2 Delta storage. This task is a resource-intensive and must be scheduled carefully.

1.2.3 Pushing application logic to the database

Although the concepts that are described in 1.2.2, “Columnar storage” on page 5 speedup processing within the database, there is still one factor that can slow down the processing of data. An application that is running the application logic on the data must get the data from the database, process it, and possibly send it back to the database to store the results. Sending data back and forth between the database and the application usually involves communication over a network, which introduces an effect on communication and latency and is limited by the speed and throughput of the network between the database and the application.

To eliminate this factor and increase overall performance, it is beneficial to process the data where it is (in the database.) If the database can perform calculations and apply application logic, less data must be sent back to the application and might even eliminate the need for the exchange of intermediate results between the database and the application. This action minimizes the amount of data transfer, and the communication between database and application adds less time to the overall processing time.

1.3 Dividing and conquering

The phrase “divide and conquer” (derived from the Latin saying divide et impera) typically is used when a large problem is divided into a number of smaller, easier-to-solve problems. Regarding performance, processing huge amounts of data is a problem that can be solved by splitting the data into smaller chunks of data, which can be processed in parallel.


1.3.1 Parallelization on multi-core systems

When chip manufacturers reached the physical limits of semiconductor-based microelectronics with their single-core processor designs, they started to increase processor performance by increasing the number of cores, or processing units, within a single processor. This performance gain can be used through parallel processing only because the performance of a single core remains unchanged.

The rows of a table in a relational database are independent of each other, which allows parallel processing. For example, when a database table is scanned for attribute values that match a query condition, the table or the set of attributes (columns) that are relevant to the query condition can be divided into subsets and spread across the cores that are available to parallelize the query processing. Compared with processing the query on a single core, this action reduces the time that is needed for processing by a factor equivalent to the number of cores that are working on the query (for example, on a 10-core processor, the time that is needed is one-tenth of the time that a single core needs).

The same principle applies for multi-processor systems. A system with eight 10-core processors can be regarded as an 80-core system that can divide the processing into 80 subsets that are processed in parallel.

1.3.2 Data partitioning and scale-out

Although servers that are available today can hold terabytes of data in memory and provide up to eight processors per server with up to 10 cores per processor, the amount of data that is stored in an in-memory database or the computing power that is needed to process such quantities of data might exceed the capacity of a single server. To accommodate the memory and computing power requirements that go beyond the limits of a single server, data can be divided into subsets and placed across a cluster of servers, which forms a distributed database (scale-out approach).

The individual database tables can be placed on different servers within the cluster. Tables bigger than what a single server can hold can be split into several partitions horizontally (a group of rows per partition) or vertically (a group of columns per partition), with each partition on a separate server within the cluster.


Chapter 2. SAP HANA and SAP S/4HANA overview

This chapter describes the SAP HANA offering, including its architecture, components, use cases, delivery model, and sizing and licensing aspects.


� SAP HANA overview� SAP S/4HANA overview� Sizing SAP HANA� SAP HANA software licensing

2


2.1 SAP HANA overview

This section gives an overview of SAP HANA. The following terms are used regarding SAP HANA:

� SAP HANA database

The SAP HANA database (which is also referred to as the SAP in-memory database) is a hybrid in-memory database that combines row-based, column-based, and object-based database technology that is optimized to use the parallel processing capabilities of current hardware. It is the heart of SAP offerings, such as SAP HANA.

� SAP HANA appliance (SAP HANA)

SAP HANA is a flexible, data source-neutral appliance with which you can analyze large volumes of data in real time without needing to materialize aggregations. It is a combination of hardware and software and is delivered as an optimized appliance with SAP’s hardware partners for SAP HANA.

For the sake of simplicity, this book uses the terms SAP HANA, SAP in-memory database, SAP HANA database, and SAP HANA appliance synonymously. It covers only the SAP in-memory database as part of the SAP HANA appliance. Where required, we ensure that the context makes it clear which part is being described.

2.1.1 SAP HANA architecture

Figure 2-1 shows the high-level architecture of the SAP HANA appliance. The most important software components of the SAP HANA database are described in 3.1, “SAP HANA software components” on page 20.

Figure 2-1 SAP HANA architecture

SAP HANA Appliance

SAP HANA Database

Session Management

Persistency LayerPageManagement Logger

Relational Engines

RowStore

ColumnStore

Persistent StorageData Volumes Log Volumes

Request processing / Execution Control

SQL Script

SQL MDX

Calculation Engine

TransactionManager

AuthorizationManager

MetadataManager

SAP CAR

JVM

LM Structure

SAP HANA Studio

SAP HANA Client

SAP HANA Client

SAP Host Agent

Software UpdateManager

SAP HANA Studio Repository


SAP HANA databaseThe heart of the SAP HANA database is the relational database engines. The SAP HANA database features the following engines:

� The column-based store

Stores relational data in columns, which are optimized holding tables with large amounts of data that can be aggregated in real time and used in analytical operations.

� The row-based store

Stores relational data in rows, as traditional database systems do. This row store is more optimized for row operations, such as frequent inserts and updates. It has a lower compression rate and query performance is much lower compared to the column-based store.

The engine that is used to store data can be selected on a per-table basis when the table is created. A table can be converted from one type to another type. Tables in the row-store are loaded at start time, but tables in the column-store can be loaded at start or on demand during normal operation of the SAP HANA database.

Both engines share a common persistency layer, which provides data persistency that is consistent across both engines. There is page management and logging, as with traditional databases. Changes to in-memory database pages are persisted through savepoints that are written to the data volumes on persistent storage, which often is hard disk drives (HDDs). Every transaction that is committed in the SAP HANA database is persisted by the logger of the persistency layer in a log entry that is written to the log volumes on persistent storage. The log volumes use flash technology storage for high I/O performance and low latency.

The relational engines can be accessed through various interfaces. The SAP HANA database supports SQL (JDBC/ODBC), MDX (ODBO), and BICS (SQL DBC). The calculation engine allows calculations to be performed in the database without moving the data into the application layer. It also includes a business functions library that can be called by applications to perform business calculations close to the data. The SAP HANA-specific SQL Script language is an extension to SQL that can be used to push down data-intensive application logic into the SAP HANA database.

2.1.2 SAP HANA appliance

The SAP HANA appliance consists of the SAP HANA database and adds components that are needed to work with, administer, and operate the database. It contains the repository files for the SAP HANA studio, which is an Eclipse-based administration and data-modeling tool for SAP HANA. It also includes the SAP HANA client, which is a set of libraries that is required for applications to connect to the SAP HANA database. The SAP HANA studio and the client libraries often are installed on a client PC or server.

The Software Update Manager (SUM) for SAP HANA is the framework that allows for the automatic download and installation of SAP HANA updates from the SAP Marketplace and other sources by using a host agent. It also allows distribution of the studio repository to the users.

The Lifecycle Management (LM) Structure for SAP HANA is a description of the current installation and is, for example, used by SUM to perform automatic updates.

For more information about software components, see 3.1, “SAP HANA software components” on page 20.

Chapter 2. SAP HANA and SAP S/4HANA overview 13

2.2 SAP S/4HANA overview

In 2015 SAP released SAP Business Suite 4 SAP HANA (or short S/4HANA) which is a Business Suite built natively on the in-memory database SAP HANA. This fourth generation of a business suite software offering from SAP is only available with SAP HANA database underneath.

The tight integration of the application components with the database allows to integrate all mission-critical processes of an enterprise within one system and provides instant insight into a business.

SAP uses SAP Fiori as the user interface platform to ensure a simple and consistent experience for SAP users across products and end-user devices including tablets and mobile phones. SAP Fiori UX is designed to replace SAP UI and supports HTML5 and JavaScript.

SAP S/4HANA can be installed on-premises or consumed as a cloud-based offering. As of May 2016, the following components are available with the SAP S/4HANA on-premise edition:

� SAP S/4HANA Enterprise Management

� SAP S/4HANA LoB Products (to enhance core functions of SAP S/4HANA Enterprise Management for specific lines of business)

� SAP S/4HANA LoB Products for specific industries (to enhance SAP S/4HANA Enterprise Management to provide industry specific benefits for certain lines of business)

� SAP S/4HANA Compatibility Packs

In addition, SAP S/4HANA Finance is available which is an on-premise edition consisting of SAP Accounting powered by SAP HANA. This edition was part of the initial release of SAP S/4HANA in March 2015.

The following SAP S/4HANA cloud offerings are available as of May 2016:

� SAP S/4HANA Marketing Cloud (for the marketing line of business)

� SAP S/4HANA Marketing Cloud, base option (core version of SAP S/4HANA Marketing Cloud)

� SAP S/4HANA Professional Services Cloud (for the professional services industry)

� SAP S/4HANA Enterprise Management Cloud (for a full ERP business scope, this is the equivalent to the on-premise Enterprise Management offering).

The on-premise edition follows a yearly release cycle while the cloud edition follows a quarterly release cycle.

2.3 SAP HANA and SAP S/4HANA delivery model

SAP deployed SAP HANA as an integrated solution that combines software and hardware, which is frequently referred to as the SAP HANA solution. As with SAP NetWeaver Business Warehouse Accelerator (SAP NetWeaver BW Accelerator), SAP partners with several hardware vendors to provide the infrastructure that is needed to run the SAP HANA software. Lenovo partners with SAP to provide an integrated solution.

Over the last few years, SAP gained more experience with running SAP HANA in production environments, so a second delivery model has been implemented, which is known as tailored data center integration (TDI). TDI aims to integrate clients’ hardware from different vendors.


Both approaches are described briefly in this chapter. The rest of this book covers only the solution delivery model.

Because SAP S4/HANA uses SAP HANA as the underlying database it uses the same infrastructure building blocks used with other SAP HANA deployments. In the following section we only use the term SAP HANA for simplicity.

2.3.1 SAP HANA as an appliance

Ensuring the highest customer satisfaction, SAP HANA is available as an appliance from different hardware partners. The partners select the components of the appliance and tune it specifically for use with SAP HANA. Infrastructure for SAP HANA must run through an SAP certification process to ensure that certain performance requirements are met. Only certified configurations are supported by SAP and the respective hardware partner. These configurations must adhere to the following requirements and restrictions to provide a common platform across all hardware providers:

� Only certain Intel Xeon processors can be used.

� All configurations must provide a certain main memory per core ratio, which is defined by SAP to balance CPU processing power and the amount of data that is processed.

� All configurations must meet minimum redundancy and performance requirements for various load profiles. SAP tests for these requirements as part of the certification process.

� The capacity of the storage devices that are used in the configurations must meet certain criteria that are defined by SAP.

� The networking capabilities of the configurations must include 10 Gb Ethernet for the SAP HANA software.

By imposing these requirements, SAP can rely on the availability of certain features and ensure a well-performing hardware platform for their SAP HANA software. These requirements give the hardware partners enough room to develop an infrastructure architecture for SAP HANA, which adds differentiating features to the solution. For more information about the benefits of the Lenovo solution, see Chapter 5, “Lenovo System x solutions for SAP HANA” on page 57.

2.3.2 SAP HANA tailored data center integration

To allow for an existing infrastructure to be integrated and reused when SAP HANA is deployed, clients can follow the TDI delivery model. Storage and networks that fulfill certain criteria can be used to run SAP HANA. Among others, these criteria include storage and network performance.

Implementing SAP HANA by following the TDI model requires close collaboration between the client, SAP, and the vendor of the infrastructure element that is integrated. For more information about this SAP delivery model, see this website:

http://www.saphana.com/docs/DOC-3633

IBM released a white paper that gives architectural guidance about how to implement an SAP HANA environment with IBM storage products while following the TDI model. The white paper is available for download at this website:

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102347



http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102347

EMC and IBM released a joint paper about the use of the TDI method to configure a scale-out environment with EMC Symmetrix VMAX Storage and IBM Spectrum Scale. It is available for download at this website:

https://community.emc.com/docs/DOC-35182

Only certain components are eligible for integration. For more information about the list of certified enterprise storage for SAP HANA, see this website:

http://scn.sap.com/docs/DOC-48516

2.3.3 SAP HANA in the cloud

A third option is to deploy SAP HANA in the cloud. Several cloud providers have Infrastructure-as-a-Service (Iaas) or Platform-as-a-Service (Paas) offerings around the SAP HANA in-memory database. SAP applications can continue to run on-premise and utilize the database in the cloud data center or they can also be migrated to the cloud.

The SAP S4/HANA software stack is available as an on-premise or a cloud based offering.

2.4 Sizing SAP HANA

In this section, we introduce the concept of T-shirt sizes and the new shortname concept for SAP HANA. Then, we give a brief introduction about how to size an SAP HANA system.

2.4.1 Memory per core ratio for SAP HANA appliances

For in-memory computing appliances, such as SAP HANA, the amount of main memory is important. In-memory computing brings data that is kept on disk into main memory. This action allows for much faster processing of the data because the CPU cores do not have to wait until the data is loaded from disk to memory, which means each CPU is better used.

There is a certain ideal memory per core ratio. If an in-memory appliance has too much memory per CPU core, the cores cannot fully use the advantage of fast data access. If an in-memory appliance has too little memory per CPU core, the cores are underused and remain idle. In this case, your system does not run as fast as it theoretically can.

This memory per core ratio is defined by SAP. Lenovo is building solutions for SAP HANA following this memory per core ratio.

When SAP introduced SAP HANA into the market, they defined so-called T-shirt sizes to simplify the sizing and to limit the number of hardware configurations to support, which reduces complexity. Each T-shirt size reflects a multiple of the memory to core ratio that was initially allowed by SAP. These T-shirt sizes were used for sizing of the Lenovo eX5™ server platform.

New concept of shortnames for Lenovo solutions for SAP HANAWith the introduction of the Intel Xeon processor E7 v2 generation and the Lenovo X6 server platform in February 2014, the maximum number of cores per CPU socket increased from 10 to 15. This number subsequently increased to 24 cores with the very latest Intel Xeon processor E7 v4 generation. Each core also provides more compute power than a previous generation core. This increase lead to various memory per core ratios that results in many different solution offerings.





To simplify the issue and avoid confusion, Lenovo introduced the following generic naming schema that replaces T-shirt sizes. Instead of S, M, L, and so on, names for different types of building blocks with the XXX-YG-NNNN-Z (dashes can also be left out) naming convention are used, where:

� XXX is the server’s model type� Y is the number of installed processors (or sockets)� G is the processor generation (S meaning Intel Xeon E7 v2, code-named IvyBridge, H

meaning Intel Xeon E7 v3, codenamed Haswell, or B meaning Intel Xeon E7 v4, codenamed Broadwell)

� NNNN is the memory amount in gigabytes� Z indicates a stand-alone (S) or cluster (C) node

For example, model AC3-2S-256-S represents a stand-alone, 4-socket server (AC3-models have four sockets) with two Intel Xeon E7 v2 processors that are installed and 256 GB of memory.

A model AC4-8B-1024-C represents an 8-socket cluster node (AC4-models have eight sockets) with eight Intel Xeon E7 v4 processors that are installed and 1 TB of main memory.

This new naming schema is used in this book for all the X6 based SAP HANA landscapes that are described in Chapter 6, “SAP HANA IT landscapes with X6 solutions” on page 103.

2.4.2 Sizing approach

The sizing of SAP HANA depends on the scenario in which SAP HANA is used. The sizing methodology for SAP HANA is described in the following SAP Notes1:

� Note 1514966 - SAP HANA 1.0: Sizing SAP In-Memory Database� Note 1637145 - SAP NetWeaver BW on HANA: Sizing SAP In-Memory Database� Note 1793345 - Sizing for SAP Suite on HANA� Note 1872170 - Suite on HANA memory sizing

These SAP notes and their attached documents and scripts provide a good starting point for sizing an SAP HANA database environment.

Detailed sizing depends on many different parameters, such as compression factor, the state of the NetWeaver BW system (size of the row store), encoding schema, or cache size, among other parameters.

Special considerations for scale-out BW systemsFor specific information about scale-out SAP NetWeaver BW systems, review the following SAP notes and attached presentations:

� Note 1637145 - SAP NetWeaver BW on SAP HANA: Sizing SAP In-Memory Database

� Note 1702409 - SAP HANA DB: Optimal number of scale out nodes for SAP NetWeaver BW on SAP HANA

� Note 1736976 - Sizing Report for BW on HANA

1 SAP Notes are available at this website (an SAP S-user ID is required):http://service.sap.com/notes


http://service.sap.com/notes

In addition to the sizing methodologies that are described in SAP Notes, SAP provides sizing support for SAP HANA in the SAP Quick Sizer. The SAP Quick Sizer is an online sizing tool that supports most of the SAP solutions that are available. For SAP HANA, it supports sizing for the following systems:

� A stand-alone SAP HANA system, which implements the sizing algorithms that are described in SAP Note 1514966.

� SAP HANA as the database for an SAP NetWeaver BW system, which implements the sizing algorithms that are described in SAP Note 1637145.

� Special sizing support for the SAP HANA rapid-deployment solutions.

The SAP Quick Sizer is available at this website (an SAP S-user ID is required):

http://service.sap.com/quicksizer

2.5 SAP HANA software licensing

As described in 2.2, “SAP S/4HANA overview” on page 14, the prevalent deployment method of SAP HANA is an appliance-like delivery model. Although the hardware partners deliver the infrastructure, including the operating system and middleware, the license for the SAP HANA software must be obtained directly from SAP.

The SAP HANA software is available in two editions (platform and enterprise edition) and the SAP HANA software licensing depends on the use case. For more information about use cases for SAP HANA, see Chapter 4, “SAP HANA integration scenarios” on page 35.

Note: SAP must be included for a detailed sizing because the result of the sizing affects the hardware infrastructure and SAP HANA licensing.

All SAP sizing documents that are described in this section are listed to provide a first estimate only.

Tip: Licensing SAP HANA is handled by SAP and there are different options available, depending on the use case. Because metrics can always change, discuss licensing options for your particular use case with SAP.


http://service.sap.com/quicksizer

Chapter 3. Software components and data replication methods

This chapter describes the purpose of individual software components of the SAP HANA solution and introduces available replication technologies.


� SAP HANA software components� Data replication methods for SAP HANA

3


3.1 SAP HANA software components

The SAP HANA solution is composed of the main software components that are described in the following sections:

� 3.1.1, “SAP HANA database” on page 20� 3.1.2, “SAP HANA client” on page 21� 3.1.3, “SAP HANA studio” on page 22� 3.1.4, “SAP HANA studio repository” on page 26� 3.1.5, “SAP HANA landscape management structure” on page 26� 3.1.6, “SAP host agent” on page 27� 3.1.7, “SAP HANA Database Lifecycle Manager” on page 27� 3.1.8, “SAP HANA Lifecycle Management tools” on page 28

The locations of these components are shown in Figure 3-1.

Figure 3-1 Distribution of software components that are related to SAP HANA

3.1.1 SAP HANA database

The SAP HANA database is the heart of the SAP HANA offering and the most important software component that runs on the SAP HANA appliance.

SAP HANA is an in-memory database that combines row-based and column-based database technology. All standard features that are available in other relational databases are supported (for example, tables, views, indexes, triggers, and SQL interface).

In addition to these standard functions, the SAP HANA database offers modeling capabilities that with which you can define in-memory transformation of relational tables into analytic views. These views are not materialized; therefore, all queries are providing real-time results that are based on the content of the underlying tables.

SAP HANA Appliance

SAP HANAclient

SAP HANAstudio

repository

SAP hostagent

SAP HANALM structure

SAP HANALifecycleManager

SAP HANAstudio

Other optional components:

SMD Agent(optional)

ApplicationServer

SAP HANAclient

Userworkstation

SAP HANAstudio

(optional)

SAP HANAclient

(optional)

ApplicationFunctionLibrary

SAPliveCacheApplicationsSAP HANA

database

Data Modeling

Row Store

Column Store


Another feature that extends the capabilities of the SAP HANA database is the SQLscript programming language, with which you can capture transformations that might not be easy to define by using simple modeling.

The SAP HANA database also can be integrated with external applications, such as an SAP application environment (for example, ERP). By using these possibilities, customers can extend their models by implementing existing statistical and analytical functions that were, for example, developed in ABAP.

For more information about the internal structures of the SAP HANA database, see Chapter 2, “SAP HANA and SAP S/4HANA overview” on page 11.

3.1.2 SAP HANA client

The SAP HANA client is a set of libraries that are used by external applications to connect to the SAP HANA database.

The following interfaces are available after the SAP HANA client libraries are installed:

� SQLDBC

An SAP native database SDK that can be used to develop new custom applications that are working with the SAP HANA database.

� OLE DB for OLAP (ODBO) (available for Windows only)

ODBO is a Microsoft driven industry standard for multi-dimensional data processing. The query language that is used with ODBO is the Multidimensional Expressions (MDX) language.

� Open Database Connectivity (ODBC)

The ODBC interface is a standard for accessing database systems, which was originally developed by Microsoft.

� Java Database Connectivity (JDBC)

JDBC is a Java based interface for accessing database systems.

The SAP HANA client libraries are delivered in 32-bit and 64-bit editions. It is important always to use the correct edition that is based on the architecture of the application that uses this client. The 32-bit applications cannot use 64-bit client libraries and vice versa.

To access the SAP HANA database from Microsoft Excel, you also can use a special 32-bit edition of the SAP HANA client that is called SAP HANA client package for Microsoft Excel.

The SAP HANA client is compatible with earlier versions; that is, the revision of the client must be the same or higher than the revision of the SAP HANA database.

The SAP HANA client libraries must be installed on every machine where connectivity to the SAP HANA database is required, including all servers and user workstations that are hosting applications that are directly connecting to the SAP HANA database (for example, SAP BusinessObjects Client Tools or Microsoft Excel).

Whenever the SAP HANA database is updated to a more recent revision, all clients that are associated with this database also must be upgraded. For more information about how to install the SAP HANA client, see the official SAP guide SAP HANA Database - Client Installation Guide, which is available at this website:

http://help.sap.com/hana_platform

Chapter 3. Software components and data replication methods 21


3.1.3 SAP HANA studio

The SAP HANA studio is a graphical user interface (GUI) that is required to work with local or remote SAP HANA database installations. It is a multipurpose tool that covers all of the main aspects of working with the SAP HANA database. The user interface is slightly different for each function.

The SAP HANA studio does not depend on the SAP HANA client.

The following main function areas are provided by the SAP HANA studio:

� Database administration

The key functions are stopping and starting the SAP HANA databases, status overview, monitoring, performance analysis, parameter configuration, tracing, and log analysis.

The SAP HANA studio user interface for database administration is shown in Figure 3-2.

Figure 3-2 Administration console (overview)


� Security management

This function area provides tools that are required to create users, define and assign roles, and grant database privileges. An example of the user definition window is shown in Figure 3-3.

Figure 3-3 User definition window

� Data management

These functions can be used to create, change, or delete database objects (such as tables, indexes, and views), and include commands to manipulate data (for example, insert, update, delete, or perform a bulk load). An example of the table definition window is shown in Figure 3-4.

Figure 3-4 Table definition window


� Modeling

This GUI is used to work with models (metadata descriptions about how source data is transformed in resulting views), including defining new custom models and adjusting or deleting models. A simple analytic model is shown in Figure 3-5.

Figure 3-5 Modeling interface (analytic view)

� Content management

By using these functions, you can organize models in packages, define delivery units for transport into an SAP HANA system, or export and import individual models or whole packages. Content management functions are accessible from the main window in the modeler perspective, as shown in Figure 3-6.

Figure 3-6 Content functions in the main window of modeler perspective


� Replication management

Data replication into the SAP HANA database is controlled from the data provisioning window in the SAP HANA studio, in which new tables can be scheduled for replication, suspended, or replication for a particular table can be interrupted.

An example of a data provisioning window is shown in Figure 3-7.

Figure 3-7 Data provisioning window

� Lifecycle Management

The SAP HANA solution can be used to download and automatically install updates to SAP HANA software components. This function is controlled from the Lifecycle Manager window in the SAP HANA studio, as shown in Figure 3-8.

Figure 3-8 Lifecycle Manager window


The SAP HANA database queries are used indirectly by using front-end components, such as SAP BusinessObjects BI 4.0 clients. Therefore, the SAP HANA studio is required only for administration or development and is not needed for users.

The SAP HANA studio runs on the Eclipse platform; therefore, every user must have a recent version of the Java Runtime Environment (JRE) installed that uses the same architecture (64-bit SAP HANA studio has 64-bit JRE as a prerequisite).

Currently, Windows 32-bit, Windows 64-bit, and Linux 64-bit are the supported platforms.

The SAP HANA studio also is compatible with earlier versions; therefore, the revision level of the SAP HANA studio must be the same or higher than the revision level of the SAP HANA database. However, based on practical experience, the preferred approach is to keep SAP HANA studio on same revision level as the SAP HANA database whenever possible. Installation and parallel usage of multiple revisions of SAP HANA studio on one workstation is possible. When one SAP HANA studio instance for multiple SAP HANA databases is used, the revision level of the SAP HANA studio must be the same or higher revision level than the highest revision level of the SAP HANA databases to which you are connecting.

SAP HANA studio must be updated to a more recent version on all workstations whenever the SAP HANA database is updated. This process can be automated by using Update Server, which can be configured by using SAP HANA Lifecycle Manager (HLM). (For more information about HLM, see 3.1.7, “SAP HANA Database Lifecycle Manager” on page 27.) The use of HLM is the best way to keep installations of SAP HANA studio synchronized with the SAP HANA database revision.

For more information about how to install the SAP HANA studio, see the official SAP guide, SAP HANA Database - Studio Installation Guide, which is available at this website:


3.1.4 SAP HANA studio repository

Because SAP HANA studio is an Eclipse-based product, it can benefit from all the standard features that are offered by this platform. One of these features is the ability to automatically update the product from a central repository on the SAP HANA server.

The SAP HANA studio repository is initially installed during the deployment of the SAP HANA appliance and must be updated manually when the SAP HANA database is updated. This repository can then be used by all SAP HANA studio installations to download and automatically install new versions of code.

For more information about how to install the SAP HANA studio repository, see the official SAP guide, SAP HANA Database - Studio Installation Guide, which is available at this website:


3.1.5 SAP HANA landscape management structure

The SAP HANA landscape management (LM) structure (lm_structure) is an XML file that describes the software components that are installed on a server. The information in this file contains the following items:

� System ID (SID) of the SAP HANA system and host name� A stack description, including the edition (depending on the license schema)� Information about the SAP HANA database, including the installation directory





� Information about the SAP HANA studio repository, including its location� Information about the SAP HANA client, including its location� Information about the host controller

The LM structure description also contains revisions of individual components; therefore, it must be upgraded when the SAP HANA database is upgraded. Information that is contained in this file is used by the System Landscape Directory (SLD) data suppliers and by the SAP HANA HLM.

3.1.6 SAP host agent

The SAP host agent is a standard part of every SAP installation. In an SAP HANA environment, it is important in the following situations:

� Automatic update by using SAP HANA LM� Remote starting and stopping of SAP HANA database instances

3.1.7 SAP HANA Database Lifecycle Manager

In older SAP HANA releases (SAP HANA SP05 and before), the following tools were available for update, configuration, and customization tasks:

� Software Update Manager (SUM) for SAP HANA that is focused on automating updates of various SAP HANA components and the SAP HANA database.

� SAP HANA On-site Configuration Tool (OCT) for SAP HANA automated customization of SAP HANA appliances, which performs postinstallation steps, such as deployment of other components (such as adding or removing a host or deploying a secondary system).

In later SAP HANA release (SPS07 and beyond), these tools are marked as deprecated and replaced by SAP HLM. The SAP HLM is a tool that automates many installation, configuration, and deployment tasks that are related to SAP HANA environments. It offers the following functions:

� Rename an SAP HANA system (SID, instance number, or host name)� Configure SAP Landscape Transformation (SLT) replication� Configure the SAP HANA connection to System Landscape Directory (SLD)� Add/Remove Solution Manager Diagnostics (SMD) agent� Add/Remove extra SAP HANA system (MCOS installation)� Add/Remove hosts to/from scale-out (or distributed) SAP HANA installation� Configure SAP HANA system internal network� Update SAP HANA System - Apply Support Package Stack� Update SAP HANA System - Apply Single Support Packages� Add/Update Application Function Libraries (AFL) on an SAP HANA system� Add/Update SAP liveCache Applications (LCA) on an SAP HANA system� Add SAP HANA Smart Data Access (SDA) on an SAP HANA system

SAP HANA Lifecycle Manager can be run in the following ways:

� Locally or remotely as part of SAP HANA studio� Locally from the Linux command line on an SAP HANA server node� Locally or remotely through a web interface

For more information about SAP HANA Lifecycle Manager, see the official SAP guide, SAP HANA Update and Configuration Guide, which is available at this website:




3.1.8 SAP HANA Lifecycle Management tools

The SAP HANA Unified Installer was a tool that was targeted to be used by SAP HANA hardware partners. It automatically installed all required software components on the SAP HANA appliance according to SAP requirements and specifications.

With the release of SAP HANA SPS07, the SAP HANA Unified Installer is marked as deprecated. However, it is still available as a part of the standard delivery package.

SAP HANA LM tools (also referred to and available in Linux as hdblcm or hdblcmgui) are delivered as the replacement for SAP HANA Unified Installer. Although hdblcm is a text-based Linux command-line utility that can be run in interactive or batch mode, its counterpart hdblcmgui delivers the same functions in a GUI.

Like Unified Installer, these two utilities are intended for hardware vendors to automate the initial deployment of SAP HANA on their appliance models. System administrators are expected to use SAP HLM, as described in 3.1.7, “SAP HANA Database Lifecycle Manager” on page 27.

The SAP HANA LM tools include the following features:

� Install SAP HANA system (single-node or scale-out)

� Add host to scale-out SAP HANA system

� Enable or disable auto-start of SAP HANA database instance

� Regeneration of SSL certificates that are used by SAP HANA LM

� Installation and update of the following SAP HANA components:

– Application Function Library– SAP HANA Client– SAP HANA Studio– SAP HANA Database server– SAP HANA Lifecycle Manager (HLM)– SAP liveCache applications

� Update of server-side studio repository

� Install and update SAP host agent

For more information about SAP HANA LM tools, see the official SAP guide, SAP HANA Server Installation Guide, which is available at this website:


3.1.9 Solution Manager Diagnostics agent

SAP HANA can be connected to SAP Solution Manager 7.1, SP03 or higher.1 It is preferable to use SP05 or higher.

The SMD provides a set of tools to monitor and analyze SAP systems, including SAP HANA. It also provides a centralized method to trace problems in all systems that are connected to an SAP Solution Manager system.

1 With monitor content update and more SAP notes for SP02




The SMD agent is an optional component, which can be installed on the SAP HANA appliance. It enables diagnostic tests of the SAP HANA appliance through SAP Solution Manager. The SMD agent provides access to the database logs and the file system, and collects information about the system’s CPU and memory consumption through the SAP host agent.

For more information about how to deploy SMD agent, see the official SAP guide, SAP HANA Update and Configuration Guide, which is available at this website:


3.2 Data replication methods for SAP HANA

Data can be written to the SAP HANA database directly by a source application or replicated by using replication technologies.

The following replication methods are available for use with the SAP HANA database:

� Trigger-based replication

This method is based on database triggers that are created in the source system to record all changes to monitored tables. These changes are then replicated to the SAP HANA database by using the SAP Landscape Transformation system.

� ETL-based replication

This method uses an Extract, Transform, and Load (ETL) process to extract data from the data source, transform it to meet the business or technical needs, and load it into the SAP HANA database. The SAP BusinessObject Data Services application is used as part of this replication scenario.

� Extractor-based replication

This approach uses the embedded SAP NetWeaver Business Warehouse (SAP NetWeaver BW) that is available on every SAP NetWeaver based system. SAP NetWeaver BW starts an extraction process by using available extractors and then redirects the write operation to the SAP HANA database instead of the local Persistent Staging Area (PSA).

� Log-based replication

This method is based on reading the transaction logs from the source database and reapplying them to the SAP HANA database.



These replication methods are shown in Figure 3-9.

Figure 3-9 Available replication methods for SAP HANA

The following sections describe these replication methods for SAP HANA.

3.2.1 Trigger-based replication with SAP Landscape Transformation

SAP Landscape Transformation (SLT) replication is based on tracking database changes by using database triggers. All modifications are stored in logging tables in the source database, which ensures that every change is captured, even when the SLT system is not available.

The SLT system reads changes from source systems and updates the SAP HANA database. The replication process can be configured as real time (continuous replication) or scheduled replication in predefined intervals.

The SLT operates on the application level; therefore, the trigger-based replication method benefits from the database abstraction that is provided by the SAP software stack, which makes it database-independent. It also features extended source system release coverage, where supported releases start from SAP R/3 4.6C up to the newest SAP Business Suite releases.

The SLT also supports direct replication from database systems that are supported by the SAP NetWeaver platform. In this case, the database must be connected to the SLT system directly (as another database) and the SLT plays the role of the source system.

The replication process can be customized by creating ABAP routines and configuring their execution during the replication process. This feature allows the SLT system to replicate more calculated columns and to scramble existing data or filter-replicated data that is based on defined criteria.

The SLT replication uses proven System Landscape Optimization (SLO) technologies, such as Near Zero Downtime, Test Data Migration Server (TDMS), and SLT, and can handle Unicode and non-Unicode source databases. The SLT replication provides a flexible and reliable replication process, fully integrates with SAP HANA Studio, and is simple and fast to set up.

Source SystemSAP ERP

Trigger-Based Replication

ETL-Based Replication

Log-Based ReplicationDatabase Log

File

Application Layer

Embedded BWExtractor-Based Replication

SAP HANAdatabase


The SLT Replication Server does not have to be a separate SAP system. It can run on any SAP system with the SAP NetWeaver 7.02 ABAP stack (Kernel 7.20EXT). However, it is preferable to install the SLT Replication Server on a separate system to avoid a high replication load that affects the base system performance.

The SLT Replication Server is the ideal solution for all SAP HANA customers who need real-time (or scheduled) data replication from SAP NetWeaver based systems or databases that are supported by SAP NetWeaver.

3.2.2 ETL-based replication with SAP BusinessObjects Data Services

An ETL-based replication for SAP HANA can be set up by using SAP BusinessObjects Data Services, which is a full-featured ETL tool that gives customers the following maximum flexibility regarding the source database system:

� Customers can specify and load the relevant business data in defined periods from an SAP ERP system into the SAP HANA database.

� SAP ERP application logic can be reused by reading extractors or by using SAP function modules.

� It offers options for the integration of third-party data providers and supports replication from virtually any data source.

Data transfers are performed in batch mode, which limits the real-time capabilities of this replication method.

SAP BusinessObjects Data Services provide several kinds of data quality and data transformation functions. Because of the rich feature set that is available, implementation time for the ETL-based replication is longer than for the other replication methods. SAP BusinessObjects Data Services offer integration with SAP HANA. SAP HANA is available as a predefined data target for the load process.

The ETL-based replication server is the ideal solution for all SAP HANA customers who need data replication from non-SAP data sources.

3.2.3 Extractor-based replication with Direct Extractor Connection

Extractor-based replication for SAP HANA is based on application logic that is available in every SAP NetWeaver system. The SAP NetWeaver BW package that is a standard part of the SAP NetWeaver platform can be used to run an extraction process and store the extracted data in the SAP HANA database.

This function requires some corrections and configuration changes to the SAP HANA database (import of delivery unit and parameterization) and on the SAP NetWeaver BW system as part of the SAP NetWeaver platform (implementing corrections by using an SAP note or installing a support package and parameterization). Corrections in the SAP NetWeaver BW system ensure that extracted data is not stored in local Persistent Staging Area (PSA), but diverted to the external SAP HANA database.

The use of native extractors instead of the replication of underlying tables can bring certain benefits. Extractors offer the same transformations that are used by SAP NetWeaver BW systems, which can decrease the complexity of modeling tasks in the SAP HANA database.

This type of replication is not real time, and only the available features and transformation capabilities that are provided by a specific extractor can be used.


Replication by using Direct Extractor Connection (DXC) can be achieved in the following basic scenarios:

� By using the embedded SAP NetWeaver BW function in the source system

SAP NetWeaver BW functions in the source system often are not used. After the implementation of the required corrections, the source system calls its own extractors and pushes data into the external SAP HANA database.

The source system must be based on SAP NetWeaver 7.0 or higher. Because the function of a specific extractor is diverted into SAP HANA database, this extractor must not be in use by the embedded SAP NetWeaver BW component for any other purpose.

� By using an SAP NetWeaver BW to drive replication

An SAP NetWeaver BW can be used to extract data from the source system and to write the result to the SAP HANA system.

The release of the SAP NetWeaver BW system that is used must be at least SAP NetWeaver 7.0, and the specific extractor must not be in use for this particular source system.

� By using a dedicated SAP NetWeaver BW to drive replication

The last option is to install a dedicated SAP NetWeaver system to extract data from the source system and store the result in the SAP HANA database. This option has a minimal effect on functions because no system is changed in any way. However, a new system is required for this purpose.

The current implementation of this replication technology allows for only one database schema in the SAP HANA database. The use of one system for controlling the replication of multiple source systems can lead to collisions because all source systems use the same database schema in the SAP HANA database.

3.2.4 Log-based replication with Sybase Replication Server

The log-based replication for SAP HANA is realized by the Sybase Replication Server. It captures table changes from low-level database log files and transforms them into SQL statements that are in turn run on the SAP HANA database. This action is similar to what is known as log shipping between two database instances.

Replication with the Sybase Replication Server is fast and uses little processing power because of its closeness to the database system. However, this mode of operation makes this replication method highly database-dependent, and the source database system coverage is limited.2 It also limits the conversion capabilities; therefore, replication with the Sybase Replication Server supports only Unicode source databases. The Sybase Replication Server cannot convert between code pages, and because SAP HANA works with Unicode encoding internally, the source database also must use Unicode encoding. Also, certain table types that are used in SAP systems are unsupported.

To set up replication with the Sybase Replication Server, the definition and content of tables that are chosen to be replicated must be copied initially from the source database to the SAP HANA database. This initial load is done with the R3Load program, which is also used for database imports and exports. Changes in tables during initial copy operation are captured by the Sybase Replication Server; therefore, no system downtime is required.

2 Only certain versions of IBM DB2 on AIX, Linux, and HP-UX are supported by this replication method.


This replication method is recommended only for SAP customers who were invited to use it during the ramp-up of SAP HANA 1.0. It was part of the SAP HANA Enterprise Extended Edition, which was discontinued with the introduction of SAP HANA 1.0 SPS05 in November 2012.3

SAP recommends that you use trigger-based data replication by using the SAP Landscape Transformation Replicator.

3.2.5 Comparing the replication methods

Each of the described data replication methods for SAP HANA has the following benefits and weaknesses:

� The trigger-based replication method with the SLT system provides real-time replication while supporting a wide range of source database systems. It can handle Unicode and non-Unicode databases and use proven data migration technology. It uses the SAP application layer, which limits it to SAP source systems. Compared to the log-based replication method, it offers a broader support of source systems while providing almost similar real-time capabilities. For this reason, it is preferred for replication from SAP source systems.

� The ETL-based replication method is the most flexible; however, its flexibility comes at a cost because it has only near real-time capabilities. With its various possible data sources, advanced transformation, and data quality functions, it is the ideal choice for replication from non-SAP data sources.

� The extractor-based replication method offers reuse of transformation capabilities that are available in every SAP NetWeaver based system. This method can decrease the required implementation effort. However, this type of replication is not real time and is limited to capabilities that are provided by the available extractors in the source system.

� The log-based replication method with the Sybase Replication Server provides the fastest replication from the source database to SAP HANA. However, it is limited to Unicode-encoded source databases and it does not support all table types that are used in SAP applications. It provides no transformation function, and the source database system support is limited.

3 For more information, see section 2.1.3 at this website:http://help.sap.com/hana/hana_sps5_whatsnew_en.pdf


http://help.sap.com/hana/hana_sps5_whatsnew_en.pdf

These replication methods are compared in Figure 3-10.

Figure 3-10 Comparison of the replication methods for SAP HANA

The replication method that you choose depends on the requirements. When real-time replication is needed to provide benefit to the business and the replication source is an SAP system, the trigger-based replication is the best choice.

Extractor-based replication might keep project costs down by reusing existing transformations.

ETL-based replication provides the most flexibility regarding data source, data transformation, and data cleansing options, but does not provide real-time replication.

Sybase Replication Server

SAP Business ObjectsData Services

SAP LT System

Real Real-Time

Real-Time

Near Real-Time

Unicode onlyVery limited DB support

Many DBs supportedUnicode and Non-Unicodeon Application Layer

Any DatasourceTransformationData Cleansing

Preferred by SAP

Rea

l-Tim

e C

apab

ilitie

s

Dat

a C

onve

rsio

n C

apab

ilitie

s

Direct Extractor Connection

SAP NetWeaver 7.0+Re-use of extractorsTransformation


Chapter 4. SAP HANA integration scenarios

This chapter describes the different ways that SAP HANA can be implemented in client landscapes and highlights various aspects of such an integration. Whenever possible, real-world examples and related offerings are included.


� Basic use case scenarios� SAP HANA as a technology platform� SAP HANA for operational reporting� SAP HANA as an accelerator� SAP products that are running on SAP HANA� Programming techniques that use SAP HANA

4


4.1 Basic use case scenarios

The following classification of use cases was presented in the “EIM205 Applications powered by SAP HANA” session during the SAP TechEd 2011 event:

� Technology platform� Operational reporting� Accelerator� In-memory products� Next generation applications

These use case scenarios are shown in Figure 4-1.

Figure 4-1 Basic use case scenarios that are defined by SAP in session EIM205

These five basic use case scenarios describe the ways that SAP HANA can be integrated. Each of these use case scenarios is described in this chapter.

SAP maintains a “SAP HANA Use Case Repository” with specific examples for how SAP HANA can be integrated. This repository is available at this website:

http://www.experiencesaphana.com/community/resources/use-cases

The use cases in this repository are divided into categories that are based on their relevance to a specific industry sector. It is a good idea to review this repository to find inspiration about how SAP HANA can be used in various scenarios.

4.2 SAP HANA as a technology platform

SAP HANA can be used even in non-SAP environments. The client can use structured and unstructured data that is derived from non-SAP application systems to use SAP HANA power. SAP HANA can be used to accelerate functions or to provide new functions that were, until now, unrealistic.

Technologyplatform

SAP HANA

Data Modeling

Column Store

Row Store

OperationalReporting

AcceleratorIn-MemoryProducts

NextGeneration

Applications


http://www.experiencesaphana.com/community/resources/use-cases

SAP HANA as a technology platform is shown in Figure 4-2.

Figure 4-2 SAP HANA as technology platform

SAP HANA is not technologically dependent on other SAP products and can be used independently as the only one SAP component in the client’s information technology (IT) landscape. However, SAP HANA can be easily integrated with other SAP products, such as SAP BusinessObjects BI platform for reporting or SAP BusinessObjects Data Services for Extract, Transform, and Load (ETL) replication, which gives clients the possibility to use only the components that are needed.

There are many ways that SAP HANA can be integrated into a client landscape, and it is not possible to describe all combinations. Software components around the SAP HANA offering can be seen as building blocks, and every solution must be assembled from the blocks that are needed in a particular situation. This approach is versatile and the number of possible combinations is growing because SAP constantly adds components to their SAP HANA-related portfolio.

Lenovo offers consulting services that help clients to choose the correct solution for their business needs.

4.2.1 SAP HANA data acquisition

There are multiple ways that data can flow into SAP HANA. This section describes the various options that are available, as shown in Figure 4-3.

Figure 4-3 Examples of SAP HANA deployment options regarding data acquisition

Non-SAPor SAP

data source

SAP HANA

Data Modeling

Column Store

Row Store SAPReporting

and Analytics

Non-SAPapplication

Non-SAPapplication

Customdatabase

Non-SAPapplication SAP HANA

SAP HANA

Non-SAPapplication

Customdatabase

SAP HANA

Data replication

Non-SAPapplication

Customdatabase

Current situation Replacing existing database

Dual database approachData replication

Chapter 4. SAP HANA integration scenarios 37

The initial situation is displayed schematically in the upper left of Figure 4-3 on page 37. In this example, a client-specific non-SAP application writes data to a custom database that is slow and is not meeting client needs.

The other three examples in Figure 4-3 on page 37 show that SAP HANA can be deployed in such a scenario. These examples show that there is no single solution that is best for every client, but that each situation must be considered independently.

Each of these three solutions has the following advantages and disadvantages, which are highlighted to show the aspects of a specific solution that might need further consideration:

� Replacing the existing database with SAP HANA

The advantage of this solution is that the overall architecture is not going to be significantly changed. The solution remains simple without the need to include more components. Customers might also save on license costs for the original database.

A disadvantage to this solution is that the custom application must be adjusted to work with the SAP HANA database. If ODBC or JDBS is used for database access, this problem is insignificant. Also, the entire setup must be tested properly. Because the original database is being replaced, a certain amount of downtime is inevitable.

Clients that are considering this approach must be familiar with the features and characteristics of SAP HANA, especially when certain requirements must be met by the database that is used (for example, special purpose databases).

� Populating SAP HANA with data replicated from the existing database

This option integrates SAP HANA as a “side-car” database to the primary database and replicates required data by using one of the available replication techniques.

An advantage of this approach is that the original solution is not touched and no downtime is required. Also, only the required subset of data must be replicated from the source database, which might allow customers to minimize acquisition costs because SAP HANA acquisition costs are linked directly to the volume of stored data.

The need for implementing replication technology can be seen as the only disadvantage of this solution. Because data is delivered only into SAP HANA through replication, this component is a vital part of the whole solution. Customers that are considering this approach must be familiar with various replication technologies (including their advantages and disadvantages) as described in 3.2, “Data replication methods for SAP HANA” on page 29.

Clients must also be aware that replication might cause extra load on the database because modified records must be extracted and then transported to the SAP HANA database. This aspect is highly dependent on the specific situation and can be addressed by choosing the proper replication technology.

� Adding SAP HANA as a second database in parallel to the existing database

This option keeps the existing database in place while adding SAP HANA as a secondary database. The custom application then stores data in the original database and in the SAP HANA database.

This option balances the advantages and disadvantages of the other options. A main prerequisite is the ability of the source application to work with multiple databases and the ability to control where data is stored. This prerequisite can be easily achieved if the source application was developed by the client and can be changed, or if the source application is going to be developed as part of this solution. If this prerequisite cannot be met, this option is not viable.


An advantage of this approach is that no replication is required because data is stored directly in SAP HANA as required. Customers also can store some of the records in both databases.

If data that is stored in the original database is not going to be changed and SAP HANA data is stored in both databases simultaneously, customers might achieve only minimal disruption to the existing solution.

A main disadvantage is the prerequisite that the application must work with multiple databases and store data according to the customer’s expectations.

Customers considering this option must be aware of the abilities that are provided by the application that is delivering data into the existing database. Also, disaster recovery plans must be carefully adjusted, especially when consistency between both databases is seen as a critical requirement.

These examples must not be seen as a complete list of integration options for an SAP HANA implementation, but rather as a demonstration of how to develop a solution that matches client needs.

The SAP HANA database can be populated with data from different sources, such as SAP or non-SAP applications, and custom databases. These sources can feed data into SAP HANA independently, each by using a different approach or in a synchronized manner by using the SAP BusinessObjects Data Services.

4.2.2 SAP HANA as a source for other applications

The second part of integrating SAP HANA is to connect existing or new applications to run queries against the SAP HANA database, as shown in Figure 4-4.

Figure 4-4 Example of SAP HANA as a source for other applications

The initial situation is shown on the left side of Figure 4-4. A customer-specific application runs queries against a custom database, which is a function that must be preserved.

A potential solution is shown on the right side of Figure 4-4. A customer-specific application runs problematic queries against the SAP HANA database. If the database is still part of the solution, specific queries that do not need acceleration can still be run against the original database.

Possible scenario

Non-SAPapplication

Customdatabase

Current situation

SAP HANA

Non-SAPapplication

SAP analytictools

SAP BOBJreporting

Customdatabase


Specialized analytic tools, such as SAP BusinessObjects Predictive Analysis, can be used to run statistical analysis on data that is stored in the SAP HANA database. This tool can run analysis directly inside the SAP HANA database, which helps avoid expensive transfers of massive volumes of data between the application and the database. The result of this analysis can be stored in SAP HANA, and the custom application can use these results for further processing; for example, to facilitate decision making.

SAP HANA can be easily integrated with products from the SAP BusinessObjects family. Therefore, these products can be part of the solution, which are responsible for reporting, monitoring critical key performance indicators (KPIs) that use dashboards, or for data analysis.

These tools can also be used without SAP HANA; however, SAP HANA is enabling these tools to process much larger volumes of data and still provide results in reasonable time.

4.3 SAP HANA for operational reporting

Operational reporting is playing a more important role. In today’s economic environment, companies must understand how various events in the globally integrated world affect their business to make proper adjustments to counter the effects of these events. Therefore, the pressure to minimize the delay in reporting is becoming higher. An ideal situation is to use a real-time snapshot of current situations within seconds of a request.

Concurrently, the amount of data that is being captured grows every year. More information is collected and stored at more detailed levels. All of these issues make operational reporting more challenging because huge amounts of data must be processed quickly to produce the preferred result.

SAP HANA is a perfect fit for this task. Required information can be replicated from transactional systems into the SAP HANA database and then processed faster than directly on the source systems.

The following use case is often referred to as a data mart or side-car approach because SAP HANA sits by the operational system and receives the operational data (often only an excerpt) from this system by using replication.

You find many systems In a typical SAP-based application landscape today, such as SAP ERP, SAP CRM, SAP SCM, and other, possibly non-SAP, applications. All of these systems contain loads of operational data, which can be used to improve business decision making by using business intelligence technology. Data that is used for business intelligence purposes can be gathered on a business unit level by using data marts or on an enterprise level with an enterprise data warehouse, such as the SAP NetWeaver Business Warehouse (SAP NetWeaver BW). ETL processes feed the data from the operational systems into the data marts and the enterprise data warehouse.


A typical landscape is shown in Figure 4-5.

Figure 4-5 Typical view of an SAP-based application landscape today

With the huge amount of data that is collected in an enterprise data warehouse, response times of queries for reports or navigation through data can become an issue, which generates new requirements for the performance of such an environment. To address these requirements, SAP introduced the SAP NetWeaver Business Warehouse Accelerator (SAP NetWeaver BW Accelerator), which speeds up queries and reports in the SAP NetWeaver BW by using in-memory technology. Although being a perfect fit for an enterprise data warehouse holding huge amounts of data, the combination of SAP NetWeaver BW and SAP NetWeaver BW Accelerator is not always a viable solution for relatively small data marts.

With the introduction of SAP HANA 1.0, SAP provided an in-memory technology that supports Business Intelligence (BI) at a business unit level. SAP HANA combined with business intelligence tools, such as the SAP BusinessObjects tools and data replication mechanisms feeding data from the operational system into SAP HANA in real time, brought in-memory computing to the business unit level. Figure 4-6 shows such a landscape with the local data marts replaced by SAP HANA.

Figure 4-6 SAP vision after the introduction of SAP HANA 1.0

SAP ERP 1(or CRM, SRM, SCM)

Database

BIDataMart

DB

Local BI

DataMart

DB

DataMart

DB

SAP ERP n(or CRM, SRM, SCM)

Database

BI

Non-SAPBusiness

Application

DatabaseETL

. . .

ETL

Enterprise Data Warehouse (BW)

Database BWA

Corporate BI

SAPHANA

1.0

SAPHANA

1.0


Database

xSAP

HANA

1.0

Local BI


Database

Non-SAPBusiness

Application

DatabaseSync

. . .


Database Accelerator

Corporate BI


BI functionality is provided by an SAP BusinessObjects BI tool, such as the SAP BusinessObjects Explorer, which communicates with the SAP HANA database through the BI Consumer Services (BICS) interface.

This use case scenario is for products from the SAP Business Suite, where SAP HANA acts as a foundation for reporting on large volumes of data.

The role of SAP HANA in an operational reporting use case scenario is shown in Figure 4-7.

Figure 4-7 SAP HANA for operational reporting

The first step in the process is the replication of data into the SAP HANA database, which often originates from the SAP Business Suite. However, some solution packages are also built for non-SAP data sources.

There are instances in which source systems must be adjusted by implementing modifications or by performing specific configuration changes.

Data often is replicated by using the SAP Landscape Transformation replication; however, other options, such as replication by using SAP BusinessObjects Data Services or SAP HANA Direct Extractor Connection (DXC), also are possible. The replication technology often is chosen as part of the package design and cannot be changed easily during implementation.

A list of tables to replicate (for SAP Landscape Transformation replication) or transformation models (for replication by using Data Services) are part of the package.

SAP HANA is loaded with models (views) that are static (designed by SAP and packaged) or automatically generated based on customized criteria. These models describe the transformation of source data into the resulting column views. These views are then used by SAP BusinessObjects BI 4.0 reports or dashboards that are delivered as final products or pre-made templates that can be finished as part of implementation process.

Some solution packages are based on more components (for example, SAP BusinessObjects Event Insight). If required, extra content that is specific to these components can also be part of the solution package.

Individual use cases, required software components, prerequisites, and configuration changes (including overall implementation processes) are properly documented and attached as part of the delivery.

SAPBusiness

Suite

RDBMSrepl.

SAP HANA

Data Modeling

Column Store

Row Store

SAPReporting

and Analytics


Solution packages can contain the following items:

� SAP BusinessObjects Data Services Content (data transformation models)

� SAP HANA Content (exported models with attribute views and analytic views)

� SAP BusinessObjects BI Content (prepared reports and dashboards)

� Transports and ABAP reports (adjusted code to be implemented in source system)

� Content for other software components, such as SAP BusinessObjects Event Insight and Sybase Unwired Platform

� Documentation

Packaged solutions such as these are being delivered by SAP under the name SAP Rapid Deployment Solutions (RDSs) for SAP HANA or by other system integrators, such as Lenovo.

Available offerings contain everything that customers need to implement the requested function. Associated services, including implementation, can also be part of delivery.

Although SAP HANA as a technology platform can be seen as an open field where every client can build their own solution by using available building blocks, the SAP HANA for operational reporting scenarios is prepared packaged scenarios that can easily and quickly be deployed on existing landscapes.

For more information about SAP RDS offerings, see this website:

http://www.sap.com/resources/solutions-rapid-deployment/solutions-by-business.epx

At the following website, click Technology → SAP HANA:

http://service.sap.com/solutionpackages

4.4 SAP HANA as an accelerator

SAP HANA in a side-car approach as an accelerator is similar to a side-car approach for reporting purposes. The difference is that the data user that is replicated to SAP HANA is not a BI tool but the source system. The source system can use the in-memory capabilities of the SAP HANA database to run analytical queries on the replicated data. This feature helps applications that are performing queries on huge amounts of data to run simulations, pattern recognition, planning runs, and so on.

SAP HANA also can be used to accelerate processes in SAP Business Suite systems, even for those systems that are not yet released to be running directly on the SAP HANA database.

Some SAP systems are processing large amounts of records that must be filtered or aggregated based on specific criteria. Results are then used as inputs for all dependent activities in a specific system.

In the case of large data volumes, the running time can be unacceptable (in number of hours). Such workloads can easily run several hours, which can cause unnecessary delays. Currently, these tasks often are processed overnight as batch jobs.

SAP HANA as an accelerator can help decrease this running time.




http://service.sap.com/solutionpackages

This use case scenario is shown in Figure 4-8.

Figure 4-8 SAP HANA as an accelerator

The accelerated SAP system must meet specific prerequisites. Before this solution can be implemented, installation of specific support packages or implementation of SAP Notes might be required, which introduces the necessary code changes in the source system.

The SAP HANA client must be installed on a specific server, and the SAP kernel must be adjusted to support direct connectivity to the SAP HANA database.

As a next step, replication of data from the source system is configured. Each specific use case features a defined replication method and a list of tables that must be replicated. The most common method is the SAP Landscape Transformation replication. However, some solutions offer alternatives. For example, for the SAP CO-PA Accelerator, replication can also be performed by an SAP CO-PA Accelerator-specific ABAP report in the source system.

The source system is configured to have direct connectivity into SAP HANA as the secondary database. The required scenario is configured according to the specifications and then activated. During activation, the source system automatically deploys the required column views into SAP HANA and activates new ABAP code that was installed in the source system as the solution prerequisite. This new code can run and then use queries against the SAP HANA database, which leads to shorter execution times.

Because SAP HANA is populated with valuable data, it is easy to extend the accelerator use case by adding operational reporting functions. More (often optional) content is delivered for SAP HANA and for SAP BusinessObjects BI 4.0 client tools, such as reports or dashboards.

SAP HANA as the accelerator and SAP HANA for operational reporting use case scenarios can be combined in to a single package. The following SAP RDSs implement SAP HANA as an accelerator:

� SAP Bank Analyzer Rapid-Deployment Solution for Financial Reporting with SAP HANA (see SAP Note 1626729):

http://service.sap.com/rds-hana-finrep

� SAP rapid-deployment solution for customer segmentation with SAP HANA (see SAP Note 1637115):

http://service.sap.com/rds-cust-seg

� SAP ERP rapid-deployment solution for profitability analysis with SAP HANA (see SAP Note 1632506):

http://service.sap.com/rds-hana-copa

SAPBusiness

Suite

RDBMSSAP HANA

Data Modeling

Column Store

Row Storerepl.

SAP UI

read

SAPReporting

and Analytics


http://service.sap.com/rds-hana-copa


http://service.sap.com/rds-cust-seg


� SAP ERP rapid-deployment solution for accelerated finance and controlling with SAP HANA (see SAP Note 1656499):

http://service.sap.com/rds-hana-fin

� SAP Global Trade Services rapid-deployment solution for sanctioned-party list screening with SAP HANA (see SAP Note 1689708):

http://service.sap.com/rds-gts

4.5 SAP products that are running on SAP HANA

Another way that SAP HANA can be deployed is to use SAP HANA as the primary database for selected products.

SAP NetWeaver BW that is running on SAP HANA was available since April 2012. The SAP ERP Central Component (SAP ECC) that is running on HANA was announced in early 2013, and the remaining products of the SAP Business Suite family became available in the second half of 2013. In 2015 SAP S/4HANA was announced, which is the new SAP Business Suite 4 SAP HANA and optimized to run on SAP HANA only.

One significant advantage of running products to use SAP HANA as the primary database is the minimal disruption to the existing system. Almost all functions, customizations, and (with SAP NetWeaver BW) client-specific modeling, are preserved because application logic that is written in ABAP is not changed. From a technical perspective, the SAP HANA conversion is similar to any other database migration.

Figure 4-9 shows the different SAP products running with SAP HANA.

Figure 4-9 SAP products running on SAP HANA - SAP S/4HANA, SAP NetWeaver BW and SAP ECC

4.5.1 SAP NetWeaver Business Warehouse that is powered by SAP HANA

SAP HANA can be used as the database for an SAP NetWeaver BW installation. In this scenario, SAP HANA replaces the traditional database server of an SAP NetWeaver BW installation. The application servers stay the same.

SAP Business

Suite

RDBMS

SAP BW

SAP HANA

Column Store

Row Store

traditional extraction SAP ECC

SAP HANA

Column Store

Row Store

SAP ECC on SAP HANA SAP BW on SAP HANA

SAP S/4HANA

SAP HANA

Column Store

Row Store

SAP S/4HANA



http://service.sap.com/rds-gts


The in-memory performance of SAP HANA improves query performance and eliminates the need for manual optimizations by materialized aggregates in SAP NetWeaver BW. Figure 4-10 shows SAP HANA as the database for the SAP NetWeaver BW.

Figure 4-10 SAP HANA as the database for SAP NetWeaver BW

In contrast to an SAP NetWeaver BW system that is accelerated by the in-memory capabilities of SAP NetWeaver BW Accelerator, an SAP NetWeaver BW system with SAP HANA as the database keeps all data in-memory. With SAP NetWeaver BW Accelerator, the client chooses the data to be accelerated, which is then copied to the SAP NetWeaver BW Accelerator. Here, the traditional database server (for example, IBM DB2 or Oracle) still acts as the primary data store.

SAP NetWeaver BW on SAP HANA often is the most popular SAP HANA use case, which achieves performance improvements with relatively small efforts.

The underlying database is replaced by the SAP HANA database, which improves data loading times and query run times. Because the application logic that is written in ABAP is not affected by this change, all investments in developing BW models are preserved. The transition to SAP HANA is a transparent process that requires minimal effort to adjust existing modeling.

In-memory optimized InfoCubesInfoCubes in SAP NetWeaver BW that is running on traditional database use the so-called Enhanced Star Schema. This schema was designed to optimize different performance aspects of working with multidimensional models on database systems.

SAPHANA


Database

SAPHANA

Local BI


Database

Non-SAPBusiness

Application

Database

. . .

Local BI


Corporate BI

Local BISAP HANA

VirtualData Mart

VirtualData Mart

VirtualData Mart


The Enhanced Star Schema in SAP NetWeaver BW with an example is shown in Figure 4-11.

Figure 4-11 Enhanced Star Schema in SAP NetWeaver BW

The core part of every InfoCube is the fact table. This table contains dimension identifiers (IDs) and corresponding key figures (measures). This table is surrounded by dimension tables that are linked to fact tables by using the dimension IDs.

Dimension tables are small tables that group logically connected combinations of characteristics, usually representing master data. Logically connected means that the characteristics are highly related to each other; for example, company code and plant. Combining unrelated characteristics leads to many possible combinations, which can have a negative effect on the performance.

Because master data records are in separate tables outside of the InfoCube, another table is required to connect these master data records to dimensions. These extra tables contain a mapping of auto-generated Surrogate IDs (SIDs) to the real master data.

KEY_0COPC_C08P

KEY_0COPC_C08T

KEY_0COPC_C08U

KEY_0COPC_C081

KEY_0COPC_C082

KEY_0COPC_C083

KEY_0COPC_C084

KEY_0COPC_C085

AMOUNTFX

AMOUNTVR

PRDPLN_QTY

LOTSIZE_CM

Fact Table:/BI0/F0COPC_C08

DIMID

SID_0CHNGID

SID_0RECORDTP

SID_0REQUID

Data PackageDimension Table:/BI0/D0COPC_C08P

DIMID

SID_0COMP_CODE

SID_0PLANT

Enterprise StructureDimension Table:/BI0/D0COPC_C081

COMP_CODE

SID

CHCKFL

DATAFL

INCFL

Company CodeSID Table:/BI0/SCOMP_CODE

PLANT

SID

CHCKFL

DATAFL

INCFL

PlantSID Table:/BI0/SPLANT

COMP_CODE

OBJVERS

CHANGED

CHRT_ACCTS

COMPANY

COUNTRY

...

Company CodeMaster Data Table:/BI0/PCOMP_CODE

PLANT

OBJVERS

CHANGED

ALTITUDE

BPARTNER

...

PlantMaster Data Table :/BI0/PPLANT

DIMID

SID_0MATERIAL

SID_0MAT_PLANT

MaterialDimension Table:/BI0/D0COPC_C082


This complex structure is required on classical databases; however, with SAP HANA, this requirement is obsolete. Therefore, SAP introduced the SAP HANA Optimized Star Schema, as shown in Figure 4-12.

Figure 4-12 SAP HANA Optimized Star Schema in an SAP NetWeaver BW system

The content of all dimensions (except for the Data Package dimension) is incorporated into the fact table. This modification brings the following advantages:

� Simplified modeling

Poorly designed dimensions (wrong combinations of characteristics) no longer can affect performance. Moving characteristics from one dimension to another is not a physical operation; instead, it is a metadata update.

� Faster loading

Because dimension tables do not exist, all overhead workload that is related to the identification of combinations or creating combinations in the dimension tables is no longer required. Instead, the required SID values are inserted directly into the fact table.

The SAP HANA Optimized Star Schema is used automatically for all created InfoCubes on the SAP NetWeaver BW system that are running on the SAP HANA database.

InfoCubes are not automatically converted to this new schema during the SAP HANA conversion of the SAP NetWeaver BW system. The conversion of standard InfoCubes to in-memory optimized InfoCubes must be done manually as a follow-up task after the database migration.

KEY_0COPC_C08P

SID_0CALDAY

SID_0FISCPER

SID_0FISCVARNT

SID_0CURRENCY

SID_0UNIT

SID_0COMP_CODE

SID_0PLANT

SID_0MATERIAL

SID_0MAT_PLANT

SID_0CURTYPE

...

AMOUNTFX

AMOUNTVR

PRDPLN_QTY

LOTSIZE_CM

Fact Table:/BI0/F0COPC_C08

DIMID

SID_0CHNGID

SID_0RECORDTP

SID_0REQUID

Data PackageDimension Table:/BI0/D0COPC_C08P

COMP_CODE

SID

CHCKFL

DATAFL

INCFL

Company CodeSID Table:/BI0/SCOMP_CODE

PLANT

SID

CHCKFL

DATAFL

INCFL

PlantSID Table:/BI0/SPLANT

COMP_CODE

OBJVERS

CHANGED

CHRT_ACCTS

COMPANY

COUNTRY

�

Company CodeMaster Data Table:/BI0/PCOMP_CODE

PLANT

OBJVERS

CHANGED

ALTITUDE

BPARTNER

�

PlantMaster Data Table :/BI0/PPLANT


SAP HANA acceleration areasThe SAP HANA database can bring performance benefits; however, it is important to set the expectations correctly. SAP HANA can improve loading and query times, but certain limits cannot be overcome.

The migration of SAP NetWeaver BW to run on SAP HANA does not improve extraction processes because extraction occurs in the source system. Therefore, it is important to understand how much of the overall load time is taken by extraction from the source system. This information is needed to correctly estimate the potential performance improvement for the load process.

Other parts of the load process are improved. The new Optimized Star Schema removes unnecessary activities from the loading process.

Some of the calculations and application logic can be pushed to the SAP HANA database, which ensures that data-intensive activities are being done at the SAP HANA database level instead of at the application level. This activity increases the performance because the amount and volume of data that is exchanged between the database and the application are reduced.

SAP HANA can calculate all aggregations in real time. Therefore, aggregates are no longer required, and roll-up activity that is related to aggregate updates is obsolete, which also reduces the overall run time of update operations.

If SAP NetWeaver BW Accelerator is used, the update of its indexes is also no longer needed. Because SAP HANA is based on technology that is similar to SAP NetWeaver BW Accelerator, all queries are accelerated. Query performance with SAP HANA can be compared to situations where all cubes are indexed by the SAP NetWeaver BW Accelerator. In reality, query performance can be even faster than with SAP NetWeaver BW Accelerator because extra features are available for SAP NetWeaver BW that is running on SAP HANA; for example, the possibility of removing an InfoCube and instead running reports against in-memory optimized DataStore Objects (DSOs) and advanced DataStore Objects (aDSOs).

4.5.2 Migrating SAP NetWeaver Business Warehouse to SAP HANA

There are several ways that an SAP NetWeaver Business Warehouse (BW) system can be moved to an SAP HANA database. It is important to distinguish between building proof of concept (POC) demonstration system and a productive migration.

The available options are divided into the following main groups:

� SAP NetWeaver Business Warehouse database migration� Transporting the content to the SAP NetWeaver Business Warehouse

These groups are the main driving ideas behind the move from a traditional database to SAP HANA. Within each group, there are still many possibilities of how a project plan can be orchestrated.

These approaches are described in the following sections.


SAP NetWeaver Business Warehouse database migrationThe following software levels are prerequisites for SAP NetWeaver BW that is running on SAP HANA1:

� SAP NetWeaver BW 7.30 SP52 or SAP NetWeaver BW 7.31 SP4 or SAP NetWeaver BW 7.40 or SAP NetWeaver BW 7.50

� SAP HANA 1.0 SPS08 for SAP NetWeaver BW 7.30, 7.31, and 7.40, and SAP HANA 1.0 SPS10 for SAP NetWeaver BW 7.50

Not all SAP NetWeaver BW add-ons are supported to run on the SAP HANA-based system. For more information, see following SAP Notes:

� Note 1600929 - SAP NetWeaver BW powered by SAP HANA DB: Information� Note 1532805 - Add-On Compatibility of SAP NetWeaver 7.3� Note 1652738 - Add-on compatibility for SAP NetWeaver EHP 1 for NW 7.30

Unless your system already meets the minimal release requirements, the first step before converting SAP NetWeaver BW is to upgrade the system to the latest available release and to the latest available support package level.

A database upgrade might be required as part of the release upgrade or as a prerequisite before database migration to SAP HANA. For a list of supported databases, see SAP Note 1600929.

Table 4-1 lists the databases that were approved as source databases for the migration to SAP HANA at the time of writing.

Table 4-1 Supported source databases for a migration to the SAP HANA database

SAP HANA is not a supported database for any SAP NetWeaver Java stack. Therefore, dual-stack installations (ABAP plus Java) must be separated into two individual stacks by using the Dual-Stack Split Tool from SAP.

Because some existing installations are still non-Unicode installations, another important prerequisite step might be a conversion of the database to Unicode encoding. This Unicode conversion can be done as a separate step or as part of the conversion to the SAP HANA database.

1 For the latest information, see SAP Note 1600929.2 According to SAP Note 1600929, SP07 or higher must be imported for your SAP NetWeaver BW Installation

(ABAP) before migration and after installation.

Database SAP NetWeaver BW 7.30

SAP NetWeaver BW 7.31



Oracle 11.2 11.2 11.2 12.1

MaxDB 7.8 7.9 7.9 7.9

MS SQL server 2008 2008 2008, 2012 2012, 2014

IBM DB2 for Linux, UNIX, and Windows

9.7 9.7 10.1 10.5

IBM DB2 for i 6.1, 7.1 6.1, 7.1 7.1 7.2, 7.3

IBM DB2 for z/OS 9, 10 9, 10 10 11

SybaseASE N/A 15.7 15.7, 16 16


All InfoCubes with data persistency in the SAP NetWeaver BW Accelerator are set as inactive during conversion, and their content in SAP NetWeaver BW Accelerator is deleted. These InfoCubes must be reloaded again from the original primary persistence; therefore, required steps must be incorporated into the project plan.

A migration to the SAP HANA database follows the same process as any other database migration. All activity in the SAP NetWeaver BW system is suspended after all preparation activities are finished. A special report is run to generate database-specific statements for the target database that is used during import. Next, the content of the SAP system is exported to a platform-independent format and stored in files on disk.

These files are then transported to the primary application server of the SAP NetWeaver BW system. The application part of SAP NetWeaver BW is not allowed to run on the SAP HANA appliance. Therefore, a minimal installation must have the following servers:

� SAP HANA appliance that hosts the SAP HANA database

The SAP HANA appliance is delivered by Lenovo with the SAP HANA database preinstalled. However, the database is empty.

� Primary application server that hosts ABAP instance of SAP NetWeaver BW

There are minimal restrictions regarding the operating system of the primary application server. For more information about available combinations, see the Product Availability Matrix (PAM) (search for SAP NetWeaver 7.3, 7.31, 7.4, or 7.5 and download the overview presentation), which is available at this website:

http://support.sap.com/pam

At the time of this writing, the operating systems that are listed in Table 4-2 are available to host the ABAP part of the SAP NetWeaver BW system.

Table 4-2 Supported operating systems for primary application server

The next step is the database import. It contains the installation of the SAP NetWeaver BW on the primary application server and the import of data into the SAP HANA database. The import occurs remotely from the primary application server as part of the installation process.

Parallel export or import by using socket connection and File Transfer Protocol (FTP) and Network File System (NFS) exchange modes are not supported. Currently, only the asynchronous file-based export/import method is available.

Win

dow

s S

erve

r 20

12 R

2x8

6_64

(64

-bit

)

Win

dow

s S

erve

r 20

08x8

6_64

(64

-bit

) (i

ncl

ud

ing

R2)

AIX

6.1

, 7.1

, 7.2

Pow

er (

64-b

it)

HP

-UX

11.

31 IA

64 (

64-b

it)

So

lari

s 11

SPA

RC

(64

-bit

)

So

lari

s 10

SPA

RC

(64

-bit

)

So

lari

s 11

x86

_64

(64-

bit

)

So

lari

s 10

x86

_64

(64-

bit

)

Lin

ux

SL

ES

12

x86_

64 (

64-b

it)

Lin

ux

SL

ES

11

x86_

64 (

64-b

it)

Lin

ux

RH

EL

7 x

86_6

4 (6

4-b

it)

Lin

ux

RH

EL

5 x

86_6

4 (6

4-b

it)

Lin

ux

RH

EL

6 x

86_6

4 (6

4-b

it)

IBM

i 7.

1, 7

.2, 7

.3 P

ower

(64

-bit

)

SAP NetWeaver BW 7.30 N Y Y Y N Y N Y N Y N Y Y N

SAP NetWeaver BW 7.31 N Y Y Y N Y N Y N Y N N Y Y

SAP NetWeaver BW 7.40 Y Y Y Y N Y N Y N Y N N Y Y

SAP NetWeaver BW 7.50 Y N Y Y Y Y Y Y Y Y Y Y N Y


http://service.sap.com/pam

http://support.sap.com/pam

After mandatory post-activities are completed, the conversion of InfoCubes and DataStore objects to their in-memory optimized form must be started so that all of the benefits that the SAP HANA database can offer can be used. This task can be done manually for each object or as a mass operation by using a special report.

Clients must plan enough time to perform this conversion. This step can be time-consuming because the content of all InfoCubes must be copied into temporary tables that have the new structure.

After all post-activities are finished, the system is ready to be tested.

Transporting the content to the SAP NetWeaver Business Warehouse Unlike with a database migration, this approach is based on performing transports of activated objects (Business Content) from the SAP NetWeaver BW system into a newly installed SAP NetWeaver BW system with SAP HANA as a primary database.

The advantage of this approach is that content can be transported across releases, as explained in following SAP Notes:

� Note 1090842 - Composite note: Transports across several releases� Note 454321 - Transports between Basis Release 6.* and 7.0� Note 1273566 - Transports between Basis Release 700/701 and 702/73*� Note 323323 - Transport of all activated objects of a system

The possibility to transport content across different releases can reduce the amount of effort that is required to build a proof of concept (POC) system because most of the prerequisite activities, such as the release upgrade, database upgrade, and dual-stack split, are not needed.

After transporting the available objects (metadata definitions), their content must also be transported from the source to the target system. The SAP NetWeaver BW consultant must assess which available options are most suitable for this purpose.

This approach is not recommended for production migration where a conventional database migration is used. Therefore, more effort that is invested in building a POC system in the same way as the production system is treated is a valuable test. This test can help customers create a realistic effort estimation for the project, estimate required runtimes, and define detailed planning of all actions that are required. All involved project team members become familiar with the system and can solve and document all specific problems.

Parallel approach to SAP HANA conversionThe preferred approach to convert an SAP NetWeaver BW system to use the SAP HANA database is a parallel approach; that is, that the new SAP NetWeaver BW system is created as a clone of the original system. The standard homogeneous system copy method can be used for this purpose.

This clone is then reconfigured in a way that the original and the cloned BW system are functional and both systems can extract data from the same sources.

For more information about how to perform this cloning operation, see scenario B2 of SAP Note 886102. To understand the full procedure that must be applied on the target BW system, see SAP Note 886102. The SAP Note includes the following statement:

“Caution: This step deletes all transfer rules and PSA tables of these source systems, and the data is lost. A message is generated stating that the source system cannot be accessed (since you deleted the host of the RFC connection). Choose ‘Ignore’.”


It is important to understand the consequences of this action and to plan the required steps to reconfigure the target BW system so that it can again read data from the source systems.

Persistent Staging Area (PSA) tables can be regenerated by the replication of DataSources from the source systems, and transfer rules can be transported from the original BW system. However, the content of these PSA tables is lost and must be reloaded from source systems.

This step might cause problems where DataStore objects are used and PSA tables contain the complete history of data.

An advantage of creating a cloned SAP NetWeaver BW system is that the original system is not affected and can still be used for productive tasks. The cloned system can be tested and results compared with the original system immediately after the clone is created and after every important project milestone, such as a release upgrade or the conversion to SAP HANA itself.

Both systems are synchronized because both systems periodically extract data from the same source systems. Therefore, after a project is finished and the new SAP NetWeaver BW system that is running on SAP HANA meets the client’s expectations, the new system can replace the original system.

A disadvantage of this approach is the extra load that is imposed on the source systems, which is caused by both SAP NetWeaver BW systems performing extraction from the same source system, and certain limitations that are described in the following SAP notes:

� Note 775568 - Two and more BW systems against one OLTP system� Note 844222 - Two OR more BW systems against one OLTP system

4.5.3 SAP Business Suite that is powered by SAP HANA

SAP announced restricted availability of SAP Business Suite (which is powered by SAP HANA) in January 2013.3 After a successful ramp-up program, SAP made this generally available during the SAPPHIRENOW conference in May 2013.

SAP HANA can be used as the database for an SAP Business Suite installation. In this scenario, SAP HANA replaces the traditional database server of an SAP Business Suite installation. The application servers stay the same, and can run on any platform that supports the SAP HANA database client. As of May 2016, the following applications of SAP Business Suite are supported by SAP HANA as their primary database:

� Enterprise Resource Planning (ERP)� Customer Relationship Management (CRM)� Supply Chain Management (SCM)� Supplier Relationship Management (SRM)

The Product Lifecycle Management (PLM) application is not available with SAP HANA, but there is a plan to make these available to run with SAP HANA.

SAP Business Suite, which is powered by SAP HANA, does not induce any functional changes. Configuration, customization, the ABAP Workbench, connectivity, security, transports, and monitoring stay unchanged. For modifications, the same upgrade requirements as with any other upgrade apply. Customized code can stay unchanged, or can be adjusted to use extra performance.

3 For more information, see this website:http://www.news-sap.com/sap-business-suite-on-sap-hana-launch


http://www.news-sap.com/sap-business-suite-on-sap-hana-launch

SAP Business Suite applications can benefit in the following ways from the in-memory technology of SAP HANA:

� Running dialog processes instead of batch

� Integration of unstructured data and machine-to-machine data (M2M) with ERP processes

� Integration of predictive analysis with ERP processes

� Running operational reports in real time, directly on the source data

� Removing the need for operational data stores

� Eliminating the need for data replication or transfers to improve operational report performance

SAP is enabling the functions in SAP applications to use the in-memory technology with the following versions:

� SAP enhancement package 6 for SAP ERP 6.0, version for SAP HANA� SAP enhancement package 2 for SAP CRM 7.0, version for SAP HANA� SAP enhancement package 2 for SAP SCM 7.0, version for SAP HANA� SAP enhancement package 2 for SAP SRM 7.0, version for SAP HANA

RestrictionsThere are certain restrictions4 in effect regarding running SAP Business Suite with SAP HANA.

Currently, multi-node support for SAP Business Suite with SAP HANA is limited.5 SAP HANA multi-node configurations can serve the following purposes:

� Achieving high-availability by the use of standby nodes� Scaling the main memory to accommodate larger databases (scale-out)

Scale-out scenarios with multiple worker nodes (as described in 5.2.5, “Scaling-out SAP HANA using Spectrum Scale” on page 90) are only supported for SAP Business Suite with SAP HANA in very special cases under a controlled environment.

High availability (HA) scenarios for SAP Business Suite with SAP HANA are supported, but are restricted to the simplest case of two servers, one being the worker node and one acting as a standby node. In this case, the database is not partitioned, but the entire database is on a single node. This configuration is sometimes also referred to as a single-node HA configuration. Because of these restrictions with regards to scalability, SAP decided to allow configurations with a higher memory per core ratio, specifically for this use case. For more information about the available configurations that are dedicated to SAP Business Suite (which is powered by SAP HANA), see Chapter 6, “SAP HANA IT landscapes with X6 solutions” on page 103.

4 See SAP Note 1774566.5 For up-to-date information about multi-node support, see SAP Note 1825774.


4.5.4 SAP S/4HANA

In 2015 SAP released SAP Business Suite 4 SAP HANA, the business suite offering that is tightly integrated with SAP HANA as the underlying database.

SAP S/4HANA is the next step of innovation towards an IT platform that provides real-time business insights. SAP S/4HANA is based on new concepts in several different areas:

� Simplified data model

� New user experience that is consistent across all access methods

� Advanced processing

� Instant Insight

� Support for Internet-of-Things (IoT)

� Connectivity to third-party systems

SAP S/4HANA only runs on SAP HANA. Traditional database management systems are not supported with SAP S/4HANA.

Transition to S/4HANAThere are three different ways to transition to S/4HANA:

� New Implementation

This method is applicable for all new customers and for customers who currently run their ERP software on non-SAP products. Existing data from a legacy system can be loaded into the new implementation with tools like SAP Data Services (SAP DS). This approach is also called greenfield implementation.

New implementations can be done either on-premise or in the cloud.

� System Conversion

This method covers the conversion of existing customers running their ERP on SAP already; for example, all customers running SAP Business Suite. System conversion works on all layers of the software stack, the database layer, SAP Netweaver, and the application. Ultimately, also a conversion of the user interface to SAP Fiori UX is required.

System conversions require an on-premise installation.

� Landscape Transformation

Landscape Transformation provides a more granular approach of converting an SAP Business Suite environment into an SAP S/4HANA environment. It allows to consolidate multiple different source ERP systems into one SAP S/4HANA system, or to only selectively transform source data into an SAP S/4HANA system. This allows for a phased migration with those components that provide the highest return on investment of a migration. SAP Landscape Transformation (SLT) is the migration tool that supports this transition method.

Landscape Transformation projects work with on-premise or cloud systems as the destination environment.

More product information about SAP S/4HANA can be found at the following website:

http://www.sap.com/s4hana


http://www.sap.com/s4hana

4.6 Programming techniques that use SAP HANA

The last use case scenario is based on recent developments from SAP where applications can be built directly against the SAP HANA database by using all its features, such as the embedded application server (XS Engine) or stored procedures, which allows logic to be directly processed inside the SAP HANA database.

A new software component can be integrated with SAP HANA directly or it can be built on top of the SAP NetWeaver stack, which can work with the SAP HANA database by using client libraries.

Because of its breadth and depth, this use case scenario is not described in detail as part of this publication.


Chapter 5. Lenovo System x solutions for SAP HANA

This chapter describes the Lenovo System x solutions for SAP HANA. It describes the configurations of the workload-optimized solutions for both generations of Intel processors. It then describes IBM Spectrum Scale and the features it provides to the solution. The final section describes the networking switches that are part of the solution.


� Lenovo X6 systems� Storage subsystem� Networking options

5


5.1 Lenovo X6 systems

In 2014, the sixth generation of the Enterprise X-Architecture® servers was announced. The X6 rack family consists of the following new flagship servers of the Lenovo System x family:

� System x3850 X6 (a 4U rack-optimized server that is scalable to four sockets)� System x3950 X6 (an 8U rack-optimized server that is scalable to eight sockets)

The x3850 X6 is shown in Figure 5-1.

Figure 5-1 Lenovo System x3850 X6

The x3950 X6 resembles two x3850 X6 servers where one is placed on top of the other. However, unlike eX5 servers, x3950 X6 uses a single chassis with a single midplane design without any external connectors and cables.

The x3950 X6 is shown in Figure 5-2.

Figure 5-2 Lenovo System x3950 X6

The X6 systems offer a new “bookshelf” design concept that is based on a fixed chassis that is mounted in a standard rack cabinet. There is no need to pull the chassis in or out of the


rack to access components because all components can be accessed from the front or from the rear, as with pulling books from a bookshelf.

The x3850 X6 server with one of the four Compute Books that is partially removed is shown in Figure 5-3.

Figure 5-3 Lenovo x3850 X6 server with a Compute Book partially removed

The modular component that can be installed in a chassis is called a Book. The following types of books are available:

� Compute Books

A Compute Book contains 1 processor, 24 DIMM slots, and 2 hot-swap fan modules. It is accessible from the front of the server.

The x3850 X6 supports up to four Compute Books; the x3950 X6 supports up to eight Compute Books.

� Storage Books

The Storage Book contains standard 2.5-inch drives or eXFlash™ 1.8-inch hot-swap SSD drives. It also provides front USB and video ports, and has two PCIe slots that are reserved for internal storage adapters. The Storage Book is accessible from the front of the server.

The x3850 X6 has one Storage Book; the x3950 X6 has two Storage Books.

� I/O Books

I/O Book is a container that provides PCIe expansion capabilities. I/O Books are accessible from the rear of the server.

The following types of I/O Books are available:

– Primary I/O Book: This book provides core I/O connectivity, including the ML2 unique slot for an onboard network, three standard PCIe 3.0 slots, Integrated Management Module II, hot-swap fan modules and USB, video, serial, and systems management ports.

– Full-length I/O Book: This hot-swap Book provides three optional full-length PCIe slots.

Chapter 5. Lenovo System x solutions for SAP HANA 59

– Half-length I/O Book: This hot-swap Book provides three optional half-length PCIe slots.

The x3850 X6 has one Primary I/O Book and supports one or two of the full or half-length I/O Books (one of each or two of either). The x3950 X6 has two Primary I/O Books and supports up to four of the full or half-length I/O Books (any combination).

For more information about X6 servers and the sixth generation of Enterprise X-Architecture technology, see in IBM X6 Servers: Technical Overview, REDP-5059, which is available at this website:

http://lenovopress.com/redp5059

The next sections introduce the technological components that are used to build the Lenovo X6-based workload-optimized solution for SAP HANA and describe the architecture of x3850 X6 and x3950 X6-based solutions. The solutions share a common concept so that you can start with a x3850 X6-based installation and later on upgrade to an x3950 X6 installation without leaving parts on the floor.

5.1.1 Intel Xeon processor E7 v2, v3, and v4 family1

The high-end line of Intel CPUs is called the Intel Xeon processor E7 family. It scales beyond two sockets (which is the limit on the Intel Xeon processor E5 family) and provides the best platform for mission-critical workloads that require the most features and advanced RAS capabilities in modern data centers.

Over the course of three years Intel has released three CPU generations that are all based on the same technology platform which allowed Intel to reuse most of its external components, such as memory and the I/O subsystem.

Table 5-1 compares the Intel Xeon E7 processors that are used in Lenovo solutions for SAP HANA X6.

Table 5-1 X6 processors comparisons

1 The Intel Xeon processor E7 v1 generation is based on a different technology platform than the E7 v2, v3, and v4 processors (“Westmere-EX”, which is used on eX5 products). Whenever the term Intel Xeon processor E7 is used in the remainder of this section it refers to only the Intel Xeon processor E7 v2, v3, and v4 generation that runs on the Lenovo X6 systems.

Feature X6 family, Xeon E7 v2 X6 family, Xeon E7 v3 X6 family, Xeon E7 v4

Processor family Intel Xeon E7-8800 v2 Intel Xeon E7-8800 v3 Intel Xeon E7-8800 v4

Processor codenames Ivy Bridge EX Haswell EX Broadwell EX

Released in 2014 2015 2016

Cores per CPU Up to 15 cores Up to 18 cores Up to 24 cores

Last level cache Up to 37.5 MB L3 cache Up to 45 MB L3 cache Up to 60 MB L3 cache

QPI QPI 1.1 at 8.0 GT/s max QPI 1.1 at 9.6 GT/s max QPI 1.1 at 9.6 GT/s max

CPU TDP rating Up to 155 W Up to 165 W Up to 165 W

DIMM sockets 24 DDR3 DIMMs per CPU 24 DDR3 DIMMs per CPU24 DDR4 DIMMs per CPU

24 DDR4 DIMMs per CPU

Maximum memory speeds 2667 MHz SMI2 3200 MHz SMI2 3200 MHz SMI2

PCIe technology PCIe 3.0 (8 GTps) PCIe 3.0 (8 GTps) PCIe 3.0 (8 GTps)


http://lenovopress.com/redp5059

Intel Xeon E7 processors include a broad set of features and extensions. Many of these technologies are common to the Intel Xeon E7 v2, v3, and v4 generation; some technologies are unique to the Intel Xeon E7 v3/v4 or Intel Xeon E7 v4 family.

Features of the Intel Xeon processor E7 v2, v3, and v4 familyThe Lenovo X6 portfolio of servers uses CPUs from the Intel Xeon processor E7 family to maximize performance. These processors are the latest in a long line of high-performance processors.

The Intel Xeon processor E7 family CPUs can be used to scale up to four processors in the x3850 X6, or up to eight processors in the x3950 X6.

The current models of the X6 systems use processors from the Intel Xeon processor E7 v4 product family. The Intel Xeon processors that are used in the X6 systems are follow-ons to the Intel Xeon processor E7 product family. They provide higher core count, larger cache sizes, higher core frequencies, and higher memory speeds. In addition, these new processors support more memory, with up to 24 DIMMs per processor, and faster low-latency I/O by using integrated PCIe 3.0 controllers.

The Intel Xeon E7 processors have many new features that improve the performance of SAP HANA workloads. For more information about the benefits of the new generation Xeon processors for SAP HANA, see the white paper Infuse your business with real-time, data-driven intelligence, which is available at this website:

http://www.intel.com/content/www/us/en/big-data/big-data-xeon-e7-sap-hana-real-time-business-platform-brief.html

Instruction set extensionWith the release of the Xeon processor E7 family, Intel added Advanced Vector Extensions (AVX and AVX2) to the CPU instruction set. AVX has several new instructions and an increased register size of 256 bits, up from 128 bits.

When it is used in single instruction, multiple data (SIMD) algorithms, AVX allows for a much higher throughput on each single CPU core because twice the number of data values can now be processed per single clock cycle. SIMD processing is perfectly suited to speed up SAP HANA because running typical data warehouse algorithms, such as aggregation or scanning on column stored tables, are inherently parallelizable.

Intel Advanced Encryption Standard - New InstructionsAdvanced Encryption Standard (AES) is an encryption standard that is widely used to protect network traffic and sensitive data. Advanced Encryption Standard - New Instructions (AES-NI), which is available with the E7 processors, implements certain complex and performance-intensive steps of the AES algorithm by using processor hardware. AES-NI can accelerate the performance and improve the security of an implementation of AES over an implementation that is performed by software.

For more information about Intel AES-NI, see this website:

http://software.intel.com/en-us/articles/intel-advanced-encryption-standard-instructions-aes-ni

Turbo Boost Technology 2.0The Intel Xeon E7-8800/4800 family of processors brings enhanced capabilities of changing processor speed with new generation of Intel Turbo Boost 2.0 technology.


http://software.intel.com/en-us/articles/intel-advanced-encryption-standard-instructions-aes-ni

http://www.intel.com/content/www/us/en/big-data/big-data-xeon-e7-sap-hana-real-time-business-platform-brief.html

Intel Turbo Boost Technology dynamically saves power on unused processor cores and increases the clock speed of the cores in use. Depending on current workload, Intel Turbo Boost Technology allows a dynamic increase in the clock speed of the active cores to gain a performance boost. For example, a 3.4 GHz 15-core processor can overclock the cores up to 3.7 GHz.

Turbo Boost Technology is available on a per-processor basis for the X6 systems. For ACPI-aware operating systems and hypervisors, such as Microsoft Windows 2008/2012, RHEL 5.6, SLES 11, VMware ESXi 4.1, and later, no changes are required to use it. Turbo Boost Technology can be used with any number of enabled and active cores, which results in increased performance of multi-threaded and single-threaded workloads.

Turbo Boost Technology dynamically saves power on unused processor cores and increases the clock speed of the cores in use. In addition, it can temporarily increase speed of all cores by intelligently managing power and thermal headroom. For example, a 2.5 GHz 15-core processor can temporarily run all 15 active cores at 2.9 GHz. With only two cores active, the same processor can run those active cores at 3.0 GHz. When the cores are needed again, they are turned back on dynamically and the processor frequency is adjusted.

When temperature, power, or current exceeds factory-configured limits and the processor is running above the base operating frequency, the processor automatically steps the core frequency back down to reduce temperature, power, and current. The processor then monitors temperature, power, and current, and reevaluates whether the current frequency is sustainable or if it must reduce the core frequency further. At any time, all active cores run at the same frequency.

For more information about Turbo Boost Technology, see this website:

http://www.intel.com/technology/turboboost/

Intel Data Direct I/O: PCI Express 3.0In the Xeon processor E7 family processors, Intel integrates the I/O subsystem into the chip to allow for lower latency and faster data transfers (compared to dedicated I/O hubs in the previous processor generation).

Intel also adds support for the PCI Express 3.0 standard, which almost doubles the theoretical maximum bandwidth while keeping compatibility with previous generations of the PCIe protocol. PCIe 1.x and 2.x cards are supported in PCIe 3.0-capable slots.

PCIe 3.0 uses the 128b/130b encoding scheme, which is more efficient than the 8b/10b encoding that is used in the PCIe 2.0 protocol. This approach reduces the processing effect to less that 2% versus the 20% of PCIe 2.0, and allows almost double bandwidth at 8 GTps speed.

Up to 32 PCIe 3.0 lanes are available per processor. These 32 lanes can be split into any combination of x4, x8, and x16.

For more information about Data Direct I/O, see this website:

http://www.intel.com/content/www/us/en/io/direct-data-i-o.html

QuickPath InterconnectThe Intel Xeon E7 processors that are implemented in Lenovo X6 servers include two integrated memory controllers in each processor. Processor-to-processor communication is carried over shared-clock or coherent quick path interconnect (QPI) links. Each processor has three QPI links to connect to other processors.




http://www.intel.com/technology/turboboost/

Figure 5-4 shows the QPI configurations. How the four sockets of the x3850 X6 are connected is shown on the left side. How all eight sockets of the x3950 X6 are connected is shown on the right side.

Figure 5-4 QPI links between processors

Each processor “owns” some amount of memory, which is connected directly to the processor. To access memory that is connected to another processor, each processor uses QPI links to pass through the other processor. This design creates a NUMA system. Similarly, I/O can be local to a processor or remote through another processor.

For QPI usage, Intel modified the MESI cache coherence protocol to include a forwarding state. Therefore, when a processor asks to copy a shared cache line, only one other processor responds.

For more information about QPI, see this website:

http://www.intel.com/technology/quickpath

Intel RunSure technologyIntel added new features to further improve the RAS of the Intel Xeon E7 family. The following features are grouped into CPU-related, memory-related, and I/O-related features:

� Cyclic redundancy checking (CRC) on the QPI links

The data on the QPI link is checked for errors.

� QPI packet retry

If a data packet on the QPI link has errors or cannot be read, the receiving processor can request that the sending processor attempt to send the packet again.

� QPI clock failover

If there is a clock failure on a coherent QPI link, the processor on the other end of the link can become the clock. This action is not required on the QPI links from processors to I/O hubs because these links are asynchronous.

� QPI self-healing

If there are persistent errors that are detected on a QPI link, the link width can be reduced dynamically to allow the system to run in a degraded mode until a repair can be performed.

QPI link can reduce its width to half-width or quarter-width and reduce its speed.

� Scalable memory interconnect (SMI) packet retry

If a memory packet has errors or cannot be read, the processor can request that the packet be sent again from the memory buffer.

4 3

21

7 8

56

4 3

21

x3950 X6 - 8 socketsx3850 X64 sockets


http://www.intel.com/technology/quickpath

Intel Transactional Synchronization eXtensionsIntel Transactional Synchronization eXtensions (TSX) feature the latest v4 and v3 generation of Intel Xeon E7 processor families and brings hardware transactional memory support. Intel TSX implements a memory-locking approach that is called Hardware Lock Elision (HLE), which facilitates running multi-threaded applications more efficiently.

Much TSX-aware software gained great performance boosts by running on Intel Xeon E7 v4 processors. For example, SAP HANA SPS 09 in-memory database showed twice as many transactions per minute with Intel TSX enabled versus TSX disabled on E7 v3 processors and three times more transactions per minute compared to Intel Xeon E7 v2 processors.

For more information about Intel TSX, see the Solution Brief, Ask for More from Your Data, which is available at this website:

http://www.intel.com/content/dam/www/public/us/en/documents/solution-briefs/sap-hana-real-time-analytics-solution-brief.pdf

Machine Check Architecture recoveryThe Intel Xeon processor E7 family also features Machine Check Architecture (MCA) recovery, a RAS feature that enables the handling of system errors that otherwise require that the operating system be halted. For example, if a dead or corrupted memory location is discovered but it cannot be recovered at the memory subsystem level and it is not in use by the system or an application, an error can be logged and the operation of the server can continue. If it is in use by a process, the application to which the process belongs can be stopped or informed about the situation.

Implementation of the MCA recovery requires hardware support, firmware support (such as found in the UEFI), and operating system support. Microsoft, SUSE, Red Hat, VMware, and other operating system vendors include or plan to include support for the Intel MCA recovery feature on the Intel Xeon processors in their latest operating system versions.

The following new MCA recovery features of the Intel Xeon processor E7-8800 product family are included:

� Execution path recovery: Ability to work with hardware and software to recognize and isolate the errors that were delivered to the execution engine (core).

� Enhanced MCA (eMCA) Generation 1: Provides enhanced error log information to the operating system, hypervisor, or application that can be used to provide better diagnostic and predictive failure analysis for the system. This feature enables higher levels of uptime and reduced service costs.

� Enhanced MCA (eMCA) Generation 2: Provides more capabilities for error handling. eMCA Gen 2 is available for E7 v3 and v4 processors only.

Security improvementsThe Intel Xeon E7 processor families feature the following important security improvements that help to protect systems from different types of security threats:

� Intel OS Guard: Evolution of Intel Execute Disable Bit technology, which helps to protect from escalation of privilege attacks by preventing code execution from user space memory pages while in kernel-mode. It helps to protect from certain types of malware attacks.

� #VE2 (Beacon Pass 2 Technology): #VE utilizes ISA-level CPU-assists to allow memory-monitoring of anti-malicious software performance to scale on virtualized and non-virtualized servers, making deep malicious software detection possible on server platforms.




� Intel Trusted Execution Technology (Intel TXT), Intel VT-x, and Intel VT-d: New hardware-based techniques with which you can isolate VMs and start VMs in a trusted environment only. In addition, malware-infected VMs cannot affect another VMs on the same host.

� Intel Secure Key: Provides hardware random numbers generation without storing any data in system memory. It keeps generated random numbers out of sight of malware, which enhances encryption protection.

For more information, see Crimeware Protection: 3rd Generation Intel Core vPro Processors, which is available at this website:

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/3rd-gen-core-vpro-security-paper.pdf

The following paragraphs point out the differences between the Intel Xeon processor E7 generations that were released since 2014.

Intel Xeon processor E7 v4 familyThe latest addition to the Intel Xeon processor E7 family is the E7 v4 generation. It is fully supported in the X6 systems and offers the following key features:

� Up to 24 cores and 48 threads (by using Hyper-Threading feature) per processor

� Up to 60 MB of shared last-level cache

� Up to 3.2 GHz core frequencies

� Up to 9.6 GTps bandwidth of QPI links

� DDR4 memory interface support, which brings greater performance and power efficiency

� Integrated memory controller with four SMI2 Gen2 channels that support up to 24 DDR4 DIMMs

� Memory channel (SMI2) speeds up to 1866 MHz in RAS (lockstep) mode and up to 3200 MHz in performance mode.

� Integrated PCIe 3.0 controller with 32 lanes per processor

� Intel Virtualization Technology (VT-x and VT-d)

� Intel Turbo Boost Technology 2.0

� Improved performance for integer and floating point operations

� Virtualization improvements with regards to posted jnterrupts, page modification logging, and VM enter/exit latency reduction

� New Intel Transactional Synchronization eXtensions (TSX)

� Intel Advanced Vector Extensions 2 (AVX2.0) with new optimized turbo behavior

� Intel AES-NI instructions for accelerating of encryption

� Advanced QPI and memory reliability, availability, and serviceability (RAS) features

� Machine Check Architecture recovery (non-running and running paths)

� Enhanced Machine Check Architecture Gen2 (eMCA2)

� Machine Check Architecture I/O

� Resource director technology: Cache monitoring technology, cache allocation technology, memory bandwidth monitoring

� Security technologies: OS Guard, Secure Key, Intel TXT, Crypto performance (ADOX/ADCX), Malicious Software (SMAP), Key generation (RDSEED)




Intel Xeon processor E7 v3 familyIn May 2015, Lenovo refreshed its X6 portfolio of servers that support the Intel Xeon processor E7 v3 family of CPUs. These processors can be used to scale up to four processors in the x3850 X6 or up to eight processors in the x3950 X6.

Upgrading a x3850 X6 or x3950 X6 system from Intel Xeon E7 v2 to Intel Xeon E7 v3 processors is an easy process, although the Intel Xeon E7 v3 processors are not pin compatible to the Intel Xeon E7 v2 generation. They both use the Intel LGA 2011 socket, but are keyed differently and can be loaded into only their respective version of the LGA 2011 socket (to distinguish between them, they are sometimes referred to as LGA 2011-1 and LGA 2011-3 sockets).

The unique X6 architecture allows you to maintain the full system (which covers the hardware and operating system installation). You need to replace only the CPU Books that are carrying the processors and purchase CPU books with Intel Xeon E7 v3 CPUs.

The Intel Xeon processors E7 v3 family supports DDR3 memory and DDR4 memory. Lenovo customers who invested in the X6 platform with DDR3 memory can reuse their memory, if desired, with the new processors and move it from the existing CPU Books into the new CPU Books.

Intel Xeon processor E7 v3 processors feature the well-established 22 nm manufacturing process to provide higher core count, larger cache sizes, faster inter-processor communication, and improved power management features over the Intel Xeon processor E7 v2 generation.

The Intel Xeon processor E7 v3 product family offers the following key features:

� Up to 18 cores and 36 threads (that use Hyper-Threading feature) per processor

� Up to 45 MB of L3 cache


� Up to 9.6 GTps bandwidth of QPI links

� Integrated memory controller with four SMI2 channels that support up to 24 DDR3 DIMMs

� Up to 1600 MHz DDR3 memory speeds or up to 1800 MHz DDR4 memory speeds and up to 3200 MHz SMI link speeds




� Intel Advanced Vector Extensions 2 (AVX2, increasing the size of vector integer instructions to 256 bit)



� Machine Check Architecture recovery (non-execution and execution paths)

� Enhanced Machine Check Architecture Gen2 (eMCA)

Note: There are two X6 CPU Books that carry the Intel Xeon processors E7 v3 family. Their only difference is the main memory that you can install. One model supports only DDR3 memory and the other model supports only DDR4 memory. Make sure to order the correct CPU Book model for your needs.



� Security technologies: OS Guard, Secure Key, and Intel TXT

� Intel Transactional Synchronization Extensions (TSX)

� Fully Integrated Voltage Regulators (FIVR)

� Two new Power Management States (C6 and C7 states)

Intel Xeon processor E7 v2 familyThe Intel Xeon processor E7 v2 product family support DDR3 memory technology only and offers the following key features:

� Up to 15 cores and 30 threads (that use Hyper-Threading feature) per processor

� Up to 37.5 MB of L3 cache


� Up to 8 GTps bandwidth of QPI links

� Integrated memory controller with four SMI2 channels that support up to 24 DDR3 DIMMs

� Up to 1600 MHz DDR3 memory speeds and up to 2667 MHz SMI link speeds




� Intel Advanced Vector Extensions (AVT)



� Machine Check Architecture recovery (non-execution and execution paths)

� Enhanced Machine Check Architecture Gen1


� Security technologies: OS Guard, Secure Key, and Intel TXT

5.1.2 Memory subsystem

The System x3850 X6 and x3950 X6 support three generations of Intel Xeon E7 processors, E7 v2, E7 v3, and E7 v4. The E7 v2-based Compute Books support up to 24 DDR3 memory modules that run at speeds up to 1333 MHz in SAP HANA workload optimized models. Compute Books with E7 v3 processors can be DDR3 based with up to 24 DDR3 DIMMs operating at speeds up to 1333 MHz or DDR4-based with up to 24 TruDDR4™ memory modules operating at speeds up to 1600 MHz in SAP HANA models. The E7 v4-based Compute Books support up to 24 TruDDR4 memory modules running at speeds up to 1600 MHz.

Lenovo TruDDR4 DIMMs can operate at greater speeds and feature extended performance over industry standards. TruDDR4 memory modules are tested and tuned to maximize performance and reliability. DDR3 and TruDDR4 DIMMs cannot be mixed. DDR3 and TruDDR4 memory types have ECC protection and support Chipkill and Redundant bit steering technologies.


The x3850 X6 supports up to a total of 96 DIMMs when all processors are installed (24 DIMMs per processor), and the x3950 X6 supports up to a total of 192 DIMMs. The processor and the corresponding memory DIMM slots are on the Compute Book.

Each processor has two integrated memory controllers, and each memory controller has two Scalable Memory Interconnect generation 2 (SMI2) links that are connected to two scalable memory buffers. Each memory buffer has two DDR3 (Intel Xeon E7 v2 and E7 v3) or DDR4 (Intel Xeon E7 v3 and E7 v4) channels, and each channel supports three DIMMs for a total of 24 DIMMs per processors.

The overall architecture is shown in Figure 5-5.

Figure 5-5 Intel Xeon processor E7 v2/v3/v4 memory architecture

The SMI2 links in Compute Books with DDR3 memory run at a frequency of up to 1333 MHz and supports two transfers per clock cycle, which leads to an effective transfer rate of 2666 megatransfers per second (MTps). The use of a data width of 64-bit results in a channel bandwidth of 21.3 GBps per SMI2 link. The SMI2 link supports two DDR3 links because it runs at twice the DDR3 frequency of 667 MHz.

The SMI2 links in Compute Books with DDR4 memory run at a frequency of up to 1600 MHz and supports two transfers per clock cycle, which leads to an effective transfer rate of 3200 megatransfers per second (MTps). The use of a data width of 64-bit results in a channel bandwidth of 25.6 GBps per SMI2 link. The SMI2 link supports two DDR4 links because it runs at twice the DDR4 frequency of 800 MHz.

ChipkillChipkill memory technology (an advanced form of ECC) is available for the X6 servers. Chipkill, which is also known as Single Device Data Correction (SDDC), protects the memory

Note: The Intel Xeon processor E7 families allow for an alternative operational mode that is called RAS mode or Lockstep mode. Although RAS mode is supported on the Lenovo X6 architecture, you cannot enable it for any of the workload-optimized solutions for SAP HANA.

Intel Xeon processor

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��


in the system from any single memory chip failure. It also protects against multi-bit errors from any portion of a single memory chip.

Chipkill on its own can provide 99.94% memory availability to the applications without sacrificing performance and with standard ECC DIMMs.

Advanced Page RetireAdvanced Page Retire is a Lenovo unique algorithm to handle memory errors. It is a built-in sophisticated error handling firmware that uses and co-ordinates memory recovery features, which balances the goals of maximum up time and minimum repair actions.

The algorithm uses short- and long-term thresholds per memory rank with leaky bucket and automatic sorting of memory pages with the highest correctable error counts. First, it uses hardware recovery features and then software recovery features to optimize recovery results for newer and older operating systems and hypervisors.

When recovery features are exhausted, the firmware issues a Predictive Failure Alert. Memory that fails is held offline during reboots until it is repaired. Failed DIMMs are indicated by light path diagnostics LEDs that are physically at the socket location.

Lenovo performs thorough testing to verify the features and co-ordination between the firmware and the operating system or hypervisor.

5.1.3 Flash technology storage

The Lenovo X6 systems support different types of flash memory. This section introduces you to the flash technology that is used by the X6 workload-optimized solutions for SAP HANA that uses hard disk drives to store SAP HANA data and logs. For the all-flash solution refer to 5.1.8, “All-flash X6 solution” on page 82.

All X6 workload-optimized solutions for SAP HANA include storage controllers to provide certain features, such as RAID support. There is at least one storage controller for the internal disk drives. Depending on the memory size of the workload-optimized solution, there also are one or more controllers for external storage expansion enclosures (for more information, see Chapter 6, “SAP HANA IT landscapes with X6 solutions” on page 103 and Chapter 7, “Business continuity and resiliency for SAP HANA” on page 169).

Both storage controllers support SSD caching for traditional HDDs. You can use this feature to accelerate the performance of HDD arrays with only an incremental investment in SSD technology. SSDs are configured as a dedicated pool of controller cache and the controller firmware automatically places the most frequently accessed (hot) data on these SSDs.

SSD caching works in a transparent manner to the operating system; that is, the SSDs are not visible as block devices by Linux. They can be seen only with the controller configuration utility.

SSD caching of the local HDD arrays means that you can accelerate all types of I/O operations that are sent to the local RAID devices. Because all storage controllers in the X6 workload-optimized solutions for SAP HANA are configured with SSD caching, all data that SAP HANA writes to disk is accelerated.

SSD caching is the most flexible way to speed up any kind of disk-based operation in an SAP HANA solution.


5.1.4 Lenovo Storage D1224 Drive Enclosure for SAP HANA

Lenovo Storage D1224 Drive Enclosure models for SAP HANA provide additional required disk storage capacity for certain SAP HANA X6 systems with large memory configurations. They are designed specifically for SAP HANA certified appliance requirements and are supported on SAP HANA X6 systems.

The D1224 for SAP HANA is a 2U rack-mount, 12 Gbps SAS expansion enclosure that has 24 SFF hot-swap drive bays and is attached to SAP HANA X6 systems via the ServeRAID™ M5225 RAID adapter. All D1224 models for SAP HANA contain single Environmental Service Module (ESM) with 3x 12 Gbps SAS x4 ports for direct-attach host connectivity.

The D1224 for SAP HANA is shown in Figure 5-6.

Figure 5-6 Lenovo Storage D1224 Disk Expansion Enclosure for SAP HANA

Table 5-2 lists the models of the D1224 drive enclosure for SAP HANA.

Table 5-2 Lenovo Storage D1224 models for SAP HANA

All D1224 models for SAP HANA contain one environmental service module (ESM) with three 12 Gb SAS x4 ports (Mini-SAS HD SFF-8644 connectors) for direct-attach host connectivity (daisy-chaining is not supported).

The D1224 models for SAP HANA have two redundant hot-swap 580 W AC power supplies, and they include two 1.5m, 10A/100-250V, C13 to IEC 320-C14 rack power cables.

The D1224 for SAP HANA is designed to support the SAP HANA data storage requirements with the following 2.5-inch SAS drives:

� High performance solid-state drives (3 DWD): 400 GB� Performance-optimized, enterprise class disk drives: 1.2 TB 10K rpm

The number of drives needed depends on the selected SAP HANA configuration, as described in Chapter 6, “SAP HANA IT landscapes with X6 solutions” on page 103.

Description Part number

Lenovo Storage D1224 Single ESM Drive Enclosure for SAP HANA (US English documentation) 4587E41a

a. Available worldwide (except China and Japan).

Lenovo Storage D1224 Single ESM Drive Enclosure for SAP HANA (Simplified Chinese documentation) 4587E4Cb

b. Available only in China.

Lenovo Storage D1224 Single ESM Drive Enclosure for SAP HANA (Japanese documentation) 4587E4Jc

c. Available only in Japan.


Table 5-3 list ordering part numbers for the drives.

Table 5-3 Lenovo Storage D1224 for SAP HANA drive options

Table 5-4 lists ordering part numbers for the external SAS connectivity cables.

Table 5-4 Lenovo Storage D1224 for SAP HANA SAS connectivity cables

5.1.5 Lenovo Storage D1024 Expansion Solution for SAP HANA

Lenovo Storage D1024 Expansion Solution for SAP HANA provides additional required disk storage capacity for certain SAP HANA X6 systems with large memory configurations. It is designed specifically for SAP HANA certified appliance requirements and is supported on SAP HANA X6 systems.

Figure 5-7 shows the D1024 Expansion Solution for SAP HANA

Figure 5-7 Lenovo Storage D1024 SFF Disk Expansion for SAP HANA

The D1024 for SAP HANA is a 2U rack-mount enclosure that has 24 SFF hot-swap drive bays and is attached to SAP HANA X6 systems via the ServeRAID M5120 or M5225 RAID adapters.

The following drive types are available for the D1024 for SAP HANA:

� 400 GB SAS SSD� 1.2 TB 10K rpm SAS HDD

The Lenovo Storage D1024 Expansion Solution for SAP HANA is offered in seven models depending on the disk configuration that are listed in Table 5-5.

Table 5-5 Lenovo Storage D1024 Storage Expansion for SAP HANA models


Lenovo Storage 1.2TB 10K 2.5" SAS HDD 01DC407

Lenovo Storage 400GB 3 DWD 2.5" SAS SSD 01DC482


External MiniSAS HD 8644/MiniSAS HD 8644 2M Cable 00YL849


Lenovo Storage D1024 Single IO Module 6411HBB

Lenovo Storage D1024 Sgl IO Module / 0x400GB SSD / 9x1.2TB 10K HDD 6411HB9

Lenovo Storage D1024 Sgl IO Module / 0x400GB SSD / 18x1.2TB 10K HDD 6411HB8

Lenovo Storage D1024 Sgl IO Module / 2x400GB SSD / 9x1.2TB 10K HDD 6411H29


All D1024 models for SAP HANA contain one I/O Module with a 6 Gbps SAS x4 port (Mini-SAS SFF-8088) that is used for direct-attach host connectivity.

The D1024 models have two AC power supplies, and they come standard with two 4.3m, 10A/100-250V, C13 to IEC 320-C14 rack power cables.

Additional 400 GB SSDs and 1.2 TB HDDs can be ordered for the D1024 base models that are listed in Table 5-5, if needed. Table 5-6 list ordering part numbers for the drives.

Table 5-6 Lenovo Storage D1024 Storage Expansion for SAP HANA drives

Table 5-7 lists ordering part numbers for the external SAS connectivity cables.

Table 5-7 SAS connectivity cables for D1024 Storage Expansion for SAP HANA models

5.1.6 x3850 X6 Workload Optimized Solution for SAP HANA

The workload-optimized solutions that are based on the x3850 X6 server cover two socket and four socket configurations. Customers can start with two socket machines and scale up over time when the database size grows by adding only new components. (There are no leftover parts.)

A standard Lenovo x3850 X6 workload-optimized solution features the following minimum components:

� Two or four CPU Books with Intel Xeon E7-8880 v2/v3/v4 processors

� Up to 6TB of DDR3 or DDR4 main memory

� Two local 1.2 TB SAS drives in a RAID 1 for the operating system (SLES or RHEL) and local housekeeping





Lenovo Storage 2.5" 1.2TB 10k SAS HDD 00MM690

Lenovo Storage 2.5" 400GB SSD (SAS) 00MM720


Mini-SAS HD (M5225) to Mini-SAS (D1024)

3m SAS Cable (mSAS HD to mSAS) 00MJ166

3m HD-miniSAS to miniSAS SAS Cable 00D5226

Mini-SAS (M5120) to Mini-SAS (D1024)

External MiniSAS 8088/MiniSAS 8088 0.6m cable 00WE754

External MiniSAS 8088/MiniSAS 8088 1m cable 00WE755

External MiniSAS 8088/MiniSAS 8088 2m cable 00WE756



� Four local 1.2 TB SAS drives in a RAID 5 to store SAP HANA data and logs

� One internal storage controller, plus locally attached 400 GB SSDs for the SSD caching feature

� Two dual-port Mellanox ConnectX-3 adapters that are running in 10 Gigabit mode for SAP HANA internal communication

� One Intel I350 quad-port 1 Gigabit network adapter for uplink into a customer network (different NIC vendor and speed supported)

When your database size grows and you must scale up the x3850 X6 solution, you add one or more of the following components:

� CPU Books with Intel Xeon E7-8880 v2/v3/v4 processors

� DDR3/DDR4 memory modules

� 1 Gbps or 10 Gbps network adapters

� A storage expansion enclosure to increase local storage capacity with more 1.2 TB SAS drives, plus one external storage controller, plus locally attached 400 GB SSDs for the SSD caching feature

The following sections describe the components.

CPU and memorySAP restricts supported CPU models to a certain subset of the Xeon E7 family, which ensures that application tuning is most effective and the highest performance of the database application.

The following CPU models are supported by SAP and Lenovo on the x3850 X6:

� Intel Xeon Processor E7-8880 v2/v3/v4: 2.5 GHz, for up to eight sockets (default)� Intel Xeon Processor E7-8890 v2/v3/v4: 2.8 GHz, for up to eight sockets2

� Intel Xeon Processor E7-4880 v2/v3/v4: 2.5 GHz, for up to four sockets3 � Intel Xeon Processor E7-4890 v2/v3/v4: 2.8 GHz, for up to four sockets3

To allow memory to scale from 128 GB up to 6 TB, memory modules with different capacities are used. The following memory modules are used with the x3850 X6:

� 8 GB DIMM DDR3 ECC RDIMM (1Rx4, 1.35 V) CL11 1600 MHz� 8 GB DIMM TruDDR4 ECC RDIMM (1Rx4, 1.2V) CL15 2133 MHz

Note: When scaling up from an x3850 X6 environment to an x3950 X6 installation, you can reuse all your parts. Only the mechanical enclosure must be changed from a 4U to an 8U frame. All active components (that is, all CPU Books, Storage Books, I/O Books, power supplies, and fans) can be reused in the new system.

However, as your system environment changes, you might be forced to re-create the RAID 1 array for the operating system. To avoid losing the operating system installation, you can use UEFI and import the RAID array configuration (that is stored on the HDDs) to the RAID controller. This process worked in the lab, but make a backup of the installation.

If you want to start with an x3850 X6 but expect to grow to an x3950 X6, purchase a four socket x3950 X6 solution, which allows for seamless growth to eight sockets.

2 Available upon request for compute-bound SAP HANA installations (such as SAP Business Suite powered by SAP HANA).

3 Available upon special request only because CPU Books with E7-48x0 v2/v3 processors cannot be reused when scaling up to an x3950 X6 installation.


� 16 GB DIMM DDR3 ECC RDIMM (2Rx4, 1.35 V) CL11 1600 MHz� 16 GB DIMM TruDDR4 ECC RDIMM (2Rx4, 1.2V) CL15 2133 MHz� 32 GB DIMM DDR3 ECC LRDIMM (4Rx4, 1.35 V) CL11 1600 MHz� 32 GB DIMM TruDDR4 ECC RDIMM (2Rx4, 1.2V) CL15 2133 MHz� 64 GB DIMM DDR3 ECC LRDIMM (8Rx4, 1.35 V) 1333 MHz� 64 GB DIMM TruDDR4 ECC LRDIMM (4Rx4, 1.2V) CL15 2133 MHz

For more information about which DIMMs are used with which memory configuration, see 6.2, “X6 based environments that use Intel Xeon E7 v2 processors” on page 105 and 6.3, “Lenovo X6 based environments that use Intel Xeon E7 v3 processors” on page 117.

For more information about the DDR4 DIMMs that are used with specific memory configurations, see 6.3, “Lenovo X6 based environments that use Intel Xeon E7 v3 processors” on page 117 and 6.5, “Lenovo X6 based environments that use Intel Xeon E7 v4 processors” on page 133.

DIMM placement is crucial for best performance. Changing module placement or memory configuration without consulting Lenovo first is not supported.

StorageSAP requires a certain amount of persistent storage capacity in each SAP HANA server to save main memory data onto a non-volatile storage media regularly. This storage space is divided into a log volume and a data volume. For binary and configuration files, SAP requires another volume. The exact storage requirements depend on the node’s memory size. Bigger nodes must provide more storage capacity.

Lenovo has the following storage devices in the x3850 X6 solution for SAP HANA:

� 3.6 TB (four 1.2 TB 2.5-inch SAS drives in a RAID 5) that uses four internal storage bays in the Storage Book.

Two more 400 GB 2.5-inch SAS SSDs are used in a RAID 1 array for the SSD caching feature of the ServeRAID adapter. They occupy two bays in the Storage Book.

� 9.6 TB (nine 1.2 TB 2.5-inch SAS drives in a RAID 5 or 10 1.2 TB 2.5-inch SAS drives in a RAID 6) that uses storage expansion enclosure (D1024 for SAP HANA or EXP2524) that is connected to a ServeRAID adapter.

Two more 400 GB 2.5-inch SAS SSDs are used in a RAID 1 array for the SSD caching feature of the ServeRAID adapter. They occupy two bays in the D1024 for SAP HANA or EXP2524.

� 9.6 TB (nine 1.2 TB 2.5-inch SAS drives in a RAID 5 or 10 1.2 TB 2.5-inch SAS drives in a RAID 6) that uses empty bays in the D1024 or EXP2524.

Extra hot-spare drives are supported. They can be added without affecting the overall performance of the solution.

For details on how the storage devices are utilized and integrated into a file system see the respective sections about single node or scale-out in Chapter 5.2, “Storage subsystem” on page 84.

NetworkLenovo solutions for SAP HANA have several different network interfaces that can be grouped into the following modes:

� Internal communication (HANA communication and Spectrum Scale communication). Redundancy is required.


� External communication (SAP data management, SAP client access, data replication, appliance management, and others, depending on the customer landscape). Redundancy is optional.

Internal communication remains internal within scale-out SAP HANA solutions. They have no connection to the customer network. The networking switches for these networks are part of the appliance and cannot be substituted with other than the validated switch models.

Lenovo uses two dual-port Mellanox ConnectX-3 adapters that are running in 10 Gbit Ethernet mode for both internal communication networks. Each of the two networks requires its own connection on two physical interfaces to allow for redundancy if there is a hardware failure in the network. The Lenovo System Networking RackSwitch™ G8272 is used as the switch for internal communication. One switch handles traffic, HANA, and Spectrum Scale communication. To allow for a switch failure, a second G8272 is required in scale-out solutions. Smaller scale-out installations can also use the G8124 switch. For more information about the switches, see 5.3.2, “Lenovo RackSwitch G8272” on page 96 and 5.3.4, “Lenovo RackSwitch G8124E” on page 99.

For the uplink into the client network, an Intel quad-port 1 Gigabit Ethernet adapter is included as the default adapter. A second adapter can be added if more ports are required (for access to more networks or for redundancy reasons) or a different vendor can be chosen if, for example, 10 Gigabit is also required towards the customer networks.

Figure 5-8 shows the back side of an x3850 X6 workload-optimized solution with one quad-port 1 GbE card installed (the rightmost PCIe slot). The network names are examples only.

Figure 5-8 Networking interfaces of the x3850 X6 solution

HANA IMM

SAP Appl. SAP Data M.

SAP Client Acc. SAP Data M

GPFS


Figure 5-9 shows how the different network interfaces are connected.

Figure 5-9 Network architecture for the x3850 X6 scale-out solution

Integrated virtualizationThe VMware ESXi embedded hypervisor software is a virtualization platform that allows multiple operating systems to run on a host system at the same time. Its compact design allows it to be embedded in physical servers.

Lenovo offers versions of VMware vSphere Hypervisor (ESXi) that are customized for select System x hardware to give you online platform management, including updating and configuring firmware, platform diagnostic tests, and enhanced hardware alerts. All models support the USB keys as options, which are listed in Table 5-8.

Table 5-8 VMware ESXi memory keys for the x3850 X6

Part number Feature code Description

00WH140 ATRM Blank USB Memory Key 4G SLC for VMware ESXi Downloads

41Y8298 A2G0 Blank USB Memory Key for VMware ESXi Downloads

41Y8382 A4WZ USB Memory Key for VMware ESXi 5.1 U1

00ML233 ASN6 USB Memory Key for VMware ESXi 5.1 Update 2


00WH138 ATRL USB Memory Key 4G for VMware ESXi 6.0 Update 1A

SAP HANA scale-out solution

10 GbE switch

10 GbE switch

node 1 node 2 node n...

C D

A B

H

E A B E A B F

C D G C D G

switch

switch

SAPBW

system

FF

G H

E

H


The x3850 X6 has one internal USB connector on the primary I/O book for the embedded hypervisor USB key. The location of this USB connector is shown in Figure 5-10.

Figure 5-10 Location of internal USB port for embedded hypervisor on the x3850 X6

For more information about the USB keys and to download the System x customized version of VMware ESXi up to ESXi 5.5, see this website:

http://www.ibm.com/systems/x/os/vmware/esxi

For ESXi 6.0 the file can be found on the VMware website as a custom ISO.

In addition to the USB key, you must license vSphere with VMware. Depending on the configuration of the physical server, you must buy different licenses. For more information, see 6.10, “Consolidation of SAP HANA instances” on page 158, which also includes more information about virtualizing an SAP HANA instance on x3850 X6 servers.

5.1.7 x3950 X6 Workload Optimized Solution for SAP HANA

The workload-optimized solutions that are based on the x3950 X6 server cover four-socket and eight-socket configurations. Customers can start with a four socket configuration and scale up over time when the database size grows by adding only new components. (There are no leftover parts.)

A standard x3950 X6 workload-optimized solution has the following minimum components:

� Four or eight CPU Books with Intel Xeon E7-8880 v2/v3/v4 processors

� Up to 12 TB of DDR3 or DDR4 main memory

� Two local 1.2 TB SAS drives in a RAID 1 for the operating system (SLES or RHEL) and local housekeeping

� Four local 1.2 TB SAS drives in a RAID 5 to store SAP HANA data and logs

� One internal storage controller, plus locally attached 400 GB SSDs for the SSD caching feature

� Two Half-length I/O Books

Internal USB port for hypervisor




� Two dual-port Mellanox ConnectX-3 adapters that are running in 10 Gigabit mode for SAP HANA internal communication

� Two Intel I350 quad-port 1 Gigabit network adapters for uplink into customer network (different NIC vendor and speed are supported)

When your database size grows and you must scale-up the x3950 X6 solution, you add one or more of the following components:

� CPU Books with Intel Xeon E7-8880 v2/v3/v4 processors

� DDR3/DDR4 memory modules

� 1 Gbps or 10 Gbps network adapters

� Six local 1.2 TB SAS drives, an internal storage controller, plus locally attached 400 GB SSDs for the SSD caching feature

� A storage expansion enclosure that increases local storage capacity with more 1.2 TB SAS drives, one external storage controller, plus locally attached 400 GB SSDs for the SSD caching feature

The following sections describe the components.

CPU and memorySAP restricts supported CPU models to a certain subset of the Xeon E7 family, which ensures that application tuning is most effective and ensures the highest performance of the database application. The following CPU models are supported by SAP and Lenovo on the x3950 X6:

� Intel Xeon Processor E7-8880 v2/v3/v4: 2.5 GHz, for up to eight sockets (default)� Intel Xeon Processor E7-8890 v2/v3/v4: 2.8 GHz, for up to eight sockets4

To allow memory to scale from 256 GB up to 12 TB, memory modules with different capacities are used. The following memory modules are used with the x3950 X6:

� 8 GB DIMM DDR3 ECC RDIMM (1Rx4, 1.35 V) CL11 1600 MHz� 8 GB DIMM TruDDR4 ECC RDIMM (1Rx4, 1.2V) CL15 2133 MHz� 16 GB DIMM DDR3 ECC RDIMM (2Rx4, 1.35 V) CL11 1600 MHz� 16 GB DIMM TruDDR4 ECC RDIMM (2Rx4, 1.2V) CL15 2133 MHz� 32 GB DIMM DDR3 ECC LRDIMM (4Rx4, 1.35 V) CL11 1600 MHz� 32 GB DIMM TruDDR4 ECC RDIMM (2Rx4, 1.2V) CL15 2133 MHz� 64 GB DIMM DDR3 ECC LRDIMM (8Rx4, 1.35 V) 1333 MHz� 64 GB DIMM TruDDR4 ECC LRDIMM (4Rx4, 1.2V) CL15 2133 MHz

For more information about the DDR3 DIMMs that are used with specific memory configuration, see 6.2, “X6 based environments that use Intel Xeon E7 v2 processors” on page 105 and 6.3, “Lenovo X6 based environments that use Intel Xeon E7 v3 processors” on page 117.

Note: x3950 X6 servers are supported by only four CPU Books installed. In this configuration, two CPU Books must be installed in the lower half of the chassis and the other two CPU Books must be installed in the upper half of the chassis. Different configurations lead to situations where not all PCIe adapters are usable.

4 Available upon request for compute-bound SAP HANA installations (like SAP Business Suite powered by SAP HANA)


For more information about the DDR4 DIMMs that are used with specific memory configurations, see 6.3, “Lenovo X6 based environments that use Intel Xeon E7 v3 processors” on page 117 and 6.5, “Lenovo X6 based environments that use Intel Xeon E7 v4 processors” on page 133.

DIMM placement is crucial for best performance. Changing the module placement or memory configuration without consulting Lenovo first is not supported.

StorageSAP requires a certain amount of persistent storage capacity in each SAP HANA server to save main memory data onto a non-volatile storage media regularly. This storage space is divided into a log volume and a data volume. For binary and configuration files, SAP requires another volume. The exact storage requirements depend on the node’s memory size. Bigger nodes must provide more storage capacity.

Lenovo has the following storage devices in the x3950 X6 solution for SAP HANA:

� 3.6 TB (four 1.2 TB 2.5-inch SAS drives in a RAID 5) that uses four internal storage bays in the lower Storage Book.

Two more 400 GB 2.5-inch SAS SSDs are used in a RAID 1 for the SSD caching feature of the ServeRAID adapter. They occupy two bays in the lower Storage Book.

� 6 TB (six 1.2 TB 2.5-inch SAS drives in a RAID 55) that uses six internal storage bays in the upper Storage Book, which requires another ServeRAID M5210 storage controller in the upper Storage Book to be installed.

Two 400 GB 2.5-inch SAS SSDs are used in a RAID 1 for the SSD caching feature of the upper ServeRAID adapter. They occupy two bays in the upper Storage Book.

� 9.6 TB (nine 1.2 TB 2.5-inch SAS drives in a RAID 5 or 10 1.2 TB 2.5-inch SAS drives in a RAID 6) that uses storage expansion enclosure (D1024 for SAP HANA or EXP2524), which is connected to an ServeRAID storage controller.

Two more 400 GB 2.5-inch SAS SSDs are used in a RAID 1 for the SSD caching feature of the ServeRAID adapter. They occupy two bays in the D1024 or EXP2524.

� 9.6 TB (nine 1.2 TB 2.5-inch SAS drives in a RAID 5 array or 10 1.2 TB 2.5-inch SAS drives in a RAID 6 array) that uses empty bays in the D1024 or EXP2524.

Extra hot spare drives are supported and they can be added without affecting the overall performance of the solution.

For details on how the storage devices are utilized and integrated into a file system see the respective sections about single node or scale-out in Chapter 5.2, “Storage subsystem” on page 84.

NetworkLenovo solutions for SAP HANA have several different network interfaces that can be grouped into the following modes:

� Internal communication (HANA communication and Spectrum Scale communication). Redundancy is required.

� External communication (SAP data management, SAP client access, data replication, appliance management, and others, depending on the customer landscape). Redundancy is optional.

5 Installations requiring RAID 6 have only 4.8 TB usable storage space instead of 6 TB.


Internal communication remains internal within scale-out SAP HANA solutions. They have no connection to the customer network. The networking switches for these networks are part of the appliance and cannot be substituted with other than the validated switch models.

Lenovo uses two dual-port Mellanox ConnectX-3 adapters that are running in 10 Gbit Ethernet mode for both internal communication networks. Each of the two networks requires its own connection on two physical interfaces to allow for redundancy if there is a hardware failure in the network. The Lenovo System Networking RackSwitch G8272 is used as the switch for internal communication. One switch handles traffic, HANA, and Spectrum Scale communication. To allow for a switch failure, a second G8272 is required in scale-out solutions. Smaller scale-out installations can also use the G8124 switch. For more information about the switches, see 5.3.2, “Lenovo RackSwitch G8272” on page 96 and 5.3.4, “Lenovo RackSwitch G8124E” on page 99.

For the uplink into the customer network, two Intel quad-port 1 Gigabit Ethernet adapters are included as the default adapters. Third and fourth adapters can be added if more ports are required (to access more networks) or a different vendor can be chosen if, for example, 10 Gigabit is required also towards the customer networks.

Figure 5-11 shows the rear of an x3950 X6 workload-optimized solution with two quad-port 1G cards installed (the rightmost PCIe slots). The network names are examples.

Figure 5-11 Networking interfaces of the x3950 X6 solution

HANA

GPFS

IMM

SAP Appl. SAP Data M.

SAP Client Acc. SAP Data M


Figure 5-12 shows how the different network interfaces are connected.

Figure 5-12 Network architecture for the x3950 X6 scale-out solution

Integrated virtualizationThe VMware ESXi embedded hypervisor software is a virtualization platform that allows multiple operating systems to run on a host system at the same time. Its compact design allows it to be embedded in physical servers.

Lenovo offers versions of VMware vSphere Hypervisor (ESXi) that are customized for selected Lenovo hardware to give you online platform management, including updating and configuring firmware, platform diagnostic tests, and enhanced hardware alerts. All models support the USB keys options that are listed in Table 5-9.

Table 5-9 VMware ESXi memory keys for the x3950 X6

Part number Feature code Description

00WH140 ATRM Blank USB Memory Key 4G SLC for VMware ESXi Downloads

41Y8298 A2G0 Blank USB Memory Key for VMware ESXi Downloads

41Y8382 A4WZ USB Memory Key for VMware ESXi 5.1 U1



00WH138 ATRL USB Memory Key 4G for VMware ESXi 6.0 Update 1A

SAP HANA scale-out solution

10 GbE switch

10 GbE switch

node 1 node 2 node n...

C D

A B

H

E A B E A B F

C D G C D G

switch

switch

SAPBW

system

FF

G H

E

H


The x3950 X6 has two internal USB connectors on each of the primary I/O Books for the embedded hypervisor USB key. The location of the USB connectors is shown in Figure 5-13.

Figure 5-13 Location of the internal USB port for the embedded hypervisor on the x3950 X6

Although the x3950 X6 has two primary I/O books, you must equip only one of them with the embedded hypervisor. Installing two hypervisors is supported only when the x3950 X6 is configured to be partitioned, where the two halves of the server operate as two independent four-socket servers.

For more information about the USB keys and to download the System x customized version of VMware ESXi up to ESXi 5.5, see this website:


For ESXi 6.0 the file can be found on the VMware website as a custom ISO.

In addition to the USB key, you must license vSphere with VMware. Depending on the configuration of the physical server, you must buy different licenses. For more information, see 6.10, “Consolidation of SAP HANA instances” on page 158, which also includes more information about virtualizing an SAP HANA instance on x3950 X6 servers.

5.1.8 All-flash X6 solution

With support of higher-capacity flash devices in a standard 2.5-inch form factor it was possible to design an X6 solution that utilizes only flash devices to store SAP HANA persistency files. No additional SSD caching feature is required on RAID-adapter level (as it is with hard disk drives). The operating system is installed either on two HDDs (operating in RAID-1 mode) or on two SSDs. This allows for true all-flash solutions based on Lenovo X6 systems.

Because each of the Enterprise Capacity flash devices can hold 3.84 TB of data fewer devices are required to achieve the same capacity as with hard disk drives. This allows all SAP HANA persistency data to be stored inside the X6 system and avoids the use of a storage expansion enclosure. Further advantages are energy savings due to a lower power consumption and eliminating moving parts to reduce potential failure points in the server.

Internal USB port for hypervisor




More information about the SSD devices can be found at:

https://lenovopress.com/tips1297

An architectural overview of the all-flash solution can be found at the following website from SanDisk, the manufacturer of the Enterprise capacity flash devices utilized in this design:

https://www.sandisk.com/content/dam/sandisk-main/en_us/assets/resources/enterprise/white-papers/all-flash-storage-solution-for-sap-hana.pdf

All-flash solutions are certified and supported with all SAP HANA single node configurations running the latest Intel Xeon processor E7 v4 generation. Single node configurations comprise both SAP Business Warehouse and SAP Business Suite on HANA. Configuration details can be found in Chapter 6.7, “All-Flash X6 solutions for SAP HANA” on page 149.

5.1.9 X6 Partitioning

The x3950 X6 server can be electronically split into the following configurations by using a process called Node Partitioning, also known as FlexNode technology:

� An x3950 X6 server with four processors can be split into two 2-socket system.

� An x3950 X6 server with eight processors can be split into two 4-socket system.

When partitioning is enabled, the 8-socket server is seen by the operating system as two independent 4-socket machines, as shown in Figure 5-14.

Figure 5-14 x3950 X6 server with partitioning

Each half of the x3950 X6 is called a node. With factory defaults, the upper and lower node of an x3950 X6 server are partitioned into a single 8-socket system image. The nodes are joined internally via QPI links in the midplane to form an 8-socket system.

Partitioning can be managed via IMM2 system management web interface or via IMM2 telnet interface.

Note: The x3850 X6 server cannot be partitioned.

Upper 4-socket node

Lower 4-socket node


https://www.sandisk.com/content/dam/sandisk-main/en_us/assets/resources/enterprise/white-papers/all-flash-storage-solution-for-sap-hana.pdf

https://lenovopress.com/tips1297

The following minimum number of components is required to support partitioning in the x3950 X6:

� Four Compute Books (two in each node)

� Two Primary I/O Books (one in each node)

� Two Storage Books (one in each node)

� Two boot devices (one in each node), such as a local drive, hypervisor key, or external boot device

� Four power supplies (two power supplies in each node)

Consider the following points regarding partitioning an x3950 X6 server:

� By default, a single partition is created out of both halves of the x3950 X6 (that is, to form a single system image that uses all eight processors).

� Each half independently maintains its own set of hardware, such as storage backplanes, power supplies, and each scalable partition can support its own independent operating system installation. However, both halves must be at the same level of UEFI firmware and IMM firmware.

� Partitioning is performed at a whole-node level of granularity. For example, if you have eight processors in your x3950 X6, you cannot have a partition of two processors and a partition of six processors. With eight processors installed, partitions can be four processors or eight processors only.

� The x3950 X6 server must be shut down before partitions can be created or changed.

� The bottom half of the x3950 X6 is considered the primary node in the complex. The primary node provides the information to the secondary node regarding partitioning.

� After being created, partitions cannot be expanded (for example, 4-socket partition to 8-socket partition). To create an 8-socket partition (the entire server), you must first remove the 4-socket partitions.

� Partition data remains consistent through reboots, shutdowns, and firmware upgrades.

5.2 Storage subsystem

This section describes the different choices that are available for the underlying storage architecture of the Lenovo X6 solution for SAP HANA. It explains the options customers have as part of the solution offering including support for the full solution stack that comprises all components. This section does not cover storage architectures under the SAP Tailored Data Center Integration (TDI) category.

Table 5-10 lists the available options as part of the Lenovo Solution for SAP HANA offerings.

Table 5-10 Storage file systems supported with the Lenovo Solution for SAP HANA

The following sections first explain the two different file system choices and then describe how the solution with each file system is built.

SUSE RedHat

Single Node Scale-out Single Node Scale-out

XFS Yes No No No

Spectrum Scale Yes Yes Yes Yes


5.2.1 XFS technology

XFS is a high-performance journaling 64-bit file system initially developed by SGI in 1994. Since 2000 it has been licensed under the GPL. It is natively included in both SUSE and RedHat enterprise distributions and is included at no additional cost to the customer.

XFS is approved by SAP for usage with SAP HANA and provides the following key features:

� 64-bit journaling file system� Online growth (no support for shrinking)� Online defragmentation� User and group quota� Extended attributes� Variable block sizes� Quick restart period after unexpected interruption, regardless of the number of files� Delayed allocation (also called allocate-on-flush)� High scalability through allocation groups that allow parallel IO to the file system

XFS is available with all single-node Lenovo X6 solutions that are installed with SLES 12 and later and that run the Intel Xeon E7 v3 or v4 processor technology.

5.2.2 IBM Spectrum Scale technology

The IBM Spectrum Scale (formerly called IBM GPFS) is a high-performance, shared-disk file management solution that provides very fast, reliable access to a common set of file data. It enables a view of distributed data with a single global namespace. It is used as the central component of the scale-out Lenovo System x solution for SAP HANA because it enables a shared-nothing scale-out architecture.

Common Spectrum Scale featuresSpectrum Scale uses its cluster architecture to provide quicker access to your file data. File data automatically is spread across multiple storage devices, which provides optimal use of your available storage to deliver high performance.

Spectrum Scale is designed for high-performance parallel workloads. Data and metadata flow from all the nodes to all of the disks in parallel under the control of a distributed lock manager. It has a flexible cluster architecture that enables the design of a data storage solution that meets current needs and can quickly be adapted to new requirements or technologies. Spectrum Scale configurations include direct-attached storage, network block I/O (or a combination of the two), and multisite operations with synchronous data mirroring.

Spectrum Scale can intelligently prefetch data into its buffer pool, issuing I/O requests in parallel to as many drives as necessary to achieve the peak bandwidth of the underlying storage-hardware infrastructure. Spectrum Scale recognizes multiple I/O patterns, including sequential, reverse sequential, and various forms of striped access patterns. In addition, for high-bandwidth environments, Spectrum Scale can read or write large blocks of data in a single operation, which minimizes the effect of I/O operations.

Expanding beyond a storage area network (SAN) or locally attached storage, a single Spectrum Scale file system can be accessed by nodes via a TCP/IP or InfiniBand connection. By using this block-based network data access, Spectrum Scale can outperform network-based sharing technologies, such as Network File System (NFS) and even local file systems, such as the EXT3 journaling file system for Linux or Journaled File System.


Network block I/O, also called network shared disk (NSD), is a software layer that transparently forwards block I/O requests from a Spectrum Scale client application node to an NSD server node to perform the disk I/O operation and then passes the data back to the client. By using a network block I/O, this configuration can be more cost-effective than a full-access SAN.

For optimal reliability, Spectrum Scale can be configured to help eliminate single points of failure. The file system can be configured to remain available automatically if there is a disk or server failure. A Spectrum Scale file system transparently fails over token (lock) operations and other Spectrum Scale cluster services, which can be distributed throughout the entire cluster to eliminate the need for dedicated metadata servers. Spectrum Scale can be configured to recover automatically from node, storage, and other infrastructure failures.

Spectrum Scale provides this function by supporting the following functions:

� Data replication to increase availability if there is a storage media failure� Multiple paths to the data if there is a communications or server failure� File system activity logging, which enables consistent fast recovery after system failures

In addition, Spectrum Scale supports snapshots to provide a space-efficient image of a file system at a specified time, which enables online backup and can help protect against user error. How snapshots can be used to create backups of SAP HANA databases is described in 7.4.4, “Database backups by using Spectrum Scale snapshots” on page 208.

Spectrum Scale extensions for shared-nothing architecturesIBM added several features to Spectrum Scale that support the design of shared-nothing architectures. A single shared storage is not necessarily the best approach when dozens, hundreds, or even thousands of servers must access the same set of data. Shared storage can impose a single point of failure (unless designed in a fully redundant way by using storage mirroring). It can limit the peak bandwidth for the cluster file system and is expensive to provide storage access to hundreds or thousands of nodes.

Spectrum Scale File Placement Optimizer (Spectrum Scale FPO) is a set of features to support big data applications on shared-nothing architectures. In such scenarios, hundreds or even thousands of commodity servers compute certain problems. They do not include shared storage to hold the data. The internal disks of the nodes are used to store all data, which requires a new way of thinking to run a cluster file system on top of a shared-nothing architecture.

The following features were introduced with Spectrum Scale FPO:

� Write affinity: Provides control over the placement of new data. It can be written to the local node or wide striped across multiple nodes.

� Locality awareness: The ability to obtain on which node certain data chunks are stored. This ability allows jobs to be scheduled on the node that is holding the data, which avoids costly transfer of data across the network.

� Metablocks: Enable two block sizes within the same file system. MapReduce workloads often have small files (below 1 MB for index files, for example) and large files (such as 128 MB, holding the actual data) in the same file system. The concept of metablocks allows for an optimal use of the available physical blocks.


� Pipelined replication: Makes the most effective use of the node interconnect bandwidth. Data that is written on node A sends data to node B, which in turn sends data to node C. In contrast to pipelined replication, the other replication schema is star replication, where node A sends data to both node B and node C. For bandwidth-intense operations or for servers with limited network bandwidth, the outgoing link of node A can limit replication performance in such a scenario. Choosing the correct replication schema is important when a shared-nothing architecture is run because this process almost always involves replicating data over the network.

� Fast recovery: An intelligent way to minimize recovery efforts after the cluster is healthy again. After an error, Spectrum Scale tracks the updates that are missing through the failed drives. In addition, the load to recover the data is distributed across multiple nodes. Spectrum Scale also allows two different recovery policies. After a drive fails, data can be rebuilt when the drive is replaced or it can immediately be rebuilt by using other nodes or disks to hold the data.

The Lenovo System x solution for SAP HANA benefits in the following ways from the features of Spectrum Scale:

� Spectrum Scale provides a stable, industry-proven, cluster-capable file system for SAP HANA.

� Spectrum Scale transparently works with multiple replicas (that is, copies) of a single file to protect from drive failures.

� Spectrum Scale adds extra performance to the storage devices by striping data across devices.

� With the FPO extensions, Spectrum Scale enables the Lenovo System x Solution for SAP HANA to grow beyond the capabilities of a single system, into a scale-out solution, without the need for external storage.

� Spectrum Scale adds high-availability and disaster recovery features to the solution.

All these features make Spectrum Scale a robust and flexible basis for the Lenovo System x Solution for SAP HANA.

5.2.3 Single node SAP HANA using XFS

Lenovo offers single node SAP HANA solutions based on the XFS file system. During installation time you can choose between XFS or Spectrum Scale. Further drive and file system setup is handled automatically by the installation wizard from Lenovo.

Figure 5-15 on page 88 shows the storage architecture on a single node x3850 X6 solution running XFS. The optional second RAID array is shown in dashed lines. Linux-internal tools manage the different sizes of the block devices and balances I/O operations to maximize the use of both devices.


Figure 5-15 Storage architecture of the x3850 X6 solution with XFS

Figure 5-16 shows the storage architecture of an x3950 X6 solution running XFS. The optional second, third, and fourth RAID arrays are shown in dashed lines.

Figure 5-16 Storage architecture of the x3950 X6 solution with XFS

Single node installations based on XFS require a reinstallation of SAP HANA when turned into scale-out due to the change of file system to Spectrum Scale.

5.2.4 Single-node SAP HANA using Spectrum Scale

Besides XFS, Lenovo offers solutions based on IBM Spectrum Scale. Spectrum Scale provides a robust, high-performance file system that you can also use to non-disruptively

x3850 X6 node

XFS

DB partition 1

Index server Statistic server SAP HANA studio

SAP HANA DB

SAP HANA DB

First replica

3.6 TB 9.6 TB

data + log

x3950 X6 node

XFS

DB partition 1

Index server Statistic server

SAP HANA studio

SAP HANA DB

SAP HANA DB

First replica

3.6 TB 6 TB 9.6 TB 9.6 TB

data + log


grow your environment as your needs increase. As you add drives to your node, Spectrum Scale transparently includes this added storage and starts to use it. No data is lost during this upgrade.

Figure 5-17 shows the storage architecture on an x3850 solution running Spectrum Scale. The optional second RAID array is shown in dashed lines. Spectrum Scale manages the different sizes of the block devices and balances I/O operations to maximize the use of both devices.

Figure 5-17 Storage architecture of the x3850 X6 solution with Spectrum Scale

Figure 5-18 shows the storage architecture of an x3950 X6 solution running Spectrum Scale. The optional second, third, and fourth RAID arrays are shown in dashed lines.

Figure 5-18 Storage architecture of the x3950 X6 solution with Spectrum Scale

x3850 X6 node

Shared file system - GPFS

DB partition 1

- Index server- Statistic server- SAP HANA® studio

- SAP HANA® DB

SAP HANA® DB

First replica

3.6 TB 9.6 TB

data + log

x3950 X6 node


DB partition 1

- Index server- Statistic server

- SAP HANA® studio

- SAP HANA® DB

SAP HANA® DB

First replica

3.6 TB 6 TB 9.6 TB 9.6 TB

data + log


Single-node installations based on Spectrum Scale can be turned into scale-out environments without reinstalling the system.

5.2.5 Scaling-out SAP HANA using Spectrum Scale

By scaling up a single node SAP HANA appliance, you can expand the capabilities of an SAP HANA installation up to a certain point when the physical limit is reached. To allow for further growth, the Lenovo System x Solution for SAP HANA supports a scale-out approach. An SAP HANA system can span multiple servers, partitioning the data to hold and process larger amounts of data than a single server can accommodate.

All scale-out solutions are based on the same building blocks, as described in section 5.1, “Lenovo X6 systems” on page 58. All Lenovo scale-out solutions for SAP HANA have the following properties:

� The scale-out solution is a cluster of servers, which are interconnected with two separate 10 Gb Ethernet networks, one for the SAP HANA application and one for the shared Spectrum Scale file system communication. Both networks are redundant.

� The SAP HANA database is split into partitions on each cluster node, which forms a single instance of the SAP HANA database.

� Each node of the cluster holds its own savepoints and database logs on the local storage devices of the server.

� The Spectrum Scale file system is a shared file system. Because Spectrum Scale spans all nodes of the cluster, it makes the data of each node available to all other nodes in the cluster despite using only local storage devices (for more information about this technology, see, “Spectrum Scale extensions for shared-nothing architectures” on page 86).

To an outside application that is connecting to the SAP HANA database, this configuration appears to be a single instance of SAP HANA. The SAP HANA software distributes the requests internally across the cluster to the individual worker nodes, which process the data and exchange intermediate results, which are then combined and sent back to the requester. Each node maintains its own set of data, persisting it with savepoints and logging data changes to the database log that are stored on local storage.

Spectrum Scale combines the storage devices of the individual nodes into one large file system, ensuring that the SAP HANA software has access to all data regardless of its location in the cluster. Spectrum Scale also ensures that savepoints and database logs of an individual database partition are stored on the appropriate storage device of the node on which the partition is located. This feature is called locality.

Although Spectrum Scale provides the SAP HANA software with the functionality of a shared storage system, it ensures maximum performance and minimum latency by using locally attached drives and flash devices.

In addition, because server-local storage devices are used, the total capacity and performance of the storage within the cluster automatically increases with the addition of nodes, which maintains the same per-node performance characteristics regardless of the size of the cluster. This kind of scalability is not achievable with external storage systems.


Scaling out a Lenovo SAP HANA solution creates a cluster of nodes. SAP HANA designates nodes in a scale-out configuration with a certain role. The node can be a worker node or a standby node. Worker nodes actively process workload. Standby nodes are part of the cluster only and do not process workload while the cluster remains in a healthy state. Standby nodes take over the role of a worker node when it fails. Standby nodes are required for scale-out clusters with high availability.

Figure 5-19 shows the networking architecture of a four-node scale-out solution. The node designation has no effect on the network connectivity of a node. All nodes are considered equal.

Figure 5-19 Network architecture of a four-node scale-out solution

The following networks span the redundant Ethernet switches:

� Spectrum Scale network, for communication and data transfer between the nodes� SAP HANA network for database communication

Every node has redundant connectivity to each of the two networks, which leads to four 10 Gbps Ethernet ports that are required per node in scale-out environments. If the SAP HANA database instance that is running on those nodes grows, clients can add nodes to extend the overall main memory of the cluster. This addition is possible without affecting any of the existing nodes, so the cluster does not have to be taken down for this operation.

Scale-out environments with only SAP HANA worker nodes have no support for HA because no standby nodes are part of the cluster.

Maximum scalability:

� With X6 nodes, SAP validated the Lenovo scale-out solution for up to 94 nodes in a cluster.

� The building block approach of Lenovo makes the solution virtually scalable without any known limitations. Clients that require scale-out configurations beyond the generally available 94 nodes can work with Lenovo and SAP to jointly validate such large clusters at the client site.

� Lenovo has shown scalability for up to 224 nodes in a single SAP HANA scale-out cluster. With current X6 servers, SAP HANA database instances of up to 896 TB are possible.

SAP HANA worker node

Two 10G Ethernet Switches:GPFS networkSAP HANA network

RedundantInter-Switch Links





The absence of failover capabilities represents a major disadvantage of this solution. The cluster acts as a single-node configuration. If one node becomes unavailable for any reason, the database partition on that node becomes unavailable, and with it the entire SAP HANA database. To cover the risk of a node failure, a standby node must be added to the cluster. This solution is described in the next section.

Scale-out solution with high-availability capabilitiesThe scale-out solution for SAP HANA with high-availability capabilities enhances the scale-out solution in the following ways:

� Making the SAP HANA application highly available by introducing SAP HANA standby nodes, which can take over from a failed node within the cluster.

� Making the data that is provided through Spectrum Scale highly available to the SAP HANA application, including its data on the local storage devices. This availability allows you to tolerate the loss of a node.

SAP HANA allows the addition of nodes in the role of a standby node. These nodes run the SAP HANA application, but do not hold any data in memory or take an active part in the processing. If one of the active worker nodes fails, a standby node takes over the role of the failed node, including the data (that is, the database partition) of the failed node. This mechanism allows the clustered SAP HANA database to continue operating.

To take over the database partition from the failed node, the standby node must load the savepoints and database logs of the failed node to recover the database partition and resume operation in place of the failed node. This process is possible because Spectrum Scale provides a shared file system across the entire cluster, which gives each individual node access to all the data that is stored on the storage devices that are managed by Spectrum Scale.

A four-node cluster with the fourth node in a standby node is shown in Figure 5-20.

Figure 5-20 Four-node clustered solution with failover capabilities

If a node has an unrecoverable hardware error, the storage devices that are holding the node’s data might become unavailable or destroyed. With high-availability features in place, the Spectrum Scale file system replicates the data of each node to the other nodes, which creates a second replica to prevent data loss if one of the nodes fails.

node04node03node01 node02


DB partition 1

- Index server- Statistic server- SAP HANA studio

- SAP HANA DBWorker node

DB partition 2



DB partition 3




- SAP HANA DBStandby node

Second replica

First replica

SAP HANA DB

local storagelocal storagelocal storagelocal storage

data01 + log01 data02 + log02 data03 + log03


Replication is done in a striping fashion. Every node has a piece of data of all other nodes. In Figure 5-20, the contents of the data storage (that is, the savepoints, here data01) and the log storage (that is, the database logs, here log01) of node01 are replicated to node02, node03, and node04.

Replication happens for all nodes that are generating data so that all information is available twice within the Spectrum Scale file system, which makes it tolerant to the loss of a single node. Replication occurs synchronously. The write operation finishes only when the data is written locally and on a remote node. This configuration ensures consistency of the data at any point. Although Spectrum Scale replication is done over the network and in a synchronous fashion, this solution still overachieves the performance requirements for validation by SAP.

The File Placement Optimizer (FPO), which is part of Spectrum Scale, ensures that the first replica always is stored local to the node that is generating the data. If SAP HANA data must be read from disk (for example, for backups or restore activity), FPO always prefers the replica that is available locally. This configuration ensures the best read performance of the cluster. By using replication, Spectrum Scale provides the SAP HANA software with the functionality and fault tolerance of a shared storage subsystem while maintaining its performance characteristics.

Example of a node takeoverTo further show the capabilities of this solution, this section provides a node takeover example. In this example, we have a four-node setup (initially configured as shown in Figure 5-20 on page 92) with three active nodes and one standby node.

First, node03 experiences a problem and fails unrecoverably. Data that is stored on this node is no longer available. The SAP HANA master node (node01) recognizes this fact and directs the standby node (node04) to take over from the failed node. The standby node is running the SAP HANA application and is part of the cluster, but in an inactive role.

To re-create database partition 3 in memory to take over the role of node03 within the cluster, node04 reads the savepoints and database logs of node03 from the Spectrum Scale, reconstructs the savepoint data in memory, and reapplies the logs so that the partition data in memory is as it was before node03 failed. Node04 is operating and the database cluster recovered. This scenario is shown in Figure 5-21.

Figure 5-21 Standby node 4 takes over from failed node 3



DB partition 1



DB partition 2




- SAP HANA DBDefunct node



Second replica

First replica

SAP HANA DB



DB partition 3


The data that node04 must load into memory is the data of node03 (which failed), including its local storage devices. For that reason, Spectrum Scale had to deliver the data to node04 from the second replica, which is spread across the cluster. Spectrum Scale handles this process transparently so that the application does not recognize from which node the data was read. If data is available locally, Spectrum Scale prefers to read from node04 and avoid going over the network.

Now, when node04 starts writing savepoints and database logs again during the normal course of operations, these savepoints are not written over the network. Instead, the savepoints are written local drives with a second replica striped across the other cluster nodes.

After fixing the cause for the failure of node03, it can be reintegrated into the cluster as the new standby system, as shown in Figure 5-22.

Figure 5-22 Node 3 is reintegrated into the cluster as a standby node

This example shows how Lenovo combines two independently operating high-availability measures (that is, the concept of standby nodes at the SAP HANA application level and the reliability features of Spectrum Scale at the infrastructure level), which results in a scalable solution that provides fully automated high availability with no administrative intervention required.

5.3 Networking options

Larger SAP HANA implementations scale beyond the limits of a single server. In those environments, the database is split into several partitions, with each partition on a separate server within the cluster. Nodes in a cluster communicate with each other through a high-speed interconnect. Network switches are crucial in such scale-out solutions.



DB partition 1



DB partition 2







Second replica

First replica

SAP HANA DB



DB partition 3


All Lenovo System x solutions for SAP HANA use network switches that meet these requirements. Thee following top-of-rack Ethernet switches are part of the scale-out solution of Lenovo:

� Lenovo RackSwitch G8296

The G8296 switch is a 10/40 Gb Ethernet switch with up to 94 10 Gb SFP+ ports. This switch is used in scale-out solutions to provide internal cluster communication for Spectrum Scale and SAP HANA networks for maximum solution scalability of up to 94 nodes.


The G8272 switch is a 10/40 Gb Ethernet switch with up to 72 10 Gb SFP+ ports in a 1U rack form factor. This switch is used in scale-out solutions to provide internal cluster communication for Spectrum Scale and SAP HANA networks for maximum solution scalability of up to 64 nodes.


The G8264 switch is a 10 Gb Ethernet switch with 64 SFP+ ports. This switch is used in scale-out solutions to provide internal cluster communication for Spectrum Scale and SAP HANA networks for maximum solution scalability of up to 56 nodes.


The G8124E switch is a 10 Gb Ethernet switch with 24 SFP+ ports. It is used for smaller installations that do not need as many ports as the G8264 switch provides.


The G8052 switch is a 1 Gb Ethernet switch with 48 10/100/1000 BASE-T RJ45 ports. It is used in scale-out environments for management and SAP client networks.

5.3.1 Lenovo RackSwitch G8296

The G8296 switch is a 10 Gb/40 Gb Top-of-Rack (TOR) switch that is designed for data center applications that require the highest performance and scalability at low latency. It combines 2.56 Tbps throughput with up to 94 10 Gb SFP+ ports with up to eight 40 Gb QSFP+ uplinks in a dense 2U form factor.

Figure 5-23 shows the Lenovo RackSwitch G8296.

Figure 5-23 Lenovo RackSwitch G8296

The G8296 switch offers the following benefits regarding SAP HANA environments:

� High performance

The 10 Gb/40 Gb switch provides the best combination of low latency, non-blocking line-rate switching, and ease of management. It has a throughput of 2.56 Tbps.


� Lower power and better cooling

The RackSwitch G8296 uses as little as 210 W of power, which is a fraction of the power consumption of most competitive offerings. Unlike side-cooled switches, which can cause heat recirculation and reliability concerns, the front-to-rear or rear-to-front cooling design of the G8296 reduces data center air conditioning costs by having airflow match the servers in the rack. In addition, variable speed fans help to automatically reduce power consumption.

� Layer 3 functionality

The G8296 includes Layer 3 functionality, which provides security and performance benefits, as inter-VLAN traffic stays within the switch. This switch also provides the full range of Layer 3 protocols from static routes for technologies such as Open Shortest Path First (OSPF) and Border Gateway Protocol (BGP) for enterprise customers.

� Seamless interoperability

RackSwitch switches perform seamlessly with other vendors' upstream switches.

� Fault tolerance

RackSwitch switches learn alternate routes automatically and perform faster convergence in the unlikely case of a link, switch, or power failure. The switch uses proven technologies like L2 trunk failover, advanced VLAN-based failover, VRRP, and Hot Links.

The following interface options are available:

� A total of 94 SFP+ ports (10 GbE)

86 SFP+ ports (10 GbE) and two QSFP+ ports (40 GbE) that are split into four 10 GbE ports each with a breakout cable can be used for nodes connections. Each cluster node uses two switch ports (one port for internal Spectrum Scale network, one port for internal HANA network); therefore, the G8296 switches support configurations of up to 94 nodes in the cluster.

� Eight QSFP+ ports (40 GbE)

Up to eight QSFP+ ports (40 GbE) are used in high-availability scale-out cluster solutions for interswitch links (ISLs).

� One 10/100/1000 Ethernet RJ45 port for out-of-band management

One dedicated RJ45 port is used as the management port of the switch.


The G8272 switch is a 10 Gb/40 Gb Top-of-Rack (TOR) switch designed for data center applications that require the highest performance and scalability at low latency in a very dense 1U rack form factor. It combines 1.44 Tbps throughput with up to 48 10 Gb SFP+ ports and up to six 40 Gb QSFP+ uplinks that can be broken out into 24 10 Gb ports.




The G8272 switch offers the following benefits in SAP HANA environments:

� High Performance

The 10 Gb/40 Gb switch provides the best combination of low latency, non-blocking line-rate switching, and ease of management. It has a throughput of 2.56 Tbps.


The RackSwitch G8296 uses as little as 210 W of power, and the choice of front-to-rear or rear-to-front cooling design of the G8296 reduces data center air conditioning costs by having airflow match the servers in the rack. In addition, variable speed fans help to automatically reduce power consumption (and noise), while increasing the lifespan of the fans.

� VM-aware networking

VMready® software on the switch simplifies configuration and improves security in virtualized environments. VMready automatically detects virtual machine movement between physical servers and instantly reconfigures each VM’s network policies across VLANs to keep the network up and running without interrupting traffic or impacting performance. VMready works with all leading VM providers, including VMware, Citrix Xen, and Microsoft Hyper-V.


The RackSwitch G8296 includes Layer 3 functionality, which provides security and performance benefits, as inter-VLAN traffic stays within the switch. This switch also provides the full range of Layer 3 protocols from static routes for technologies such as Open Shortest Path First (OSPF) and Border Gateway Protocol (BGP) for enterprise customers.


RackSwitch switches perform seamlessly with other vendors' upstream switches.

� Fault tolerance

RackSwitch switches learn alternate routes automatically and perform faster convergence in the unlikely case of a link, switch, or power failure. The switch uses proven technologies, such as L2 trunk failover, advanced VLAN-based failover, VRRP, and Hot Links.


� A total of 64 SFP+ ports (10 GbE) for SAP HANA node connections

48 SFP+ ports (10 GbE) and four QSFP+ ports (40 GbE) that are split into four 10 GbE ports each with a breakout cable can be used for node connections. Each cluster node uses two switch ports (one port for internal Spectrum Scale network, one port for internal HANA network); therefore, the G8272 switches support configurations of up to 64 nodes in an HA cluster with two redundant switches.

� Two QSFP+ ports (40 GbE) for interswitch links

In an HA scale-out solution, the first two QSFP+ ports are used for forming interswitch links (ISLs) between a pair of G8272 switches.





The G8264 switch is a 10 Gb/40 Gb Top-of-Rack (TOR) switch that is designed for applications that require the highest performance at low latency. It combines 1.28 Tbps throughput with up to 64 10 Gb SFP+ ports in an ultra-dense 1U form factor.



The G8264 switch offers the following benefits in SAP HANA environments:


The 10 Gb/40 Gb Low Latency Switch provides the best combination of low latency, non-blocking line-rate switching, and ease of management. It also has a throughput of 1.2 Tbps.


The G8264 switch uses as little as 275 W of power, which is a fraction of the power that is used by most competitive offerings. Unlike side-cooled switches that can cause heat recirculation and reliability concerns, the G8264 switch’s front-to-rear or rear-to-front cooling design reduces data center air conditioning costs by having airflow match the servers in the rack. In addition, variable speed fans assist in automatically reducing power consumption.


RackSwitch switches include Layer 3 functionality, which provides security and performance benefits because inter-VLAN traffic stays within the switch. These switches also provide the full range of Layer 3 protocols from static routes for technologies, such as Open Shortest Path First (OSPF) and Border Gateway Protocol (BGP) for enterprise customers.


RackSwitch switches interoperate seamlessly with other vendors’ upstream switches.

� Fault tolerance

RackSwitch switches learn alternative routes automatically and perform faster convergence in the unlikely event of a link, switch, or power failure. The switches use proven technologies, such as L2 trunk failover, advanced VLAN-based failover, VRRP, and Hot Links.


� A total of 56 SFP+ ports (10 GbE) for SAP HANA node connections

48 SFP+ ports and two QSFP+ ports (40 Gb) that are split into four 10 GbE ports each with a breakout cable can be used for nodes connections. Each cluster node uses two switch ports (one port for internal Spectrum Scale network, one port for internal HANA network); therefore, the G8264 switches support up to 56 nodes in the cluster.


� Two QSFP+ ports (40 GbE) for interswitch links

In an HA scale-out solution, the first two QSFP+ ports are used for forming ISLs between a pair of G8264 switches.



5.3.4 Lenovo RackSwitch G8124E

The RackSwitch G8124E is designed with top performance in mind. This low-latency switch provides line-rate, high-bandwidth switching, filtering, and traffic queuing without delaying data.

The RackSwitch G8124E TOR switch is shown in Figure 5-26.

Figure 5-26 Lenovo RackSwitch G8124E

The G8124E switch offers the following features regarding SAP HANA environments:


The 10G Low Latency (<700 ns) switch provides the best combination of low latency, non-blocking line-rate switching, and ease of management. It also has a throughput of 480 Gbps.


The G8124E switch uses as little power as two 60 W light bulbs, which is a fraction of the power consumption of most competitive offerings. Unlike side-cooled switches, which can cause heat recirculation and reliability concerns, the G8124 switch’s rear-to-front cooling design reduces data center air conditioning costs by having airflow match the servers in the rack. In addition, variable speed fans assist in automatically reducing power consumption.


This RackSwitch includes Layer 3 functionality, which provides security and performance benefits because inter-VLAN traffic stays within the switch. This switch also provides the full range of Layer 3 protocols from static routes for technologies, such as OSPF and BGP for enterprise customers.

� Active MultiPath (AMP)

Effectively doubles bandwidth by allowing all uplink ports to be active/active, which eliminates cross-stack traffic and provides up to 900 Gbps aggregate bandwidth between servers. Built-in fault tolerance constant health checking ensures maximum availability.




� Fault tolerance

RackSwitch switches learn alternative routes automatically and perform faster convergence in the unlikely event of a link, switch, or power failure. The switch uses proven technologies, such as L2 trunk failover, advanced VLAN-based failover, VRRP, and Hot Links.


� A total of 24 10 Gb SFP+ fiber connectors

In HA scale-out configurations, the first four ports of the switch should be used for ISL between switches; the other 20 ports can be used for nodes connections. Each cluster node uses two switch ports (one port for internal Spectrum Scale network, one port for internal HANA network); therefore, the G8124 switches supports up to 20 nodes in the cluster.

� 2x 10/100/1000 Ethernet RJ-45 ports for management

Two dedicated RJ-45 ports are used as the management ports of the switch.


The Lenovo RackSwitch G8052 combines great performance, server-like airflow for cooling, and low-power consumption in a 1U package. The RackSwitch G8052 TOR switch is shown in Figure 5-27.


The G8052 switch is a TOR data center switch that delivers unmatched line-rate Layer 2/3 performance at an attractive price. It features 48 10/100/1000 BASE-T RJ45 ports and four 10 Gb Ethernet SFP+ ports. It also includes hot-swap redundant power supplies and fans standard, which minimizes your configuration requirements. Unlike most rack equipment that cools from side-to-side, the G8052 switch has rear-to-front or front-to-rear airflow that matches server airflow.

For 10 Gb uplinks, there is a choice of SFP+ transceivers (SR or LR) for longer distances or more cost-effective and lower-power-consuming options, such as SFP+ direct-attached cables (DAC or Twinax cables), which can be 1 - 7 meters (3.28 - 22.97 feet) in length and are ideal for connecting to another TOR switch or to an adjacent rack.

The G8052 switch provides the following features regarding SAP HANA environments:


The G8052 switch provides up to 176 Gbps throughput and supports four SFP+ 10 Gb uplink ports for a low oversubscription ratio, in addition to a low latency of 1.7 ms.


The G8052 switch typically uses only 120 W of power, a fraction of the power consumption of most competitive offerings. Unlike side-cooled switches that can cause heat recirculation and reliability concerns, the G8052 switch’s rear-to-front or front-to-rear cooling design reduces data center air conditioning costs by matching airflow to the server’s configuration in the rack. Variable speed fans assist in automatically reducing power consumption.



The G8052 switch includes Layer 3 functionality, which provides security and performance benefits because inter-VLAN traffic can be processed at the access layer. This switch also provides the full range of Layer 3 static and dynamic routing protocols, including OSPF and BGP for enterprise customers at no extra cost.

� Fault tolerance

These switches learn alternative routes automatically and perform faster convergence in the unlikely event of a link, switch, or power failure. The switch uses proven technologies, such as L2 trunk failover, advanced VLAN-based failover, VRRP, Hot Link, Uplink Failure Detection (UFD), IGMP v3 Snooping, and OSPF.



The G8052 switch features the following Interface options:

� A total of 48 10/100/1000BaseT ports (RJ-45)� Four 10 GbE SFP+ ports

G8052 switches are used in all System x offerings for SAP HANA as switches for the management network (connecting IMM interfaces of the nodes) and as the default choice for SAP HANA appliances to the client network. Clients that have 10 Gbps Ethernet backbone for their SAP environments can also choose the G8264 switch for uplink connectivity.


Chapter 6. SAP HANA IT landscapes with X6 solutions

This chapter describes IT landscapes in which SAP HANA can be deployed and shows the corresponding System x workload-optimized solution that is based on the building blocks that are introduced in Chapter 5, “Lenovo System x solutions for SAP HANA” on page 57.

This chapter also describes implementations based on X6 models using different Intel Xeon processor generations, and mixed scale-out environments based on eX5 servers and X6 servers. The portfolio of all-flash solutions based on X6 models is described and the benefits of the X6 partitioning feature are laid out.

This chapter also describes solutions how to consolidate multiple database instances on one SAP HANA system and introduces data encryption for SAP HANA solutions.


� 6.1, “eX5 based environments” on page 104� 6.2, “X6 based environments that use Intel Xeon E7 v2 processors” on page 105� 6.3, “Lenovo X6 based environments that use Intel Xeon E7 v3 processors” on page 117� 6.4, “Upgrading X6 from Intel Xeon E7 v2 to E7 v3 CPUs” on page 131� 6.5, “Lenovo X6 based environments that use Intel Xeon E7 v4 processors” on page 133� 6.6, “Upgrading X6 to Intel Xeon E7 v4 CPUs” on page 147� 6.7, “All-Flash X6 solutions for SAP HANA” on page 149� 6.8, “Partitioning the x3950 X6” on page 153� 6.9, “Migrating from eX5 to X6 servers” on page 156� 6.10, “Consolidation of SAP HANA instances” on page 158� 6.11, “Security and encryption of an SAP HANA system” on page 165

6


6.1 eX5 based environments

Following the appliance-like delivery model for SAP HANA, Lenovo created several custom server models for SAP HANA. These workload-optimized models are designed to match and exceed the performance requirements and the functional requirements that are specified by SAP. With a small set of System x workload-optimized models for SAP HANA, all sizes of SAP HANA solutions can be built, from the smallest to large installations.

eX5 workload-optimized models for SAP HANAIn October 2012, a set of eX5 workload-optimized models for SAP HANA was announced that were built on the Intel Xeon processor E7-8800 product families. Because there is no direct relationship between the workload-optimized models and the SAP HANA “T-shirt” sizes, these models are referred to as building blocks.

The building blocks are configured to match the SAP HANA sizing requirements. The main memory sizes match the number of processors to give the correct balance between processing power and data volume. Also, the storage devices in the systems provide the storage capacity that is required to match the amount of main memory.

eX5 workload-optimized models for SAP HANA are available with SUSE Linux Enterprise Server (SLES) for SAP Applications only. Red Hat Enterprise Linux is not available with these eX5 models.

Figure 6-1 shows all of eX5-based building blocks for SAP HANA. You can see that they are based on two different servers: the System x3690 X5™ and the System x3950 X5.

Figure 6-1 eX5 based building blocks for SAP HANA

��

��

��

��

��

�� !�� !�" ��" ��"

�#

�

�

�

$

%�

& %& %%&

'(��)��)��) ��)�*��)��)

+�,� ��+)��*��-�,��-'.'

�) �/�� +��

+0��-

+0��-

+0��-

��-

��- ��-

��

��

��

�

!�$��

%�

��(��12�%��(�1�2�%�

+0��- +0��-


In some cases, there are several building blocks that are available for one T-shirt size. In some cases, two-building blocks must be combined to build a specific T-shirt size. Table 6-1 lists all eX5-based building blocks and their features.

Table 6-1 Overview of all eX5 workload-optimized models for SAP HANA

All models include preinstalled software that is composed of SUSE Linux Enterprise Server for SAP Applications (SLES for SAP) 11, IBM Spectrum Scale, and the SAP HANA software stack. Licenses and maintenance fees (for three years) for SLES for SAP and Spectrum Scale are included. For more information about which type of Spectrum Scale license is included for a specific model, see “Spectrum Scale license information” on page 220.

XM, XL, and XXL building blocks are specific to and limited for use with SAP S/4HANA and SAP Business Suite powered by SAP HANA. The building blocks feature a different memory to core ratio than the regular models, which is suitable for this specific workload only, as described in 4.5.3, “SAP Business Suite that is powered by SAP HANA” on page 53.

6.2 X6 based environments that use Intel Xeon E7 v2 processors

With the introduction of X6, builds of the building block concept gives customers the most flexibility in selecting components to fit their needs while protecting investments through an unmatched upgrade path when workload increases. Lenovo defines workload-optimized models that are based on X6 servers that match and exceed the performance and the functional requirements that are specified by SAP.

Lenovo X6 workload-optimized models for SAP HANAIn spring 2014, Lenovo announced two additions to the workload-optimized models for SAP HANA on X6. The additions are built on two new servers with processors from the Intel Xeon processor E7 v2 family. The two servers are the x3850 X6, with up to four processor sockets,

Buildingblock

Server CPUs Mainmemory

Business Warehouse

S/4HANA and Suite on HANA

Upgrade options

XS x3690 X5 2x Intel XeonE7-2870

128 GB Yes Yes XS → S

S x3690 X5 2x Intel XeonE7-2870

256 GB Yes Yes None

S+ x3950 X5 2x Intel XeonE7-8870

256 GB Yes Yes S+ → M

M x3950 X5 4x Intel XeonE7-8870

512 GB Yes Yes M → XMM → L

XM x3950 X5 4x Intel XeonE7-8870

1 TB No Yes XM → XL

L x3950 X5 +x3950 X5

8x Intel XeonE7-8870

1 TB Yes Yes L → XL

XL x3950 X5 +x3950 X5


2 TB No Yes XL → XXL

XXL x3950 X5 +x3950 X5


4 TB No Yes None

Chapter 6. SAP HANA IT landscapes with X6 solutions 105

and the x3950 X6, with up to eight processor sockets. Both servers use the same components that can be reused when growing from an x3850 X6 to an x3950 X6 solution. Only the mechanical chassis must be replaced to accommodate more components, such as processors and I/O. For more information about the X6 server and its components, see 5.1, “Lenovo X6 systems” on page 58.

Figure 6-2 shows an overview of the available workload-optimized solutions that are based on the x3850 X6 server, which covers the entry-level and mid-range requirements.

Figure 6-2 x3850 X6 based building blocks for SAP HANA that use Intel Xeon E7 v2 processors

Figure 6-3 shows the building blocks for SAP HANA solutions that are based on the x3950 X6 server, which covers the mid-range to high-end requirements.

Figure 6-3 x3950 X6 based building blocks for SAP HANA that uses Intel Xeon E7 v2 processors

34�5677�47

89 <�= 89 <�= 89 <�= 89 <�= <�=<�=

>��4�?6;��;�;�4�5677�47;7��4�6?��;� @��;AB;5��777;C��;��4�7;46�7�D�6E

<�=

F6��4�;7G6�H�;I8 H�B;I8 ��;I8 ��H;I8 �B�;I8

H

�

89:;<�=

89:;<�=

�;�8 �J�;�8 H;�8

89:;<�=

89:;<�= 89:;<�= 89:;<�= <�=<�=

89:;<�=

89;K;8�7�677;9�46��76<�=;K;<��6;��;=�.�

<5��6L��;��4;89;��

<��6;��4�?67;46M�46;��46��56;�6��4�;��?��67;��64;5��5��;��?��6789:;<�=

�8

<�=

34�5677�47

89 <�= 89 <�= 89 <�= 89 <�= <�=<�= <�=

�6��4�;7��6

��;8 ��;8 � ;8 �;�8 ��;�8

�

89;�;8�7��677;9�46��76<�=;�;<��6;��;=��

<5��6��;��4;89;��

<��6;��4��67;46��46;��46��56;�6��4�;��67; ��64;5��5��;��67

89:;<�=

89:;<�=

�;�8 !;�8 �;�8

89:;<�= 89:;<�= 89:;<�=

89:;<�= 89:;<�= 89:;<�= <�=<�=

;�8

<�=

<�= <�= <�=


Table 6-2 lists the technical details of all x3850 X6-based workload-optimized models for SAP HANA running the Intel Xeon E7 v2 family processors. They cover the range of 2- and 4-socket solutions.

Table 6-2 x3850 X6 workload-optimized models for SAP HANA that use Intel Xeon E7 v2 processors

Table 6-3 lists the technical details of all x3950 X6 based workload-optimized models for SAP HANA running the Intel Xeon E7 v2 family processors. They cover the range of 4- and 8-socket solutions.


Model CPUsa

a. Alternative CPU types E7-4880 v2, E7-4890 v2 (both support no upgrade to eight sockets), and E7-8890 v2 are available upon request. For more information, see “CPU and memory” on page 73.

Mainmemory

Business Warehouse

Scale-out BW


Total rack space

AC3-2S-128 2x Intel XeonE7-8880 v2

128 GB DDR3 Yes No Yes 4U


256 GB DDR3 Yes Yes, up to 4 nodes

Yes 4U




512 GB DDR3 Yes Yes, up to 4 nodes

Yes 4Uc




512 GB DDR3 Yes Yes, up to 94 nodesb

b. Support for up to 224 nodes is verified by Lenovo development and presented to SAP.

Yes 4Uc

c. 6U when used in a scale-out environment.



Yes 6U


1 TB DDR3 Yes Yes, up to 94 nodesb

Yes 6U


1.5 TB DDR3 No N/A Yes 6U


2 TB DDR3 No N/A Yes 6U



Model CPUsa Mainmemory

Business Warehouse

Scale-out BW


Total rack space



Yes 8U



Yes 8U


6.2.1 Single-node X6 solution for Business Warehouse

A single-node solution is the simplest possible implementation of an SAP HANA environment. Depending on the sizing requirements, you choose the appropriate building block from the available set of X6 workload-optimized solutions.



Yes 8U



Yes 8U











Yes 8U


1.5 TB DDR3 Yes Yes, up to 94 nodesb

Yes 8U



Yes 8Uc







a. Alternative CPU type E7-8890 v2 is supported for enhanced performance. Available upon request.b. Support for up to 224 nodes is verified by Lenovo development and presented to SAP.c. 10U when used in a scale-out environment.


Business Warehouse

Scale-out BW


Total rack space

Note: Consider the following points:

� For non-production database instances, SAP maintains relaxed hardware requirements, with which you can have more memory in a server; for example, for test or training instances of SAP HANA.

� The following sections list only solutions that are certified by SAP for productive environments.

� All X6 servers physically support the installation of memory beyond what is listed in this section. At the time of this writing, Lenovo can provide X6 systems with 3 TB, 4 TB, 6 TB, 8 TB, or 12 TB of main memory for non-productive SAP HANA landscapes.


Figure 6-4 on page 109 shows all X6 based models that are available for a single-node SAP BW environment.

Figure 6-4 Single-node X6 based solutions for Business Warehouse that use Intel Xeon E7 v2 processors

In the lower half of Figure 6-4 (below the dashed line), the available models that are based on the x3850 X6 server are shown. In the upper half of Figure 6-4, the models that are based on the x3950 X6 server are shown. There are four socket models for both. You can physically upgrade an x3850 X6 to an x3950 X6 server by replacing the mechanical enclosure that houses the CPU Books, Storage Books, and I/O Books. The Books can be used in both server types.

Table 6-4 lists the technical details of all x3850 X6 workload-optimized models for SAP HANA running the Intel Xeon E7 v2 family processors.

Table 6-4 x3850 X6 workload-optimized models for single-node BW that use Intel Xeon E7 v2 processors

34�5677�47

"

#

$%&'(��

��4�6�7��4�6� �

�6��4��7��6

��

�

�

��

!��"��

46��5��65��5��5��77�7

#��6��4�67�46$�46��46��56��6��4��67�%��64�5��5��67


Storage(data and log)

Upgrade options


128 GB DDR3(16x 8 GB)

3.6 TB (2x 400 GB SAS SSD,4x 1.2 TB 10K SAS HDD)

128 → 256


256 GB DDR3(32x 8 GB or16x 16 GB)


256 → 384256 → 5122S → 4S


384 GB DDR3(48x 8 GB)


256 → 5122S → 4S-512


512 GB DDR3(32x 16 GB or 16x 32 GB)


2S → 4S


256 GB DDR3(32x 8 GB)


256 → 512


Table 6-5 lists the technical details of all x3950 X6 workload-optimized models for SAP HANA running the Intel Xeon E7 v2 family processors.



512 GB DDR3(64x 8 GB or32x 16 GB)


512 → 768512 → 1 TB


768 GB DDR3(96x 8 GB)

13.2 TB (4x 400 GB SAS SSD,13x 1.2 TB 10K SAS HDD)b

None


1 TB DDR3(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB1 TB → 2 TB


b. Extra drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.



Upgrade options

Model CPUsa

a. Alternative CPU type E7-8890 v2 is supported for enhanced performance. Available upon request.

Mainmemory


Upgrade options


256 GB DDR3(32x 8 GB)


256 → 512


512 GB DDR3(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB4S → 8S


768 GB DDR3(96x 8 GB)


4S → 8S-1 TB


1 TB DDR3(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB1 TB → 2 TB4S → 8S


1.5 TB DDR3(96x 16 GB)


4S → 8S


2 TB DDR3(64x 32 GB or 32x 64 GB)


None


512 GB DDR3(64x 8 GB)


512 → 1 TB


1 TB DDR3(128x 8 GB or 64x 16 GB)


1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR3(192x 8 GB)


None


2 TB DDR3(128x 16 GB or 64x 32 GB)


Scale-out


You can also grow into a clustered scale-out environment without changing any of the hardware of the single-node solution. The upgrade to scale-out is supported for memory configurations with 256 GB, 512 GB, 768 GB, 1 TB, 1.5 TB, or 2 TB (as denoted by the boxes in Figure 6-4 on page 109). Scale-out of other memory sizes is not supported because adding memory first (to scale up) to reach one of the supported memory sizes gives better performance than adding nodes.

Upgrading the server requires downtime of the system. However, because of the capability of the underlying storage subsystem (XFS or Spectrum Scale) to add capacity online simply by adding devices only, the data on the system remains intact. It is recommended that data is backed up before the system’s configuration is changed.

Support for business continuityAll of the single-node solutions that are listed in Table 6-4 on page 109 and Table 6-5 on page 110 support the following features for business continuity:

� HA within a single data center� HA across data centers (metro distance)� HA within a single data center plus DR to a secondary site� HA across data centers (metro distance) plus DR to a third site� DR to a secondary site

All HA and DR solutions can be implemented by using storage based replication or SAP HANA System Replication. You also can use the standby nodes in HA and DR solutions to run a second SAP HANA instance for non-productive purposes. For more information about these architectures, see 7.2, “HA and DR for single-node SAP HANA” on page 177.

Adding business continuity features after the initial implementation is possible. Only extra hardware must be bought, and no parts are retired.

6.2.2 Single-node X6 solution for Business Suite on HANA

Business Suite is a set of applications that are bundled and designed to run an entire business. When Business Suite on SAP HANA is run, either as SAP Business Suite powered by SAP HANA or as SAP S/4HANA, the memory-per-core ratio is different compared to a BW scenario because the type of workload on the database server is different. By using this setup, Lenovo can offer more building blocks with more main memory than for SAP BW.


Figure 6-5 shows the building blocks that you can choose from when you plan for SAP S/4HANA or SAP Business Suite powered by SAP HANA.

Figure 6-5 Single-node X6 based models for SAP S/4HANA and SAP Business Suite powered by SAP HANA that use Intel Xeon E7 v2 processors

In the lower half of Figure 6-5 (below the dashed line at four processors), the available models that are based on the x3850 X6 server are shown. In the upper half of Figure 6-5, the models that are based on the x3950 X6 server are shown. There are four socket models for both. You can physically upgrade an x3850 X6 to an x3950 X6 server by replacing the mechanical enclosure that houses the CPU Books, Storage Books, and I/O Books. The Books can be used in both server types.

Table 6-6 lists the technical details of all x3850 X6-based models for SAP HANA running the Intel Xeon E7 v2 family processors when installing SAP S/4HANA or SAP Business Suite on top of the SAP HANA system.

Table 6-6 x3850 X6 workload-optimized models for S/4HANA and Suite on HANA that use Intel Xeon E7 v2 processors

34�5677�47

�

��&��

�

6��4 �7��6

��

�

�

��

��6��4�67�46��46��46��56��6��4 ��67� ��!!��!64�5��5�� 67

��

"��#��

��4�67��4�6$ �46��5�%��65!�%�5��5!�77�7



Upgrade options


128 GB DDR3(16x 8 GB)

3.6 TB(2x 400 GB SAS SSD,4x 1.2 TB 10K SAS HDD)

128 → 256


256 GB DDR3(32x 8 GB or 16x 16 GB)


256 → 384256 → 5122S → 4S


384 GB DDR3(48x 8 GB)


256 → 5122S → 4S-512


512 GB DDR3(32x 16 GB or 16x 32 GB)


2S → 4S


Table 6-7 shows the technical details of all x3950 X6-based models for SAP S/4HANA and SAP Business Suite powered by SAP HANA running the Intel Xeon E7 v2 family processors.



256 GB DDR3(32x 8 GB)


256 → 512


512 GB DDR3(64x 8 GB or32x 16 GB)

3.6 TB 2x 400 GB SAS SSD,4x 1.2 TB 10K SAS HDD)

512 → 768512 → 1TB


768 GB DDR3(96x 8 GB)


None


1 TB DDR3(64x 16 GB or 32x 32 GB)

13.2 TB 4x 400 GB SAS SSD,13x 1.2 TB 10K SAS HDD)b

1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR3(96x 16 GB)


None


2 TB DDR3(64x 32 GB or 32x 64 GB)


2 TB → 3TB


3 TB DDR3(96x 32GB)


None





Upgrade options



Upgrade options


256 GB DDR3(32x 8 GB)


256 → 512


512 GB DDR3(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB4S → 8S


768 GB DDR3(96x 8 GB)


4S → 8S- 1 TB


1 TB DDR3(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB1 TB → 2 TB4S → 8S


1.5 TB DDR3(96x 16 GB)


4S → 8S


2 TB DDR3(64x 32 GB or 32x 64 GB)


2 TB → 3 TB


3 TB DDR3(96x 32 GB)


4S → 8S- 4 TB


512 GB DDR3(64x 8 GB)


512 → 1 TB


Upgrading the server requires downtime of the system. However, because of the capability of the underlying storage subsystem (XFS or Spectrum Scale) to add storage capacity online simply by adding devices, the data on the system remains intact. We recommend backing up data before changing the system’s configuration.

Support for business continuityImplementing some level of business continuity features is highly recommended because business suite constitutes the entire business and its processes. Downtime of these applications most often has a significant effect on the business. Clients in the manufacturing industry are often faced with a halt of their production lines if the supply chain system is not available.

All single-node solutions that are listed in Table 6-6 on page 112 and Table 6-7 on page 113 support the following business continuity features:

� HA within a single data center� HA across data centers (metro distance)� HA within a single data center plus DR to secondary site� HA across data centers (metro distance) plus DR to third site� DR to secondary site

All HA and DR solutions can be implemented by using storage-based replication or SAP HANA System Replication. You also can use the standby nodes in HA and DR solutions to run a second SAP HANA instance for non-productive purposes. For more information about these architectures, see 7.2, “HA and DR for single-node SAP HANA” on page 177.


6.2.3 Scale-out X6 solution for Business Warehouse

Customer whose BW databases do not fit into a single node must scale out their SAP HANA installation, which means clustering multiple single nodes with a high-speed interconnect (10 Gb Ethernet) and spreading out the database tables across the participating nodes.


1 TB DDR3(128x 8 GB or 64x 16 GB)


1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR3(192x 8 GB)


None


2 TB DDR3(128x 16 GB or 64x 32 GB)


2 TB → 3 TB2 TB → 4 TB


3 TB DDR3(192x 16 GB)


None


4 TB DDR3(128x 32 GB or 64x 64 GB)


4 TB → 6 TB


6 TB DDR3(192x 32 GB)


Upgrade to Intel Xeon E7 v4c

a. Alternative CPU type E7-8890 v2 is supported for enhanced performance. Available upon request.b. Extra drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.c. Upgrading to latest Intel Xeon processor technology allows it to use up to 8 TB of data.



Upgrade options


Figure 6-6 shows all X6 based building blocks that are supported in such scale-out scenarios.

Figure 6-6 X6 based solutions for scale-out Business Warehouse that use Intel Xeon E7 v2 processors

In the lower half of Figure 6-6, models that are based on the x3850 X6 server are shown. In the upper half of Figure 6-6, models that are based on the x3950 X6 server are shown. Models with four sockets exist by using both servers. You can physically upgrade a x3850 X6 server to a x3950 X6 server by replacing the x3850 4U mechanical chassis with an 8U version for the x3950 X6. All CPU Books, I/O Books, and Storage Books can be reused in the 8U chassis.

Table 6-8 lists the technical details of all workload-optimized x3850 X6 models in a scale-out configuration running the Intel Xeon E7 v2 family processors.

Table 6-8 x3850 X6 models for a scale-out SAP Business Warehouse that uses Intel Xeon E7 v2 processors

34�5677�47

�

"�&�#��

�6��4��7��6��

�

�

��

��6��4��67�46��46��46��56��6��4��67��64�5��5��67

� � �!

"��4��6�7��4�6��#�46��5�$��65��$�5��5��77�7

Model CPUsa

a. Alternative CPU types E7-4880 v2, E7-4890 v2 (both support no upgrade to eight sockets), and E7-8890 v2 are available upon request. For more information, see“CPU and memory” on page 73.

Mainmemory


Upgrade options

Max nodes

AC3-2S-256-C 2x Intel XeonE7-8880 v2

256 GB DDR3(32x 8 GB or 16x 16 GB)

3.6 TB (2x 400 GB SAS SSD,4x 1.2 TB SAS HDD)

256 → 512 4


512 GB DDR3(32x 16 GB or 16x 32 GB)


2S → 4S 4


512 GB DDR3(64x 8 GB or 32x 16 GB)

13.2 TB (4x 400 GB SAS SSD,13x 1.2 TB SAS HDDb)


512 → 768512 → 1 TB

94


768 GB DDR3(96x 8 GB or 48x 16 GB)


768 → 1 TB 94


1 TB DDR3(64x 16 GB or 32x 32 GB)


94


Table 6-9 lists the technical details of all workload-optimized x3950 X6 models in a scale-out configuration running the Intel Xeon E7 v2 family processors.

Table 6-9 x3950 X6 models for scale-out SAP Business Warehouse that use Intel Xeon E7 v2 processors

An SAP HANA cluster environment can consist of building blocks with the same memory size only. Mixing different memory sizes in a cluster is not supported. For example, when a cluster with 512 GB nodes is upgraded to 1 TB nodes, you must add the memory to every node.

Model CPUsa


Mainmemory


Upgrade options

Max nodes


256 GB DDR3(32x 8 GB)

3.6 TB(2x 400 GB SAS SSD,4x 1.2 TB SAS HDD)

256 → 512 94


512 GB DDR3(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB

94


768 GB DDR3(96x 8 GB)


768 → 1 TB4S → 8S

94


1 TB DDR3(64x 16 GB or 32x 32 GB)


4S → 8S 94


1 TB DDR3(128x 8 GB or 64x 16 GB)


1 TB → 1.5 TB1 TB → 2 TB

94


1.5 TB DDR3(192x 8 GB)


1.5 TB → 2 TB 94


2 TB DDR3(128x 16 GB or 64x 32 GB)

19.2 TB(6x 400 GB SAS SSD,19x 1.2 TB SAS HDDb)


94

Scalability:

� The minimum number of nodes in any SAP HANA scale-out environment is three worker nodes. A cluster of two worker nodes is not supported. Adding one standby node for business continuity results in a four-node cluster that is the minimum that is supported.

� The maximum number of nodes that is available for implementation is 94 (except for model AC3-2S-256-C, which is limited to four nodes). However, Lenovo validated feasibility for up to 224 nodes in its labs to accommodate for growing customer demand.

� If you are interested in solutions beyond 94 nodes, contact your Lenovo representative or email the Lenovo SAP Center of Competence at: [email protected].


Support for business continuityAll X6 scale-out solutions that are listed in Table 6-8 on page 115 and Table 6-9 support the following features for business continuity:

� HA within the cluster to protect against node failures� DR that replicates data to a secondary data center� Combination of HA and DR above

HA can be implemented by using storage based replication. DR solutions can be implemented by using storage based replication or SAP HANA System Replication. You also can use the idling DR site nodes to run a second SAP HANA instance for non-productive purposes. For more information about these solutions, see 7.3, “HA and DR for scale-out SAP HANA” on page 194.

Single-node to scale-out solution upgrade requirementsWith X6 systems, the scale-out solution for the Lenovo Systems Solution for SAP HANA builds upon the same building blocks as they are used in a single-server installation. There are more hardware and software components that are needed to complement the basic building blocks when a single-node solution is upgraded to a scale-out solution, as described in this section.

More hardware must be purchased only when scaling out using models AC3-4S-512 or AC4-8S-2048. For these two models, more storage capacity is required for the second Spectrum Scale replica that is generated in scale-out environments. This second replica does not exist for single nodes.

Depending on the building blocks that are used, more Spectrum Scale licenses might be needed for the scale-out solution. The Spectrum Scale on x86 Single Server for Integrated Offerings V4.1 provides file system capabilities for single-node integrated offerings. This Spectrum Scale license does not cover use in multinode environments, such as the scale-out solution described here. To use building blocks that are included with the Spectrum Scale on x86 Single Server for Integrated Offerings licenses, Spectrum Scale on x86 Server licenses must be obtained for these building blocks for a scale-out solution. For more information about which type of license is included with a specific model and the number of PVUs, see “Spectrum Scale license information” on page 220. Alternatively, Spectrum Scale File Placement Optimizer licenses can be used with Spectrum Scale on x86 Server licenses.

In a scale-out configuration, a minimum of three nodes must use Spectrum Scale on x86 Server licenses, and the remaining nodes can use Spectrum Scale File Placement Optimizer licenses. Other setups, such as the disaster recovery solution described in 7.3, “HA and DR for scale-out SAP HANA” on page 194, might require more nodes that use Spectrum Scale on x86 Server licenses, depending on the role of the nodes in the setup. For more information about the Spectrum Scale license types, which type of license is included with a specific model, and the number of PVUs needed, see “Spectrum Scale license information” on page 220.

6.3 Lenovo X6 based environments that use Intel Xeon E7 v3 processors

With the introduction of X6 in the beginning of 2014 Lenovo, set the foundation to support more than one Intel CPU generation in the same server. Initially launched with the Intel Xeon processor E7 v2 family of CPUs, the same hardware platform is used to run the successor generation; that is, the Intel Xeon processor E7 v3 family of CPUs.


Customers who invested in the X6 platform can upgrade from Intel Xeon E7 v2 processors to Intel Xeon E7 v3 processors by purchasing new CPU Books. DDR3 main memory installed on the CPU Books can be uninstalled and used in the new CPU Books.

The Intel Xeon E7 v3 family includes more cores on each CPU, which translates to more main memory being supported on SAP HANA workload-optimized X6 models. They are built on two servers: the x3850 X6, with up to four processor sockets, and the x3950 X6, with up to eight processor sockets. Both servers use the same components that can be reused when growing from an x3850 X6 to an x3950 X6 solution. Only the mechanical chassis must be replaced to accommodate more components, such as processors and I/O. For more information about the X6 server architecture and components, see 5.1, “Lenovo X6 systems” on page 58.

Figure 6-7 shows an overview of the available workload-optimized solutions that are based on the x3850 X6 server that is running Intel Xeon E7 v3 processors, which covers the entry-level and mid-range requirements (2-socket and 4-socket).


Figure 6-8 on page 119 shows the building blocks for SAP HANA solutions that are based on the x3950 X6 server that are running Intel Xeon E7 v3 processors, which covers the mid-range to high-end requirements (4-socket and 8-socket).

Memory size

Processors

128 GB 256 GB 384 GB 512 GB 768 GB

2

4

BW, SoH

BW, SoH

1 TB 1.5 TB 2 TB

BW, SoH

BW, SoH BW, SoH BW, SoH

Upgrade to 8 processors supported with x3950 X6 chassis (all parts reusable)

BW, SoH BW, SoH BW, SoH SoH SoH

3 TB

SoH BW, SoH

4 TB

SoH

BW = Business Warehouse SoH = Suite on HANA, S/4HANA Scale-out for BW only Some upgrades require to replace memory modules with higher capacity modules

BW, SoH



Table 6-10 lists the technical details of all x3850 X6-based workload-optimized models for SAP HANA that is running Intel Xeon E7 v3 processors. They cover the range of 2- and 4-socket solutions.

Table 6-10 x3850 X6 workload-optimized models for SAP HANA that uses Intel Xeon E7 v3 processors

BW, SoH

Memory size

Processors

256 GB 512 GB 768 GB 1 TB 1.5 TB

4

8

BW, SoH

BW, SoH

2 TB 3 TB 4 TB

BW, SoH BW, SoH

BW, SoH

BW, SoH BW, SoH BW, SoH

6 TB

SoH

SoH

BW, SoH

8 TB

SoH

SoH


BW, SoH

BW, SoH

Notes:

� The Intel Xeon E7 v3 processors provide up to 18 cores, which is three cores more than the Intel Xeon E7 v2, which has up to 15 cores. This increase in core count allows more main memory to be installed in an SAP HANA X6 system when the latest Intel Xeon E7 v3 processors are run.

� Increased memory capacity is supported when running SAP HANA SPS11 or later. Please see the respective sections to identify the actual limit.

� The architecture of the Lenovo System x Solution for SAP HANA on X6 remains identical, regardless of the processor generation. This configuration ensures a seamless upgrade path and allows for mixed environments that uses both generations to work without any special considerations.


Business Warehouse

Scale-out BW


Total rack space

AC3-2H-128 2x Intel XeonE7-8880 v3

128 GB DDR3/4 Yes No Yes 4U






Table 6-11 lists the technical details of all x3950 X6 based workload-optimized models for SAP HANA that is running Intel Xeon E7 v3 processors. They cover the range of four and eight socket solutions.







1024 GB DDR3/4 No N/A Yes 6U


1536 GB DDR3/4 No N/A Yes 6U




512 GB DDR3/4 Yes Yes, up to 94 nodesb

Yes 4Uc




1 TB DDR3/4 Yes Yes, up to 94 nodesb

Yes 6U


1.5 TB DDR3/4 Yes Yes, up to 94 nodesb

Yes 6U



Yes 6U


3 TB DDR3/4 No N/A Yes 6U




b. Support for up to 224 nodes is verified by Lenovo development and presented to SAP.c. 6U when used in a scale-out environment.


Business Warehouse

Scale-out BW


Total rack space


Business Warehouse

Scale-out BW


Total rack space



Yes 8U



Yes 8U



Yes 8U




Yes 8U



Yes 8U



Yes 8U









Yes 8U



Yes 8U



Yes 8Uc



Yes 10U



Yes 10Ud



a. Alternative CPU type E7-8890 v3 is supported for enhanced performance. Available upon request.b. Support for up to 224 nodes is verified by Lenovo development and presented to SAP.c. 10U when used in a scale-out environment.d. 12U when used in a scale-out environment


Business Warehouse

Scale-out BW


Total rack space

Notes:

� For non-productive database instances, SAP maintains relaxed hardware requirements. As a result, you can have more memory in a server; for example, for test or training instances of SAP HANA.

� The next sections list only the solutions that are certified by SAP for productive environments.

� All X6 servers physically support the installation of more memory beyond what is listed in this section. At the time of this writing, Lenovo can provide X6 systems with up to 12 TB of main memory for non-productive SAP HANA landscapes.




Figure 6-9 on page 122 shows all X6 based models available for a single-node SAP BW environment when the Intel Xeon processors of the E7 v3 family are run. The highest-capacity configurations (2 TB on the 4-socket and 4 TB on the 8-socket systems) are only supported when running SAP HANA SPS11 or later.

Figure 6-9 Single-node X6 based solutions for Business Warehouse that uses Intel Xeon E7 v3 processors

In the lower half of Figure 6-9 (below the dashed line), the available models that are based on the x3850 X6 server are shown. In the upper half of Figure 6-9, the models that are based on the x3950 X6 server are shown. There are four socket models for both. You can physically upgrade an x3850 X6 to an x3950 X6 server by replacing the mechanical enclosure that houses the CPU Books, Storage Books, and I/O Books. The Books can be used in both server types.

Memory size

Processors

128 GB 256 GB 384 GB 512 GB 768 GB

2

4

1 TB 1.5 TB 2 TB

8

x3850 X6 with Intel Xeon E7 v3


Upgrade supported by replacing mechanical chassis

3 TB 4 TB


Table 6-12 lists the technical details of all x3850 X6 workload-optimized models for SAP HANA when the Intel Xeon processors of the E7 v3 family are run.


Model) CPUsa

a. Alternative CPU type E7-8890 v3 is available upon request.

Mainmemory


Upgrade options


128 GB DDR3/4(16x 8 GB)


128 → 256


256 GB DDR3/4(32x 8 GB or16x 16 GB)


256 → 384256 → 5122S → 4S


384 GB DDR3/4(48x 8 GB)


2S → 4S-512


512 GB DDR3/4(32x 16 GB or 16x 32 GB)


512 → 768512 → 1 TB2S → 4S


768 GB DDR3/4(48x 16 GB)


2S → 4S- 1 TB


256 GB DDR3/4(32x 8 GB)


256 → 512


512 GB DDR3/4(64x 8 GB or32x 16 GB)


512 → 768512 → 1 TB


768 GB DDR3/4(96x 8 GB)

13.2 TB(4x 400 GB SAS SSD,13x 1.2 TB 10K SAS HDD)b


None


1 TB DDR3/4(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR3/4(96x 16 GB)


Grow into 8-socket


2 TB DDR3/4 (64x 32 GB or 32x 64 GB)


Grow into 8-socket


Table 6-13 lists the technical details of all x3950 X6 workload-optimized models for SAP HANA when the Intel Xeon processors of the E7 v3 family are run.


You can also grow into a clustered scale-out environment without changing any of the hardware of the single-node solution. The upgrade to scale-out is supported for memory configurations with 512 GB, 1 TB, 1.5 TB (4-socket only), 2 TB, 3 TB (8-socket only) or 4 TB (8-socket only) as denoted by the multiple boxes in Figure 6-9 on page 122. Scale-out of other memory sizes is not supported because adding memory first (to scale up) to reach one of the supported memory sizes gives better performance than adding nodes.

Model CPUsa


Mainmemory


Upgrade options


256 GB DDR3/4(32x 8 GB)


256 → 512


512 GB DDR3/4(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB4S → 8S


768 GB DDR3/4(96x 8 GB)


4S → 8S-1 TB


1 TB DDR3/4(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB4S → 8S


1.5 TB DDR3/4(96x 16 GB)


4S → 8S- 2TB4S → 8S- 3TB


2 TB DDR3/4(64x 32 GB or 32x 64 GB)


4S → 8S-4TB


512 GB DDR3/4(64x 8 GB)


512 → 1 TB


1 TB DDR3/4(128x 8 GB or 64x 16 GB)


1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR3/4(192x 8 GB)


None


2 TB DDR3/4(128x 16 GB or 64x 32 GB)


2 → 3 TB2 → 4TB


3 TB DDR3/4(192x 16 GB)


Scale-out


4 TB DDR3/4 (128x 32 GB or 64x 64 GB)


Scale-out


Upgrading the server requires downtime of the system. However, because of the capability of the underlying storage subsystem (XFS or Spectrum Scale) to add storage capacity online simply by adding only devices, data that is on the system remains intact. It is recommended that data is backed up before the system’s configuration is changed.







Figure 6-10 shows the building blocks you can choose from when you plan for SAP S/4HANA or SAP Business Suite powered by SAP HANA.

Figure 6-10 Single-node X6 based models for SAP S/4HANA or SAP Business Suite powered by SAP HANA

Memory size

Processors

128 GB 256 GB 384 GB 512 GB 768 GB

2

4

1 TB 1.5 TB 2 TB 3 TB 4 TB 6 TB

8



*

Some upgrades require to replace memory modules with higher capacity modules

8 TB


In the lower half of Figure 6-10 (below the dashed line at four processors), the available models that are based on the x3850 X6 server are shown. In the upper half of Figure 6-10, the models that are based on the x3950 X6 server are shown. There are four socket models for both. You can physically upgrade an x3850 X6 to an x3950 X6 server by replacing the mechanical enclosure that houses the CPU Books, Storage Books, and I/O Books. The Books can be used in both server types.

Table 6-14 lists the technical details of all x3850 X6-based models for SAP HANA with Intel Xeon processors of the E7 v3 family when SAP S/4HANA or SAP Business Suite is run on top.

Table 6-14 x3850 X6 workload-optimized models for S/4HANA or Suite on HANA that use Intel Xeon E7 v3 processors



Upgrade options


128 GB DDR3/4(16x 8 GB)


128 → 256


256 GB DDR3/4(32x 8 GB or16x 16 GB)


256 → 384256 → 5122S → 4S


384 GB DDR3/4(48x 8 GB)


256 → 5122S → 4S-512


512 GB DDR3/4(32x 16 GB or 16x 32 GB)


2S → 4S512 → 768


768 GB DDR3/4(48x 16 GB)


2S → 4S- 1 TB


1024 GB DDR3/4(32x 32 GB or 16x 64 GB)


1 → 1.5 TB2S → 4S


1536 GB DDR3/4(48x 32 GB)


2S → 4S-2TB


256 GB DDR3/4(32x 8 GB)


256 → 512


512 GB DDR3/4(64x 8 GB or32x 16 GB)


512 → 768512 → 1 TB


768 GB DDR3/4(96x 8 GB)


None


1 TB DDR3/4(64x 16 GB or 32x 32 GB)


1 → 1.5 TB1 → 2 TB


1.5 TB DDR3/4(96x 16 GB)


None


Table 6-15 on page 127 shows the technical details of all x3950 X6-based models for SAP S/4HANA and SAP Business Suite powered by SAP HANA when Intel Xeon processors of the E7 v3 family are run.

Table 6-15 x3950 X6 workload-optimized models for S/4HANA or Suite on HANA that use Intel Xeon E7 v3 processors


2 TB DDR3/4(64x 32 GB or32x 64 GB)


2 → 3 TB


3 TB DDR3/4(96x 32 GB)


None

a. Alternative CPU type E7-8890 v3 is available upon request.b. Extra drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.



Upgrade options



Upgrade options


256 GB DDR3/4(32x 8 GB)


256 → 512


512 GB DDR3/4(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB4S → 8S


768 GB DDR3/4(96x 8 GB)


4S → 8S- 1 TB


1 TB DDR3/4(64x 16 GB or 32x 32 GB)


1 → 1.5 TB1 → 2 TB4S → 8S


1.5 TB DDR3/4(96x 16 GB)


4S → 8S


2 TB DDR3/4(64x 32 GB or 32x 64 GB)


2 → 3 TB4S → 8S


3 TB DDR3/4(96x 32 GB)


4S → 8S- 4 TB


512 GB DDR3/4(64x 8 GB)


512 → 1 TB


1 TB DDR3/4(128x 8 GB or 64x 16 GB)


1 → 1.5 TB1 → 2 TB


1.5 TB DDR3/4(192x 8 GB)


None


2 TB DDR3/4(128x 16 GB or 64x 32 GB)


2 → 3 TB2 → 4 TB


3 TB DDR3/4(192x 16 GB)


None


Upgrading the server requires downtime of the system. However, because of the capability of the underlying storage subsystem (XFS or Spectrum Scale) to add storage capacity simply by adding devices only, data that is on the system remains intact. It is recommended that data is backed up before the system’s configuration is changed.

Support for business continuityImplementing some level of business continuity features is highly recommended because Business Suite constitutes the entire business and its processes. Downtime of these applications most often has a significant effect on the business. Clients in the manufacturing industry are often faced with a halt of their production lines if the supply chain system is not available.






Customers whose BW databases do not fit into a single node must scale out their SAP HANA installation, which means clustering multiple single nodes with a high-speed interconnect (10 Gb Ethernet) and spreading out the database tables across the participating nodes.

Figure 6-11 on page 129 shows all X6 based building blocks that are supported in such scale-out scenarios when the Intel Xeon processors of the E7 v3 family are run. The highest-capacity configurations (2 TB on the 4-socket and 4 TB on the 8-socket systems) are only supported when running SAP HANA SPS11 or later.


4 TB DDR3/4(128x 32 GB or 64x 64 GB)


4 → 6 TB


6 TB DDR3/4(192x 32 GB)


See footnotec

a. Alternative CPU type E7-8890 v3 is supported for enhanced performance. Available upon request.b. More drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.c. If your SAP database exceeds 6 TB in main memory size, email: [email protected]



Upgrade options


Figure 6-11 X6 based solutions for scale-out Business Warehouse using Intel Xeon E7 v3 processors

In the lower half of Figure 6-11 on page 129, the models that are based on the x3850 X6 server are shown. In the upper half of Figure 6-11 on page 129, the models that are based on the x3950 X6 server are shown. Models with four sockets are available that use both servers. You can physically upgrade a x3850 X6 server to a x3950 X6 server by replacing the x3850 4U mechanical chassis with an 8U version for the x3950 X6. All CPU Books, I/O Books, and Storage Books can be reused in the 8U chassis.

Table 6-16 lists the technical details of all workload-optimized x3850 X6 models in a scale-out configuration. The highest-capacity configuration with 2 TB is only supported when running SAP HANA SPS11 or later.

Table 6-16 x3850 X6 models for a scale-out SAP Business Warehouse that use Intel Xeon E7 v3 processors

Memory size

Processors

512 GB 768 GB

4

1 TB 1.5 TB 2 TB

8




3 TB 4 TB

Model CPUsa

a. Alternative CPU type E7-8890 v3 is available upon request. For more information, see“CPU and memory” on page 73.

Mainmemory


Upgrade options

Maximum cluster size

AC3-4H-512-C 4x Intel XeonE7-8880 v3

512 GB DDR3/4(64x 8 GB or32x 16 GB)



512 → 1 TB 94 nodes


1 TB DDR3/4(64x 16 GB or 32x 32 GB)


1 → 1.5 TB 94 nodes


1.5 TB DDR3/4(96x 16 GB)


Grow into x3950 X6

94 nodes


2 TB DDR3/4 (64x 32 GB or 32x 64 GB)


Grow into x3950 X6

94 nodes


Table 6-17 lists the technical details of all workload-optimized x3950 X6 models in a scale-out configuration when the Intel Xeon processors of the E7 v3 family are run. The highest capacity configurations with 2 TB on the 4-socket and 4 TB on the 8-socket system are only supported when running SAP HANA SPS11 or later.



Model CPUsa


Mainmemory


Upgrade options



512 GB DDR3/4(64x 8 GB or 32x 16 GB)


512 → 1 TB 94 nodes


1 TB DDR3/4(64x 16 GB or 32x 32 GB)


4S → 8S 94 nodes


1.5 TB DDR3/4(96x 16 GB)


4S → 8S-2T 94 nodes


2 TB DDR3/4 (64x 32 GB or 32x 64 GB)


4S → 8S-4T 94 nodes


1 TB DDR3/4(128x 8 GB or 64x 16 GB)


1 TB → 2 TB

94 nodes


2 TB DDR3/4(128x 16 GB or 64x 32 GB)



1 TB → 3 TB

94 nodes


3 TB DDR3/4(192x 16 GB)


94 nodes


4 TB DDR3/4 (128x 32 GB or 64x 64 GB)


94 nodes

Notes:

� The minimum number of nodes in any SAP HANA scale-out environment is three worker nodes. A cluster of two worker nodes is not supported. Adding one standby node for business continuity results in a four-node cluster being the minimum that is supported.

� The maximum number of nodes that is available for implementation is 94 (except for model AC3-2H-512-C, which is limited to four nodes). However, Lenovo validated feasibility for up to 224 nodes in its labs to accommodate for growing customer demand.



Support for business continuityAll X6 scale-out solutions that are listed in Table 6-16 on page 129 and Table 6-17 on page 130 support the following features for business continuity:


HA can be implemented by using storage-based replication. DR solutions can be implemented by using storage-based replication or SAP HANA System Replication. You also can use the idling DR site nodes to run a second SAP HANA instance for nonproductive purposes. For more information about these solutions, see 7.3, “HA and DR for scale-out SAP HANA” on page 194.

Single-node to scale-out solution upgrade requirementsWith X6 systems, the Lenovo scale-out solution for SAP HANA builds upon the same building blocks as they are used in a single-server installation. There are hardware and software components that are needed to complement the basic building blocks when a single-node solution is upgraded to a scale-out solution, as described in this section.

Extra hardware needs to be bought only when scaling out using models AC3-4H-512 or AC4-8H-2048. For these models, more storage capacity is required for the second Spectrum Scale replica that is generated in scale-out environments. This second replica does not exist for single nodes.

Depending on the building blocks that are used, more Spectrum Scale licenses might be needed for the scale-out solution. The Spectrum Scale on x86 Single Server for Integrated Offerings V4.1 provides file system capabilities for single-node integrated offerings. This Spectrum Scale license does not cover the usage in multinode environments, such as the scale-out solution that is described here. To use building blocks that are included with the Spectrum Scale on x86 Single Server for Integrated Offerings licenses, Spectrum Scale on x86 Server licenses must be obtained for these building blocks for a scale-out solution. For more information about the type of license included with a specific model and the number of PVUs needed, see “Spectrum Scale license information” on page 220. Alternatively, Spectrum Scale File Placement Optimizer licenses can be used with Spectrum Scale on x86 Server licenses.

In a scale-out configuration, a minimum of three nodes must use Spectrum Scale on x86 Server licenses, and the remaining nodes can use Spectrum Scale File Placement Optimizer licenses. Other setups, such as the disaster recovery solution that is described in 7.3, “HA and DR for scale-out SAP HANA” on page 194, might require more nodes that use Spectrum Scale on x86 Server licenses, depending on the role of the nodes in the setup. For more information about the Spectrum Scale license types, which type of license is included with a specific model, and the number of PVUs needed, see “Spectrum Scale license information” on page 220.

6.4 Upgrading X6 from Intel Xeon E7 v2 to E7 v3 CPUs

Customers that invested in earlier Lenovo X6 servers and want to upgrade to the Intel Xeon processor E7 v3 family have different options, depending on their needs.

Running Intel Xeon E7 v3 processors in one single cluster with Intel Xeon E7 of the first generation (installed in eX5 systems) is not supported.


Table 6-18 lists the different paths customers can follow to introduce the Intel Xeon processor E7 v3 family into their data center.

Table 6-18 Upgrade options from Intel Xeon E7 v2 to Intel Xeon E7 v3

6.4.1 Upgrading a single node

A single node can be upgraded by physically replacing the CPU Books only and carrying over all DIMM modules. No reinstallation of the operating system must be done. The following process is used:

1. Confirm that your current uEFI and IMM code levels supports E7 v3 processors.

2. Shut down the server and pull out the AC power connection.

3. Uninstall all DIMM modules from the CPU Books and reinstall them in the correct order into the new CPU Books.

4. Restart the server.

6.4.2 Converting a single node into a scale-out environment

A single node with Intel Xeon E7 v2 processors can be integrated into a scale-out environment by using one of the following methods:

� Not upgrading the Intel Xeon E7 v2 node and add nodes with Intel Xeon E7 v3 processors

� Upgrading the single node to Intel Xeon E7 v3 processors and then add nodes that are of the same generation

Not upgrading the node to latest Intel Xeon E7 v3 processors leads to a mixed cluster with different processor generations. This configuration can be easily done by adding nodes only and following the instructions about how to turn a single node into a scale-out node (as described in the Lenovo Implementation Guide for the System x X6 solution).

Upgrading a single node before turning it into a scale-out environment is the second possibility. Follow the steps that were described in 6.4.1, “Upgrading a single node” on page 132.

6.4.3 Upgrading a scale-out environment

Upgrading a scale-out environment can be done without upgrading the nodes to the latest Intel Xeon processor generation.

To →From

Single node E7 v3 Scale-out E7 v3 Scale-out with mixed E7 v2 and E7 v3

Single node E7 v2 Yes Yes Yes

Scale-out E7 v2 N/A Yes Yes

Note: When operating a mixed scale-out environment, most of the servers must have the newer CPU generation installed. A two-node environment with one server of each processor generation is not allowed. A three-node environment with two new generation servers and one current generation server is allowed.


Upgrading a scale-out environment into a mixed scale-outNot upgrading the nodes to latest Intel Xeon E7 v3 processors leads to a mixed cluster with different processor generations. This process can be accomplished by adding nodes to your scale-out environment only.

Upgrading a scale-out environment into a Intel Xeon E7 v3-only environment

For all nodes, follow the upgrade instructions for a single node. Upgrade one node at a time. If you have HA capabilities in your cluster, you can complete the upgrade without shutting down your SAP HANA instance (rolling upgrade).

It is also possible to shut down the database instance and upgrade all nodes at the same time.

6.5 Lenovo X6 based environments that use Intel Xeon E7 v4 processors

With the introduction of X6 in the beginning of 2014 Lenovo, set the foundation to support more than one Intel CPU generation in the same server. Initially launched with the Intel Xeon processor E7 v2 family of CPUs, the same hardware platform is used to run the 2015 generation (Xeon processor E7 v3 family) and the 2016 generation (Xeon processor E7 v4 family).

Customers who have invested in the X6 platform can upgrade from Intel Xeon E7 v2 processors to Intel Xeon E7 v3 processors and from Intel Xeon E7 v3 to Intel Xeon E7 v4 processors by purchasing new CPU Books. DDR3 main memory installed on the CPU Books can be uninstalled and used in the new CPU Books, or replaced with DDR4 memory (if using the Xeon E7 v4 platform).

The new Intel Xeon E7 v4 family includes more cores on each CPU, which translates to more main memory being supported on SAP HANA workload-optimized X6 models. They are built upon two servers: the x3850 X6, with up to four processor sockets, and the x3950 X6, with up to eight processor sockets. Both servers use the same components, which can be reused when growing from an x3850 X6 to an x3950 X6 solution. Only the 4U mechanical chassis must be replaced with an 8U chassis to accommodate more components, such as processors, memory, and I/O. For more information about the X6 server architecture and components, see 5.1, “Lenovo X6 systems” on page 58.

Note: When operating a mixed scale-out environment, most of the servers must have the newer CPU generation installed. A two-node environment with one server of each processor generation is not allowed. A three-node environment with two new-generation servers and one previous generation server is allowed.


Figure 6-12 shows an overview of the available workload-optimized solutions based on the x3850 X6 server that is running Intel Xeon E7 v4 processors, which covers the entry-level and mid-range requirements (2-socket and 4-socket).


Figure 6-13 on page 134 shows the building blocks for SAP HANA solutions based on x3950 X6 servers running Intel Xeon E7 v4 processors, which covers the mid-range to high-end requirements (4-socket and 8-socket).


Memory size

Processors

128 GB 256 GB 384 GB 512 GB 768 GB

2

4

BW, SoH

BW, SoH

1 TB 1.5 TB 2 TB

BW, SoH

BW, SoH BW, SoH BW, SoH BW*, SoH

Upgrade to 8 processors supported with x3950 X6 chassis (all parts reusable)

BW, SoH


BW, SoH BW, SoH SoH SoH

3 TB

SoH BW, SoH

4 TB

SoH**

SoH** * Requires SPS11 for BW ** Requires SPS12

BW, SoH

Memory size

Processors

256 GB 512 GB 768 GB 1 TB 1.5 TB

4

8

BW, SoH

BW, SoH

2 TB 3 TB 4 TB

BW, SoH BW, SoH

BW, SoH

BW, SoH BW, SoH BW, SoH BW*, SoH

6 TB

SoH

BW*, SoH SoH

BW, SoH

8 TB

SoH**

SoH** * Requires SPS11 for BW ** Requires SPS12



Table 6-19 lists the technical details of all x3850 X6-based workload-optimized models for SAP HANA running Intel Xeon E7 v4 processors. They cover the range of 2- and 4-socket solutions.


Notes:

� The new Intel Xeon E7 v4 processors provide up to 24 cores, which is six cores more than on the Intel Xeon E7 v3 that has up to 18 cores. This increase in core count allows more main memory to be installed in an SAP HANA X6 system when the latest Intel Xeon E7 v4 processors are run.

� Increased memory capacity is supported when running SAP HANA SPS11 or later. Please see the respective sections to identify the actual limit.

� The architecture of the Lenovo System x Solution for SAP HANA on X6 remains identical, regardless of the processor generation. This configuration ensures a seamless upgrade path and allows for mixed environments that use both generations, to work without any special considerations.


Business Warehouse

Scale-out BW


Total rack space

AC3-2B-128 2x Intel XeonE7-8880 v4






















Yes 4Uc





Yes 6U


Table 6-11 lists the technical details of all x3950 X6 based workload-optimized models for SAP HANA running Intel Xeon E7 v4 processors. They cover the range of 4- and 8-socket solutions.




Yes 6U



Yes 6U







a. Alternative CPU type E7-8890 v4 is available upon request. For more information, see “CPU and memory” on page 73.

b. Support for up to 224 nodes is verified by Lenovo development and presented to SAP.c. 6U when used in a scale-out environment.


Business Warehouse

Scale-out BW


Total rack space


Business Warehouse

Scale-out BW


Total rack space



Yes 8U



Yes 8U



Yes 8U



Yes 8U



Yes 8U



Yes 8U









Yes 8U






Yes 8U


1.5 TB DDR4 Yes N/A Yes 8U



Yes 8Uc



Yes 10U



Yes 10U







a. Alternative CPU type E7-8890 v4 is supported for enhanced performance. Available upon request.b. Support for up to 224 nodes is verified by Lenovo development and presented to SAP.c. 10U when used in a scale-out environment.


Business Warehouse

Scale-out BW


Total rack space

Notes:

� For non-productive database instances, SAP maintains relaxed hardware requirements. As a result, you can have more memory in a server; for example, for test or training instances of SAP HANA.

� The next sections list only the solutions that are certified by SAP for productive environments.

� All X6 servers physically support the installation of more memory beyond what is listed in this section. At the time of this writing, Lenovo can provide X6 systems with up to 12 TB of main memory for non-productive SAP HANA landscapes.


Figure 6-14 shows all X6 based models that are available for a single-node SAP BW environment when the latest Intel Xeon processors of the E7 v4 family are run.

Figure 6-14 Single-node X6 based solutions for Business Warehouse that use Intel Xeon E7 v4 processors

In the lower half of Figure 6-14 (below the dashed line), the available models based on the x3850 X6 server are shown. In the upper half of Figure 6-14, the models based on the x3950 X6 server are shown. There are four-socket models for both. You can physically upgrade an x3850 X6 to an x3950 X6 server by replacing the mechanical enclosure that houses the CPU Books, Storage Books, and I/O Books. The Books can be used in both server types.

Table 6-21 lists the technical details of all x3850 X6 workload-optimized models for SAP HANA when the latest Intel Xeon processors of the E7 v4 family are run.


Memory size

Processors

128 GB 256 GB 384 GB 512 GB 768 GB

2

4

1 TB 1.5 TB 2 TB

8




3 TB 4 TB

Model) CPUsa Mainmemory


Upgrade options


128 GB DDR4(16x 8 GB)


128 → 256


256 GB DDR4(32x 8 GB or16x 16 GB)


256 → 384256 → 5122S → 4S


384 GB DDR4(48x 8 GB)


2S → 4S-512


512 GB DDR4(32x 16 GB or 16x 32 GB)


512 → 768512 → 1 TB2S → 4S


Table 6-22 lists the technical details of all x3950 X6 workload-optimized models for SAP HANA when the latest Intel Xeon processors of the E7 v4 family are run.



768 GB DDR4(48x 16 GB)


2S → 4S- 1 TB


256 GB DDR4(32x 8 GB)


256 → 512


512 GB DDR4(64x 8 GB or32x 16 GB)


512 → 768512 → 1 TB


768 GB DDR4(96x 8 GB)


None


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR4(96x 16 GB)


1 TB → 1.5 TB1 TB → 2 TB


2 TB DDR4(64x 32 GB or 32x 64 GB)


Scale-out or grow into x3950 X6

a. Alternative CPU type E7-8890 v4 is available upon request for all models.b. Extra drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.



Upgrade options



Upgrade options


256 GB DDR4(32x 8 GB)


256 → 512


512 GB DDR4(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB4S → 8S


768 GB DDR4(96x 8 GB)


4S → 8S-1 TB


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB4S → 8S


1.5 TB DDR4(96x 16 GB)


1.5 → 2TB


You can also grow into a clustered scale-out environment without changing any of the hardware of the single-node solution. The upgrade to scale-out is supported for memory configurations of 512 GB, 1 TB, 1.5 TB, 2 TB (4-socket only), 3 TB, or 4 TB (8-socket only) as denoted by the multiple boxes in Figure 6-14 on page 138. Scale-out of other memory sizes is not supported because adding memory first (to scale up) to reach one of the supported memory sizes gives better performance than adding nodes.

Upgrading the server requires downtime of the system. However, because of the capability of the underlying storage subsystem (XFS or Spectrum Scale) to add capacity online simply by adding only devices, the data on the system remains intact. It is recommended that data is backed up before the system’s configuration is changed.



All HA and DR solutions can be implemented by using storage based replication or SAP HANA System Replication. You also can use the standby nodes in HA and DR solutions to run a second SAP HANA instance for non-productive purposes. For more information about these architectures, see 7.2, “HA and DR for single-node SAP HANA” on page 177.


2 TB DDR4(64x 32 GB or 32x 64 GB)


4S → 8S-4TBScale-out


512 GB DDR4(64x 8 GB)


512 → 1 TB


1 TB DDR4(128x 8 GB or 64x 16 GB)


1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR4(192x 8 GB)


None


2 TB DDR4(128x 16 GB or 64x 32 GB)


2 → 3 TB2 → 4 TB


3 TB DDR4(192x 16 GB)


Scale-out


4 TB DDR4(128x 32 GB or 64x 64 GB)


Scale-out




Upgrade options





Figure 6-15 shows the building blocks that you can choose from when you plan for SAP S/4HANA or SAP Business Suite powered by SAP HANA.

Figure 6-15 Single-node X6 based models for SAP S/4HANA or SAP Business Suite powered by SAP HANA

In the lower half of Figure 6-15 (below the dashed line at four processors), the available models that are based on the x3850 X6 server are shown. In the upper half of Figure 6-15, the models that are based on the x3950 X6 server are shown. There are four socket models for both. You can physically upgrade an x3850 X6 to an x3950 X6 server by replacing the mechanical enclosure that houses the CPU Books, Storage Books, and I/O Books. The Books can be used in both server types. Table 6-23 lists the technical details of all x3850 X6-based models for SAP HANA with latest Intel Xeon processors of the E7 v4 family when Business Suite is run on top.


Memory size

Processors

128 GB 256 GB 384 GB 512 GB 768 GB

2

4

1 TB 1.5 TB 2 TB 3 TB 4 TB 6 TB

8




8 TB

Model CPUsa Main memory Storage (data and log) Upgrade options


128 GB DDR4(16x 8 GB)


128 → 256


256 GB DDR4(32x 8 GB or16x 16 GB)


256 → 384256 → 5122S → 4S



384 GB DDR4(48x 8 GB)


256 → 5122S → 4S-512


512 GB DDR4(32x 16 GB or 16x 32 GB)


2S → 4S512 → 768


768 GB DDR4(48x 16 GB)


2S → 4S- 1 TB


1024 GB DDR4(32x 32 GB or 16x 64 GB)


1 → 1.5 TB2S → 4S


1536 GB DDR4(48x 32 GB)


1.5 → 2TB


2048 GB DDR4(32x 64 GB)


2S → 4S2S → 4S- 4 TB


256 GB DDR4(32x 8 GB)


256 → 512


512 GB DDR4(64x 8 GB or32x 16 GB)


512 → 768512 → 1 TB


768 GB DDR4(96x 8 GB)


None


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 → 1.5 TB1 → 2 TB


1.5 TB DDR4(96x 16 GB)


None


2 TB DDR4(64x 32 GB or32x 64 GB)


2 → 3 TB2 → 4 TB




None




None

a. Alternative CPU type E7-8890 v4 is available upon request.b. Extra drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.

Model CPUsa Main memory Storage (data and log) Upgrade options


Table 6-24 shows the technical details of all x3950 X6-based models for SAP S/4HANA and SAP Business Suite powered by SAP HANA when latest Intel Xeon processors of the E7 v4 family are run.


Model CPUsa


Main memory Storage (data and log) Upgrade options


256 GB DDR4(32x 8 GB)


256 → 512


512 GB DDR4(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB4S → 8S


768 GB DDR4(96x 8 GB)


4S → 8S- 1 TB


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 → 1.5 TB1 → 2 TB4S → 8S


1.5 TB DDR4(96x 16 GB)


4S → 8S


2 TB DDR4(64x 32 GB or 32x 64 GB)


2 → 3 TB4S → 8S




4S → 8S- 4 TB




4S → 8S-4S → 8S- 8 TB


512 GB DDR4(64x 8 GB)


512 → 1 TB


1 TB DDR4(128x 8 GB or 64x 16 GB)


1 → 1.5 TB1 → 2 TB


1.5 TB DDR4(192x 8 GB)


None


2 TB DDR4(128x 16 GB or 64x 32 GB)


2 → 3 TB2 → 4 TB


3 TB DDR4(192x 16 GB)



None


4 TB DDR4(128x 32 GB or 64x 64 GB)


4 → 6 TB


6 TB DDR4(192x 32 GB)


None


8 TB DDR4(128x 64 GB)


See footnotec

c. If your SAP database exceeds 8 TB in main memory size, email: [email protected]


Upgrading the server requires downtime of the system. However, because of the capability of the underlying storage subsystem (XFS or Spectrum Scale) to add capacity simply by adding devices only, data that is on the system remains intact. It is recommended that data is backed up before the system’s configuration is changed.

Support for business continuityImplementing some level of business continuity features is highly recommended because SAP Business Suite constitutes the entire business and its processes. Downtime of these applications most often has a significant effect on the business. Clients in the manufacturing industry are often faced with a halt of their production lines if the supply chain system is not available.



All HA and DR solutions can be implemented by using storage based replication or SAP HANA System Replication. You also can use the standby nodes in HA and DR solutions to run a second SAP HANA instance for non-productive purposes. For more information about these architectures, see 7.2, “HA and DR for single-node SAP HANA” on page 177. Adding business continuity features after the initial implementation is possible. Only extra hardware must be bought, and no parts are retired.


Customers whose BW databases do not fit into a single node must scale out their SAP HANA installation, which means clustering multiple single nodes with a high-speed interconnect (10 Gb Ethernet) and spreading out the database tables across the participating nodes. Figure 6-16 shows all X6 based building blocks that are supported in such scale-out scenarios when the latest Intel Xeon processors of the E7 v4 family are run.

Figure 6-16 X6 based solutions for scale-out Business Warehouse using Intel Xeon E7 v4 processors

Memory size

Processors

512 GB 768 GB

4

1 TB 1.5 TB 2 TB


8




3 TB 4 TB


In the lower half of Figure 6-16 on page 144, the models that are based on the x3850 X6 server are shown. In the upper half of Figure 6-16 on page 144, the models that are based on the x3950 X6 server are shown. Models with four sockets are available that use both servers. You can physically upgrade a x3850 X6 server to a x3950 X6 server by replacing the x3850 4U mechanical chassis with an 8U version for the x3950 X6. All CPU Books, I/O Books, and Storage Books can be reused in the 8U chassis.

Table 6-25 lists the technical details of all workload-optimized x3850 X6 models in a scale-out configuration.

Table 6-25 x3850 X6 models for a scale-out SAP Business Warehouse that use Intel Xeon E7 v4 processors

Table 6-26 lists the technical details of all workload-optimized x3950 X6 models in a scale-out configuration when the latest Intel Xeon processors of the E7 v4 family are run.


Model CPUsa

a. Alternative CPU type E7-8890 v4 is available upon request. For more information, see“CPU and memory” on page 73.

Mainmemory


Scale-up options


AC3-4B-512-C 4x Intel XeonE7-8880 v4

512 GB DDR4(64x 8 GB or32x 16 GB)



512 → 1 TB 94 nodes


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 → 1.5 TB 94 nodes


1.5 TB DDR4(96x 16 GB)


Grow into x3950 X6

94 nodes


2 TB DDR4(64x 32 GB or 32x 64 GB)


Grow into x3950 X6

94 nodes



Scale-up options



512 GB DDR4(64x 8 GB or 32x 16 GB)


512 → 1 TB 94 nodes


1 TB DDR4(64x 16 GB or 32x 32 GB)


4S → 8S 94 nodes


1.5 TB DDR4(96x 16 GB)


1.5T → 2TB 94 nodes




4S → 8S-2TB

94 nodes


1 TB DDR4(128x 8 GB or 64x 16 GB)


1 TB → 2 TB

94 nodes



Support for business continuityAll X6 scale-out solutions that are listed in Table 6-25 on page 145 and Table 6-26 on page 145 support the following features for business continuity:


HA can be implemented by using storage based replication. DR solutions can be implemented by using storage based replication or SAP HANA System Replication. You also can use the idling DR site nodes to run a second SAP HANA instance for non-productive purposes. For more information about these solutions, see 7.3, “HA and DR for scale-out SAP HANA” on page 194.

Single-node to scale-out solution upgrade requirementsWith X6 systems, the Lenovo scale-out solution for SAP HANA uses the same building blocks as a single-server installation. There are hardware and software components that are needed to complement the basic building blocks when a single-node solution is upgraded to a scale-out solution, as described in this section.


2 TB DDR4(128x 16 GB or 64x 32 GB)


1 TB → 3 TB

94 nodes


3 TB DDR4(192x 16 GB)


None 94 nodes


4 TB DDR4(128x 32 GB or 64x 64 GB)


None 94 nodes

a. Alternative CPU type E7-8890 v4 is supported for enhanced performance. Available upon request.b. Extra drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.



Scale-up options


Notes:

� The minimum number of nodes in any SAP HANA scale-out environment is three worker nodes. A cluster of two worker nodes is not supported. Adding one standby node for business continuity results in a four-node cluster being the minimum that is supported.

� The maximum number of nodes that is available for implementation is 94. However, Lenovo validated feasibility for up to 224 nodes in its labs to accommodate for growing customer demand.



Extra hardware must be bought only when scaling out using models AC3-4B-512 or AC4-8B-2048. For these models, more storage capacity is required for the second Spectrum Scale replica generated in scale-out environments. This second replica does not exist for single nodes.

Depending on the building blocks that are used, more Spectrum Scale licenses might be needed for the scale-out solution. The Spectrum Scale on x86 Single Server for Integrated Offerings V4.1 provides file system capabilities for single-node integrated offerings. This Spectrum Scale license does not cover the usage in multinode environments, such as the scale-out solution described here. To use building blocks that are included with the Spectrum Scale on x86 Single Server for Integrated Offerings licenses, Spectrum Scale on x86 Server licenses must be obtained for these building blocks for a scale-out solution. For more information about the type of license that is included with a specific model and the number of PVUs needed, see “Spectrum Scale license information” on page 220. Alternatively, Spectrum Scale File Placement Optimizer licenses can be used with Spectrum Scale on x86 Server licenses.

In a scale-out configuration, a minimum of three nodes must use Spectrum Scale on x86 Server licenses, and the remaining nodes can use Spectrum Scale File Placement Optimizer licenses. Other setups, such as the disaster recovery solution that is described in 7.3, “HA and DR for scale-out SAP HANA” on page 194, might require more nodes that uses Spectrum Scale on x86 Server licenses, depending on the role of the nodes in the setup. For more information about the Spectrum Scale license types, which type of license is included with a specific model, and the number of PVUs needed, see “Spectrum Scale license information” on page 220.

6.6 Upgrading X6 to Intel Xeon E7 v4 CPUs

Customers that invest in Lenovo X6 servers and want to upgrade to the latest Intel Xeon processor E7 v4 family have different options, depending on the existing environment.

Running Intel Xeon E7 v4 processors in one single cluster with Intel Xeon E7 of the first generation (installed in eX5 systems) is not supported. Technology has advanced too much to make a parallel operation meaningful.

Table 6-27 lists the different paths customers can follow to introduce the Intel Xeon processor E7 v4 family into their data center. Keep in mind that upgrading from Intel Xeon E7 v2 implies that all memory modules must be exchanged because only DDR4 modules are supported with the Intel Xeon E7 v4 family.

Table 6-27 Upgrade options from Intel Xeon E7 v2 or E7 v3 to Intel Xeon E7 v4

To →From

Single node E7 v4 Scale-out E7 v4 Scale-out with mixed E7 v2/v3 and E7 v4






6.6.1 Upgrading a single node

A single node can be upgraded by physically replacing the CPU Books only and carrying over all DIMMs (if coming from an Intel Xeon E7 v3 platform with DDR4 technology). No reinstallation of the operating system is required. The following process is used:

1. Confirm that your current uEFI and IMM code levels support E7 v4 processors.

2. Shut down the server and pull out the AC power connection.

3. Uninstall all DIMMs from the CPU Books and reinstall them in the correct order within the new CPU Books. If a change of memory technology is required (DDR3 to DDR4) then use new modules instead.

4. Restart the server.

6.6.2 Converting a single node into a scale-out environment

A single node with Intel Xeon E7 v2/v3 processors can be integrated into a scale-out environment by using one of the following methods:

� Not upgrading the Intel Xeon E7 v2/v3 node but adding nodes with Intel Xeon E7 v4 processors

� Upgrading the single node to Intel Xeon E7 v4 processors and then adding nodes that are of the same generation

Not upgrading the node to latest Intel Xeon E7 v4 processors leads to a mixed cluster with different processor generations. This configuration can be easily done by adding nodes only and following the instructions about how to turn a single node into a scale-out node (as described in the Lenovo Implementation Guide for the System x X6 solution).

Upgrading a single node before turning it into a scale-out environment is the other option. Follow the steps described in 6.6.1, “Upgrading a single node”.

6.6.3 Upgrading a scale-out environment

Upgrading a scale-out environment can be done without upgrading the nodes to the latest Intel Xeon processor generation.

Upgrading a scale-out environment into a mixed scale-outNot upgrading the nodes to latest Intel Xeon E7 v4 processors leads to a mixed cluster with different processor generations. This process can be accomplished by adding nodes to your scale-out environment only.

Note: When operating a mixed scale-out environment, most of the servers must have the newer CPU generation installed. A two-node environment with one server of each processor generation is not allowed. A three-node environment with two new generation servers and one current generation server is allowed.

Note: When operating a mixed scale-out environment, most of the servers must have the newer CPU generation installed. A two-node environment with one server of each processor generation is not allowed. A three-node environment with two new generation servers and one previous generation server is allowed.


Upgrading a scale-out environment into a Intel Xeon E7 v4-only environment

For all nodes, follow the upgrade instructions for a single node. Upgrade one node at a time. If you have HA capabilities in your cluster, you can complete the upgrade without shutting down your SAP HANA instance (rolling upgrade).

It is also possible to shut down the database instance and upgrade all nodes at the same time.

6.7 All-Flash X6 solutions for SAP HANA

The advantages of flash technology allow for higher-capacity flash devices in a standard 2.5-inch slot. This enables SAP HANA solutions that are purely based on flash storage to store operational SAP HANA data.

Lenovo supports all-flash configurations in parallel to the disk drive-based configurations. The all-flash portfolio, however, is only certified for standalone use cases. SAP Business Warehouse scale-out can not be implemented using all-flash models.

6.7.1 All-flash X6 solution for single-node BW

Table 6-28 lists the technical details of all x3850 X6 workload-optimized models for SAP HANA when flash drives are used instead of disk drives.

Table 6-28 x3850 X6 workload-optimized all-flash models for single-node BW

Note: The higher capacity flash drives used in the all-flash X6 solution allow sufficient storage capacity inside the server, negating the need for external storage expansion.

This makes the all-flash solution an attractive offering for customers with limited rack space, because all configurations occupy either just 4U (x3850 X6) or 8U (x3950 X6).



Upgrade options

AC3-2B-128F 2x Intel XeonE7-8880 v4

128 GB DDR4(16x 8 GB)

7.6 TB(3x 3.8 TB SAS SSD)

128 → 256


256 GB DDR4(32x 8 GB or16x 16 GB)


256 → 384256 → 5122S → 4S


384 GB DDR4(48x 8 GB)


2S → 4S-512


512 GB DDR4(32x 16 GB or 16x 32 GB)


512 → 768512 → 1 TB2S → 4S


768 GB DDR4(48x 16 GB)


2S → 4S- 1 TB


256 GB DDR4(32x 8 GB)


256 → 512


Table 6-29 lists the technical details of all x3950 X6 workload-optimized models for SAP HANA running BW workload when flash drives are used instead of disk drives.

Table 6-29 x3950 X6 workload-optimized all-flash models for single-node BW


512 GB DDR4(64x 8 GB or32x 16 GB)


512 → 768512 → 1 TB


768 GB DDR4(96x 8 GB)


None


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR4(96x 16 GB)


1 TB → 1.5 TB1 TB → 2 TB


2 TB DDR4(64x 32 GB or 32x 64 GB)


Scale-out or grow into 8-socket

a. Alternative CPU type E7-8890 v4 is available upon request for all models.



Upgrade options



Upgrade options


256 GB DDR4(32x 8 GB)


256 → 512


512 GB DDR4(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB4S → 8S


768 GB DDR4(96x 8 GB)


4S → 8S-1 TB


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 TB → 1.5 TB4S → 8S


1.5 TB DDR4(96x 16 GB)


4S → 8S- 2TB


2 TB DDR4(64x 32 GB or 32x 64 GB)


4S → 8S-4TBScale-out


512 GB DDR4(64x 8 GB)


512 → 1 TB


1 TB DDR4(128x 8 GB or 64x 16 GB)


1 TB → 1.5 TB1 TB → 2 TB


1.5 TB DDR4(192x 8 GB)


None


6.7.2 All-flash X6 solution for Business Suite on HANA

Business Suite is a set of applications that are bundled and designed to run an entire business. When Business Suite on SAP HANA is run, either as SAP Business Suite powered by SAP HANA or as SAP S/4HANA, the memory-per-core ratio is different from a BW scenario, because the type of workload on the database server is different. By using this setup, Lenovo can offer more building blocks with more main memory than for SAP BW.

Table 6-30 lists the technical details of all x3850 X6-based models with flash devices when running SAP S/4HANA or SAP Business Suite on HANA on top.

Table 6-30 x3850 X6 workload-optimized all-flash models for S/4HANA and Suite on HANA


2 TB DDR4(128x 16 GB or 64x 32 GB)


2 → 3 TB2 → 4 TB


3 TB DDR4(192x 16 GB)


Scale-out (hard drive based)


4 TB DDR4(128x 32 GB or 64x 64 GB)

19 TB(6x 3.8 TB SAS SSD)

Scale-out (hard drive based)




Upgrade options



Upgrade options


128 GB DDR4(16x 8 GB)


128 → 256


256 GB DDR4(32x 8 GB or16x 16 GB)


256 → 384256 → 5122S → 4S


384 GB DDR4(48x 8 GB)


256 → 5122S → 4S-512


512 GB DDR4(32x 16 GB or 16x 32 GB)


2S → 4S512 → 768


768 GB DDR4(48x 16 GB)


2S → 4S- 1 TB


1024 GB DDR4(32x 32 GB or 16x 64 GB)


1 → 1.5 TB2S → 4S


1536 GB DDR4(48x 32 GB)


2S → 4S-2TB


2048 GB DDR4(32x 64 GB)


2S → 4S2S → 4S- 4 TB


256 GB DDR4(32x 8 GB)


256 → 512


Table 6-31 shows the technical details of all x3950 X6-based models for SAP S/4HANA and SAP Business Suite on HANA when using only flash devices.

Table 6-31 x3950 X6 workload-optimized all-flash models for S/4HANA and Suite on HANA


512 GB DDR4(64x 8 GB or32x 16 GB)


512 → 768512 → 1 TB


768 GB DDR4(96x 8 GB)


Grow into x3950 X6


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 → 1.5 TB1 → 2 TB


1.5 TB DDR4(96x 16 GB)


Grow into x3950 X6


2 TB DDR4(64x 32 GB or32x 64 GB)


2 → 3 TB




Grow into x3950 X6




Grow into x3950 X6

a. Alternative CPU type E7-8890 v4 is available upon request.



Upgrade options



Upgrade options


256 GB DDR4(32x 8 GB)


256 → 512


512 GB DDR4(64x 8 GB or 32x 16 GB)


512 → 768512 → 1 TB4S → 8S


768 GB DDR4(96x 8 GB)


4S → 8S- 1 TB


1 TB DDR4(64x 16 GB or 32x 32 GB)


1 → 1.5 TB1 → 2 TB4S → 8S


1.5 TB DDR4(96x 16 GB)


4S → 8S


2 TB DDR4(64x 32 GB or 32x 64 GB)


2 → 3 TB4S → 8S




4S → 8S- 4 TB




4S → 8S-4S → 8S- 8 TB


6.8 Partitioning the x3950 X6

The Lenovo x3950 X6 platform can be electronically split into two independent servers. Details are explained in Chapter 5.1.8, “All-flash X6 solution” on page 82.

Partitioning a server is especially appealing for environments that might change in the near future, or growing environments, or even between interactive day shifts and batch night shifts. One can start with a split system and later on decide to create a single system image to support a growing environment.


512 GB DDR4(64x 8 GB)


512 → 1 TB


1 TB DDR4(128x 8 GB or 64x 16 GB)


1 → 1.5 TB1 → 2 TB


1.5 TB DDR4(192x 8 GB)


None


2 TB DDR4(128x 16 GB or 64x 32 GB)


2 → 3 TB2 → 4 TB


3 TB DDR4(192x 16 GB)


None


4 TB DDR4(128x 32 GB or 64x 64 GB)


4 → 6 TB


6 TB DDR4(192x 32 GB)


None


8 TB DDR4(128x 64 GB)


See footnoteb

a. Alternative CPU type E7-8890 v4 is supported for enhanced performance. Available upon request.b. If your SAP database exceeds 8 TB in main memory size, email [email protected] for more information.



Upgrade options

Note: Partitioning is supported on the x3950 X6 with all three CPU generations of the X6 systems (that is Intel Xeon E7 v2, v3, and v4 generation).

In the following tables of this paragraph every use of a ‘?’ is a placeholder for the CPU generation supported as written in the next column.


6.8.1 Single-node X6 solutions for SAP BW supporting Partitioning

Table 6-32 lists the configurations that are supported together with SAP HANA when running SAP Business Warehouse in a single-server configuration:

Table 6-32 x3950 X6 models supporting FlexNode Partitioning for SAP BW on HANA single-node

Model CPU families Mainmemory


FlexNode config

AC4-4?-256 4x Intel XeonE7 v2, v3, v4

256 GB (32x 8 GB)


Two 2S-128 GB or single 4S-256 GB


512 GB(64x 8 GB or 32x 16 GB)


Two 256 GB or single 4S-512 GB


768 GB (96x 8 GB)




1 TB(64x 16 GB or 32x 32 GB)


Two 2S-512 GB or single 4S-1 TB


1.5 TB (96x 16 GB)


Two 2S-768 GB or single 4S-1.5 TB

AC4-4?-2048 4x Intel XeonE7 v3, v4

2 TB (64x 32 GB or 32x 64 GB)


Two 2S-1 TBa orsingle 4S-2 TB

a. Only for SAP Suite on HANA, not for BW. When running as a single 4-socket system with 2 TB BW workload is supported.


512 GB (64x 8 GB)




1 TB (128x 8 GB or 64x 16 GB)




1.5 TB (192x 8 GB)




2 TB (128x 16 GB or 64x 32 GB)


Two 4S-1 TB orsingle 8S-2 TB


3 TB (192x 16 GB)



Two 4S-1.5 TB orsingle 8S-3 TB

AC4-8?-4096 8x Intel XeonE7 v3, v4

4 TB(128x 32 GB or 64x 64 GB)


Two 4S-2 TB or single 8S-4 TB


6.8.2 Single-node X6 solutions for Business Suite supporting X6 Partitioning

Business Suite is a set of applications that are bundled and designed to run an entire business. When Business Suite on SAP HANA is run, either as SAP Business Suite powered by SAP HANA or as SAP S/4HANA, the memory-per-core ratio is different from a BW scenario, because the type of workload on the database server is different. By using this setup, Lenovo can offer more building blocks with more main memory than for SAP BW.

The following configurations are supported together with SAP HANA when running SAP S/4HANA or SAP Business Suite on HANA in a single node configuration:

Table 6-33 x3950 X6 models supporting FlexNode Partitioning for SAP S/4HANA and SAP Business Suite on HANA



FlexNode config


256 GB (32x 8 GB)




512 GB(64x 8 GB or 32x 16 GB)


Two 256 GB or single 4S-512 GB


768 GB (96x 8 GB)




1 TB(64x 16 GB or 32x 32 GB)




1.5 TB (96x 16 GB)

26.4 TB (8x 400 GB SAS SSD,26x 1.2 TB 10K SAS HDD)a



2 TB (64x 32 GB or 32x 64 GB)




3 TB(96x 32 GB)



AC4-4?-4096 4x Intel XeonE7 v4

4 TB(64x 64 GB)




512 GB (64x 8 GB)




1 TB (128x 8 GB or 64x 16 GB)




1.5 TB (192x 8 GB)




2 TB (128x 16 GB or 64x 32 GB)




3 TB (192x 16 GB)




4 TB(128x 32 GB or 64x 64 GB)




6.8.3 Scale-out X6 solutions for SAP BW supporting Partitioning

The following configurations are supported together with SAP HANA when running SAP Business Warehouse in a scale-out configuration:

Table 6-34 x3950 X6 models supporting FlexNode Partitioning for SAP Business Warehouse scale-out

6.9 Migrating from eX5 to X6 servers

With the introduction of the latest Intel processor generation, new server models are introduced. All of these models feature processors with 15 cores each (in the older generation, each processor had only 10 cores). In addition, each current generation core is more powerful than a previous generation core. These reasons lead to a shift in the memory-per-core ratio that must be taken into account when eX5-based models are compared with X6-based models.

The following sections describe migration scenarios to X6 servers for customers that have eX5 based SAP HANA models in their data center.

6.9.1 Disruptive migration

SAP HANA environments with only one node server or cluster environments that can accommodate downtime of their SAP HANA database instance have two methods available to migrate from eX5 to X6 technology.


6 TB (192x 32 GB)



AC4-8?-8192 8x Intel XeonE7 v4

8 TB (128x 64 GB)

84 TB (12x 400 GB SAS SSD,80x 1.2 TB 10K SAS HDD)a


a. Extra drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.



FlexNode config



FlexNode config


AC4-8?-1024-C 8x Intel XeonE7 v2, v3, v4

1 TB (128x 8 GB or64x 16 GB)

26.4 TB(8x 400 GB SAS SSD,26x 1.2 TB SAS HDDa)

a. Extra drives are installed in the D1024 for SAP HANA or EXP2524 expansion unit.

Two 4S-512GB orsingle 8S-1 TB

94 nodes

AC4-8?-2048-C 8x Intel XeonE7 v2, v3, v4

2 TB(128x 16 GB or 64x 32 GB)



94 nodes

AC4-8?-3072-C 8x Intel Xeon E7 v2, v3, v4

3 TB (192x 16 GB)



94 nodes

AC4-8?-4096-C 8x Intel XeonE7 v3, v4

4 TB (128x 32 GB or 64x 64 GB)

84 TB(12x 400 GB SAS SSD,80x 1.2 TB SAS HDDa)


94 nodes


The first approach is to take a backup of the eX5 environment and restore it on an X6-based workload-optimized model. This approach works for single nodes and scale-out landscapes; however, for scale-out landscapes, the number of nodes cannot change from a backup to a restore. Therefore, if you plan to migrate from a cluster with, for example, five eX5 nodes to three X6 nodes, this approach does not work.

The second migration strategy involves reloading the database with data from the outside replication source. This approach does not require any preparations on the eX5 nodes before they are decommissioned. You replace the eX5 environment with X6 nodes and start from scratch. This approach supports a modified number of X6 nodes in a scale-out environment.

6.9.2 Hybrid SAP HANA cluster with eX5 and X6 nodes (using Intel Xeon E7 v2 processors)

Customers who have a scale-out SAP HANA environment that is running on eX5 based models can grow their database size with X6 nodes. Adding X6-based workload-optimized servers to a running eX5-based cluster is supported. For more information about the step-by-step instructions of the necessary commands, see the Operations Guide in 8.2, “Lenovo SAP HANA Operations Guide” on page 212.

A hybrid SAP HANA scale-out environment must have more than 50% of the nodes still being eX5 models. When most of the nodes are X6-based, all eX5 servers must be excluded from the cluster. A hybrid cluster with an equal number of eX5 and X6 nodes is not supported. Node designation (SAP HANA worker or standby) is not important for this configuration. For example, if your cluster has five worker nodes and one standby node that are running on eX5 technology, you can add up to five X6 building blocks to it.

This situation leads to different upgrade possibilities. Slow-growing environments can add X6 nodes to their cluster without hitting the 50% rule anytime soon. Faster growing environments must take a migration scenario into account.

Figure 6-17 shows an example of how to plan for a transition in which no migration-specific spare capacity is acquired and all server elements can be reused. The cluster in this figure consists of five eX5 nodes with eight sockets and 1 TB each (T-shirt size L), which gives a total memory size of 5 TB for the cluster, as shown in the upper part of Figure 6-17 on page 158.

Note: At the time of this writing, SAP required the same number of cores and amount of memory on every eX5 and X6 model. Following the new memory-per-core ratio on the X6 nodes is not supported in hybrid clusters.

Therefore, hybrid scale-out environments are supported by the following models only:

� Two sockets: 256 GB memory (T-shirt size S with AC3-2S-256-C)� Four sockets: 512 GB memory (M with AC3-4S-512-C or AC4-4S-512-C)� Eight sockets: 1 TB memory (L with AC4-8S-1024-C)


Figure 6-17 Example transition of an eX5 based cluster into an X6 based cluster

The database then grows and the customer decides to add X6 models to prepare for a migration at some point. Up to four X6 based workload-optimized solutions can be added to the 5-node cluster. The new X6 nodes must be eight sockets and 1 TB memory each (as shown in the middle section of Figure 6-17), which allows the cluster to grow up to 9 TB total.

Adding X6 models breaks the 50% rule. It is the correct point to exclude all eX5 models from the cluster, which leaves the database instance on X6 nodes only. To accommodate for the missing 5 TB of the eX5 nodes, you add memory to each of the X6 servers and use the new memory-per-core ratio on X6. You add 1 TB per X6 node. Depending on the fill level of the database before the split, you might need one more X6 node (as shown in the lower section of Figure 6-17). The total memory size of the cluster is then 8 TB (four X6 nodes only) or 10 TB (with one new X6 node).

The five decommissioned eX5 based workload-optimized servers can be coupled to an eX5- only cluster and used for another production SAP HANA instance or any other non-production instance, such as development or test.

Although this approach (including the example) allows for a migration from eX5 to X6 that does not involve taking down your SAP HANA database instance, it is preferable to reconfigure SAP HANA to use the extra features of the X6 models. This configuration can, for example, be combined with a regular SAP HANA database maintenance window.

6.10 Consolidation of SAP HANA instances

Deployment of SAP HANA databases on dedicated hardware can lead to many SAP HANA appliances in the data center, such as production, disaster recovery, quality assurance (QA), test, and sandbox systems, and possibly for multiple application scenarios, regions, or lines of business. Therefore, the consolidation of SAP HANA instances seems desirable. The following sections discuss different approaches for how to run more than one SAP HANA database instance per physical machine.

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

1TB X6node

1TB X6node

1TB X6node

1TB X6node

HANA database grows, add X6 nodes

5TB total memory

9TB total memory

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

1TB eX5node

2TB X6node

2TB X6node

2TB X6node

2TB X6node

2TB X6node

8 or 10 TB total memory

Split cluster into eX5 only and X6 only.Add additional memory to X6 nodesto make up for excluded eX5 nodes.


6.10.1 Multitenancy built into SAP HANA (MDC)

With the release of SAP HANA SPS09 in 2014 SAP introduced support for multiple tenant databases within one single SAP HANA system with a new feature called multitenant database containers (MDC).

SAP HANA is installed natively on the operating system of a server (or multiple servers in case of a scale-out installation) and runs without a hypervisor or other virtualization software. All system resources are assigned to this SAP HANA system. Inside SAP HANA then are multiple isolated containers each comprising a tenant database.

During installation time a decision has to be made whether the system should be installed in a single-container mode or in a multiple-container mode. Systems installed in single-container mode can still be converted later on into a multiple-container configuration. This is also how upgraded systems are configured for MDC. Once a system is running in multiple-container mode it cannot rolled back into a single-container mode. A reinstallation is required in this case.

MDC uses only one single system ID (SID) and each tenant database is identified with a dedicated TCP port to which applications can connect.

A single system database is always present in an MDC-enabled environment. This system DB is shared among all tenant databases and is used to manage tenants (like create, start, stop, or drop databases) and to perform database administration tasks like creating backups.

It is supported to run SAP Business Suite or SAP S/4HANA and SAP Business Warehouse in one SAP HANA system using MDC technology. Cross-database queries are supported, which enable cross-application reporting from within the same physical server.

If MDC is enabled on a scale-out SAP HANA installation then it is supported to create tenant databases spanning more than one physical node.

Backup and restore procedures can be implemented on a per-tenant level. System Replication, however, can only be defined on a per-system level (this may change in the future with newer SAP HANA revisions). Keep in mind that all tenants run on the SAP HANA revision as they use the same binaries. If different SAP HANA revisions are required, for example as a test environment for SAP HANA updates, then MDC is not the right feature to use.

Latest information about supported SAP HANA features with MDC enabled can be found in SAP note 2096000.

6.10.2 Consolidation options of SAP HANA (pre SPS09 release)

Another way of consolidating is to install more than one instance of SAP HANA database onto one SAP HANA appliance. However, there are major drawbacks when multiple SAP HANA instances are consolidated on one appliance; therefore, it is not supported for production systems.


For non-production systems, the support status depends on the following scenarios:

� Multiple Components on One System (MCOS)

Having multiple SAP HANA instances on one system, which is also referred to as Multiple Components on One System (MCOS), is not recommended because this poses conflicts between different SAP HANA databases on a single server; for example, common data and log volumes, possible performance degradations, and interference of the systems with each other. SAP and Lenovo support this scenario under certain conditions (see SAP Note 1681092). However, if issues arise, SAP or Lenovo might ask you to stop all but one of the instances as part of the troubleshooting process to see whether the issue persists.

Consider using SAP HANA multi-tenancy database containers (MDC) instead as explained in 6.10.1, “Multitenancy built into SAP HANA (MDC)” on page 159.

� Multiple Components on One Cluster (MCOC)

Running multiple SAP HANA instances on one scale-out cluster (which is also referred to as Multiple Components on One Cluster (MCOC), is supported if each node of the cluster runs only one SAP HANA instance. A development and a QA instance can run on one cluster, but with dedicated nodes for each of the two SAP HANA instances, for example, each of the nodes runs either the development instance, or the QA instance, but not both. Only the Spectrum Scale file system is shared across the cluster.

� Multiple Components in One Database (MCOD)

Having one SAP HANA instance containing multiple components, schemas, or application scenarios, which is also referred to as Multiple Components in One Database (MCOD), is supported. To have all data within a single database, which is also maintained as a single database, can lead to limitations in operations, database maintenance, backup and recovery, and so on. For example, bringing down the SAP HANA database affects all of the scenarios. It is impossible to bring it down for only one scenario. For more information about the implications, see SAP Note 1661202.

Consider the following factors when SAP HANA instances are consolidated on one SAP HANA appliance:

� An instance filling up the log volume causes all other instances on the system to stop working correctly. This situation can be addressed by monitoring the system closely.

� Installation of another instance might fail when there are other instances that are installed and active on the system. The installation procedures check the available space on the storage and refuse to install when there is less free space than expected. This situation also might occur when you are trying to reinstall an installed instance.

� Installing a new SAP HANA revision for one instance might affect other instances that are installed on the system. For example, new library versions with the new installation might break the installed instances.

� The performance of the SAP HANA system becomes unpredictable because the individual instances on the system are sharing resources, such as memory and CPU.

When asking for support for such a system, you might be asked to remove the extra instances and to recreate the issue on a single instance system.


6.10.3 SAP HANA on VMware vSphere

Another way of consolidating multiple SAP HANA instances on one system1 is virtualization.

In 2014, VMware and SAP announced2 controlled availability for deploying SAP HANA in virtual environments for production use by using VMware vSphere 5.5 and SAP HANA SPS07 (or later).

SAP and VMware have gathered more experience since then and as of May 2016 SAP HANA on VMware is supported under the following conditions:

� Multiple SAP HANA virtual machines (VM) per physical server are allowed, as long as all prerequisites of SAP note 2024433 are fulfilled.

� Scale-up single-servers and scale-out configurations are supported, as long as all requirements of SAP note 2157587 are fulfilled.

� The vCPUs of a single SAP HANA VM must be pinned to physical cores in a way that the CPU cores of any given socket are exclusively assigned to only one single virtual machine (for example assigning two SAP HANA VMs five cores each of a 10 core CPU is not supported).

� SAP HANA is virtualized with VMware vSphere on single-node hardware configurations that are validated for SAP HANA or on SAP HANA-tailored data center integration application-verified hardware.

� The installation of the SAP HANA appliance software into the VMware guest must be done by certified SAP HANA appliance vendors or their partners. Cloning of such VMs can then be done by the clients as needed.

� Up to 4 TB and 128 vCPUs per virtual SAP HANA instance are supported with VMware vSphere 6.0. This is supported running SAP HANA SPS11 or later, and for single-node VMs only−−no scale-out support. See SAP note 2315348.

� No overprovisioning of CPU or memory is allowed, regardless of the vSphere version.

Additionally, SAP supports VMware vSphere 5.1 since the release of SAP HANA 1.0 SPS 05 (that is, revision 45 and higher) for non-production use only. Relaxed configuration rules apply to non-production environments. Consolidation of multiple SAP HANA VMs onto one physical server is allowed for non-production environments and 8-socket servers are supported as host systems.

VMware Tools is approved by SAP for installation inside the VM operating system.

Table 6-35 lists the VMware scenarios that are available with SAP HANA.

Table 6-35 VMware scenarios with SAP HANA

1 One SAP HANA system, as referred to in this section, can consist of one single server or multiple servers in a clustered configuration.

2 See http://global12.sap.com/news-reader/index.epx?articleID=22775 and http://www.saphana.com/community/blogs/blog/2014/05/06/at-the-seasside-with-sap-hana-on-vmware

Feature Non-productionSAP HANA instance

ProductionSAP HANA instance

Required SAP HANA SPS SPS05 or later SPS07 or later

Required VMware version vSphere 5.1, 5.5, or 6.0 vSphere 5.5 or 6.0a

Single node deployment Yes Yes

Scale-out deployment Yes Yes (only with vSphere 5.5)


http://global12.sap.com/news-reader/index.epx?articleID=22775

http://www.saphana.com/community/blogs/blog/2014/05/06/at-the-seasside-with-sap-hana-on-vmware

For more information about SAP HANA virtualization with VMware vSphere, see SAP Note 1788665, which includes an attached FAQ document. This SAP Note is available at this website:

http://service.sap.com/sap/support/notes/1788665

For more information, see this website:


VMware released a preferred practices guide for running SAP HANA on vSphere 5.5. It is available at this website:

http://www.vmware.com/files/pdf/SAP_HANA_on_vmware_vSphere_best_practices_guide.pdf

vSphere on eX5 workload-optimized models for SAP HANAThe System x solution for SAP HANA supports virtualization with VMware vSphere on eX5 systems by using an embedded hypervisor. The SAP HANA Virtualization FAQ states that SAP allows for non-production environments with multiple VMs to be installed by using a concept of slots.

Each slot is a VM that is created with 10 virtual CPUs (vCPUs) and 64 GB memory. The standard rules for the operating system and sizes for SAP HANA Data and Log file systems must be followed. Because of the resources that are taken by the VMware ESXi server, one slot is reserved by SAP definition because no CPU or memory overcommitment is allowed in an SAP HANA VM.

Therefore, a maximum of 15 slots are available on the x3950 X5 workload-optimized system for SAP HANA appliance. A maximum of three slots is available on the x3690 X5 workload-optimized system for SAP HANA appliance. Figure 6-18 on page 163 shows the maximum number of slots that are available per eX5 system.

Maximum number of VMs per physical server

Multiple Multiple (only with vSphere 5.5)

CPU overprovisioning No No

Memory overprovisioning No No

a. SAP HANA SPS11 or later required for vSphere 6.0

Feature Non-productionSAP HANA instance

ProductionSAP HANA instance



http://service.sap.com/sap/support/notes/1788665



Figure 6-18 SAP HANA possible slots per T-shirt size (x = reserved) on eX5 servers

Table 6-36 lists the configuration parameters for VMs from slot sizes 1 - 8. Sizes 7 and 8 are not supported by VMware because of the restriction of a maximum number of vCPUs per guest of 64 in their latest VMware ESX server software. You can install one or more non-production VMs of slot sizes 1 - 6 on any x3950 X5 workload-optimized solution for SAP HANA appliance.

Table 6-36 SAP HANA VM sizes when virtualizing eX5 models

For production environments on vSphere on eX5, only one VM is allowed per physical server.

vSphere on rack-mount X6 workload-optimized models for SAP HANAWith the release of the Intel Xeon E7 v2 processor family, SAP revised their configuration method for virtualizing SAP HANA on VMware vSphere. Sizing a virtual environment now is identical to bare-metal sizing. This applies to Intel Xeon E7 v2, E7 v3, and E7 v4 processor- based systems. To achieve the preferred performance, the main memory and number of cores must follow a certain ratio, which is the same ratio that bare-metal environments follow. Disk storage requirements also were relaxed.

SAP Slots T-Shirt Size (hosting system)

X 1 XSS / S+

M

L

2 3

4 5

6 7

8 9

10 11

12 13

14 15

SAP T-shirt

SAP HANA support pack

Lenovo name

vCPUs(HT on)

Virtual memory

Required no. of slots

Total HDD Total SSD

XXS SPS 05 VM1 10 64 GB 1 352 GB 64 GB

XS SPS 05 VM2 20 128 GB 2 608 GB 128 GB

None Manually VM3 30 192 GB 3 864 GB 192 GB

S SPS 05 VM4 40 256 GB 4 1120 GB 256 GB



None N/A VM7a

a. This slot size is not possible because of the limitations of the VMware ESXi 5 hypervisor.

70 448 GB 7 1888 GB 448 GB

M N/A VM8a 80 512 GB 8 2144 GB 512 GB


For more information, about preferred practices, see the white paper from VMware, which is available at this website:


Running VMware on X6 workload-optimized models is supported on 2-, 4-, and 8-socket machines.

VMware vSphere licensingVMware vSphere 5 is available in the following editions, which differ in their set of features.3 For the deployment of SAP HANA on VMware vSphere, the number of supported vCPUs per VM is the most important difference within the set of supported features:

� The Standard edition supports up to eight vCPUs per VM, which is less than the number of vCPUs of the smallest SAP HANA VM size. Therefore, this edition cannot be used.

� The Enterprise edition supports up to 32 vCPUs per VM, and can be used for SAP HANA VMs with up to 30 vCPUs and 192 GB of virtual memory. This edition might be a cost-saving choice when only smaller SAP HANA VMs are deployed; for example, for training or test purposes.

� The Enterprise Plus edition supports up to 64 vCPUs per VM, and thus can be used for SAP HANA VMs with up to 60 vCPUs and 384 GB of virtual memory, which is the maximum supported VM size.

VMware vSphere is licensed on a per-processor basis. Each processor on a server must have a valid license that is installed to run vSphere. For more information about the number of processors for each of the building blocks of the Lenovo Systems Solution for SAP HANA, see the section about the respective Intel processor generation in Chapter 5.1, “Lenovo X6 systems” on page 58.

With the release of VMware vSphere 5.5, the licensing model was revised and there are no longer any restrictions on the number of vCPUs per VM. As an example, if you want to deploy SAP HANA on an eX5 M building block (for example, 7143-HBx), you need four licenses of the vSphere edition of your choice. An X6 model AC48S1024 needs eight licenses.

VMware vSphere 6.0 follows the vSphere 5.5 licensing model.

SizingThe sizing is the same for virtualized and nonvirtualized SAP HANA deployments. Although there is a small performance effect because of the virtualization, the database size and the required memory size are not affected.

SupportAs with any other deployment type of SAP HANA, clients are asked to open an SAP support ticket by using the integrated support model that is described in 8.7.1, “Lenovo and SAP integrated support” on page 216. Any non-SAP related issue is routed to VMware first, and it eventually is forwarded to the hardware partner. In certain but rare situations, SAP or its partners might need to reproduce the workload on bare metal.

3 For a complete overview of the available editions and their feature sets, see this website: http://www.vmware.com/files/pdf/vsphere_pricing.pdf




http://www.vmware.com/files/pdf/vsphere_pricing.pdf

http://www.vmware.com/files/pdf/vsphere_pricing.pdf

6.11 Security and encryption of an SAP HANA system

SAP environments contain a company’s most critical data and must be protected from unauthorized access physically and virtually. Although restricting physical access is obvious and relatively easy to implement in today’s data centers, virtual access is more difficult to control.

Some industries must also obey to certain standards that define rules and processes for handling certain data sets. Three of the more well-known standards are the Payment Card Industry Data Security Standard (PCI DSS), the Sarbanes-Oxley Act (SOX), and the Health Insurance Portability and Accountability Act (HIPAA). These standards regulate banks, credit card companies, insurance companies, pharmaceutical companies, and many more.

They describe in different levels of detail how IT processing must be implemented and what safety measures must be in place to cater for maximum data privacy and to prevent data leaks. Some standards also enforce data encryption to ensure safety and to prevent unauthorized data access. The portfolio of Lenovo System x Solutions for SAP HANA provides several levels of security, including data encryption, communication channels encryption, and data access security.

6.11.1 Encrypting SAP HANA data

SAP HANA systems work with data on the following technologies:

� Volatile storage (main memory that uses DRAM technology)� Non-volatile storage (disks or flash storage)

These technologies feature different characteristics and require different levels of precaution to ensure the safety of the data that is on them.

Volatile storageVolatile storage requires a constant supply of power to continuously refresh the content of the DRAM cells. When power is lost, the data in main memory also is lost. If you turn off your System x server, all data in main memory is gone. This fact is a good enough safety measure for situations in which data must be prevented from being physically stolen. No encryption must be applied in this case.

Preventing main memory data from being virtually stolen (that is, someone can login to the SAP HANA system and work on it) is almost impossible with today’s technology. Data is encrypted in main memory and is available as encrypted data without ever decrypting it.

Although it is technically feasible to encrypt data in main memory (however, this encryption imposes a large performance penalty), the data must be decrypted at some point during processing. Every security measure is only as good as its weakest link. Therefore, encrypting main memory is not sufficient if data still must be decrypted before being processed by the CPU. The performance penalty of encrypting main memory defeats the purpose of main-memory computing.

Non-volatile storageNon-volatile storage can and should be protected whenever possible because threats are real and safety breaches can easily lead to data leaks. SAP HANA regularly writes out data from main memory to non-volatile storage, such as disk drives or flash drives. This write is required to reload data whenever the system is restarted. This restart can be the result of any event, from a scheduled maintenance window to unplanned outages or failures.


The most obvious threat are stolen disks or improperly decommissioned disks. When a disk is replaced, you must ensure that data that is on the disk is properly destroyed. This requirement is difficult to fulfill for failed drives that cannot be accessed.

By using the Lenovo System x Solution for SAP HANA, such data can be encrypted on a file system level. It uses the encryption capabilities that are built into Spectrum Scale 4.1 code. Any data that is written out by an SAP HANA database to its persistency layer (that is, to flash or disk storage) is encrypted as it goes through the file system. No unencrypted data is on physical media.

Spectrum Scale encrypts data blocks when it receives them from the application. As a result, Spectrum Scale data that travels over the local network also is encrypted before it is sent out by a node. All Lenovo System x Solutions for SAP HANA that are running with the Spectrum Scale data replication feature for high-availability or disaster recover (Spectrum Scale parameter r=2/3, m=2/3) has their data traffic encrypted in addition to the data on their local storage devices. Spectrum Scale metadata is not encrypted; it comprises the names and permissions of the files, directories, and other elements, such as Spectrum Scale Extended Attributes (EAs) that are not used in a System x solution for SAP HANA.

Figure 6-19 shows what parts of a scale-out System x solution are secured. All the orange elements in the Figure 6-19 on page 166 are encrypted. These elements include data that is written to local disks and data that is sent out over the network to other nodes to be stores on their disks (single-node installations do not send out Spectrum Scale traffic to the network).

Figure 6-19 Encrypting SAP HANA data by using Spectrum Scale

Spectrum Scale Encryption includes the following features:

� Encryption is built in Spectrum Scale without dependency on third-party products.

� Spectrum Scale encryption feature is compliant with NIST SP800-131A and FIPS 140-2.

� Encryption uses AES algorithm and key lengths of 128, 192, or 256 bits.

� Support for multiple encryption keys in a single file system (perfectly suited for multitenancy or multipurpose SAP HANA environments).

� Fine-grained Spectrum Scale policies enable encrypting only subsets of your file system.

NIST SP800-131A compliantFIPS 140-2 compliant

Scale-out SAP HANA Database Instance

Lenovo System xw/ Spectrum Scale

diskdrive(s)

block device/NSD


diskdrive(s)

block device/NSD


diskdrive(s)

block device/NSD


diskdrive(s)

block device/NSD

Ethernet

EncryptedSpectrum Scale

data traffic

Key serverKey server (1..n)

Ethernet


data traffic

Ethernet


data traffic

All elements in orangeare encrypted.


Running an encrypted SAP HANA environment safeguards you from the following threats:

� Prevention of data leaks from stolen or lost disks.

� Prevention of data leaks from improperly discarded disks.

� Prevention of eavesdropping on the network, which is especially important in HA and DR scenarios when data flows over a network that is owned by a third-party ISP.

� Secure delete is a cryptographic operation (as opposed to digital shredding by overwriting), which gives the ability to destroy arbitrarily large subsets of a file system in a predetermined amount of time.

The key server that is holding the encryption keys is external to the environment. Spectrum Scale uses the Key Management Interoperability Protocol (KMIP), which is an open standard by OASIS. At the time of this writing, the only verified and supported key server by Spectrum Scale is the IBM Security Key Lifecycle Management (ISKLM) server (formerly Tivoli Key Lifecycle Manager), which supports KMIP.

ISKLM is available on the following hardware platforms and operating systems:

� x86-64 bit; Windows and Linux (RHEL and SLES)� Power; AIX� System z; z/OS and Linux (RHEL and SLES)� SPARC; Solaris 10

ISKLM is validated to Federal Information Processing Standard (FIPS) 140-2 Level 1 and supports FIPS 140-2 Level 3-validated hardware for enhanced security requirements. To use the Spectrum Scale encryption feature, clients must also purchase the Advanced Edition license of Spectrum Scale (by default, the Lenovo SAP HANA installation includes the Spectrum Scale Standard Edition license).

Customers who have encryption in place for the storage or tape environment often have an ISKLM or Tivoli Key Lifecycle Manager installation that can be used to store the Spectrum Scale encryption keys.

Implementing encryption on the active production system is one step in the overall security policy. Do not forget to also include your backup and archival systems in the overall data security strategy. You can operate an end-to-end encrypted environment with ISKLM and its capability to also manage encrypted backup and tape archives. By using ISKLM, you can transfer your encrypted SAP HANA data between the different systems and storage devices without the need to decrypt it along the way.

For more information about Spectrum Scale Encryption, see this website:

http://www-01.ibm.com/support/knowledgecenter/SSFKCN_4.1.0/com.ibm.cluster.gpfs.v4r1.gpfs200.doc/bl1adv_encryption.htm

For more information about how to use and implement Spectrum Scale Encryption for System x SAP HANA solutions, contact your account representative, business partner, or the Lenovo SAP Center of Competency at: [email protected].

6.11.2 Securing SAP HANA communication channels

Encrypting SAP HANA data is only one piece to the overall security puzzle. You also must ensure that data that is in transit is secure and cannot be eavesdropped. This requirement applies to the following communication channels:

� SAP HANA external communication� SAP HANA internal communication


http://www-01.ibm.com/support/knowledgecenter/SSFKCN_4.1.0/com.ibm.cluster.gpfs.v4r1.gpfs200.doc/bl1adv_encryption.htm

For more information about securing external SAP HANA communication channels, see 6.11.3, “Securing SAP HANA access” on page 168.

Securing internal communication channels is applicable to all SAP HANA deployments, except for single-node installations without HA or DR. All installations with more than one node contributing to an SAP HANA database instance must think about securing the internal 10 Gbit Ethernet communication network.

SAP supports the Secure Sockets Layer (SSL) protocol for communication between multiple hosts in a distributed SAP HANA environment. SSL protects the following communication channels:

� Cluster-internal communication in a distributed, scale-out SAP HANA deployment� System Replication traffic between two SAP HANA systems

To implement SSL, you must deploy certificates on every participating host. All certificates must be signed by a central certificate authority (CA). The CA can be an external, trusted company, a CA internal to the customer, or a self-signed environment that is for SAP HANA only.

SSL provides authentication of the communication peer and encrypts data that is going across the network.

6.11.3 Securing SAP HANA access

SAP groups accounts that access SAP HANA data into the following logical categories:

� Database user� Technical database user

This categorization is purely logical and does not manifest in different parameters or rights. The purpose is to determine if SAP HANA accounts correspond to a real person or an account that is used for communication between different SAP systems.

The following methods of authentication are supported by SAP HANA:

� User name with password� Kerberos� Security Assertion Markup Language (SAML)� Logon and Assertion tickets� X.509 client certificates

User authentication can be audited.

For more information about security in an SAP HANA system, see the SAP HANA Security Guide, which is available at this website:

http://help.sap.com/hana/SAP_HANA_Security_Guide_en.pdf




Chapter 7. Business continuity and resiliency for SAP HANA

This chapter describes individual SAP HANA high availability (HA) and disaster recovery (DR) deployment options. It explains basic terminology and deployment options for single-node systems and scale-out systems. It also describes the backup options of SAP HANA.


� 7.1, “Overview of business continuity options” on page 170� 7.2, “HA and DR for single-node SAP HANA” on page 177� 7.3, “HA and DR for scale-out SAP HANA” on page 194� 7.4, “Backup and restore” on page 205

7


7.1 Overview of business continuity options

Because of the relevance of SAP software environments in today’s world, it is critical that these systems do not experience unexpected downtime. Hardware manufacturers, such as Intel or Lenovo, invest much into the reliability and availability features of their products; however, IT systems are still exposed to many different sources of errors.

For that reason, it is crucial to consider business continuity and reliability aspects of IT environments when SAP HANA systems are planned. Business continuity in this context refers to designing IT landscapes with failure in mind. Failure spans from single component errors, such as a hard disk drive (HDD) or a network cable, up to the outage of the whole data center because of an earthquake or a fire. Different levels of contingency planning must be done to cope with these sources of error.

Developing a business continuity plan highly depends on the type of business a company is doing, and it differs (among other factors) by country, regulatory requirements, and employee size.

This section introduces the three main elements of business continuity planning:

� Implementing HA� Planning for DR� Taking backups regularly

These elements have different objectives for how long it takes to get a system online again, for the state in which the system is after it is online, and for the end-to-end consistency level of business data when an IT environment comes online again. These values feature the following definitions:

� Recovery Time Objective (RTO) defines the maximum tolerated time to get a system online again.

� Recovery Point Objective (RPO) defines the maximum tolerated time span to which data must be restored. It also defines the amount of time for which data is tolerated to be lost. An RPO of zero means that the system must be designed to not lose data in any of the considered events.

The most common approach to achieve an RPO of zero is to implement HA within the primary data center plus an optional synchronous data replication to an offsite location (usually a second data center).

� Recovery Consistency Objective (RCO) defines the level of consistency of business processes and data that is spread out over multitier environments.1

It is important to understand the difference between HA and DR. HA covers a hardware failure (for example, one node becomes unavailable because of a faulty processor, memory DIMM, storage, or network failure). DR covers the event when multiple nodes in a scale-out configuration fail, or a whole data center goes down because of a fire, flood, or other disaster, and a secondary site must take over the SAP HANA system.

The ability to recover from a disaster, or to “tolerate” a disaster without major effect, is sometimes also referred to as disaster tolerance (DT).

1 This publication focuses on SAP HANA; that is, the database layer only. For that reason, this chapter does not describe RCO any further.


All of the single-node and scale-out solutions described in Chapter 6, “SAP HANA IT landscapes with X6 solutions” on page 103 can be enhanced with HA and DR capabilities. All scenarios require at least one other copy of the data to be available in the system so that the SAP HANA application can survive the outage of a server, including the data that is on it.

HA is implemented by introducing standby nodes. During normal operation, these nodes do not actively participate in processing data, but they do receive data that is replicated from the worker nodes. If a worker node fails, the standby node takes over and continues data processing. For more information about node takeover, see “Example of a node takeover” on page 93.

When an SAP HANA side-car scenario is run (for example, SAP CO-PA Accelerator, sales planning, or smart metering), the data still is available in the source SAP Business Suite system. Planning or analytical tasks run slower without the SAP HANA system being available, but no data is lost. More important is the situation in which SAP HANA is the primary database, such as when SAP Business Suite with SAP HANA or Business Warehouse is used with SAP HANA as the database. In those cases, the production data is available solely within the SAP HANA database, and according to the business service level agreements, prevention of a failure is necessary.

HA and DR solutions for SAP HANA can be at the following levels:

� On the infrastructure level

By replicating data that is written to disk by the SAP HANA persistency layer synchronously or asynchronously, standby nodes can recover lost data from failed nodes. Data replication can happen within a data center (for HA), across data centers (for DR), or both (for any combination of HA and DR). This feature is known as Spectrum Scale-based storage replication.

At this level backups that are replicated or otherwise shipped from the primary site to the secondary site, are used.

� On the application level

All actions that are performed on an active SAP HANA instance to a passive instance that is running in receive-only mode are replicated. Essentially, the passive instance runs the same instructions as the active instance, except for accepting user requests or queries. This feature is known as SAP HANA System Replication (SSR).

Note: When referring to DR, the words primary site, primary data center, active site, and production site mean the same thing. Similarly, secondary site, back up site, and DR site are also used interchangeably.

The primary site hosts your production SAP HANA instance during normal operation.

Chapter 7. Business continuity and resiliency for SAP HANA 171

The available features to implement business continuity are listed in Table 7-1. Also listed are the scenarios to which the features are applicable.

Table 7-1 Business continuity options for SAP HANA

The Spectrum Scale based storage replication and SSR are described next.

The rest of this chapter then describes all supported scenarios for HA and DR of single-node and scale-out SAP HANA environments that are based on Lenovo System x solutions. For more information about conventional backup methods, see 7.4, “Backup and restore” on page 205.

7.1.1 Spectrum Scale based storage replication

Lenovo uses Spectrum Scale in all of its solutions for SAP HANA. The file system includes built-in replication features with which you can have multiple copies of one file that is stored on disk that is spread out on different servers in a multi-server environment. Those copies are referred to as replicas and are all identical to each other. For more information about replicas, see , “Spectrum Scale extensions for shared-nothing architectures” on page 86.

Having multiple replicas available enables HA and DR solutions to be built by using these replicas. Environments can also grow into more reliable environments, if required. For example, clients can start with a single data center solution with HA and add DR capabilities later on when needed.

Lenovo uses Spectrum Scale in its SAP HANA solutions in a way that allows any node in a cluster to fail without losing data. For more information about what happens when a node fails and how a standby node takes over the workload without taking down the service, see 5.2.5, “Scaling-out SAP HANA using Spectrum Scale” on page 90.

Level Technology RTO RPO Suitable for HA

Suitable for DR

Infrastructure level (storage)

Spectrum Scale based synchronous replication

HA: ZeroDR: Minutes

Zero Yes Yes

Spectrum Scale based asynchronous replication

Minutes to hours Above zero (Seconds to hours)

No Yes

Backup - Restore Usually hours Hours to days No Yes

Application level (system)

SSR (synchronous) Minutes Zero Yesa

a. With SUSE HA Extension only. For more information, see the following Note.

Yes

SSR (asynchronous) Minutes Seconds No Yes

Note: SSR does not support automatic failover to a standby system. As the name implies, it replicates data to a standby system only. Manual intervention is required for the failover to occur. As an alternative, the tasks can be automated using the cluster manager that is part of SUSE HA Extension.

SAP uses the term Host Auto-Failover to describe the capability of an automatic failover to a standby system. At the time of this writing, Host Auto-Failover is not available with the SAP HANA product.


Expanding into a DR setup involves another replica of the data to be stored at a remote location.

The major difference between a single site HA (as described in 5.2.5, “Scaling-out SAP HANA using Spectrum Scale” on page 90) and a multi-site DR solution is the placement of the replicas within Spectrum Scale. In a single-site HA configuration, there are two replicas2 of each data block in one cluster. In contrast, a multi-site DR solution holds a third replica in the remote or secondary site. This configuration ensures that when the primary site fails, a complete copy of the data is available at the second site and operations can be resumed at this site.

A two-site solution implements the concept of a synchronous or asynchronous data replication on a file system level between both sites by using the replication capabilities that are built into Spectrum Scale.

For more information about the wording and concepts that are used by Spectrum Scale to implement HA, see the paper that is available at this website:

http://www-03.ibm.com/systems/resources/configure-gpfs-for-reliability.pdf

7.1.2 SAP HANA System Replication

SSR is a feature that was introduced with SAP HANA SPS05 and was improved in subsequent revisions. In an environment that uses SSR, the primary and the secondary system must be configured identically in terms of SAP HANA worker nodes. The number of standby nodes can differ (since SAP HANA SPS06). Every SAP HANA process that is running on the primary system’s worker nodes must have a corresponding process on a secondary worker node to which it replicates its activity.

The only difference between the primary and secondary system is the fact that one cannot connect to the secondary HANA installation and run queries on that database. They can also be called active and passive systems.

Upon start of the secondary HANA system, each process establishes a connection to its primary counterpart and requests the data that is in main memory, which is called a snapshot. After the snapshot is transferred, the primary system continuously sends the log information to the secondary system that is running in recovery mode. At the time of this writing, SSR does not support replaying the logs immediately as they are received; therefore, the secondary site system acknowledges and persists the logs only. To avoid having to replay hours or days of transaction logs upon a failure, SSR asynchronously transmits a new incremental data snapshot periodically.

Among other criteria, the following criteria must be met to enable the SSR feature:

� The SAP HANA software revision of the target environment must be the same or higher than the source environment.

� Both systems must use the same SID and instance numbers.

� On both systems, “instance number + 1” must be free because it is used for replication purposes.

2 In Spectrum Scale terminology, each data copy is referred to as a replica. This term also applies to the primary data, which is called the first replica. This term indicates that all data copies are equal.




Replication modesSSR can be set to one of the following modes:

� Synchronous mode

This mode makes the primary site system wait until the change is committed and persisted on the secondary site system. If the link between the sites goes down, only the primary site writes the log to the persistency layer. The system stays operational during link failures. Changes must be replayed after the link is restored. Data loss can occur when a failover is run after a link failure.

� Synchronous with Full Sync mode (available since SAP HANA SPS08)

This mode makes the primary site system wait until the change is committed and persisted on the secondary site system. Log write returns success if the log is written on the secondary site as well. If the link between the sites goes down, database transaction suspends until the link is restored. Although no data is lost, the system is not usable until the link is restored.

� Synchronous in-memory mode

This mode makes the primary site system acknowledge the change after it is committed in main memory on the secondary site system, but not yet persisted on disk.

� Asynchronous mode (available since SAP HANA SPS06)

This mode makes the primary site system commit transaction when the replicated log is sent to the DR site. The system does not wait for an acknowledgment from the remote site.

Asynchronous replication allows for greater distances because a high latency between the primary and secondary system does not prevent a production workload from running at maximum performance as with synchronous replication.

In all synchronous modes, the effect is defined by the transmission time from the primary to its corresponding secondary system process. When SSR is run in synchronous mode, you must add the time that it takes to persist the change on a disk in addition to the transmission delay.

Live-replication stops if the connection between the two data centers is lost. Then, after a (configurable) timer expires on the primary site system, it resumes work without replication.

When the connection is restored, the secondary site system requests a delta snapshot of what changes were done since the connection was lost. Live replication can then continue after this delta is received on the secondary site system.

Operating ModesThere are two operating modes that SSR can be set to:

� Delta data shipping

This mode instructs the secondary system to receive the data from the primary system only but not replay it into the standby system.

� Log replay (available since SAP HANA SPS11)

This mode instructs the secondary system to immediately replay all received transactions from the primary system. This reduces the takeover time.

Log based operation is the foundation for active-active setups that is a feature on the roadmap for a future SAP HANA release.


System failoverIf there is a failover to the secondary system (system takeover), manual intervention is required to change the secondary site system from recovery mode to active mode. SAP HANA automatically loads all row-based tables into memory and rebuilds the row store indexes. In the next step, all logs since the last received snapshot are replayed. After this step finishes, the system can accept incoming database connections. A restart of the SAP HANA database instance is not required.

An optional feature (called table preload) enables the primary system to share information about which columns are loaded into main memory. The secondary system can use this information to preload those tables in main memory. Preloading reduces the duration of a system takeover operation.

Hosting a non-productive instance at the secondary systemIf the secondary system is intended to host a non-productive instance, preloading must be disabled. In such a scenario, SAP HANA operates with a minimal main memory footprint on the secondary system so that the remaining memory can be used for a non-productive SAP HANA installation.

If a system takeover is triggered, both instances (the one receiving the replication data and the non-productive instance) must be stopped. The secondary system must be reconfigured to use all available main memory, and then a takeover operation is run. Because you must restart the SAP HANA processes, the time for a system takeover and a subsequent system performance ramp-up is longer when compared to when no non-productive instance is hosted and tables preload is enabled.

A non-productive instance cannot share storage with production data. For this reason, Lenovo uses the Lenovo Storage D1024 for SAP HANA or System x Storage EXP2524 to extend the locally available storage capacity to hold the data and log files of the non-productive system.

Multitier System ReplicationMultitier System Replication was introduced with SAP HANA SPS07, with which you can cascade replication over several databases.

At the time of this writing, only one scenario is supported. The primary system synchronously replicates (by using synchronous or synchronous in-memory mode) to the secondary system that asynchronously replicates the information to a tertiary system, which can be physically far away from the primary and secondary system, as shown in Figure 7-1.

Figure 7-1 SAP HANA Multitier System Replication

��N�2��N�2! ��N�2�

�*)��N�� /��O��) �-�� )�*

N)�)2! �P2!

� �O)��!

/ ��N�(��Q��/ ��)��Q��/ �'��-'.' ��N��

/ �'��-'.' � +��N�

-�� )�*

� �O)��

/ ��N�(��Q��/ ��)��Q��

/ �'��-'.' � +��N�

N)�)2� �P2�

-�� )�*

/ ��N�(��Q��/ ��)��Q��

/ �'��-'.' � ��)�N��N�

�'��-'.'R �

��O �)

��)��N)�)

SAP HANA��N�2��N�2! ��N�2�

�*)��N�� /�O��) �-�� )�*

N)�)2! �P2!

� �O)��!

/ ��N�(��Q��/ ��)��Q��/ �'��-'.' ��N��

/ �'��-'.' � +��N�

-�� )�*

� �O)��

/ ��N�(��Q��/ ��)��Q��

/ �'��-'.' � +��N�

N)�)2� �P2�

-�� )�*

/ ��N�(��Q��/ ��)��Q��

/ �'��-'.' � ��)�N��N�

�'��-'.'R �

��O �)

��)��N)�)

SAP HANA��N�2��N�2! ��N�2�

�*)��N�� /�O��) �-�� )�*

N)�)2! �P2!

� �O)��!

/ ��N�(��Q��/ ��)��Q��/ �'��-'.' ��N��

/ �'��-'.' � +��N�

-�� )�*

� �O)��

/ ��N�(��Q��/ ��)��Q��

/ �'��-'.' � +��N�

N)�)2� �P2�

-�� )�*

/ ��N�(��Q��/ ��)��Q��

/ �'��-'.' � ��)�N��N�

�'��-'.'R �

��O �)

��)��N)�)

SAP HANA

��*��O �)��

)��*��O �)��


Multitenant Database ContainersWhen multiple databases are consolidated into one single SAP HANA system using the MDC feature, then System Replication has to be implemented at the system level. That means no replication on tenant level is supported.

The primary and secondary systems must be identical at the time System Replication is initially set up. This requirement includes the tenant databases.

For more information about SSR, see the following publications:

� Introduction to High Availability for SAP HANA:


� How to Perform System Replication for SAP HANA:

https://scn.sap.com/docs/DOC-47702

� SAP HANA Administration Guide, SAP HANA Security Guide:


7.1.3 Special considerations for DR and long-distance HA setups

The distance between the data centers that are hosting the SAP HANA servers must be within a certain range to keep network latency to a minimum. This range allows synchronous replication to occur with limited affect on the overall application performance (which is also referred to as Metro Mirror distance). It does not matter for what exact purpose the data is replicated to the second data center (for DR or for long-distance HA). In both cases, data must be transferred between the two locations, which affects the process.

Application latency is the key indicator for how well a long-distance HA or DR solution performs. The geographical distance between the data centers can be short. However, the fiber cable between them might follow another route. The Internet service provider (ISP) usually routes through one of its hubs, which leads to a longer physical distance for the signal to travel, and therefore a higher latency. Another factor is the network equipment between the two demarcation points on each site. More routers and protocol conversions along the line introduce a higher latency.

Data can be replicated from one data center to the other data center synchronously or asynchronously.

Synchronous data replication refers to any write request that is issued by the application is committed to the application only after the request is successfully written on both sides. To maintain the application performance within reasonable limits, the network latency (and

Attention: Regarding latency, make sure to specify the layer at which you are measuring it. Network engineers refer to network latency, but SAP prefers to use application latency.

Network latency refers to the low-level latency that network packets experience when traveling over the network from site A to site B. Network latency does not necessarily include the time that it takes for a network packet to be processed on a server.

Application latency refers to the delay that an SAP HANA database transaction experiences when it occurs in a DR environment. This value is sometimes also known as end-to-end latency. It is the sum of all delays as they occur while the database request is in flight and includes, in addition to network latency, packet extraction in the Linux TCP/IP stack, Spectrum Scale code execution, or processing the SAP HANA I/O code stack.




https://scn.sap.com/docs/DOC-47702

therefore the distance) between the sites must be limited to Metro Mirror distances. The maximum achievable distance depends on the performance requirements of the SAP HANA system. In general, an online analytical processing (OLAP) workload can work with higher latencies than an online transaction processing (OLTP) workload. The network latency mainly is dictated by the connection between the two SAP HANA clusters. This inter-site link typically is provided by a third-party ISP.

7.2 HA and DR for single-node SAP HANA

Customers that are running their SAP HANA instance on a single node can still implement redundancy to protect against a failure of this node. Two options are available to achieve this level of protection. Lenovo uses Spectrum Scale to replicate data to a second or third server (storage replication) and SAP uses SSR to replicate data to another SAP HANA system that is running on another server.

Table 7-2 lists the different installations that can be implemented with Lenovo workload-optimized solutions for SAP HANA. It also lists Spectrum Scale and SSR-based solutions and their combinations.

Table 7-2 Overview of HA and DR options for single-node SAP HANA solutions

Characteristic Single-node SAP HANA installation with

HA Stretched HA

DR(by using Spectrum Scale)

DR(by using SSR)

HA and DR (by using Spectrum Scale only)

HA and DR (by using Spectrum Scale and SSR)

Required data centers 1 2 or 3 (metro distance)

2 or 3 (metro distance or higher)

2(metro distance or higher)

2 or 3(metro distance or higher)

2 or 3(metro distance or higher)

RTO Seconds Seconds Minutes Minutes Seconds, for HA, minutes for DR

Seconds, for HA, minutes for DR

RPO Zero Zero Zero (sync) or higher (async)

Zero or higher

Zero Zero or higher

Replication method Spectrum Scale (sync)

Spectrum Scale (sync)

Spectrum Scale (sync or async)

SAP HANA (sync or async)

Spectrum Scale (sync or async)

Spectrum Scale (sync) plus SSR (sync or async)

Automatic failover Yes Yes No Yes, with SUSE HA

Yes, for HA node

Yes, for HA node

Can host non-production No No Yes Yes Yes Yes

Number of SAP HANA server nodes

2 2 2 2 3 3

Number of quorum servers (for Spectrum Scale or SUSE HA)

1 1 1 1 0 1


The following sections describe each of the solutions.

7.2.1 High availability (by using Spectrum Scale)

This setup implements HA for a single node within a single data center. It protects your SAP HANA system from being offline when the server experiences any failure, such as a hardware or a software failure.

A single node with HA installation consists of three physical nodes with the following designations:

� Active (or worker) node� Standby node� Quorum node

From a Spectrum Scale perspective, single-node HA setups can be treated as scale-out solutions with only one node that is running SAP HANA workload and producing data. The second node only receives data from the active node and is ready to take over if the active node experiences any failure.

The worker and the standby nodes must be identical to ensure a successful takeover of SAP HANA operations.

The third node, called the quorum node, must be added so that Spectrum Scale can decide which node survives if there is an outage anywhere in the network between the active and standby node (so called split-brain situations).

During normal operation, all database queries are run on the active node only. The standby node remains inactive and does not respond to any queries.

Spectrum Scale replication ensures that all SAP HANA persistency files on the active node are copied over to the standby node. Under normal operation, there is a synchronous data copy on each of the two servers at any point, which means an RPO of zero is ensured. If a failure occurs, the standby node takes over and continues operation by using its local data copy. The use of synchronous replication mode ensures that every write request that is issued by the application on the worker node returns only after it successfully is written on both nodes. This method ensures that you always have a consistent data set on both nodes.

The quorum node is not running any SAP HANA processes. It runs a small Spectrum Scale process only; therefore, it does not need connectivity to the SAP HANA network. Only Spectrum Scale access is required. Quorum functionality can be implemented on different servers (you do not need to use a Lenovo SAP HANA building block for this task). In most cases, a smaller system, such as the System x3550 M4, is suitable for this task. Without this quorum node, if only the communication between the SAP HANA nodes is interrupted but otherwise the nodes are running fine, Spectrum Scale cannot tell which side should continue to operate and it unmounts the file system on both nodes to protect from inconsistent data

Tolerated node failures 1 1 1 1 2 2

Characteristic Single-node SAP HANA installation with

HA Stretched HA

DR(by using Spectrum Scale)

DR(by using SSR)

HA and DR (by using Spectrum Scale only)

HA and DR (by using Spectrum Scale and SSR)


(split brain). When Spectrum Scale is down, SAP HANA cannot continue to operate because data and logs are inaccessible. Administrative intervention is required in that case.

Figure 7-2 shows a single-node HA solution. Although the Spectrum Scale quorum node is part of the Spectrum Scale cluster, it does not contribute storage capacity to it. For that reason, we extend the shared file system only half into the quorum node in this example.

Figure 7-2 Single-node HA solution

For a fully redundant solution, the network also must be built by using two redundant Ethernet switches. Figure 7-3 shows the network setup of such a single-node HA installation.

Figure 7-3 Network setup for single-node high availability

No manual intervention is required for the takeover to complete because it is handled internally by SAP HANA processes. The takeover usually happens within seconds or minutes, depending on the environment and workload on the system. Clients experience a delay only in their queries.

It is not possible to host any non-production SAP HANA instances on the standby node because it is working in hot-standby mode and must be ready to take over operation at any time.

quorum nodenode02node01


data01 + log01

Worker



Stand-by



Second replica

First replica

SAP HANA DB

local storage local storagedata01 + log01

G8264 switches

Inter-Switch Link (ISL)

GPFS Links

SAP HANA Links

Standby NodeWorker NodeQuorum Node


7.2.2 Stretched high availability (by using Spectrum Scale)

This solution extends the physical distances between two servers of a single-node HA setup to metro distance. This configuration is called stretched HA. It usually spans two data centers at separate sites. If circumstances allow, it also is possible to host each server at one end of a company’s campus.

Allowing for longer distances between server nodes usually means that the network in between is also different from what is available within a single data center. For more information about latency and throughput requirements for running SAP HANA over longer distances, see 7.1.3, “Special considerations for DR and long-distance HA setups” on page 176.

This solution requires two identical SAP HANA building blocks and a Spectrum Scale quorum node. One SAP HANA building block is installed at each site. The node at the primary site is installed as a worker node and the node at the secondary site is installed as a standby node.

Spectrum Scale ensures that all data that is written on the active node is replicated over to the server in the second data center, which is running SAP HANA in standby mode.

Spectrum Scale needs a quorum node to act as tie-breaker if the network between the servers breaks. This quorum node should be in a separate, third location to ensure maximum reliability of the solution.

Figure 7-4 shows a stretched HA implementation for a single-node SAP HANA solution. The Spectrum Scale quorum node is placed at a dedicated third site C.

Figure 7-4 Single-node stretched HA solution (three-site approach)

The Spectrum Scale quorum node does not contribute disk space to the file system; it is connected to the Spectrum Scale network only to decide on the surviving site in split-brain situations. This configuration is why the yellow box that is shown in Figure 7-4 partially spans to the quorum node. The network view that is outlined in Figure 7-5 on page 181 shows that the quorum node must be connected only to the Spectrum Scale network.

Site C Site BSite Aquorum node node02node01


Worker





SAP HANA DB

Second replica

First replica

Stand-by

local storage

data01 + log01

local storage

data01 + log01


Figure 7-5 Network setup of single node with stretched HA

Depending on the customer network preconditions, different scenarios are possible for implementing the inter-site network. Some customers might not have two dedicated links available between the G8264 switches at the primary and secondary sites.

Figure 7-6 shows an alternative approach with only one link. Different versions of virtual private networks (VPN) are used today. In those situations, the G8264 switches can be connected to the VPN gateways or distribution layer switches.

Figure 7-6 Network setup of single node with stretched HA (three-site approach with one inter-site link)


GPFS Links

SAP HANA Links

Site C

Quorum Node

Site B

G8264 switches

Worker Node Standby Node

Site B

G8264 switches


GPFS Links

SAP HANA Links (usually MPLS LER or other dist switches)

Standby Node

Site C

Quorum Node

Worker Node

MPLSor other campus/metro switching


If a separate third location is not available, the quorum node must be placed in the primary data center with the active worker node. This configuration gives the active site a higher weight so that Spectrum Scale continues to operate, even if the server that is hosting the SAP HANA standby process at the second site is no longer reachable. The standby server can become unreachable, for example, because of a server hardware failure or because of a broken link between the two data centers.

Figure 7-7 shows a stretched HA solution with the quorum node in the primary site A with the active worker node.

Figure 7-7 Detailed view of single-node stretched HA solution (two-site approach)

The corresponding network architecture is shown in Figure 7-8.

Figure 7-8 Network setup of single node with stretched HA (two-site approach with dual inter-site links)

Site BSite Aquorum node node02node01


Worker





SAP HANA DB

Second replica

First replica

Stand-by

local storage

data01 + log01

local storage

data01 + log01

G8264 switches


GPFS Links

SAP HANA Links

Quorum Node

Site B

Worker Node Standby Node


This scenario assumes that the customer has two links between the primary and secondary sites that are available for SAP HANA usage. If the links are not there, a consolidated single network link also can be used, as shown in Figure 7-9.

Figure 7-9 Network setup of single node with stretched HA (two-site approach with one inter-site link)

The disadvantage of having the Spectrum Scale quorum node in the primary data center and not in a separate third location comes is important only if the primary data center experiences an outage.

If the Spectrum Scale quorum node is at a third independent site, it can constitute (with the standby node at site B) most of the cluster nodes, so the shared file systems stays up at the standby node and SAP HANA can automatically fail over.

If the Spectrum Scale quorum node is in the same failure domain as the SAP HANA worker node, the file system on the standby node unmounts and manual intervention is required to bring it back up if there is a failure in this domain. In this situation, the Spectrum Scale cluster loses most of the nodes. Automatic failover is still possible in any failure situation that does not affect the network subsystem. If the quorum node can still communicate to the standby node, automatic failover works. This situation protects against any failure of the worker node, such as hardware failures, firmware faults, or operating system and user errors.

Only if there is a total site outage (when the SAP HANA worker node and the Spectrum Scale quorum node are unavailable) is manual intervention required on site B to bring the database instance backup.

Because the node at site B is not idling (it runs SAP HANA standby processes that monitor the worker node), it is not possible to host non-production database instances on this node.

7.2.3 Disaster recovery (by using Spectrum Scale)

An environment that is running single-node DR spans two data centers in two distinct locations. There is a primary and a secondary node and SAP HANA is active on the primary node only. Under normal conditions, SAP HANA is not running on the secondary node, which means no SAP HANA processes are running on it. The secondary node is used only to store

Site B

G8264 switches


GPFS Links

SAP HANA Links

Quorum Node

(usually MPLS LER or other dist switches)

Standby Node

MPLSor other

campus/metro switching

Worker Node


data that is replicated through Spectrum Scale from the primary node (sometimes called Spectrum Scale based storage replication or storage replication). This replication happens in a synchronous manner, which means that disk operations must be completed on both sites before an I/O operation is marked as complete. Synchronous replication ensures an RPO of zero.

If there is a disaster, SAP HANA must be started manually on the secondary node because no standby processes are running in the operation system that monitors the active worker node. The two nodes that are running SAP HANA must be identical; different memory configurations are not allowed.

Another server is required that acts as a Spectrum Scale quorum node. This server ideally is placed at a third location to ensure maximum reliability of the file system. However, if a third data center is not available, this extra server must be placed at the primary data center where it ensures that when the primary node cannot communicate with the secondary node (because of a hardware failure or broken network link), the primary site server stays up and Spectrum Scale runs without experiencing any downtime.

Figure 7-10 shows a single-node solution for DR by using Spectrum Scale based storage replication.

Figure 7-10 Single-node DR solution by using Spectrum Scale (three-site approach)

Site C Site BSite Aquorum node node02node01


Worker



- Index server- Statistic server- SAP HANA Studio


Second replica

First replica

Worker

local storage

data01 + log01

local storage

data01 + log01

SAP HANA DB SAP HANA DB


The network can be implemented in several different ways. The basic version is shown in Figure 7-11, with fully redundant network connections towards satellite sites B and C.

Figure 7-11 Network setup of a single node with DR installation (three-site approach with dual links)

If no dedicated links to site B or C are available, a different architecture can be implemented. An example is shown in Figure 7-12. Combinations of the different approaches are possible. The combination that is decided upon affects the level of redundancy that is provided by a solution.

Figure 7-12 Network setup of a single node with DR (three-site with alternative inter-site connectivity)


GPFS Links

SAP HANA Links

Site C

Quorum Node

G8264 switches

Worker Node DR Node

Site BStorage expansion for non-prod DB instance

G8264 switches


GPFS Links

SAP HANA Links (usually MPLS LER or other dist switches)

DR Node

MPLSor other

campus/metro switching

Site C

Worker NodeQuorum Node



In the absence of a third site that hosts the Spectrum Scale quorum server, this machine must be installed at the primary site A (as shown in Figure 7-13) to ensure continuous availability of the file system if the link between sites A and B is broken.

Figure 7-13 Detailed view of single-node DR solution by using Spectrum Scale (two site approach)

Similar to the approach with three data centers, there is more than one way that networking can be implemented with two data centers, depending on customer requirements. Figure 7-14 on page 186 shows the fully redundant networking architecture with two links between site A and B.

Figure 7-14 Network setup of single node with DR installation (two-site approach with dual links)

Different approaches that are similar to what is shown in Figure 7-12 on page 185 can be implemented for a two-site solution. Details must be agreed upon by the client.

Site BSite Aquorum node node02node01


Worker



- Index server- Statistic server- SAP HANA Studio


Second replica

First replica

Worker

local storage

data01 + log01

local storage

data01 + log01

SAP HANA DB SAP HANA DB

G8264 switches


GPFS Links

SAP HANA Links

Quorum Node

Site B

Worker Node DR Node

Storage expansion for non-prod DB instance


During normal operations, SAP HANA is running only on site A’s node. No SAP HANA processes are running on the DR node at site B. The only Spectrum Scale processes that are running are the processes that receive replicated data from the active worker node from site A.

If the worker node at site A goes down, the file system stays up on the DR node if the Spectrum Scale quorum node is still healthy and available. If the quorum node also goes down, Spectrum Scale unmounts the file system on the DR node because most of the Spectrum Scale cluster nodes are down. If this situation occurs, human intervention is required. An administrator can manually override Spectrum Scale cluster logic and mount the file system on the DR node only by using the second replica.

After the file system is available on the DR node, SAP HANA processes can be started that load database data and replay log files so that the database instance can resume operation with no data loss (RPO of zero).

Because during normal operation there are no SAP HANA processes that are running on the secondary node (not even in standby mode), there is an option to host a non-productive SAP HANA instance (such as development or test) there. In such a scenario, if there is a disaster, this non-productive SAP HANA instance must be shut down before a takeover can happen from the primary system.

This extra SAP HANA instance needs its own storage space for persistency and logs. Lenovo uses the D1024 for SAP HANA or EXP2524 unit to provide this needed space. The expansion units add up to 24 2.5-inch HDDs that are connected directly to the server through an SAS interface. A second file system is created over those drives for the extra SAP HANA instance. This second file system is visible to this node only. Data is not replicated to a second node.

If a disaster occurs and the productive SAP HANA instance is switched to run on the DR server at site B, the non-productive instance must be shut down first. After the primary server at site A is repaired, customers can choose to switch back their productive instance to site A or it can continue to run on the DR node at site B.

If customers choose to keep their production instance on site B, the non-production instance must now be started on site A; that is, the former primary server. For that reason, the site A server also must have a D1024 for SAP HANA or EXP2524 attached (not only site B’s machine). Non-production data on the D1024 for SAP HANA or EXP2524 must be copied manually to the other side before you can start the non-production SAP HANA instance.

7.2.4 Disaster recovery (by using SAP HANA System Replication)

Section 7.2.3, “Disaster recovery (by using Spectrum Scale)” on page 183 describes DR for a single SAP HANA node by using Spectrum Scale-based storage replication. As an alternative, you can use SSR. Database activity is replicated on the application layer to a second SAP HANA instance that is running on a DR node.

Coupling the two single nodes is done at the application level, not on the Spectrum Scale level. Therefore, it is two distinct single-node Spectrum Scale clusters from a file system perspective. No shared file system exists across the two nodes. No Spectrum Scale quorum node is required.


A DR setup that uses SSR is shown in Figure 7-15.

Figure 7-15 Detailed view of single-node DR solution by using SAP HANA System Replication

A failover to site B always requires manual intervention and does not occur automatically.

SSR can be configured to work in synchronous or asynchronous mode. Different physical distances can be realized in each mode. For more information, see “Replication modes” on page 174.

Different architectures are possible for the network that is connecting the two SAP HANA building blocks. Figure 7-16 shows a setup with redundant switches for the SAP HANA replication network.

Figure 7-16 Network setup of single node with SAP HANA System Replication (dual inter-site links)

Because SAP HANA processes are not in an active standby mode on the DR node, a non-production instance can be hosted by using extra dedicated storage. Lenovo uses the D1024 for SAP HANA or EXP2524 unit to provide this extra space. The expansion units add up to 24 2.5-inch HDDs that are directly connected to the server through an SAS interface. A

Site A Site Bnode01


data01 + log01

Worker


- SAP HANA DB

SAP HANA DB

First replica

node01


Worker


- SAP HANA DB

SAP HANA DB

First replica

SAP HANASystem

Replication

local storage

data01 + log01

local storage

G8264 switches


SAP HANA Links

Worker Node DR Node



second file system is created over those drives for the extra SAP HANA instance. This second file system is visible to this node only. Data is not replicated to a second node.

Designs without the network switches are also supported, as shown in Figure 7-17. It is possible to have these inter-site links going across VPNs (or other forms of overlay networks) if the latency and throughput requirements are met.

Figure 7-17 Network setup of single node with SAP HANA System Replication (no switches)

Intermediate architectures with redundant switches in the primary data center on site A and only one link to site B are possible (similar to Figure 7-12 on page 185). Implementations with only one link for SSR can be implemented when no redundancy is required. Details should be discussed with the customer.

7.2.5 HA plus DR (by using Spectrum Scale)

Customers that require the highest level of reliability for their SAP HANA installation can combine HA and DR into one design. This design merges the HA and DR principles that are described in 7.2.1, “High availability (by using Spectrum Scale)” on page 178 and 7.2.3, “Disaster recovery (by using Spectrum Scale)” on page 183. HA also can be implemented as a stretched HA, as described in 7.2.2, “Stretched high availability (by using Spectrum Scale)” on page 180.

A single-node solution with HA and DR by using Spectrum Scale for both features requires three SAP HANA nodes with the same memory configuration. Two nodes are installed at the primary site and one node at the DR site. There is one Spectrum Scale cluster that spans all three nodes. The SAP HANA instance spans only a single node. Spectrum Scale ensures that there are always three replicas available of the data, one on each node. This setup allows the DR node to work even when both primary site nodes are down because it has its own data replica. Spectrum Scale replication occurs in a synchronous manner, which means disk operations are flagged as complete only if all three nodes wrote their replica successfully.

SAP HANA Links


Worker Node DR Node


Figure 7-18 shows a single node with a HA and DR solution. No extra Spectrum Scale quorum node is required because the configuration includes three nodes that are running Spectrum Scale processes. No split-brain situation can occur in a cluster with three members.

Figure 7-18 Detailed view of single-node HA and DR solution (by using Spectrum Scale)

During normal operation, SAP HANA is running only on the nodes at the primary data center when the setup is healthy. From these two nodes, only the worker node responds to user queries. The HA node does not respond to requests. The DR node at site B has no active SAP HANA processes. Only Spectrum Scale processes are running that receive data (third replica) and write it to local disk.

If the primary site worker node experiences an outage, the standby node detects it and SAP HANA fails over and resumes operation on this node. This node is promoted from a standby to a worker designation and responds to user requests. Now, Spectrum Scale has only two replicas that are left in the cluster. When the first node is repaired, it rejoins the cluster and its local replica is restored.

If there is a disaster in which both nodes at the primary site are lost, manual intervention is required to get SAP HANA running at the DR site B. Because Spectrum Scale lost most of its cluster nodes, it must be told that it is safe to operate by using only the surviving node (this temporary procedure is called relaxing the node quorum). The file system can then be mounted again and SAP HANA can be started. It reads data and log files from local disk, loads it into memory, and resumes database operation. Because Spectrum Scale replication happens synchronously, no data is lost and the DR node continues with the last committed transaction before the disaster occurred.

Site BSite Anode03node02node01

Worker



Stand-by



SAP HANA DB


Second replica

First replica

SAP HANA DB

Worker



Third replica

local storage

data01 + log01

local storage

data01 + log01

local storagedata01 + log01


Figure 7-19 shows the networking architecture with redundant Ethernet switches.

Figure 7-19 Network setup of a single node with HA plus DR by using Spectrum Scale

Because the node at the secondary site is not hosting any running instance during normal operation, there is an option to host a non-productive SAP HANA instance (such as development or test) on this DR site node. If there is a disaster, this non-productive SAP HANA instance must be stopped before the production database instance can be started on this node.

This extra SAP HANA instance needs its own storage space for persistency and logs. Lenovo uses the D1024 for SAP HANA or EXP2524 unit to provide this added space. The expansion units add up to 24 2.5-inch HDDs that are directly connected to the server through an SAS interface. A second file system is created over those drives for the extra SAP HANA instance. This second file system is visible only to this node and is not replicated to another node.

7.2.6 HA (by using Spectrum Scale) plus DR (by using SSR)

Single-node HADR solutions can be built with Spectrum Scale-based storage replication and with a combination of storage replication and SSR. In this scenario, Spectrum Scale storage replication is used to provide seamless HA capabilities and SSR is used to implement DR capabilities on a different site, as shown in Figure 7-20 on page 192. A dedicated Spectrum Scale quorum node is required because node01 and node02 can experience a split-brain situation in which an external decision maker is required to decide on the surviving node.

G8264 switches


GPFS Links

SAP HANA Links

Standby Node DR Node


Worker Node


Figure 7-20 Single-node solution with HA by using Spectrum Scale and DR by using SSR

HA with Spectrum Scale can be implemented as base HA (as described in 7.2.1, “High availability (by using Spectrum Scale)” on page 178) or as stretched HA spanning bigger distances (as described in 7.2.2, “Stretched high availability (by using Spectrum Scale)” on page 180). For stretched HA setups, place the Spectrum Scale quorum node at a third location.

There are two data replicas available at site A at any point. Synchronous Spectrum Scale replication ensures that I/O operations are marked as successfully complete if the operation is written to disk on node01 and node 02 of site A.

Depending on the mode in which SSR runs, the third data copy on site B is a synchronous or asynchronous copy. If it is configured to be a synchronous copy, restrictions on the distance of the DR site apply. If SSR runs in asynchronous mode, longer distances can be covered (as described in 7.1.3, “Special considerations for DR and long-distance HA setups” on page 176). Data replication to the DR site occurs on SAP HANA level. The DR node feeds Spectrum Scale with the incoming replication data. Spectrum Scale stores this data on its local disk (as indicated by first replica as shown in the right in Figure 7-20).

Site BSite Anode01


Worker


- SAP HANA DB

SAP HANA DB

First replica

quorum nodenode02node01


Worker



Stand-by



Second replica

First replica

SAP HANA DB

SAP HANASystem

Replication

data01 + log01

local storage

data01 + log01

local storage local storage

data01 + log01


The networking architecture for such a single-node setup is shown in Figure 7-21. Redundant switches are used at both sites. At the DR site B, there is no need to connect the Spectrum Scale interfaces to the Ethernet switch because from a Spectrum Scale perspective, the DR node is creating its own single-node Spectrum Scale cluster. This server is not a member of site A’s Spectrum Scale cluster.

Figure 7-21 Network setup of single-node HA (by using Spectrum Scale) plus DR (by using SSR)

The DR node does not need Spectrum Scale connectivity to site A; therefore, you can leave out the Ethernet switches at site B and connect the node directly to the primary site switches, as shown in Figure 7-22.

Figure 7-22 Network setup of single-node HA (by using Spectrum Scale) plus DR (by using SSR)

G8264 switches


GPFS Links

SAP HANA Links

DR Node


Worker Node

Quorum Node

Standby Node

G8264 switchesInter-Switch Link (ISL)

GPFS Links

SAP HANA Links


Worker Node

Quorum Node

Standby Node DR Node


Depending on the type of inter-site link that connects the primary and secondary data centers, different scenarios are possible. For example, if only one inter-site link is available, only one network interface of the DR node can be cabled. The different options and their implications to redundancy and availability must be discussed with the customer.

During normal operation, the node at the DR site B is running in recovery mode; therefore, you can host a non-productive SAP HANA instance (such as development or test) on this idling node. If there is a disaster, this non-productive SAP HANA instance must be stopped before the production database instance can be started on this node.

This extra SAP HANA instance needs its own storage space for persistency and logs. Lenovo uses the D1024 for SAP HANA or EXP2524 unit to provide this extra space. The expansion units add up to 24 2.5-inch HDDs that are directly connected to the server through an SAS interface. A second file system is created over those drives for the added SAP HANA instance. This second file system is visible only to this node and is not replicated to another node.

7.3 HA and DR for scale-out SAP HANA

Scale-out SAP HANA installations can implement two levels of redundancy to keep their database instance from going offline. The first step is to add a server node to the scale-out cluster that acts as a hot-standby node. The second step is to set up another scale-out cluster in a distinct data center that takes over operation if there is a disaster at the primary site. This DR capability can be implemented by using two different technologies: Spectrum Scale-based replication or SSR. Both products replicate all of the required data to the DR site nodes.

Table 7-3 lists the available options for protecting scale-out SAP HANA environments when Lenovo workload-optimized solutions for SAP HANA are used.

Table 7-3 HA and DR options for scale-out SAP HANA installations

Characteristic Scale-out SAP HANA installation with

HA(by using Spectrum Scale)

HA and DR(by using Spectrum Scale)

HA and DR(by using Spectrum Scale and SSR)

Required data centers 1 2 or 3a 2

Geographical distance Not applicable Metro distance or higher Metro distance or higher

RTO Seconds Seconds for HA, minutes for DR

Seconds for HA, minutes for DR

RPO Zero Zero or higher Zero or higher

Replication method Spectrum Scale (synchronous)

Spectrum Scale (synchronous or asynchronous)

Spectrum Scale (synchronous) plus SSR (synchronous or asynchronous)

Automatic HA failover Yes Yes Yes

Automatic DR failover Not applicable Yes Yes, requires SUSE HA

Can host non-production No Yes, at DR site Yes, at DR site


Although technically it is feasible to implement DR with no HA exclusively in the primary site cluster, it is not a preferred practice to do so and it can cause issues. The first fault that occurs in the primary site triggers a DR event that leads to a failover of the production instance to the DR site. Therefore, always implement HA capabilities alongside DR to avoid these failovers. With HA, the first fault is handled within the primary site and no failover to the DR data center is triggered.

7.3.1 HA by using Spectrum Scale storage replication

SAP HANA uses the concept of standby nodes, which take over operation if a node that is designated as an active worker node fails. Standby nodes can replace any cluster node’s role. For this reason, the standby node must have the same memory configuration as the other cluster nodes.

For a takeover to occur, the standby node requires access to the file system of the failed node. This requirement is fulfilled by Spectrum Scale, which is a shared file system. By using the FPO extension of Spectrum Scale, you can keep the shared file system feature in a local storage only environment. For more information about FPO, see , “Spectrum Scale extensions for shared-nothing architectures” on page 86.

To achieve HA in a scale-out SAP HANA installation, Lenovo uses the data replication feature that is built into the file system. Spectrum Scale replication ensures that two valid physical copies of the data are always in the file system. The concept of replicas is transparent to the application, which means that SAP HANA is not affected if a server that is holding one replica goes offline. Access requests to a file that lost one replica are automatically served from the second replica.

Any file system operation is run on multiple nodes (the number depends on different parameters, such as the size of the data to be written). If both data replicas are persisted successfully, Spectrum Scale signals an I/O operation as complete. This measure ensures an RPO of zero.

For more information about HA in a scale-out SAP HANA installation, see “Scale-out solution with high-availability capabilities” on page 92. This section also describes what happens when one cluster member goes offline and its data is no longer accessible.

More than one standby node in a cluster is a supported scenario. Even with multiple standby nodes, there still is a small time window in which only one data replica exists. This duration depends on the speed of the network, load on the system, and amount of data in the database. Multiple standby nodes are not a replacement for a backup strategy.

7.3.2 DR by using Spectrum Scale storage replication

To implement DR capability, Lenovo includes a feature that is built into the file system. By using the Spectrum Scale replication feature, another data copy (replica) can be stored in a secondary data center location. The use of multiple replicas for DR is identical to the use of HA in a single-site scale-out environment. Spectrum Scale is configured to write another third replica to a remote site.

a. A third data center is required only for automatic DR failover. If no third site is available, manual failover can be implemented instead.


The next section describes a multi-site DR solution by using Spectrum Scale replication. It also describes the advantages of having a quorum node in a separate location, and how the idling servers on the secondary site can be used for hosting more installations of SAP HANA.

For a DR setup, identical scale-out configurations are necessary on the primary and secondary sites. There also can be a third site, which has the sole responsibility to act as a quorum site. We describe the differences between a two-site and a three-site setup later in this section.

Basic architectureDuring normal operation, there is one active, running SAP HANA instance. The SAP HANA instance on the secondary site is not active. The architecture on each site is identical to a standard scale-out cluster with HA, as described in 7.3.1, “HA by using Spectrum Scale storage replication” on page 195. The architecture must include standby servers for HA. A server failure is handled completely within one site and does not enforce a site failover, as shown in Figure 7-23.

Figure 7-23 Basic setup of the disaster recovery solution by using Spectrum Scale synchronous replication

The connection between the two main sites, A and B, depends on the client’s network infrastructure. Use a dual link dark Fibre Connection to allow for redundancy in the network switch at each site. For full redundancy, another link pair is required to fully mesh the four switches.

Figure 7-24 shows a connection with one link pair in between. It also shows that only the Spectrum Scale network must span both sites because they make up one stretched Spectrum Scale cluster. The SAP HANA internal network is kept within each site because SAP HANA does not need to communicate to the other site.

Site C

Site Bnode08node07node05 node06

Site Anode04node03node01 node02

Partition 1 Partition 2 Partition 3 Standby

SAP HANA DB (active)

Partition 1 Partition 2 Partition 3 Standby

SAP HANA DB (inactive)

GPFSquorum

node


Second replica

First replica

Third replica

synchronousreplication


Figure 7-24 Networking details for SAP HANA Spectrum Scale based DR solutions

Within each site, the 10 Gb Ethernet network connections for the internal SAP HANA and internal Spectrum Scale network are implemented in a redundant layout.

Depending on where the demarcation point is between the SAP HANA installation, client network, and inter-site link, different architectures can be used. In the preferred case, there is a dedicated 10 Gb Ethernet connection going out of each of the two SAP HANA switches at each site towards the demarcation point. There is no requirement with which technology the data centers are connected if the technology can route IP traffic across the link. In general, low-latency interconnect technologies are the preferred choice. For more information about latency, see 7.1.3, “Special considerations for DR and long-distance HA setups” on page 176.

Depending on the client infrastructure, the inter-site link might be the weakest link and no full 10 Gb Ethernet can be carried across it. SAP validated the Lenovo solution by using a 10 Gb interconnect; however, an HANA cluster might not generate that much traffic and a value much smaller than 10 Gb is sufficient depending on the workload. This choice must be decided upon by each client. For example, the initial database load might take hours by using a 1 Gbit connection, or minutes when a 10 Gbit network connection to the remote site is used. During normal operation, latency is more critical than bandwidth for the overall application performance.

As with a standard scale-out implementation, the DR configuration relies on Spectrum Scale functionality to enable the synchronous data replication between sites. A single-site solution holds two replicas of each data block. This function is enhanced with a third replica in the dual-site DR implementation. A stretched Spectrum Scale cluster is implemented between the two sites. Figure 7-23 on page 196 shows that there is a combined cluster on Spectrum Scale level spanning both sites, where each site’s SAP HANA clusters are independent from each other on different sites. Spectrum Scale file placement policies ensure that there are two replicas on the primary site and a third replica on the secondary site. If there is a site failure, the file system can stay active with a complete data replica in the secondary site. The SAP HANA database can then be made operational with the one remaining replica of persistency and log files.

Spectrum Scale is a cluster file system. As such, it is vulnerable to a split-brain situation. A split brain occurs when the connection between the two data centers is lost but both clusters can still communicate internally. Each surviving cluster then thinks that the nodes at the other site are down and it is safe to continue writing to the file system. In the worst case, this situation can lead to inconsistent data at the two sites.

To avoid this situation, Spectrum Scale requires a quorum of nodes to communicate. This quorum is called a Spectrum Scale cluster quorum. Not every server that is designated as a Spectrum Scale quorum node is elected to act as one. Spectrum Scale chooses an odd number of servers to act as quorum nodes. The exact number depends on the number of total servers within the Spectrum Scale cluster. For an SAP HANA DR installation, the primary

ISL ISL

Site A Site B

HANA internal VLANGPFS VLAN




Four IBM G8264 Ethernet switches


active site always has one more node that is assigned as a quorum than the backup site. This configuration ensures that if the inter-site link goes down, Spectrum Scale stays up on the primary site nodes.

In addition to the Spectrum Scale cluster quorum, each file system in the Spectrum Scale cluster stores vital information in a small structure that is called a file system descriptor (shown as FSdesc in the output of Spectrum Scale commands). This file system descriptor contains information, such as file system version, mount point, and a list of all the disks that make up this file system. When the file system is created, Spectrum Scale writes a copy of the file system descriptor on to every disk. From then on, Spectrum Scale updates them only on a subset of disks upon changes to the file system.

Depending on the Spectrum Scale configuration, three, five, or six copies are kept up to date.3 Most valid file system descriptors are required for the file system to be accessible.4 If disks fail over time, Spectrum Scale updates another copy on different disks to ensure that all copies are alive. If multiple disks fail concurrently and each was holding a valid copy of the file system descriptor, a cluster loses the file system descriptor quorum and the file system is unmounted automatically.

Site failoverDuring normal operation, there is a running SAP HANA instance active on the primary site. The secondary site includes an installed SAP HANA instance that is inactive. A failover to the remote SAP HANA installation is a manual procedure; however, the steps in a script can be automated. Depending on the reason for the site failover, you can decide whether the secondary site becomes the new production site or a failback must happen after the error in the primary site is fixed.

To ensure the highest level of safety, the Spectrum Scale file system is not mounted on the secondary site during normal operation. This action ensures that there is no read nor write access to the file system. However, the file system can be mounted, for example, read-only, to allow for backup operations on the DR site.

A failover is defined as anything that brings the primary site down. Single errors are handled within the site by using the fully redundant local hardware (such as a spare HA node and second network interface).

The following events are handled within the primary site:

� Single-server outage� Single-switch outage� Accidentally pulled network cable� Local disk failure

The following events cause a failover to the DR site:

� Power outage in the primary data center, which causes all nodes to be down� Two servers going down at the same time (not necessarily because of the same problem)

The time for the failover procedure depends on how long it takes to open SAP HANA on the backup site. The data is in the data center and ready to be used. Any switch from one site to the other includes a downtime of SAP HANA operations because the two independent instances on either site must not run concurrently because of the sharing of the persistency and log files on the file system.

3 For environments with only one disk, Spectrum Scale uses only one file system descriptor. This scenario does not apply to SAP HANA setups.

4 Even if all disks fail, if there is at least one valid file system descriptor that is accessible, you still have a chance to recover data manually from the file system.


Spectrum Scale provides the means to restore HA capabilities on the backup site. During normal operation, only one replica is pushed to the backup site. However, clients that consider their data centers to be equal might choose to not gracefully fail back to the primary data center after it is repaired, but instead continue to run production from the backup site. For this scenario, a Spectrum Scale restripe is triggered that creates a second replica on the backup site out of the one available replica. This procedure is called restoring HA capabilities. The exact commands are described in the Lenovo SAP HANA Operations Guide (see 8.2, “Lenovo SAP HANA Operations Guide” on page 212). The duration of this restriping depends on the amount of data in the file system.

When the SAP HANA instance is started, the data is loaded into main memory. The SAP HANA database is restored to the latest savepoint and the available logs are recovered. This procedure can be automated, but it depends on the client environment. The commands in the Operations Guide (see 8.2, “Lenovo SAP HANA Operations Guide” on page 212) provide a template for such automation.

Clients who choose to continue running production out of the former backup data center can easily add the former primary site after it is restored. The nodes are integrated back into the Spectrum Scale cluster and a resynchronization of the most recent version of the data occurs between the new primary data center and the new secondary data center. One replica is held in the new secondary site. The overall picture looks as before a failover only with the data centers having switched their designation.

Site failbackA site failback is defined as a graceful switch of the production SAP HANA instance from the secondary data center back to the primary data center.

To understand the procedure for a failback, it is important to know the following possible initial states of the DR environment:

� You have two replicas that are local in the data center and one replica in the remote site’s data center, as shown in the following examples:

– A disaster occurred and you restored HA on the backup site; now you are ready to fail back production to the primary site again.

– During normal operations, you want to switch production to the backup site for maintenance reasons.

� You have one replica that is local in the data center and two replicas are in the remote site’s data center. For example, a disaster occurred and you are running production from the backup data center without having HA restored.

Environments with only one working replica must restore HA first before failing back gracefully, which ensures the highest level of safety for your data during the failback procedure.

When SAP HANA is running from a file system with two local replicas, the failback procedure is identical to a controlled failover procedure. The data center is assumed to be down, therefore the remote site is now the active site. HA is restored (by using Spectrum Scale restriping), and the second site is attached again with one single replica of the data. SAP HANA can be started when it is shut down on the other site, but it will experience a performance degradation during HA restore (that is, Spectrum Scale restriping). Spectrum Scale restriping is an I/O-heavy operation.

DR environments with dedicated quorum nodeThe most reliable way to implement DR is with the use of a dedicated quorum node in a third site. The sole purpose of the quorum node is to decide which site can run the production


instance after the link that is connecting the primary and secondary data centers is lost (known as a split brain). The quorum node, which is placed at a third site, has a separate connection to the primary and to the secondary site, as shown in Figure 7-25 on page 200.

Figure 7-25 Network setup of Spectrum Scale based DR for a scale-out system with a dedicated quorum node

The quorum node is configured to act as a Spectrum Scale quorum server without storing any SAP HANA data. The only requirement is to have a small disk or partition available that can hold a file system descriptor. The quorum node needs network access to only the Spectrum Scale network. SAP HANA network access is not required.

In a standard setup, two nodes from the primary data center act as quorum nodes, two nodes from the secondary data center act as quorum nodes, plus the quorum node at the third site. The number of quorum nodes is shown as the weight of a data center in Figure 7-26.

Figure 7-26 Three-site environment with a dedicated Spectrum Scale quorum node

In the unlikely event that two inter-site links are interrupted at the same time, most of the quorum nodes are available to communicate with each other over the one connection that is


GPFS Links

SAP HANA Links

Site B

WorkerNode

StandbyNode

WorkerNode

WorkerNode

DRNode

DRNode

DRNode

DRNode

Site C

QuorumNode

GPFS quorum functionality

Tertiary siteGPFS quorum node only

Primary siteServers running production

Weight = 2

Secondary siteServers waiting to take over

Weight = 2

Weight = 1


left to decide from which data center to run production. In terms of weight, any of these situations a minimum of three can be ensured to always be up.

If the links between the primary and secondary and between the secondary and tertiary data centers go down, SAP HANA keeps running out of the primary site without any downtime. If the links between the primary and tertiary and between the secondary and tertiary data centers go down, the primary and secondary data center can still communicate and SAP HANA keeps running out of the primary data center.

If the links between the primary and secondary and the primary and tertiary data centers go down, it means that the production SAP HANA instance in the primary data center is isolated and loses Spectrum Scale quorum. As a safety measure, Spectrum Scale prevents any writing to the file system on the primary site and SAP HANA stops. Spectrum Scale stays up and running on the secondary site because the quorum node still has a connection to it. Depending on client requirements, HA must be restored first before SAP HANA can be started and production continues to run out of the secondary site data center.

It is a valid use case to set up a server to act as a Spectrum Scale quorum node for the SAP HANA DR installation.5 Spectrum Scale must have root access on this machine to run. Three Spectrum Scale server licenses are required on the primary and on the secondary sites for the first three servers.6 Extra servers need Spectrum Scale FPO licenses.

The main advantage of having a dedicated quorum node is that the file system always is available during failover and failback without any manual intervention if there is at least one site and the quorum nodes can communicate with each other.

DR environments without a dedicated quorum nodeEnvironments that do not have a third site to host a dedicated quorum node can still implement a Spectrum Scale-based DR solution for SAP HANA. The difference between environments with and without a dedicated quorum node is the procedure that is required upon a failover. Figure 7-27 shows such a scenario with no third site.

Figure 7-27 Network setup of Spectrum Scale based DR for a scale-out system without a dedicated quorum

5 The applicability of this statement must be verified for each installation by Lenovo.6 The currently validated minimum number of servers on each site for DR is three, which is required by SAP to set up

a scale-out environment with HA. Therefore, the requirement is for at least three Spectrum Scale server licenses per site.


GPFS Links

SAP HANA Links

Site B

WorkerNode

StandbyNode

WorkerNode

WorkerNode

DRNode

DRNode

DRNode

DRNode

GPFS quorum functionality


A dedicated quorum node at a third site allows the file system to remain accessible even if the primary or secondary site goes down because Spectrum Scale has most of its nodes still available. Without a dedicated quorum node, the quorum node’s functionality must be put on the primary site to ensure that Spectrum Scale in the primary data center continues to operate even if the inter-site link is interrupted.

Figure 7-28 shows the Spectrum Scale view of an environment without a dedicated quorum node. The weight symbolizes quorum designation. There are three servers with a quorum designation in the primary data center and two servers in the backup data center.

Figure 7-28 A two-site environment without a dedicated Spectrum Scale quorum node

If a disaster occurs at the primary site, the Spectrum Scale cluster loses its quorum because the two quorum nodes at the backup site do not meet the minimum of three. Another procedure is required to relax the Spectrum Scale quorum on the surviving secondary site before Spectrum Scale comes up again. The exact procedure is described in the Operations Guide for Lenovo Systems Solution for SAP HANA (see 8.2, “Lenovo SAP HANA Operations Guide” on page 212).

After Spectrum Scale is running again, the procedure is identical to a failover with a dedicated quorum node. It is optional to restore HA capabilities within the secondary site. SAP HANA can be started while this restore procedure is still running; however, performance can be affected because this process is an I/O-intensive operation.

You need three Spectrum Scale server licenses on the primary and on the backup sites, even though during normal operation only two of them are required on the backup site. If a disaster occurs and you must failover the SAP HANA production instance to the backup data center, this data center becomes the main SAP HANA cluster and requires three Spectrum Scale server licenses. More servers on either site receive Spectrum Scale FPO licenses.7

Backup site hosting non-production SAP HANAIn the environments that are described thus far, all servers on the secondary site receive data over the network from the primary site only and store it on local disks; otherwise, they are idling. There are no SAP HANA processes running on them.

To use the idling compute power, SAP supports hosting non-production SAP HANA instances on the backup site; for example, a quality assurance (QA) or training environment. When a disaster occurs, this non-production instance must be shut down before the failover procedure of the production instance can be started.

The non-production SAP HANA instances need more space for persistency and log data. Lenovo uses the D1024 for SAP HANA or EXP2524 to extend the locally available disk storage space. The expansion units directly connect through an SAS interface to one server

7 The currently validated minimum number of servers on each site for DR is three, which is required by SAP to set up a scale-out environment with HA. Therefore, the requirement is for at least three Spectrum Scale server licenses per site.

Primary siteServers running production

Secondary siteServers waiting to take over

Weight = 3 Weight = 2


and provide up to 24 extra 2.5-inch disks. You need one D1024 or EXP2524 for each secondary site server that is to participate in hosting a non-production instance.

Figure 7-29 shows an example of the overall architecture with four SAP HANA appliances per site.

Figure 7-29 Running non-production SAP HANA instances on the idling backup site

If a failover occurs from the primary site to the secondary site and you are planning to keep running production from the secondary data center, you must host non-production instances in the primary data center. To accommodate for this added storage space on the primary site servers, you also must connect D1024 or EXP2524 expansion units to them.

If you plan to fail back gracefully your production instance from the backup site to the primary site after it is repaired, you do not need to have D1024 or EXP2524 expansion units on the primary site servers. There might be unforeseen outages that take a long time to repair. You cannot run your non-production site instances during this outage.

There is one new file system that spans all expansion units of the backup site servers. This new file system runs with a Spectrum Scale replication factor of two, meaning that there are always two copies of each data block. The first replica is stored locally in the node writing the data. The second replica is stored in a striped round-robin fashion over the other nodes. This configuration is identical to a scale-out HA environment.

One server can fail and the data is still available on the other nodes’ expansion units. Figure 7-29 shows this from node5’s perspective. If node5 writes data into the non-production file system, it stores the first replica on local disks in the expansion unit. The second replica is striped across node6, node7, and node8’s expansion unit drives (symbolized as a long blue box in Figure 7-29).

Although Lenovo does not support a multi-SID configuration, it is a valid scenario to run different SAP HANA instances on different servers. If you had a cluster of six nodes on each site, you can, for example, run QA on two nodes and development on four nodes, or you can run QA on three nodes and development on three nodes. All non-production instances must use the same file system, which means that in this example, QA and development must be configured to use different directories.

Primary site

node4

Localstorage

Secondary site

node1 node2 node3 node5 node6 node7

Localstorage

Localstorage

Localstorage

Localstorage

Localstorage

Localstorage

node8

Localstorage

seco

ndre

plic

a

third

repl

ica

first

repl

ica

... ... ... ...

Second file system spanning only expansion unit drives (metadata and data)

Produc-tionfilesystem

First replicaSecond replica

Non-productionfile system


SummaryThe DR solution for the Lenovo Systems Solution for SAP HANA uses the advanced replication features of Spectrum Scale, which creates a cross-site cluster that ensures availability and consistency of data across two sites. It does not impose the need for more storage systems; instead, it builds upon the scale-out solution for SAP HANA. This simple architecture reduces the complexity in maintaining such a solution while keeping the possibility of adding nodes over time if the database grows.

7.3.3 HA by using Spectrum Scale replication plus DR by using SAP HANA Replication

As an alternative to Spectrum Scale only environments, customers can choose SSR to implement DR capabilities. In such a scenario, HA within the primary data center still uses Spectrum Scale storage replication, but data replication to the DR site is handled by SAP HANA on application level with the SSR feature.

From a Spectrum Scale perspective, such a setup requires two distinct clusters, one at each site. SSR replicates data from the primary cluster to the DR site cluster. Figure 7-30 shows a four-node scenario with three worker nodes and one standby node (for HA) at each site. SSR replicates data from each node to its corresponding node at the DR site.

Figure 7-30 DR of a scale-out configuration by using SAP HANA System Replication

This replication can be configured to work in synchronous or asynchronous mode. Synchronous mode ensures an RPO of zero, but limits the maximum distance between the two sites. Asynchronous mode allows for bigger distances, but with the risk of losing data that is still unacknowledged when the disaster happens (RPO > 0).



data01 + log01

DB partition 1 DB partition 2

data02 + log02

DB partition 3

data03 + log03

Standby node

Second replica

First replica

SAP HANA® DB

Site A

local storage local storage local storage local storage



data01 + log01


data02 + log02

DB partition 3

data03 + log03

Standby node

Second replica

First replica

SAP HANA® DB


SAP HANASystem Replication

Site B


For more information details about SSR and the two replication modes, see 7.1.2, “SAP HANA System Replication” on page 173.

Single-node failures are handled within the primary data center. The standby node takes over operation of the failed node. This process occurs automatically without administrative intervention.

Multi-node failures, such as a full data center outage or other disasters, require a failover of SAP HANA to the DR site. This failover is a manual activity. SAP HANA administrators must promote DR site nodes from running in recovery mode to active workers (or standby, respectively).

With the release of SAP HANA 1.0 SPS 08 in June 2014, it is possible to use different networks for SSR traffic. You can use the front-end client network, back-end SAP HANA network, or a new dedicated replication network that spans both sites. By having the choice between multiple networks, you can better adapt to different customer situations.

During normal operation, the nodes at the DR site B are running in recovery mode. By using this mode, you can host a non-productive SAP HANA instance (such as development or test) on these idling nodes. If there is a disaster, this non-productive SAP HANA instance must be stopped before the production instance can be started at site B.

This extra SAP HANA instance needs its own storage space for persistency and logs. The D1024 for SAP HANA or EXP2524 unit is used to provide this extra space. The expansion units add up to 24 2.5-inch HDDs that are connected directly to the server through a SAS interface. A second file system is created over those drives for the added SAP HANA instance. Depending on the size of this extra database instance, one or more DR site nodes must be used. Every node in scope requires an external storage expansion. This second file system is created with a replication factor of two to implement HA in the non-production instance.

7.4 Backup and restore

Because SAP HANA plays a critical role in the overall SAP IT landscape, it is important to back up the data in the SAP HANA database and restore it. This section described the basic information about backing up and recovery for the operating system, for SAP HANA, the available products that are certified for use with SAP HANA, and the integration of SAP HANA and IBM Tivoli Storage Manager for ERP.

7.4.1 Basic operating system backup and recovery

On the X6 solutions, Lenovo implemented a concept of a backup operating system partition that a known-good state can be fallen back to after something occurs on the primary active operating system installation. This feature was made available with the summer 2014 release of the X6 solution. It requires more partitions on the RAID 1 on which the operating system is installed. The feature includes a set of tools with which data can be synced from the active operating system partition onto the backup partition and vice versa. Operating system-level updates or change to certain components can be tested and easily rolled back if not successful.

This feature does not replace a full backup of the operating system installation because the data remains on the server and even on the same disks. For entry-level environments or less critical environments, it provides an easy-to-use tool at no added cost that is included as part of the solution.


7.4.2 Basic database backup and recovery

Saving the savepoints and the database logs technically is impossible in a consistent way, and thus does not constitute a consistent backup from which it can be recovered. Therefore, a simple file-based backup of the persistency layer of SAP HANA is insufficient.

Backing up the databaseA backup of the SAP HANA database must be triggered through the SAP HANA Studio or through the SAP HANA SQL interface. SAP HANA then creates a consistent backup, which consists of one file per SAP HANA service on each cluster node. SAP HANA always performs a full backup. Incremental backups are not supported by SAP HANA.

SAP HANA internally maintains transaction numbers, which are unique within a database instance, especially in a scale-out configuration. To create a consistent backup across a scale-out configuration, SAP HANA chooses a specific transaction number and all nodes of the database instance write their own backup files, including all transactions up to this transaction number.

The backup files are saved to a defined staging area that might be on the internal disks, an external disk on an NFS share,8 or a directly attached SAN subsystem. In addition to the data backup files, the SAP HANA configuration files and backup catalog files must be saved to be recovered. For point-in-time recovery, the log area also must be backed up.

With the System x solution for SAP HANA, one of the 1 Gbit network interfaces of the server can be used for NFS connectivity, or another 10 Gbit network interface must be installed (if a PCIe slot is available). You can add a Fibre Channel host bus adapter (HBA) for SAN connectivity. For more information about supported hardware additions to provide more connectivity, see the Quick Start Guide for the Lenovo Systems Solution for SAP HANA, is available at this website:

http://www-947.ibm.com/support/entry/myportal/docdisplay?lndocid=MIGR-5087035

Restoring a database backupIt might be necessary to recover the SAP HANA database from a backup in the following situations:

� The data area is damaged.

If the data area is unusable, the SAP HANA database can be recovered up to the last committed transaction if all the data changes after the last complete data backup are still available in the log backups and log area. After the data and log backups are restored, the SAP HANA databases uses the data and log backups and the log entries in the log area to restore the data and replay the logs to recover. It also is possible to recover the database by using an older data backup and log backups if all relevant log backups that are made after the data backup are available.9

� The log area is damaged.

If the log area is unusable, the only way to recover is to replay the log backups. Therefore, any transactions that are committed after the most recent log backup are lost, and all transactions that were open during the log backup are rolled back.

After the data and log backups are restored, the log entries from the log backups automatically are replayed to recover. It is also possible to recover the database to a specific point if it is within the existing log backups.

8 SAP Note 1820529 lists network file systems that are unsuitable for backup and recovery.9 For help with determining the files that are needed for a recovery, see SAP Note 1705945.



� The database must be reset to an earlier point because of a logical error.

To reset the database to a specific point, a data backup from before that point to recover to and the subsequent log backups must be restored. During recovery, the log area might be used as well, depending on the point to which the database is reset. All changes that are made after the recovery time are (intentionally) lost.

� You want to create a copy of the database.

You might want to create a copy of the database for various purposes, such as creating a test system.

A database recovery is started from the SAP HANA studio or, starting with SAP HANA 1.0 SPS07, from the command line.

Certain restrictions apply when a backup is restored. Up to and including SAP HANA 1.0 SPS06, the target SAP HANA system was required to be identical to the source regarding the number of nodes and node memory size. Starting with SPS07, it is possible to recover a backup that is taken from an m-node scale-out system and restore it on an n-node scale-out environment. Memory configuration also can be different. You must configure m-index server instances on the n-node target environment to restore the backup, which means that nodes can have more than one index server. Such a configuration does not provide the best performance, but it might be sufficient for test or training environments.

When a backup image is restored from a single-node configuration into a scale-out configuration, SAP HANA does not repartition the data automatically. The correct way to bring a backup of a single-node SAP HANA installation to a scale-out solution is by using the following process:

1. Back up the data from the stand-alone node.

2. Install SAP HANA on the master node.

3. Restore the backup into the master node.

4. Install SAP HANA on the subordinate and standby nodes as appropriate and add these nodes to the SAP HANA cluster.

5. Repartition the data across all worker nodes.

For more information about the backup and recovery processes for the SAP HANA database, see SAP HANA Backup and Recovery Guide, which is available at this website:


7.4.3 File-based backup tool integration

By using the mechanisms that are described in 7.4.1, “Basic operating system backup and recovery” on page 205, virtually any backup tool can be integrated with SAP HANA. Backups can be triggered programmatically by using the SQL interface, and the resulting backup files that are written locally then can be moved into the backup storage by the backup tool. Backup scheduling can be done by using scripts that are triggered by the standard Linux job scheduling capabilities or other external schedulers. Because the Backint backup interface was introduced to SAP HANA with SPS05, a file-based backup tool integration is the only option for pre-SPS05 SAP HANA deployments.

For more information about a file-based integration of IBM Tivoli Storage Manager for ERP V6.4 with SAP HANA, see A.2, “File-based backup with IBM Tivoli Storage Manager for ERP” on page 221.



7.4.4 Database backups by using Spectrum Scale snapshots

All System x solutions for SAP HANA use Spectrum Scale as the file system on which SAP HANA runs. Spectrum Scale supports a snapshot feature with which you can take a consistent and stable view of the file system that can then be used to create a backup (which is similar to enterprise storage snapshot features). While the snapshot is active, Spectrum Scale stores any changes to files in a temporary delta area. After the snapshot is released, the delta is merged with the original data and any further changes are applied on this data.

Taking only a Spectrum Scale snapshot does not ensure that you have a consistent backup that you can use to perform a restore. SAP HANA must be instructed to flush out any pending changes to disk to ensure a consistent state of the files in the file system. With the release of SAP HANA 1.0 SPS07, a snapshot feature is introduced that prepares the database to write a consistent state to the data area of the file system (the log area is merged into the data area so only data must be considered for snapshotting). While this snapshot is active, a Spectrum Scale snapshot must be triggered. SAP HANA can then be instructed to release its snapshot. By using Linux copy commands or other more sophisticated backup tools, the data can then be stored in a backup place (NFS share, SAN storage, or other places).

The use of snapshots has much less effect on the performance of the running database than to trigger a file-based backup. Triggering a Spectrum Scale snapshot works in single-node and scale-out environments. The time that it takes to activate a snapshot depends on the amount of data in the file system and the current load on it.

7.4.5 Backup tool integration with Backint for SAP HANA

Starting with SAP HANA 1.0 SPS05, SAP provides an application programming interface (API) that can be used by manufacturers of third-party backup tools to back up the data and redo logs of an SAP HANA system.10 By using this “Backint for SAP HANA” API, a full integration with SAP HANA studio can be achieved with which backups can be configured and run by using Backint for SAP HANA.

With Backint, instead of writing the backup files to local disks, dedicated SAN disks, or network shares, SAP HANA creates data stream pipes. Pipes are a way to transfer data between two processes: one is writing data into the pipe, and the other is reading data out of the pipe. This configuration makes a backup by using Backint a one-step backup. No intermediate backup data is written (unlike with a file-based backup tool integration that writes to local disk first), which relieves the local I/O subsystem from the backup workload.

Backing up through BackintThe third-party backup agent runs on the SAP HANA server and communicates with the third-party backup server. SAP HANA communicates with the third-party backup agent through the Backint interface. After the user starts a backup through the SAP HANA Studio or by running hdbsql, SAP HANA writes a set of text files that describe the parameterization for this backup, including version and name information, stream pipe location, and the backup policy to use. Then, SAP HANA creates the stream pipes. Each SAP HANA service (for example, index server, name server, statistics server, and XS engine) has its own stream pipe to which to write its own backup data. The third-party backup agents read the data streams from these pipes, and pass them to the backup server.

10 For more information, see SAP Note 1730932 “Using backup tools with Backint”.


Currently, SAP HANA does not offer backup compression; however, third-party backup agents and servers can compress the backup data and further transform it, for example, by applying encryption. Finally, SAP HANA transmits backup catalog information before the third-party backup agent writes a file reporting the result and administrative information, such as backup identifiers. This information is made available in SAP HANA Studio.

Restoring through BackintAs described in “Restoring a database backup” on page 206, a database restore might be necessary when the data area or log area is damaged to recover from a logical error or to copy the database. This process can be achieved by using data and log backups that were performed previously.

A restore operation can be started through the SAP HANA Studio only. For the first step, SAP HANA shuts down the database. SAP HANA then writes a set of text files that describe the parameterization for this restore, including a list of backup identifiers and stream pipe locations. After receiving the backup catalog information from the third-party backup tool, SAP HANA performs a series of checks to ensure that the database can be recovered with the backup data available. Then, SAP HANA establishes the communication with the third-party backup agents by using stream pipes, and requests the backup data from the backup server. The backup agents then stream the backup data that is received from the backup server through the stream pipes to the SAP HANA services. As a final step, the third-party backup agent writes a file reporting the result of the operation for error-handling purposes. This information is made available in SAP HANA Studio.

Backint certificationBackup tools that use the Backint for SAP HANA interface are subject to certification by SAP. For more information about the certification process, see this website:


To determine which backup tools are certified for Backint for SAP HANA, see the following website and enter the search term “HANA-BRINT”

http://global.sap.com/community/ebook/2013_09_adpd/enEN/search.html

As of May 2016, the following tools are certified by SAP:

� Tivoli Storage Manager for ERP 6.411

� Veritas NetBackup 7.7� Commvault Simpana 10.0� HP Data Protector 7.0, 8.1, and 9.0� HP StoreOnce� EMC Networker 8.2� EMC Interface for Data Domain Boost for Databases and Applications 1.0� ASG Time Navigator 4.4� Hitachi Data Protection Suite 10� Libelle BusinessShadow 6.0.6� SEP sesam 4.4

7.4.6 Backup and restore as a DR strategy

The use of backup and restore as a DR solution is a basic way of providing DR. Depending on the RPO, it might be a viable way to achieve DR. The basic concept is to back up the data on the primary site regularly (at least daily) to a defined staging area, which might be an external

11 IBM Spectrum Protect 7.1 (formerly called TSM for ERP) is only certified for HANA on Power, not on Intel x86.



http://global.sap.com/community/ebook/2013_09_adpd/enEN/search.html



disk on an NFS share or a directly attached SAN subsystem (this subsystem does not need to be dedicated to SAP HANA). After the backup is done, it must be transferred to the secondary site, for example, by a simple file transfer (can be automated) or by using the replication function of the storage system that is used to hold the backup files.

Following a company’s DR strategy production, SAP HANA must run on the backup site. Therefore, an SAP HANA system must be on the secondary site, which is similar to the system on the primary site at minimum regarding the number of nodes and node memory size.

During normal operations, this system can run other non-productive SAP HANA instances, for example, quality assurance (QA), development (DEV), test, or other second-tier systems. If the primary site goes down, the system must be cleared from these second-tier HANA systems and the backup can be restored.

Upon configuring the application systems to use the secondary site instead of the primary one, operation can be resumed. The SAP HANA database recovers from the latest backup if there is a disaster.

Figure 7-31 shows the concept of the use of backup and restore as a basic DR solution.

Figure 7-31 Scale-out installation that uses backup and restore as a DR solution



data01 + log01


data02 + log02

DB partition 3

data03 + log03

Standby node

Second replica

First replica

SAP HANA® DB

Primary Site

Secondary Site




data01 + log01


data02 + log02

DB partition 3

data03 + log03

Standby node

Second replica

First replica

SAP HANA® DB

local storage local storage local storage local storageStorage

Storage

backup

restore

copy or otherwisetransfer the backup files


Chapter 8. SAP HANA operations

This chapter describes the operational aspects of running an SAP HANA system.


� Installation services� Lenovo SAP HANA Operations Guide� Interoperability with other platforms� Monitoring SAP HANA� Installing more agents� Software and firmware levels� Support process

8


8.1 Installation services

The Lenovo System x Solution for SAP HANA features the complete software stack, including the operating system, IBM Spectrum Scale, and the SAP HANA software. Because of the nature of the software stack, and dependencies on how the Lenovo System x Solution for SAP HANA is used at the client location, the software stack cannot be preinstalled completely at manufacturing. Therefore, installation services are required. The Lenovo System x Solution for SAP HANA often includes the following installation services:

� Performing an inventory and validating the delivered system configuration

� Verifying and updating the hardware to the latest level of basic input/output system (BIOS), firmware, device drivers, and operating system patches as required

� Verifying and configuring the Redundant Array of Independent Disks (RAID) configuration

� Finishing the software preinstallation according to the client environment

� Configuring and verifying network settings and operation

� Performing system validation

� Providing onsite skills transfer (when required) of the solution and preferred practices, and delivering postinstallation documentation

To ensure the correct operation of the appliance, installation services for the Lenovo System x solution for SAP HANA must be performed by trained personnel who are available from Lenovo Services or Lenovo Business Partners depending on your geography.

8.2 Lenovo SAP HANA Operations Guide

The Hardware, Operating System & GPFS Operations Guide describes the operations of a Lenovo System x Solution for SAP HANA appliance. Other guides are available for the supported platform (eX5, X6, Flex X6). Each guide covers the following topics:

� Cluster operations:

– Actions to take after a server node failure, such as recovering the Spectrum Scale file system, removing the SAP HANA node from the cluster, and installing a replacement node.

– Recovering from a temporary node failure by bringing that Spectrum Scale that is on that node back to a fully operational state, and restarting SAP HANA on the node.

– Adding a cluster node by integrating it into the private networks of the appliance, and into the Spectrum Scale and SAP HANA clusters.

– Reinstalling the SAP HANA software on a node.

� Disaster recovery cluster operations:

– Common operations that deviate from the procedures on a normal installation, such as system shutdown and start.

– Planned failover and failback procedures.

– Site failover procedures after a site failure for various scenarios.

– How to deal with node failures, disk failures, and network failures.

– How to operate non-production instances that are running on the secondary site in a disaster recovery scenario.


� Drive operations:

– Checking the drive configuration and the health of the drives.

– Replacing failed hard disk drives (HDDs), solid-state drives (SSDs), and High input/output operations per second (IOPS) devices and reintegrating them into Spectrum Scale.

– Driver and firmware upgrades for the High IOPS devices, HDDs, and SSDs.

� System health checks

How to obtain Spectrum Scale cluster status and configuration, file system status and configuration, disk status and usage, quotas, SAP HANA application status, and network information from the switches.

� Software updates

Checklists for what to do to update the Linux kernel, the drivers for the High IOPS drivers, and Spectrum Scale, including instructions on how to perform a rolling upgrade, where applicable.

� References to related documentation, pointing to important documentation from Lenovo, SAP, and SUSE.

The Lenovo SAP HANA Operations Guide is being optimized and extended continuously based on new developments and client feedback. The latest version of this document can be found in SAP Note 1650046.1

8.3 Interoperability with other platforms

To access the SAP HANA database from a system (SAP or non-SAP), the SAP HANA database client must be available for the platform on which the system is running. Platform availability of the SAP HANA database client is documented in the Product Availability Matrix (PAM) for SAP HANA, which is available at the following website (search for “HANA”):


At the time of this writing, the SAP HANA database client is available on all major platforms, including (but not limited to) the following platforms:

� Microsoft Windows Server 2008, Windows Server 2008 R2, and Windows Server 2012� Microsoft Windows Vista and Windows 7 (both 32-bit and 64-bit)� SUSE Linux Enterprise Server 11 on 32 and 64-bit x86 platforms, and IBM System z � SUSE Linux Enterprise Server 12 on 64-bit x86 platforms only� Red Hat Enterprise Linux 6 on 64-bit x86 platforms and IBM System z� IBM AIX V6.1, and V7.1 on the IBM POWER platform� IBM i V7R1 and V7R2 on the IBM POWER platform� HP-UX 11.31 on Itanium� Oracle Solaris 10 and 11 on x86 and SPARC

For up-to-date and detailed availability information, see this website:


If there is no SAP HANA database client available for a certain platform, SAP HANA can still be used in a scenario with replication by using a dedicated SAP Landscape Transformation server (for SAP Business Suite sources) or an SAP BusinessObjects Data Services server that is running on a platform for which the SAP HANA database client is available. By using

1 SAP Notes can be accessed at http://service.sap.com/notes. An SAP S-user ID is required.

Chapter 8. SAP HANA operations 213

http://service.sap.com/notes



this configuration, data can be replicated into SAP HANA, which then can be used for reporting or analytic purpose by using a front end that supports SAP HANA as a data source.

8.4 Monitoring SAP HANA

In a productive environment, the administration and monitoring of an SAP HANA appliance play an important role.

The SAP tool for the administration and monitoring of the SAP HANA appliance is the SAP HANA Studio, with which you can monitor the following aspects of the overall system state:

� General system information (such as software versions).

� A warning section that shows the latest warnings that are generated by the statistics server. Detailed information about these warnings is available as a tooltip.

� Bar views that provide an overview of important system resources. The amount of available memory, CPUs, and storage space is displayed, in addition to the amount of these resources that is used.

In a distributed landscape, the amount of available resources is aggregated over all servers.

8.5 Installing more agents

Many organizations have processes and supporting software in place to monitor, back up, or otherwise interact with their servers. Because SAP HANA is delivered in an appliance-like model, there are restrictions regarding extra software (for example, monitoring agents) to be installed on to the appliance. SAP permits the installation and operation of external software if the prerequisites that are described in SAP Note 1730928 are met.

Only the software that is installed by the hardware partner is recommended on the SAP HANA appliance. For the System x solution for SAP HANA, Lenovo defines the following categories of agents:

� Supported

Lenovo provides a solution that covers the respective areas; no validation by SAP is required.

� Tolerated

Solutions that are provided by a third party can be used on the Workload Optimized Solution for SAP HANA. It is the clients’ responsibility to obtain support for such solutions. Such solutions are not validated by Lenovo and SAP. If issues with such solutions occur and cannot be resolved, the use of such solutions might be prohibited in the future.

� Prohibited

These types of solutions must not be used on the Lenovo System x solution for SAP HANA. The use of these solutions might compromise the performance, stability, or data integrity of SAP HANA.

Note: For more information about the administration and monitoring of SAP HANA, see the SAP HANA Administration Guide, which is available at this website:




Do not install more software on the SAP HANA appliance that is classified as prohibited for use on the SAP HANA appliance. For example, initial tests show that some agents can decrease performance or possibly corrupt the SAP HANA database (for example, virus scanners).

In general, all added software must be configured to not interfere with the functions or performance of the SAP HANA appliance. If any issues with the SAP HANA appliance occur, you might be asked by SAP to remove all added software and to reproduce the issue.

For more information about the agents that are supported, tolerated, or prohibited for use on the SAP HANA appliance, see the Quick Start Guide that is available at this website:


8.6 Software and firmware levels

The Lenovo System x solution for SAP HANA appliance includes several components that might need to be upgraded (or downgraded) depending on different support organizations’ recommendations. These components can be split up into the following general categories:

� Firmware� Operating system� Hardware drivers� Software

The System x SAP HANA support team reserves the right to perform basic system tests on these levels when they are deemed to have a direct effect on the SAP HANA appliance. In general, Lenovo does not give specific recommendations about which levels are allowed for the SAP HANA appliance.

The Lenovo System x SAP HANA development team provides new images for the SAP HANA appliance at regular intervals. Because these images include dependencies regarding the hardware, operating system, and drivers, use the latest image for maintenance and installation of SAP HANA systems. These images can be obtained through Lenovo Support. Part number information is contained in the Quick Start Guide.

If the firmware level recommendations for the hardware components of the SAP HANA appliance are given through the individual System x support teams that fix known code bugs, it is the client’s responsibility to upgrade or downgrade to the recommended levels as instructed by System x Support.

If the operating system recommendations for the SUSE or RedHat Linux components of the SAP HANA appliance are given through the SAP, SUSE, RedHat, or Lenovo support teams that fix known code bugs, it is the client’s responsibility to upgrade or downgrade to the recommended levels, as instructed by SAP through an explicit SAP Note or allowed through an OSS Customer Message. SAP describes their operational concept, including updating of the operating system components, in SAP Note 1599888 - SAP HANA: Operational Concept. If the Linux kernel is updated, it is recommended that extra care is taken to recompile the High IOPS drivers and Spectrum Scale software as well, as described in the Lenovo SAP HANA Operations Guide (see 8.2, “Lenovo SAP HANA Operations Guide” on page 212).

If a High IOPS driver or Spectrum Scale recommendation to update the software is given through the individual support teams (System x, Linux, or Spectrum Scale) to fix code bugs, it is not recommend to update these drivers without first asking the Lenovo System x SAP HANA support team through an SAP OSS Customer Message.



If recommendations for upgrades of other hardware or software components of the SAP HANA appliance are given through the individual Support teams that fix code bugs, it is the client’s responsibility to upgrade or downgrade to the recommended levels as instructed by Support.

8.7 Support process

The deployment of SAP HANA as an integrated solution, which combines software and hardware from Lenovo and SAP, also is reflected in the support process for the Lenovo System x Solution for SAP HANA. All SAP HANA models that are offered by Lenovo include SUSE Linux Enterprise Server (SLES) for SAP Applications with SUSE 1-year or 3-year priority support or Red Hat Enterprise Linux for SAP HANA with 1-year or 3-year support and IBM Spectrum Scale with 1-year or 3-year support. The hardware features a 1-year or 3-year limited warranty, including customer-replaceable unit (CRU) and onsite support.

8.7.1 Lenovo and SAP integrated support

SAP integrates the support process with SUSE, RedHat, and Lenovo as part of the HANA appliance solution-level support. If you encounter software problems on your SAP HANA system, see the following SAP Online Service System (SAP OSS) website:

https://service.sap.com

At the website, create a service request ticket by using a subcomponent of BC-HAN or BC-DB-HDB as the problem component. Lenovo support works closely with SAP, SUSE, and RedHat and is dedicated to supporting SAP HANA software and hardware issues.

Send all questions and requests for support to SAP by using their OSS messaging system. A dedicated Lenovo representative is available at SAP to work on this solution. Even if it is clearly a hardware problem, an SAP OSS message should be opened to provide the best direct support for the Lenovo System x Solution for SAP HANA.

When an SAP support message is opened, use the text template that is provided in the Quick Start Guide when it is obvious that you have a hardware problem. This procedure expedites all hardware-related problems within the SAP support organization. Otherwise, the SAP support teams helps you with the questions regarding the SAP HANA appliance in general.

Lenovo provides a script to get an overview of the current system status and the configuration of the running system. The saphana-support-lenovo.sh script is preinstalled in the /opt/lenovo/saphana/bin directory. The most recent version can be found in SAP Note 1661146. Previously, the script was called saphana-support-ibm.sh and located in /opt/ibm/saphana/bin directory. Both scripts can coexist on one system.

Before you contact support, ensure that you take the following steps to try to solve the problem:

� Use the troubleshooting information in your system documentation and the diagnostic tools that is included with your system. Information about diagnostic tools is available in the Problem Determination and Service Guide on the Documentation CD that is included with your system.

� See the following support website to check for technical information, hints, tips, and new device drivers, or to submit a request for information:

http://www.ibm.com/supportportal


http://www.ibm.com/supportportal

https://service.sap.com

� For SAP HANA software-related issues, you can search the SAP OSS website for problem resolutions. The OSS website includes a knowledge database of known issues and can be accessed at the following website:

https://service.sap.com/notes

The main SAP HANA information source is available at the following website:

https://help.sap.com/hana_platform

If you have a specific operating system question or issue, contact SUSE regarding SUSE Linux Enterprise Server for SAP Applications. For more information, see the following SUSE website:

http://www.suse.com/products/prioritysupportsap

Media is available for download at the following website:

http://download.suse.com/index.jsp?search=Search&families=2658&keywords=SAP

If you are running RedHat Enterprise Linux for SAP HANA, contact your RedHat support representative or online at the following website:

https://access.redhat.com/home

8.7.2 Lenovo SAP Center of Competence

The Lenovo SAP Center of Competence (Lenovo SAP CoC) is the key support function of the Lenovo and SAP alliance. It serves as a single point of entry for all SAP-related questions for clients that are using System x hardware. As a managed question and answer service, it has access to a worldwide network of experts on technology topics about System x products in SAP environments. You can contact the Lenovo SAP CoC by using the following email address:

[email protected]

Note: Registration is required before you can download software packages from the SUSE website.

Note: The Lenovo SAP CoC does not provide product support. For more information about product support for the Lenovo System x solution for SAP HANA, see 8.7.1, “Lenovo and SAP integrated support” on page 216.

If you need support for other Lenovo products, consult the product documentation for more information about how to get support.


http://www.suse.com/products/prioritysupportsap

https://access.redhat.com/home


https://service.sap.com/notes


http://download.novell.com/index.jsp?search=Search&families=2658&keywords=SAP

http://download.novell.com/index.jsp?search=Search&families=2658&keywords=SAP

mailto:[email protected]

Appendix A. Additional topics

This appendix includes the following topics:

� Spectrum Scale license information� File-based backup with IBM Tivoli Storage Manager for ERP

A


A.1 Spectrum Scale license information

The models of the Lenovo System x solution for SAP HANA feature Spectrum Scale licenses, including three years of Software Subscription and Support. Software Subscription and Support contracts, including Subscription and Support renewals, are managed through IBM Passport Advantage or Passport Advantage Express.

The following types of Spectrum Scale licenses are available:

� The Spectrum Scale on x86 Single Server for Integrated Offerings license provides file system capabilities for single-node integrated offerings. This Spectrum Scale license does not cover the use in multi-node environments, such as the scale-out solution that is described here. To use the building blocks that are included with the Spectrum Scale on x86 Single Server for Integrated Offerings licenses for a scale-out solution, Spectrum Scale on x86 Server licenses or Spectrum Scale File Placement Optimizer licenses must be obtained for these building blocks.

� The Spectrum Scale Server license permits the licensed node to perform Spectrum Scale management functions, such as cluster configuration manager, quorum node, manager node, and network shared disk (NSD) server. In addition, the Spectrum Scale Server license permits the licensed node to share Spectrum Scale data directly through any application, service, protocol, or method, such as Network File System (NFS), Common Internet File System (CIFS), File Transfer Protocol (FTP), or Hypertext Transfer Protocol (HTTP).

� The Spectrum Scale File Placement Optimizer license permits the licensed node to perform NSD server functions for sharing Spectrum Scale data with other nodes that have a Spectrum Scale File Placement Optimizer or Spectrum Scale Server license. This license cannot be used to share data with nodes that have a Spectrum Scale Client license or non-Spectrum Scale nodes.

� The Spectrum Scale Client license permits exchange of data between nodes that locally mount the same file system (for example, through a shared storage). No other export of the data is permitted. The Spectrum Scale Client cannot be used for nodes to share Spectrum Scale data directly through any application, service, protocol or method, such as NFS, CIFS, FTP, or HTTP. For these functions, a Spectrum Scale Server license is required. Because of the architecture of the Lenovo System x solution for SAP HANA (not having a shared storage system), this type of license cannot be used for the solution.

Table A-1 lists the types of Spectrum Scale licenses and the processor value units (PVUs) that are included for each of the models.

Table A-1 Spectrum Scale licenses that are included in the custom models for SAP HANA

MTM Type of Spectrum Scale license included PVUsincluded

7147-H1x Spectrum Scale on x86 Server 1400






7147-HAx Spectrum Scale on x86 Single Server for Integrated Offerings 1400


Licenses for IBM Spectrum Scale on x86 Single Server for Integrated Offerings V3 (referred to as “Integrated” in the table) cannot be ordered independently of the select hardware for which it is included. This type of license provides file system capabilities for single-node integrated offerings. The model 7143-HAx includes 4000 PVUs of Spectrum Scale on x86 Single Server for Integrated Offerings V3 licenses; therefore, an upgrade to the 7143-HBx model does not require more licenses. The PVU rating for the 7143-HAx model to consider when other Spectrum Scale license types are purchased is 1400 PVUs.

Clients with highly available, multi-node clustered scale-out configurations must purchase the Spectrum Scale on x86 Server and Spectrum Scale File Placement Optimizer product.

A.2 File-based backup with IBM Tivoli Storage Manager for ERP

IBM Tivoli Storage Manager for ERP is a simple, scalable data protection solution for SAP HANA and SAP ERP. Tivoli Storage Manager for ERP V6.4 includes a one-step command that automates file-based SAP HANA backup and Tivoli Storage Manager data protection.

Tivoli Storage Manager clients that are running SAP HANA appliances can back up their instances by using their Tivoli Storage Manager backup environment, even if the level of the SAP HANA code does not allow use of the Backint interface for SAP HANA, and only a file-based backup tool integration can be used. Tivoli Storage Manager for ERP – Data Protection for SAP HANA V6.4 provides such file-based backup and restore functions for SAP HANA.

A.2.1 Setting up Data Protection for SAP HANA

Data Protection for SAP HANA features a setup.sh command, which is a configuration tool that prepares the Tivoli Storage Manager for ERP configuration file, creates the SAP HANA backup user, and sets all necessary environment variables for the SAP HANA administration user. The setup.sh command guides you through the configuration process. Data Protection for SAP HANA stores a backup user and its password in the SAP HANA keystore called hdbuserstore to enable unattended operation of a backup.

7147-HBx Spectrum Scale on x86 Single Server for Integrated Offerings 1400






7143-HAx Spectrum Scale on x86 Single Server for Integrated Offerings 4000

7143-HBx Spectrum Scale on x86 Single Server for Integrated Offerings 4000

7143-HCx Spectrum Scale on x86 Single Server for Integrated Offerings 5600

MTM Type of Spectrum Scale license included PVUsincluded

Appendix A. Additional topics 221

A.2.2 Backing up the SAP HANA database

SAP HANA writes its backup (logs and data) to files at pre-configured directories. The Data Protection for SAP HANA command (backup.sh) reads the configuration files to retrieve these directories (if the default configuration is not used). Upon backup, the files that are created in these directories are moved to the running Tivoli Storage Manager instance and are deleted afterward from these directories (except for the HANA configuration files).

This backup process is shown in Figure A-1.

Figure A-1 Backup process with Data Protection for SAP HANA that uses local storage for backup files

The backup process includes the following steps:

1. The backup.sh command triggers a log or data backup of the SAP HANA database.2. The SAP HANA database performs a synchronized backup on all nodes.3. The SAP HANA database writes a backup file on each node.4. The backup.sh command collects the file names of the backup files.5. The backup files are moved to Tivoli Storage Manager (and deleted on the nodes).

Tivoli Storage ManagerStorage

node03node01 node02



SAP HANA DB

DB partition 3

data03 + log03First replica

Backup files backup backup

backup.sh

12 2 2

3 3 3

4

TivoliStorageManagerServer

backup restore

5move files to TSM

local storage

data01 + log01

backup

data02 + log02



Instead of having the backup files of the individual nodes written to the local storage of the nodes, an external storage system can be used to provide space to store the backup files. All nodes must access this storage; for example, by using NFS, as shown in Figure A-2.

Figure A-2 Back up process with Data Protection for SAP HANA using external storage for backup files

Running log and data backups requires the Data Protection for SAP HANA backup.sh command to be run as the SAP HANA administration user (“<sid>adm”).

The backup.sh command provides the following basic functions:

� Completes the data-backup (including HANA instance and landscape configuration files).� Completes log-backup and removes successfully saved redo log files from disk.

The functions can be selected by using the following command-line arguments to schedule the backup script with a specific parameter:

� backup.sh --data

Performs complete data and configuration file backup.

� backup.sh --logs

Performs complete log backup followed by a “LOG RECLAIM”.

By using this command, a backup of the SAP HANA database into Tivoli Storage Manager can be fully automated.

A.2.3 Restoring the SAP HANA database

The SAP HANA database requires the backup files to be restored to start a recovery process by using the SAP HANA Studio. For SAP HANA database revisions 30 and higher, Data Protection for SAP HANA provides a restore.sh command that moves all required files back to the file system location automatically so that the user is not required to search these files manually.

SAP HANAbackup file storage

node03node01 node02



SAP HANA DB

DB partition 3

First replica

backup backup backup

backup.sh

12 2 2

3

4

5move files to TSM

3 3

Tivoli Storage ManagerStorage

TivoliStorageManagerServer

backup restore

data03 + log03

local storage

data01 + log01 data02 + log02



For earlier revisions of the SAP HANA database, this task must be done manually by using the Tivoli Storage Manager BACKUP-Filemanager. The SAP HANA database expects the backup files to be restored to the same directory as they were written during backup. The recovery can then be triggered by using SAP HANA Studio.

To restore data backups, including SAP HANA configuration files and log file backups, the Tivoli Storage Manager BACKUP-Filemanager is used. A sample panel of the BACKUP-Filemanager is shown in Figure A-3.

Figure A-3 The BACKUP-Filemanager interface

BACKUP-Filemanager V6.4.0.0, Copyright IBM 2001-2012 .------------------+---------------------------------------------------------------. | Backup IDs | Files stored under TSM___A0H7K1C4QI | |------------------+---------------------------------------------------------------| | TSM___A0H7KM0XF4 | */hana/log_backup/log_backup_2_0_1083027170688_1083043933760 | | TSM___A0H7KLYP3Z | */hana/log_backup/log_backup_2_0_1083043933760_1083060697664 | | TSM___A0H7KHNLU6 | */hana/log_backup/log_backup_2_0_1083060697664_1083077461376 | | TSM___A0H7KE6V19 | */hana/log_backup/log_backup_2_0_1083077461376_1083094223936 | | TSM___A0H7K9KR7F | */hana/log_backup/log_backup_2_0_1083094223936_1083110986880 | | TSM___A0H7K7L73W | */hana/log_backup/log_backup_2_0_1083110986880_1083127750848 | | TSM___A0H7K720A4 | */hana/log_backup/log_backup_2_0_1083127750848_1083144513792 | | TSM___A0H7K4BDXV | */hana/log_backup/log_backup_2_0_1083144513792_1083161277760 | | TSM___A0H7K472YC | */hana/log_backup/log_backup_2_0_1083161277760_1083178040064 | | TSM___A0H7K466HK | */hana/log_backup/log_backup_2_0_1083178040064_1083194806336 | | TSM___A0H7K1C4QI | */hana/log_backup/log_backup_2_0_1083194806336_1083211570688 | | TSM___A0H7JX1S77 | */hana/log_backup/log_backup_2_0_1083211570688_1083228345728 | | TSM___A0H7JSRG2B | */hana/log_backup/log_backup_2_0_1083228345728_1083245109824 | | TSM___A0H7JOH1ZP | */hana/log_backup/log_backup_2_0_1083245109824_1083261872960 | | TSM___A0H7JK6ONC | */hana/log_backup/log_backup_2_0_1083261872960_1083278636608 | | TSM___A0H7JJWUI8 | */hana/log_backup/log_backup_2_0_1083278636608_1083295400384 | | TSM___A0H7JJU5YN | */hana/log_backup/log_backup_2_0_1083295400384_1083312166016 | | TSM___A0H7JFWAV4 | */hana/log_backup/log_backup_2_0_1083312166016_1083328934016 | | TSM___A0H7JBG625 | */hana/log_backup/log_backup_2_0_1083328934016_1083345705856 | | TSM___A0H7JBAASN | */hana/log_backup/log_backup_2_0_1083345705856_1083362476352 | | TSM___A0H7J7BLDK | */hana/log_backup/log_backup_2_0_1083362476352_1083379244416 | | TSM___A0H7J5U8S7 | */hana/log_backup/log_backup_2_0_1083379244416_1083396008064 | | TSM___A0H7J5T92O | */hana/log_backup/log_backup_2_0_1083396008064_1083412772928 | | TSM___A0H7J4TWPG | */hana/log_backup/log_backup_2_0_1083412772928_1083429538688 | | | */hana/log_backup/log_backup_2_0_1083429538688_1083446303424 | | | */hana/log_backup/log_backup_2_0_1083446303424_1083463079488 | | | */hana/log_backup/log_backup_2_0_1083463079488_1083479846528 V |------------------+---------------------------------------------------------------| | 24 BIDs | 190 File(s) - 190 marked | `------------------+---------------------------------------------------------------' TAB change windows F2 Restore F3 Mark all F4 Unmark allF5 reFresh F6 fileInfo F7 redireCt F8 Delete F10 eXit ENTER mark file


Data and log backups can be selected and then restored to the wanted location. If no directory is specified for the restore, the BACKUP-Filemanager restores the backups to the original location from which the backup was done.

After the backup files are restored, the recovery process must be started by using SAP HANA Studio. For more information about this process and the various options for a recovery, see the SAP HANA Backup and Recovery Guide, which is available this website:


After successfully completing the recovery process, the backup files must be removed manually from the disk if the files are no longer needed.



ronyms

ABAP Advanced Business Application Programming

ACID atomicity, consistency, isolation, durability

APO Advanced Planner and Optimizer

BI Business Intelligence

BICS BI Consumer Services

BM bridge module

BW Business Warehouse

CD compact disc

CPU central processing unit

CRC cyclic redundancy checking

CRM customer relationship management

CRU customer-replaceable unit

DB database

DEV development

DIMM dual inline memory module

DSO DataStore Object

DR disaster recovery

DXC Direct Extractor Connection

ECC ERP Central Component

ECC error checking and correcting

ERP enterprise resource planning

ETL extract, transform, and load

FTSS Field Technical Sales Support

GB gigabyte

GPFS General Parallel File System

GTS Global Technology Services

HA high availability

HDD hard disk drive

HPI Hasso Plattner Institute

I/O input/output

IBM International Business Machines

ID identifier

IDs identifiers

IMM integrated management module

IOPS I/O operations per second

JDBC Java Database Connectivity

Abbreviations and ac

© Copyright Lenovo 2013, 2014, 2015, 2016. All rights reserve

JRE Java Runtime Environment

KPIs key performance indicators

LM landscape management

LUW logical unit of work

MB megabyte

MCA Machine Check Architecture

MCOD Multiple Components in One Database

MCOS Multiple Components on One System

MDX Multidimensional Expressions

NOS Notes object services

NSD Network Shared Disk

NUMA non-uniform memory access

ODBC Open Database Connectivity

ODBO OLE DB for OLAP

OLAP online analytical processing

OLTP online transaction processing

OS operating system

OSS Online Service System

PAM Product Availability Matrix

PC personal computer

PCI Peripheral Component Interconnect

POC proof of concept

PSA Persistent Staging Area

PVU processor value unit

QA quality assurance

QPI QuickPath Interconnect

RAID Redundant Array of Independent Disks

RAM random access memory

RAS reliability, availability, and serviceability

RDS Rapid Deployment Solution

RHEL Red Hat Enterprise Linux

RPM revolutions per minute

RPO Recovery Point Objective

RTO Recovery Time Objective

SAN storage area network

SAPS SAP Application Benchmark Performance Standard

d. 227

SAS serial-attached SCSI

SATA Serial ATA

SCM supply chain management

SCM software configuration management

SD Sales and Distribution

SDRAM synchronous dynamic random access memory

SLD System Landscape Directory

SLES SUSE Linux Enterprise Server

SLO System Landscape Optimization

SMI scalable memory interconnect

SQL Structured Query Language

SSD solid-state drive

SSR SAP HANA System Replication

STG Systems and Technology Group

SUM Software Update Manager

TB terabyte

TCO total cost of ownership

TCP/IP Transmission Control Protocol/Internet Protocol

TDMS Test Data Migration Server

TREX Text Retrieval and Information Extraction

UEFI Unified Extensible Firmware Interface


Related publications

The publications that are listed in this section are considered suitable for a more detailed description of the topics that are covered in this book.

Lenovo Press publications

The following Lenovo Press publications provide more information about the topics in this document. Some publications that are referenced in this list might be available in softcopy only:

� Lenovo System x3850 X6 and x3950 X6 Planning and Implementation Guide, SG24-8208

You can search for, view, download, or order these documents and other Lenovo Press publications, papers, drafts, and other materials, at the following website:

http://lenovopress.com

Online resources

The following websites are also relevant as further information sources:

� System x solution for SAP HANA:

http://shop.lenovo.com/us/en/systems/solutions/alliances/sap/#tab-sap_hana

� SAP In-Memory Computing - SAP Help Portal:

http://help.sap.com/hana

Help from Lenovo

Lenovo support and downloads:

http://www.lenovo.com/support

Lenovo services:

http://www.lenovo.com/services



http://www.lenovo.com/services

http://lenovopress.com

http://shop.lenovo.com/us/en/systems/solutions/alliances/sap/#tab-sap_hana

http://help.sap.com/hana


In-mem

ory Computing w

ith SAP HANA on Lenovo X6 Systems

BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE

At Lenovo Press, we

bring together experts

to produce technical

publications around

topics of importance to

you, providing

information and best

practices for using

Lenovo products and

solutions to solve IT

challenges.

lenovopress.com SG24-8086-03

In-memory Computing with SAP HANA on Lenovo X6 Systems

Introduces System x solution for SAP HANA

Explores SAP HANA features and use cases

Discusses operational aspects for SAP HANA

Describes SAP HANA high availablity and disaster recovery scenarios

The forth edition of this Lenovo Press publication describes in-memory computing appliances from Lenovo and SAP that are based on Lenovo eX5™ and X6 flagship systems and SAP HANA. It covers the basic principles of in-memory computing, describes the Lenovo eX5 and X6 hardware offerings, and explains the corresponding SAP HANA IT landscapes using these offerings.

This book also describes the architecture and components of the Lenovo System x® solution for SAP HANA. The SAP HANA operational disciplines are explained in detail: Scalability options, high availability and disaster recovery, backup and restore, and virtualization possibilities for SAP HANA appliances.

This book is intended for SAP administrators and technical solution architects. It is also for Lenovo Business Partners and Lenovo employees who want to know more about the SAP HANA offering and other available Lenovo solutions for SAP clients.

Back cover

sap hana on lenovo x6 systems - suse.com · 3.1.8 sap hana lifecycle management ... 5.1.6 x3850 x6...

Documents