151375083-sap-hana-architecture.pdf

SAP HANA Appliance 1.0

RKT for Applications Consultants

© SAP AG 2011

Agenda

SAP HANA Appliance 1.0Lesson 1: Introductions

Lesson 3: Architecture

Lesson 2: Overview

Lesson 4: Data Provisioning

Lesson 5: Modeling

Lesson 6: Reporting

Lesson 7: User Management

Lesson 8: Scenario: CO-PA

Lesson Objectives

© SAP AG 2011

After completing this lesson, you will be able to understand the:Architecture of the SAP HANA Appliance1.0Concept of MVCC - Multi Version Concurrency ControlConcept of Column Store Concept of Row Store Persistance Layer of SAP HANA Appliance1.0Concept of Backup & Recovery

SAP HANA ApplianceERP

Architecture OverviewSAP HANA Appliance and Surroundings

LogERP DB

Clients BI4 Explorer

Dashboard Design

SAP BI4 universes (WebI,...)

MS Excel

BI4 Analysis

SAP HANA Database

Request Processing / Execution ControlSQL Parser MDXSQL Script Calc Engine

Transaction Manager

Session Management

Relational EnginesRow Store Column Store

Persistence LayerPage Management Logger

Disk StorageLog VolumesData Volumes

Authorization Manager

Metadata Manager

SAP HANA Studio

Administration Modeling

Load Controller

Replication Agent

Replication Server

SAP Business Objects BI4

Data Services Designer

SBO BI4 servers

( program for client)

SBO BI4 Information Design Tool

Other Source Systems

SAP NetWeaver

BW3rd Party

Data Services

SLT Add-on

© 2011 SAP AG. All rights reserved. 13Confidential

SAP HANA Appliance – Real Time Replication

Landscape Option 1: (SAP ERP 4.6c or SAP ECC 6.0 on a NW release below NW ABAP 7.02)

For any customers on SAP ERP 4.6c, utilizing SLT will require setting up an intermediary system with at least NW ABAP 7.02 load controller for replication into HANA

For example, a solution manager system could be used for the SLT Add-on

Supports Non-Unicode or MDMP systems for SAP ERP as long as SLT is installed on a NW 7.02 Unicode system

Landscape Option 2: (SAP ECC 6.0+, running on at least NW ABAP 7.02)

For any customers on SAP ECC 6.0+ running on NW ABAP 7.02 (must be fully Unicode), utilizing SLT can be done directly for replication into HANA


LT Replication Concept: Trigger-Based ApproachArchitecture and Key Building Blocks

SAP HANA systemLT Replication Server

Application Tables

Source system

Write Modules

Controler Modules

DBConnection

RFCConnection

Read Modules

LoggingTables

Application Tables

Efficient initialization of data replication based on DB trigger

and delta logging concept(as with NearZero downtime approach)

Flexible and reliable replication process, incl. data migration

(as used for TDMS and SAP LT)

Fast data replication via DB connectLT replication functionality is fully integrated with HANA Modeler UI

LT replication server does not have to be a separate SAP system and can run on any SAP system with SAP

NetWeaver 7.02 ABAP stack (Kernel 7.20EXT)

DB Trigger


SAP HANA Appliance – Data Services

SAPERP

Any Source

BWData Load

Metadata

Repository Server

Open Hub

SAP BusinessObjectsData Services 4.0

HANA

Designer and Management

Console


SAP ECC

SAP HANA as Accelerator for SAP ApplicationsExample: CO-PA Accelerated by SAP HANA

Aggregation Levels

COPA Application

Redundant copy of ECC tables

Data duplicated into SAP HANARead interfaces accesses SAP HANA if available

Aggregation EngineAnalytic indexes

Calculation Engine

DBMS

SAP ECC on legacy DB with secondary DB Connection into HANA

SAP HANA Database

This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.

Read Interface

Primary DB connectionRead / write

Secondary DB connectionRead / write

SAP GUI

Drill-down reporting

BI 4.0 (optional)

Analytics


SAP HANA as Primary DB in Application ServerFirst example: BW 7.30 running on HANA …expected soon

DBMS

SAP NetWeaver BW

Metadata

DW Management Analytic

Engine

Single Data Management for Row- and Column based storage

Integrated engine for data management and in-memory processing of analytical capabilitiesPure DB conversion. No re-implementation required.

Aggregation Engine

Data Store Objects

Analytic indexes

Master Data

InfoCube Index

Calculation Engine

DBMS

SAP NetWeaver BW as a HANA based system (with “built-in” BWA)

HANA

Incl

udes

SA

P N

etW

eave

r B

W A

ccel

erat

or fu

nctio

nalit

y

This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.

SLT Replication Concept: Trigger-Based ApproachArchitecture and Key Building Blocks

SAP HANA systemSLT system (NW7.02)

Application Tables

Source system

Write Modules

Controler Modules

DBConnection

RFCConnection

Read Modules

LoggingTables

Application Tables

DB Trigger

Initialization of data replication based on DB trigger

and delta logging concept(as with NearZero downtime

approach)

Flexible and reliable replication process, incl. data conversion

(as used for TDMS and ByDesign) SLT can also be installed on source

system or Solution Manager

Fast data replication via DB connection

(no other interaction)


HANA 1.0 Appliance ModelSizing etc


SAP HANA Appliance ModelHow is HANA delivered

SAP HANA only comes as an applianceCollaboration with strong hardware partners

Partners create servers based on certified hardwareNo other hardware supportedProductive Systems: only one instance of SAP HANA per physical serverSAP Customer Messages are single point of entry for all support questionsSAP support collaborates with Partners’ support organizations

What is installed on the Server?Operating System: SuSE Linux Enterprise Server 11

responsibility of Hardware PartnerSAP HANA Database SoftwareAdditional SAP software, e.g. SAP Host Control3rd party programs/agents, e.g. monitoring agents, backup agents, …

for each hardware partner there is a certified set of such software


HANA Appliance sizing guidelines(Scale Up configurations - Rack Servers)

CPU core to Memory ratio 16 GB / CPU core

Memory requirements Column store: 2 x data storedRow store: 1.25 x data stored1

Storage requirements(Local storage)

Log volume : 1 x total memory (PCIe-Flash / SSD) 2Data volume: 4 x total memory (SAS drives / SSD) 3Stable Queue for SYRS : 100 – 500 GB on Data volume

Network requirementsDedicated 1 GB networks between HANA & ERP , HANA & BI Platform, HANA & data sources Redundant infrastructure for Fail over

1. 25% extra buffer to manage unicode conversion, initial R3 data loads etc..2. Log volumes need to have 100K i/o per second to prevent slow boot up or writing logs 3. Data volumes need to have at a minimum 10K spindles with sequential read speeds of 800 MB - 1 GB / sec


HANA Appliance sizing guidelines(Scale-out configurations - Clustered Rack Servers)

CPU core to Memory ratio 16 GB / CPU core

Memory requirements Column store: 2 x data storedRow store: 1.25 x data stored1

Storage requirements (External storage)

Log Volume : 1 x total memory (SSD RAID) 2

Data Volume: 4 x total memory ( SAS / FC drives) 3

Stable Queue for SYRS: 10 x hourly data loads

Network requirements10 GB n/w trunked for providing dedicated n/w as above Dedicated 10 GB n/w between HANA nodes & storage Redundant infrastructure for Failover

1. 25% extra buffer to manage unicode conversion, initial R3 data loads etc..2. Log volumes need to have 100K i/o per second to prevent slow boot up or writing logs 3. Data volumes need to have at a minimum 10K spindles with sequential read speeds of 800 MB - 1 GB / sec


• 2 x 8 core Intel Nehalem EX ( 2 socket system) • 128 GB Main memory • 160 GB PCIe-Flash / SSD for Log volume• 1 TB SAS / SSD for Data volume• 3 x 1 GB n/w or 1 x 10GB n/w (trunk) • Redundant n/w infrastructure

• Uncompressed Data ~ 256 GB to ~500 GB

• Replication Data load5GB / hr

• 60,000 USD to 75,000 USD

• 2 x 8 core Intel Nehalem EX ( 2 or 4 sockets system) • 256 GB Main memory • 320 GB PCIe-Flash / SSD for Log volume• 1 TB SAS / SSD for Data volume• 3 x 1 GB n/w or 1 x 10GB n/w (trunk) • Redundant n/w infrastructure

• Uncompressed Data ~ 500 GB to ~1.25TB

• Replication Data load5GB / hr

• 85,000 USD to 125,000 USD

• 4 x 8 core Intel Nehalem EX ( 4 or 8 socket system) • 512 GB Main memory • 640 GB PCIe-Flash / SSD • 2 TB SAS / SSD for Data volume• 3 x 1 GB n/w or 1 x 10GB n/w (trunk) • Redundant n/w infrastructure

• Uncompressed Data~1.25TB to ~2.5 TB

• Replication Data load5GB - 20 GB/ hr

• 125,000 USD to 200,000 USD

HANA Appliance T-shirt sizes Specifications & approximate data volumes

M

S

XS


• 8 x 8 core Intel Nehalem EX • 1 TB Main memory • 1.2 TB PCIe-Flash / SSD • 4 TB SAS / SSD for Data volume• 3 x 1 GB n/w or 1 x 10GB n/w (trunk) • Redundant n/w infrastructure

• Uncompressed Data~ 2.5TB to ~5TB

• Replication Data load5GB – 20 GB / hr

• 235,000 USD to 300,000 USD

• Cluster multiple Large Racks • SSD based shared storage for Log Volume• High speed SAS / FC drives based storage for Data volume • 10 GB n/w for BI clients, Data access & Replication• 10 GB n/w between clustered nodes & storage• Redundant n/w infrastructure

• Uncompressed Data> 5 TB

• Replication Data load> 20 GB / hr

• TBD

HANA Appliance T-shirt sizes Specifications & approximate data volumes

XL

Clus

ter e

nv.

L

MVCC: Multi Version Concurrency Control

© SAP AG 2011

Time

T1 Read D

All transactions want to access the same entry with value D

T5 Read ???

T6 Read A

T2 Read D

T3 Read D

T4 Change D -> A

There must be a mechanism which synchronizes theses accesses

Commit


update personalset name = ‘Jones'

where userid = '4711'and name =

'McEwen'

select *from personal

where name like 'M%'

PNR NAME

T1

T2

MVCC transaction level Snapshot

4711 McEwen 5200 Miller

Version n

MVCC statement level Snapshot

commit



PNR NAME

T3



5200 Miller

PNR NAME



PNR NAME



© SAP AG 2011


Transaction ID (TID)

Commit ID (CID)

• identifies the starting point of write transactions• is increased whenever a write transaction starts• is increased when a read transaction is transformed into a write transaction• is used by COLUMN store for version consolidation

the TID is assigned to write transactions as its unique identifier

• reflects the commit sequence of write transactions• is equivalent to a commit timestamp• maximum CID -> last recent Commit ID• is used by ROW store for version consolidation

When a transaction is committed the max. CID is increased and the new value is assigned to the committed transaction

© SAP AG 2011

© SAP AG 2011

Agenda

Lesson 3:

Architecture

Multi Version Concurrency Control MVCC

Row Store

Column Store

Persistence Layer

Backup & Recovery

Row Store ArchitectureRow store block diagram

Row Store Block DiagramTransactional version memory

Contains temporary versionsNeeded for Multi-Version Concurrency Control (MVCC)

SegmentsContain the actual data (content of row-store tables) in pages

Page ManagerMemory allocationKeeping track of free/used pages

Version Memory ConsolidationThink ‘garbage collector for MVCC’

Persistence LayerInvoked in write operations (log)And in performing savepointscheckpoint writer

© SAP AG 2011

Row Store ArchitectureRow store block diagram

Write operationsMainly go into “Transactional Version Memory”“INSERT” also writes to Persisted Segment

Read Operations

Write Operations

Transactional Version Memory

Main Memory

Persisted Segment

Data that may be

seen by all active

transactions

Recent versions of changed records

Version Memory Consolidation

Version ConsolidationMoves “visible version” from Transaction Version Memory into Persisted Segment (based on Commit ID)Clears “outdated” record versions from Transactional Version Memory

Memory handlingRow store tables are linked list of memory pagesPages are grouped in segmentsPage size: 16 kB

Persisted SegmentContains data that may be seen by any ongoing transaction

© SAP AG 2011

Indexes for Row Store Tables

Each row-store table has a primary indexPrimary index maps the ROW ID and the primary key of a tableROW ID:

Each record has an identifier called ROW IDContains the number of the segment and the slot’s offset in this segment

How to find the memory page for a table record?Access on primary index delivers the Row IDThe ROW ID tells about the Segment and the Page offset where the record is located in memory

Secondary indexes can be created if needed

Persistence of indexes in row storeIndexes in row store only exist in memory

No persistence of index dataIndex definition stored with table metadataIndexes filled on-the-fly when system loads tables into memory on system start-up

Primary Index / Row ID / Index Persistence

© SAP AG 2011

© SAP AG 2011

Agenda

Lesson 3:Lesson 3:

Architecture


Row Store

Column Store

Persistence Layer

Backup & Recovery

In-Memory Computing Engine (IMCE)Column Store Block Diagram

© SAP AG 2011

column store optimizer & OLAP optimizer • create physical execution plans• column store optimizer is cost based• OLAP Optimizer is rule based

column store execution control• some column store operation can be parallelized

delta storage• exists only in main memory• change operations are stored here

main storage• data structure that contains the main part of the data• efficient compressed data

history storage • contains data versions which are not current• normal queries don’t access history data

Load/Unload agent• loading data from persistency layer• preloaded on startup or loaded on demand

Column Store

Storage separation (Main & Delta)Enables high compression and high write performance at the same time

High lights

Delta merge operationTo move changes in delta storage into the main storage because main storage is read optimized...Happens asynchronously. (Several events to trigger it.)

Read Operations

Write Operations

Main

Main Memory

Delta

Write optimized

Compressed and

Read optimized

Read operationsAlways have to read from both main & delta storages and merge the results.IMCE uses multi version concurrency control (MVCC) to ensure consistent read operations.

Data compression in Main storage

Compression by creating dictionary and applying further compression methodsSpeed up

Data load into CPU cacheEquality check Search

The compression is computed during delta merge operation.

Write operationsOnly in delta storage because write optimized.

The update is performed by inserting a new entry into the delta storage.

© SAP AG 2011

Row Store vs. Column Store

Modeling only possible for column tablesThis answers the frequently asked question:“where should I put a table – row store or column store”

Modeller only works with column tablesReplication server, Data Services and SLT creates tables in column store per defaultSQL to create column table: “CREATE COLUMN TABLE ...”

(HANA) System tables are created where they fit bestAdministrative tables in Row Store:

Schema SYS Caches, administrative tables of IMCE...Tables from statistics server

Administrative tables in column store:Schema _SYS_BI metadata of created views + Master data for MDXSchema _SYS_BIC Some generated tables for MDXSchema _SYS_REPO e.g. Lists of active/modified versions of models.

When to use which store

© SAP AG 2011

Persistence Layer in In-memory Comp. Engine

Why does an in-memory database need a persistence layerMain Memory is volatile. What happens upon

Database restart?Power outage?...

Data need to be stored in a non-volatile way

Backup and restore

IMCE offers one persistence layer which is used by row store and column storeRegular “savepoints” full persisted image of DB at time of savepointLogs capturing all DB transactions since last savepoint (redo logs and undo logs written)

restore DB from latest savepoint onwardsCreate Snapshots ( backup)

Purpose and Scope

© SAP AG 2011

SAP In-Memory Computing Engine

Persistence Layer in In-memory Comp. Engine

Memory

Data

Persistent Storage

Regular automatic savepoints

Information aboutdata changes

LogVolume

DataVolumes

© SAP AG 2011

Data is saved to disk in intervals

Persistence Layer in In-memory Comp. EngineSavepoint – writing data in IMCE

© SAP AG 2011

DATA&

Undo

DATA&

Undo

Redo Log

Page Buffer

Log queueData Cache Other

Data PagesData pages of virtual files

Savepoint Coordinator

DATA&

UndoDATA

&Undo

4712

4713

Converter

Persistence Layer in In-memory Comp. EngineSystem Restart

© SAP AG 2011

Reboot or Power failure deletes in-memory dataSystem is normally restarted („lazy“ restart to keep downtime short: tables with preload flag +

subsequently requested tables are loaded first)System is restored to the state just before the failure (except non-committed transactions)

Used for recovery:Last data savepointLog between the last data savepoint and the time of failure(contains the data changes of all commited transactions up to that point)

Time

Data savepointto persistent storage

1Log written

to persistent storage(committed transactions)

2Power failure

3

Backup & Recovery

Data backup:From persistent storage to external backup destinationsUsing database functions (SAP in-memory computing studio)

Log backup:Not supported in SAP HANA 1.0 SPS2Planned for Hana 1.0 SPS3

Configuration backupManual copy of configuation files to external backup destination

DDData Backup

Persistent Storage

LogVolume

DataVolumes

conf

Configuration Backup

Save to External Backup Destinations

© SAP AG 2011

Backup & Recovery

Disk failure (data volumes are damaged)System is restored to the state just before the failure (except non-committed transactions)Used for recovery:

Last data backupLog since the last data backupAssumption: log area undamaged, all log entries still available (not yet overwritten)

Time

Data backupto external backup

destination

1Log written

to persistent storage(committed transactions)

2Disk failure

(data volumes)

3

Recovery scenario – Disk Failure (Data Volume)

© SAP AG 2011

Data Backup

Log Backup

Recovery to last Data Backup

Point in Time Recovery

Backup & Recovery

SAP HANA 1.0

Recovery to status before crash ( )If log is not damaged

Feature Overview

© SAP AG 2011

© SAP AG 2011

Summary

In this lesson, you learned about the:Architecture of HANA 1.0Concept of MVCCConcept of Column Store Concept of Row Store Persistance Layer of HANA 1.0Concept of Backup & Recovery

© SAP AG 2011

Copyright 2011 SAP AGAll rights reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned and associated logos displayed are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence.The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages

Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind.SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Services sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit. Alle anderen in diesem Dokument erwähnten Namen von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zu Informationszwecken. Produkte können länderspezifische Unterschiede aufweisen.

Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderen Vereinbarung mit SAP. Dieses Dokument enthält nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP®-Produkts und ist für SAP nicht bindend, einen bestimmten Geschäftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien. SAP garantiert nicht die Richtigkeit oder Vollständigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewähr, weder ausdrücklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschließlich, hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zweck sowie für die Gewährleistung der Nichtverletzung geltenden Rechts.SAP übernimmt keine Haftung für Schäden jeglicher Art, einschließlich und ohne Einschränkung für direkte, spezielle, indirekte oder Folgeschäden im Zusammenhang mit der Verwendung dieser Unterlagen. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit.Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Die Informationen, auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinks zugreifen, unterliegen nicht dem Einfluss von SAP, und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen über Internetseiten Dritter ab.Alle Rechte vorbehalten.

151375083-sap-hana-architecture.pdf

Documents

sap erp

sap applicationsexample

confidential sap eccsap

sap ltfast data replication

separate sap system

agendasap hana appliance

copalesson objectives

nw abap