CIBER creates next-generation IT infrastructure solution for SAP hosting services

IBM SAP International Competence Center

“XIV is simply amazing. The usage of XIV real time copy, which is able to copy many terabytes of data very fast, reduces the downtime for provisioning drastically. The main focus as a service provider is the sector of customer satisfaction, which is especially improved by the usage of the new XIV system.”

Markus Peltz Data center Manager CIBER Managed Services GmbH

“Through the 100 percent virtualization you gain a dynamic infrastructure and you get the ability for an easy billing system.”

Carl-Philipp Müller Lead Sales Operations Services CIBER Managed Services GmbH

CIBER creates next-generation IT infrastructure solution for SAP hosting services

About this paper

CIBER Managed Services GmbH built a dynamic Cloud Infrastructure for SAP landscapes, based on a combination of IBM server,

storage, database, and system management software. The new storage infrastructure, which consists of the IBM XIV Storage System

and IBM SAN Volume Controller (SVC), offers individual service classes to suit different customer needs, maximizing operational

efficiency and minimizing expenses. The SAP application landscapes run on both IBM Power Systems servers and IBM System x

Blade Server technologies.

CIBER creates next-generation

IT infrastructure solution for

SAP hosting services

IBM SAP International Competence Center

“IBM was chosen as a partner, because of the very good long-term experience with them. Especially their integrated concepts and their highly skilled consultants for very special challenges, plus the capability to implement the required standards within time and budget convinced us to do this project with IBM.”

Mats Johansson, Head of Operations, CIBER Managed Services GmbH

4

Customer Objectives

Provide a highly scalable and dynamic infrastructure to host •customer-oriented SAP landscapes for small- and medium-

sized businesses in a private cloud environment that will

support the future growth of the business

Establish high-performance infrastructure, featuring •transparent and easy billing models for high customer

satisfaction

Reduce CIBER’s IT operational costs without sacrificing •service quality

Operate a complete range of SAP Business Suite •components on an integrated and shared storage

infrastructure and powered by either IBM Power servers or

IBM System x BladeServer technology depending on

customer requirements

Introduce reliable and comprehensive backup, resilience, •and disaster recovery solutions

Increase the speed of new SAP application provisioning, to •help respond faster to business requests.

Customer Benefits

Ability to provide faster and more flexible and scalable •SAP hosting services to businesses, with 99.9 percent

virtualization and improved Service Level Agreements.

Improved business continuity and higher availability even •across two locations, using IBM SVC storage virtualization

and the IBM XIV Storage System

Reduced workload for IT staff through simplified setup and •administration of the XIV Storage Systems

Reduced operating costs with improved service quality •supported by three levels of storage service classes, an

integrated and virtualized storage architecture, and new

server hardware

Reduced time for provisioning and deployment of new •systems from weeks to hours with the new storage solution

based on IBM XIV technology, using the XIV Snapshot

functionality and the SVC FlashCopy functionality.

Improved database performance with DB2 by a factor of •eight in online operations

Achieved higher level of automation in database •environment

Gained disk space savings and reduced data growth by •deploying DB2 compression and the reclaimable Storage

features (reduced size of tablespaces from about 1 TB

down to 300 MB for SAP NetWeaver Business Warehouse)

The BladeCenter technology allows an option to upgrade •online to current BladeServer models, such as HX5 and

also offers hot swap technology for blades to ensure high

availability with the BladeCenter Open Fabric Manager, if

needed.

IBM Solution

Worked with IBM and SVA, an IBM Premier Business Partner, •to design and implement a highly flexible and cost-efficient

infrastructure and a next-generation storage solution for SAP

in only four months

Set up two IBM Power 560 servers, each with two POWER6 •processors with four cores, and two IBM Power 750 servers,

each with four POWER7 processors with eight cores, running

SAP applications on IBM AIX in about 40 LPARs, sharing

processor resources and direct mappings of LPARs to

LUNs. The two POWER7 machines provide 170,000 SAPS

and process 12,500 sales order positions per hour between

them

Installed two BladeCenter H, both with six HS22 •BladeServers powered by quad-core 2.40 GHz Intel Xeon

L5530 processors, and four HS22 Blade servers powered by

six-core 2.60 GHz Intel Xeon X5640 processors, running

VMware vSphere 4, forming several SAP clusters and

hosting guest operating systems Novell SUSE Linux

Enterprise Server 10 and 11 as well as Windows Server 2003

and 2008

DB2 for Linux, UNIX and Windows is the most appropriate •database platform to support the SAP applications at CIBER

Installed six IBM System x3650 M2 systems, powered by •quad-core 2.26 GHz Intel Xeon Processor L5520 series

processors, running Novell SUSE Linux Enterprise Server

10 and 11 as well as Windows Server 2003 and 2008

Implemented a next generation three service class •storage solution based on IBM SAN Volume Controller 5.x,

IBM XIV Storage System and IBM System Storage DS3400

Implemented IBM Tivoli Storage Manager and two IBM •System Storage TS3500 tape library systems, each with

five LTO4 tape drives, 260 slots, and about 200 cartridges

(migration speed to tape of 350 GB/h).

5

Application Services provide comprehensive SAP-functional

and business process-related support. With Applications

Services, CIBER supports customers in their daily business.

Furthermore Application Services support the continuous

improvement process. Additionally, CIBER renders SAP

Software Support for all SAP Software Maintenance customers.

For these customers, the CIBER support organization works as

an integral part of the Global SAP support organization.

Service Management ensures comprehensive and reliable

service delivery. CIBER’s customers benefit from the efficiency

of individually assigned Service Teams delivering the agreed

services. A dedicated Service Manager coordinates the delivery

of the agreed services, provides transparent reporting of all

activities and KPIs, and represents the customer and their needs

inside the CIBER service organization.

Background, starting point and objectives

CIBER is a full service IT partner founded in 1974 and

headquartered in Denver, USA. It employs more than 8,500 staff

at 85 locations in 18 countries, and generates annual revenues of

about $1.037 billion. CIBER specializes in implementing and

supporting complex SAP and similar solutions for a broad

customer base in multiple industry sectors.

About CIBER

More than 400 of CIBER’s total of 1,300 SAP consultants work for

CIBER Germany, which has particular expertise in retail and

consumer products, chemicals and pharmaceuticals,

automotive industry, and financial services. With more than 20

years of SAP solution implementation experience, CIBER

Germany achieved sales of €55 million in 2009.

CIBER Managed Services GmbH is based in Freiburg,

Germany, and was established in January 2010 to provide SAP

application and operation services for small- and medium-sized

businesses from its two data centers, with more than 120

experienced consultants.

The Managed Services business specializes in post-

implementation-services, designed to preserve the initial

investments that customers make in their SAP implementations.

Managed Services consist of Operations Services, Application

Services, Software Support and comprehensive Service

Management.

Operations Services provide customers with a full range of SAP

system maintenance capabilities, covering the provision and

operation of modern data centers, the necessary IT

infrastructure, and the administration and maintenance of SAP

and IT-related applications.

As a VMware Solution Provider with a special expertise in SAP

implementations, CIBER has competencies and proven

processes in architecture and sizing, and also in deployment

and analyzing complex virtualized SAP landscapes.

Figure 1: CIBER Services

6

Initial IT environment

In order to offer operation services for SAP as a cloud provider,

CIBER Managed Services needed to build up two completely

new data centers. The new and flexible technologies allow

CIBER to satisfy growing customer needs. As a future service,

CIBER wants to implement a self-service portal to allow

customers to request SAP services. Subsequently, the

necessary server capacity, operating system, applications and

storage could be provisioned and charged back automatically.

This would keep costs low and enable CIBER to serve smaller

clients cost-effectively with enterprise-class solutions.

CIBER selected SAP Solution Manager and its own CIBER

Solution Suite to provide the central interface, which includes a

full service desk based on ITIL® V3. The CIBER tool and SAP

Solution Manager enable the creation of a virtual server (either

using VMware vSphere 4 technology on IBM System x or using

LPARs on IBM Power servers), the installation of the operating

system, assignment of the LUNs for storage and the deployment

of the SAP applications. The CIBER team needed next to

consider the best infrastructure to support this business strategy.

With very positive experience of IBM System x and IBM Power

Systems servers, CIBER chose to continue on the same strategic

path, and selected IBM XIV Storage Systems with IBM SAN

Volume Controller to address performance, high availability

scenarios, and premium support for data center management.

Redundant FC connection

Figure 2: SAP private cloud

7

Business challenges and project objectives

The thinking behind the technology strategy was based on the

view that although customer workload was growing

continuously, the nature of the demand was highly volatile.

Providing an industry-standard IT services solution with server

capacity sold in fixed amounts would not deliver the kind of

flexibility that customers wanted, and CIBER wanted to create a

solution that could scale up and down in performance – and

allow customers to pay on a more cost-effective usage basis.

As a starting point, CIBER needed an infrastructure capable of

hosting 120 SAP instances with about 8,000 SAP users, and up

to 20 systems for one customer in a private cloud environment.

The customers are offered free choice among different operating

systems and platforms, from IBM AIX on IBM Power Systems to

SUSE Linux Enterprise Server or Windows Server on IBM

System x BladeServer technology. While CIBER recommends

that its customers use IBM DB2 for Linux, UNIX and Windows,

some customers run MaxDB, Oracle, and MS SQL Server

databases. The infrastructure is capable of running all these

operating systems and databases, and the CIBER team can

support all of these systems.

CIBER operates the complete range of SAP ERP software

components, as well as SAP applications such as Supplier

Relationship Management (SRM), SAP ERP Human Capital

Management (HCM), Advanced Planning and Optimization

(APO), Customer Relationship Management (CRM), SAP

NetWeaver Business Warehouse (BW) and SAP NetWeaver

Portal. Additionally, CIBER offers its own solutions based on SAP

All-in-One software, including Rapid Retail, QuickWear,

Fabricated Metals, Industrial Machinery and Components,

Aerospace and Defense, and CIBER Solution Suite, a new

intuitive front-end for SAP Solution Manager. Each SAP instance

runs in its own LPAR.

If an additional Java Stack is needed, this is installed in a

separate LPAR. The largest SAP environment provides up to

170,000 SAPS in a total of 20 instances, processing up to 12,500

sales order positions per hour. On customer request the

environment is set up as a Power HA for AIX cluster to support

high availability of logical partitions for business critical systems

in a two-node cluster with automatic failover.

Continuing the theme of continuous availability, CIBER uses Live

Partition Mobility, available on Power servers with POWER6

processors and upwards. Live Partition Mobility enables the

migration of entire running AIX partitions and hosted applications

from one physical server to another without disrupting services

and loads. Combined with LPARs, SAN Volume Controller and

other virtualization technologies, Live Partition Mobility will

further enhance CIBER’s ability to respond to customer

demands.

Alongside the scale-up scale-down flexibility for individual

customers, CIBER planned for overall systems growth that would

equip them to respond to the total business demand. The

solutions must therefore be able to expand in capacity as a

whole, while preserving the ability to offer each customer exactly

the right services.

Knowledge of new customer contracts, the customer queue,

and planned growth predicted that the infrastructure at the

current data centers would reach capacity in less than six

months. CIBER planned to build the new data center in less than

four months to allow for sufficient transition time and maintain the

delivery of high service levels for its SAP hosting customers.

8

Server architecture

Working with SVA and IBM, the CIBER team designed and

implemented a highly flexible and fully virtualized environment,

capable of hosting almost every kind of SAP application

landscape.

SVA, an IBM Premier Business Partner, provided implementation

services. SVA installed the SAN Volume Controller, XIV, x3650

and BladeCenters. Ten SVA and IBM technicians completed the

implementation on time and on budget in a six-week period –

during part of which one of the data centers did not have a fully

functioning electricity supply.

In essence, everything is virtualized – the servers, storage and

network. When an application grows in workload and functional

reach, its processes are distributed among multiple virtual

machines to ensure performance is maintained, using either

VMware virtual machines on System x or logical partitions

(LPARs) on Power servers. VMware delivers the freedom to be

fully supported for SAP production environments on both Linux

and Windows operating system and multiple x86-Hardware.

VMware virtual machines, whereby each allows at most 8 virtual

CPUs and 255 GB RAM, are smaller than the virtual machines

available in LPARs, which support maximum of 256 virtual CPUs

and 1 TB RAM. When applications grow, this scale-out

distribution of processes happens earlier on System x than on

Power servers.

The result is that CIBER has gathered a great deal of experience

in the scale-out of SAP applications, particularly on System x.

Examples include separation of database and application server

(that is, moving from two-tier to three-tier configurations),

separation into multiple application servers, and separation of

SAP Central Services. The SAP Central Services include the

unique SAP application system services, including enqueue

server and message server.

IBM System x and IBM BladeCenter

CIBER runs IBM BladeCenter H chassis featuring IBM

BladeServer HS22, each of them powered by two 2.40

GHz quad-core Intel Xeon Processor E5530 series

processors and four HS22 Blade servers powered by

hexa-core Intel Xeon Processor X5640 series processors

at 2.6 GHz. The blades run multiple virtual servers –

collectively around 500 systems – using a combination of

VMware vSphere 4 as hypervisor, and Novell SUSE Linux

Enterprise Server, and Windows Server as the SAP

platform.

CIBER uses IBM System x3650 M2 machines powered

by 2.26 GHz quad-core Intel Xeon Processor L5520

series processors. These x3650 M2 servers provide

infrastructure services, running Novell SUSE Linux

Enterprise Server 10 and 11, as well as Microsoft

Windows 2003 and 2008. These servers run without

virtualization, and monitor the virtualized systems with

applications such as IBM Director, cloud monitoring

tools, and SAP monitoring applications.

IBM Power Systems

CIBER uses two IBM Power 560 and two IBM Power 750

servers, running the IBM AIX operating system and IBM

DB2 for Linux, UNIX and Windows. Both Power 750

systems are equipped with four POWER7 processors

with 8 cores each and 512 GB of memory. The IBM Power

560 servers are equipped with two POWER6 processors

with 4 cores each and 128 GB of memory. The servers

each host around 40 logical partitions (LPARs).

Ultimately, however, CIBER’s clients select the platform: either

Power Systems and AIX or System x and Linux or Windows,

depending on the combination of reliability, scalability and

predicted workload.

Technical solution

9

IBM BladeCenter and System x landscapes

For many customers, CIBER provides SAP hosting services on

IBM System x BladeServers.

For systems powered by Intel processors, CIBER follows a rule,

based on experience: There is an optimum relationship between

processor capacity consumed and memory required, which

means that more memory does not necessarily increase the

performance of the SAP applications hosted on that server.

CIBER found that 8 GB memory per core is sufficient. In the

future, CIBER may upgrade the blades with IBM BladeServer

HX5 technology.

Provisioning of new SAP instances is completed using VMware

templates, making it fast and easy to set up standardized

systems ready for immediate customer deployment. Additional

tools such as TSM backup agents, monitoring agents, and

special SAP configurations are already integrated for quick

deployment.

CIBER uses VMware vCenter Converter (also named

vConverter) to automate physical to virtual machine conversions

as well as conversions between virtual machine formats. In

particular, the converter is used to migrate native physical

systems into CIBER’s virtual environments.

Virtualization using VMware in the Intel processor-based systems

Almost all applications, perhaps 99.9 percent, on the Intel

processor-based platforms are virtualized using VMware

vSphere 4. The only exceptions are monitoring and systems

management applications.

The Blade Servers host guest operating systems such as Novell

SUSE Linux Enterprise Server 10 and 11, as well as Microsoft

Windows Server 2003 and 2008.

For the System x servers, CIBER recommends the Linux

operating system to customers for the following reasons:

Lower license costs•

Linux installation can be configured to comprise just the •

software components needed, creating a low-footprint

operating system with relatively low maintenance

requirements (Windows includes many software

components not actually required by the applications,

increasing the operating system footprint and increasing the

patch requirements).

CIBER believes that Linux Logical Volume Manager (LVM) is •

more flexible than the Microsoft Logical Disk Manager; for

example, Linux LVM allows data replication (mirroring) at

server level

RESOURCE POOL

<<<  ESX HA  /  VMWare  Vmotion >>>

…

SVC MDISK

SAP ERP P

SAP SolMan D

SAP  BI DEV

SAP CRM

Mirrored vDisksProductive

1 TB

Direct Mappingnon productive

2.5 TB

…

SVC MDISK

SAP ..

SAP NW QA

SAP EP DEV

SAP ERP 

Direct Mappingnon productive

2.5 TB

ESX ESX ESX ESXESXESX ESX ESX ESXESX

SAP SAP SAP SAP SAP SAP SAP SAPNonSAP

SAP SAP SAP SAP SAP SAP SAPNonSAP

SAP SAP SAP SAP SAP SAP SAP SAPNonSAP

SAP SAP SAP SAP SAP SAP SAP SAP

SAP SAP SAP SAP SAP SAP SAP SAP

SAP SAP SAP SAP SAP SAP SAP SAP

Linux/DB2ERP 6.0

Intel/DB2Win/OracleERP 6.0

Intel/DB2Win/MaxDBERP 6.0

NonSAP

NonSAP

Win/MS SQLERP 6.0 

PowerVM LPAR LPAR LPAR LPARLPARLPAR LPAR LPAR LPARLPAR

Data center 1 Data center 2

Figure 3: High Availability Concept and storage cloud operating system

10

The conversion takes place either at the customer site or in

CIBER’s central data center. CIBER uses Acronis True Image to

create a backup image of the customer’s application systems

environment and transport the image files from the client’s site to

the data center, which eliminates the need for risky physical

movement of hardware from place to place.

Once customer systems have been virtualized, CIBER uses

VMware Virtual Symmetric Multi-Processing (SMP) to enable a

single virtual machine to use multiple physical processors

simultaneously. This allows CIBER to maximize utilization of its

physical processor resources, and provide the virtual machines

with the capacity required easily and quickly.

With the applications and servers hosted by CIBER, the next

question is how to deliver them to the customers. VMware View is

built on VMware vSphere for Desktops. CIBER runs its own data

center management and administration applications using

VMware View virtualized desktops.

For data storage, security and management, CIBER uses

VMware vStorage VMFS (virtual machine storage file system), a

high-performance cluster file system optimized for VMware

virtual machines. A VMware virtual machine stores the contents

of a virtual machine’s hard disk drive in a virtual disk file with the

extension .vmdk (virtual machine disk). The vmdk specification

describes and documents the virtual machine environment, and

how the contents of the hard disk drives is stored. CIBER uses

.vmdk files rather than storing the virtual disks on raw devices to

ensure isolation of the data stored in separate virtual disks.

VMware Storage vMotion enables Live Migration of .vmdk files –

in other words, a .vmdk file can be moved from one location to

another even while it is in use. This ability helps CIBER to

minimize downtime during disk array migrations and storage

upgrades, or when the virtual machines require larger-capacity

storage devices.

For disaster tolerance and disaster recovery capabilities, as well

as for high availability across locations, CIBER implemented an

enterprise-class storage design. It consists mainly of the IBM

SAN Volume Controller (SVC) and the IBM XIV Storage System.

The SVC has been set up with the Split I/O Group design to

achieve high availability even if one location fails completely.

In 2011, CIBER plans to deploy VMware vCloud Director data

center management software, which pools virtual infrastructure

resources and delivers them to users as a catalog-based

service. The target users are internal consultants who administer

the local data center systems.

CIBER also plans to implement VMware vCenter Chargeback,

which logs resource requests and consumption and enables

usage-based chargeback and accounting for hosted data

center operations.

Figure 4: Providing and consuming IT as a service

VMware vCloud Director

Customer 1

VMwarevShield

VMwarevCenter Server

VMware vSphere

VMwarevCenter Server

VMware vSphere

CIBER Solution Suite

Datacenter Configuration (Silver)Datacenter Configuration(Gold)

SecurityCatalogs

Linux/SQLBI

Linux/ DB2BS7

Win/MaxDBERP 6.0

11

IBM Power Systems landscapes

Typically, for larger customers CIBER hosts SAP ERP and SAP

NetWeaver applications on IBM Power systems. For existing

clients, CIBER has two IBM Power 560 and two IBM Power 750

servers, running the IBM AIX operating system and IBM DB2 for

Linux, UNIX and Windows.

LPARs are subsets of a computer’s hardware resources,

virtualized as a separate computer using the hypervisor. A

physical machine can be partitioned into multiple LPARs, each

hosting a separate AIX operating system. The hypervisor

assigns a specific range of virtual processors to each of the

LPARs. CIBER does not configure dedicated assignments of

processors, and shares uncapped processor resources across

all the LPARs.

The VIOS provides full virtual paths from each LPAR to the

switched fabric in the data center. Switched fabric network

topology allows nodes to connect with each other via one or

more network switches, which provides both high total capacity

and a high degree of resilience if a network switch fails.

CIBER has assigned a separate LPAR for storage management,

with a dedicated Converged Network Adapter to support

concurrent LAN (TCP/IP) and SAN (FCoE, iSCSI) traffic over a

shared 10 Gb Ethernet link. CIBER currently uses only TCP/IP

and iSCSI protocols, and wants to be ready for FCoE in the

future.

PowerHA for AIX (previously named High Availability Cluster

Multiprocessing HACMP) provides high availability of logical

partitions (LPARs) in a two-node cluster by way of automatic

failover. When one server node goes down, the other server

node takes over. Depending on clients’ requirements, most

production systems and some of the quality assurance systems

running on AIX use PowerHA to ensure continuous availability.

CIBER uses IBM Virtual I/O Server (VIOS) to enable the sharing

of physical I/O resources between client logical partitions within

the server. The total physical bandwidth can be allocated in fixed

amounts to each server, or policies set to enable full capacity to

be used by high-priority workload at peak times.

For large SAP systems, CIBER prefers to use POWER

technology and virtualization over VMware vSphere. This is for

two reasons: first, because AIX LPARs can scale to higher total

processor and memory limits; and second, because AIX

enables much higher processor utilization. By using AIX LPARs,

the SAP application can grow much larger before it is necessary

to scale-out over multiple servers.

The migration from IBM POWER5 to POWER6 and POWER7 was

without any problems, CIBER simply connected the new servers

to the SAN storage system and restarted the new servers and the

SAN storage system.

12

Storage architecture

CIBER implemented an enterprise-class storage solution, based

on two IBM XIV enterprise storage systems, IBM SAN Volume

Controller (SVC) as Split I/O Group, IBM System Storage DS3400

for the SVC quorum disk, and IBM Tivoli Storage Manager. The

storage infrastructure has been designed to provide a

combination of performance and resilience, as well as the ability

to allocate storage volumes to multiple customers exactly as their

needs dictate with the least possible administration and

expense. CIBER evaluated different solutions and chose XIV

technology because of better IO performance, improved

administrative handling, and easier maintenance support than

its competitors.

At the core of the storage landscape are two IBM XIV model A14

storage systems, each equipped with 79 TB storage. The XIV

systems act as the principal SAP application data storage.

CIBER can handle almost three-fold storage growth as customer

data volumes grow. In practice, Tivoli Storage Manager enables

sophisticated management of data, moving archive and less

active data onto tape, to help reduce the need to expand disk

systems.

CIBER implemented the two XIV systems, located at the primary

and secondary data centers. XIV technology offers high,

consistent performance and hotspot-free service due to mass

parallelism and the distribution of logical volumes across all hard

disks in the system, eliminating distribution planning.

POWER SUPPLY 4

POWER SUPPLY 3

POWER SUPPLY 2

POWER SUPPLY 1

10 9 8 7 6 5 4 3 2 1

POWERSUPPLY4+2POWERSUPPLY3+1

0

1

2

3

24

25

26

27

4

5

6

7

2 8

2 9

30

31

8

9

10

11

32

33

34

35

12

13

1 4

1 5

3 6

3 7

38

39

16

17

18

19

40

41

42

43

20

21

22

23

44

4 5

4 6

4 7

System Storage

D

POWER SUPPLY 4

POWER SUPPLY 3

POWER SUPPLY 2

POWER SUPPLY 1

10987654321

POWER SUPPLY 4 + 2 POWER SUPPLY 3 + 1

01

23

2425

2627

45

67

2 829

3031

89

1011

3 233

3435

1 213

1415

3637

3839

1617

1 81 9

404 1

4 243

2021

2223

444 5

4647

POWER SUPPLY 4

POWER SUPPLY 3

POWER SUPPLY 2

POWER SUPPLY 1

10987654321

POWER SUPPLY 4 + 2 POWER SUPPLY 3 + 1

01

23

2425

2627

45

67

2 829

3031

89

1011

3 233

3435

1 213

1415

3637

3839

1617

1 81 9

404 1

4 243

2021

2223

444 5

4647

System Storage

TotalStorageStorageEngine336

POWER SUPPLY 4

POWER SUPPLY 3

POWER SUPPLY 2

POWER SUPPLY 1

10 9 8 7 6 5 4 3 2 1

POWERSUPPLY4+2POWERSUPPLY3+1

0

1

2

3

24

25

26

27

4

5

6

7

28

29

30

3 1

8

9

10

1 1

3 2

3 3

3 4

3 5

12

13

14

15

36

37

38

39

16

1 7

1 8

19

40

41

4 2

43

20

2 1

22

23

44

45

46

4 7

System Storage

41

85

129

DS3400

1923

1822

1721

1620

1115

913

1014

812

37

15

26

40

2498-B24

1923

1822

1721

1620

1115

913

1014

812

37

15

26

40

2498-B24

1923

1822

1721

1620

1115

913

1014

812

37

15

26

40

2498-B24

1923

1822

1721

1620

1115

913

1014

812

37

15

26

40

2498-B24

19 2318 2217 2116 2011 159 13 10 148 123 71 5 2 6402498-B24

TotalStorage Storage Engine 336

System x3650 M21 2

3 4

0 1 2 3 4 5 6 7 8 9 10 11

System x3650 M21 2

3 4

0 1 2 3 4 5 6 7 8 9 10 11

System x3650 M21 2

3 4

0 1 2 3 4 5 6 7 8 9 10 11

System x3650 M21 2

3 4

0 1 2 3 4 5 6 7 8 9 10 11

System x3650 M21 2

3 4

0 1 2 3 4 5 6 7 8 9 10 11

System x3650 M21 2

3 4

0 1 2 3 4 5 6 7 8 9 10 11

35 3934 3833 3732 36

2498-B40

27 3126 3025 2924 2819 2318 2217 2116 2011 1510 149 138 123 762510 4 35 3934 3833 3732 36

2498-B40

27 3126 3025 2924 2819 2318 2217 2116 2011 1510 149 138 123 762510 4

Bladecenter H

Infrastrukturserver

SVC Node

SVC Node

Quorum

Dark Fiber

Dark Fiber

FC CWDM

`STAT

CONSOLE

L1

L2

MGMT 0

MGMT 1 4 8 12 16

3 7 11 15

2 6 10 14

1 5 9 13

20 24 28 32

19 23 27 31

18 22 26 30

17 21 25 29

SLO

T2

34 38

33 37

36 40

35 39

SLO

T3

Cisco Nexus 5020

PS

1P

S2

200-240v-6A50~60Hz

`STAT

CONSOLE

L1

L2

MGMT 0

MGMT 1 4 8 12 16

3 7 11 15

2 6 10 14

1 5 9 13

20 24 28 32

19 23 27 31

18 22 26 30

17 21 25 29

SLO

T2

34 38

33 37

36 40

35 39

SLO

T3

Cisco Nexus 5020

PS

1P

S2

200-240v-6A50~60Hz

`STAT

CONSOLE

L1

L2

MGMT 0

MGMT 1 4 8 12 16

3 7 11 15

2 6 10 14

1 5 9 13

20 24 28 32

19 23 27 31

18 22 26 30

17 21 25 29

SLO

T2

34 38

33 37

36 40

35 39

SLO

T3

Cisco Nexus 5020

PS

1P

S2

200-240v-6A50~60Hz

`STAT

CONSOLE

L1

L2

MGMT 0

MGMT 1 4 8 12 16

3 7 11 15

2 6 10 14

1 5 9 13

20 24 28 32

19 23 27 31

18 22 26 30

17 21 25 29

SLO

T2

34 38

33 37

36 40

35 39

SLO

T3

Cisco Nexus 5020

PS

1P

S2

200-240v-6A50~60Hz

1 2 3 4 5 6 7 8 9 10 11 12 13 14

1 2 3 4 5 6 7 8 9 10 11 12 13 14

BladeCenter H

1 2 3 4 5 6 7 8 9 10 11 12 13 14

1 2 3 4 5 6 7 8 9 10 11 12 13 14

BladeCenter H

1 2 3 4 5 6 7 8 9 10 11 12 13 14

1 2 3 4 5 6 7 8 9 10 11 12 13 14

BladeCenter H

1 2 3 4 5 6 7 8 9 10 11 12 13 14

1 2 3 4 5 6 7 8 9 10 11 12 13 14

BladeCenter H

560

D3 D4 D5 D6 D7 D8 D9 D10

Power 750

Power 750

Infrastrukturserver

560

D3 D4 D5 D6 D7 D8 D9 D10

Power 750

Power 750

Datacenter 1 Datacenter 2

Datacenter 3

10GE LAN Backbone

8 GB FC Backbone

8 GB FC Backbone

10GE LAN Backbone

FC CWDMFC CWDMFC 850nm

FC 850nmXIV Storage 1

Infrastructure Mgmt Bladecenter H

XIV Storage 2

Infrastructure Mgmt

Fiberchannel Single Mode Fiberchannel Multi Mode Dark Fiber Local Area Network

Figure 5: Data center landscape

Working with the service partner Netzwerktechnik Röver, CIBER

implemented a redundant and geographically separated metro

data center connection based on CWDM technology. All storage

mirroring is performed through multiplexed Dark Fiber with a

bandwidth of 60 GBit/s, which can be scaled up to 300 GBit/s.

The XIV system is controlled through an intuitive graphical user

interface (GUI), and a powerful command line interface (CLI) for

complex scripting. Training on XIV systems usage is usually a

matter of hours rather than weeks, saving costs and speeding

time to full production.

13

The very high I/O, reliability and availability performance is

enabled by a grid-based controller and disk architecture with

active-active component redundancy, which enables non-

disruptive hot-swapping of hardware and firmware. XIV systems

provide incremental scaling from partial to full rack, with 27-79

TB usable, and with every capacity increase the performance

scales linearly. Automatic re-distribution of volumes ensures that

the high performance and fast access continues even as data

volumes grow. There is no tuning necessary, and the XIV system

does not need to tune itself during operation.

Using the SVC’s FlashCopy function and XIV point-in-time copy

CIBER clones SAP systems as customers request new

instances. Combined with automated SAP post-processing,

CIBER is able to create a completed cloned SAP instance in

about 30 minutes. XIV Snapshot gives CIBER the ability to

respond to customer demands in ways and at a speed that even

in recent years would have been impossible to achieve.

In selecting XIV systems, a new technology for the CIBER team,

they soon discovered that only three hours’ training was needed

to understand the easy-to-use XIV system’s graphical interface,

compared with older systems that had taken weeks to fully learn

(redundant).

The DS3400 storage system serves two purposes:

As a storage device to host the active quorum disk (tie •

breaker) for the SVC clusters

As a storage pool for the TSM server•

CIBER uses IBM Tivoli Storage Manager, running in a separate

LPAR on the Power servers, to manage backup, archive and

restore. A dedicated physical 10 Gbps Ethernet adapter allows

data transfer to run on fully virtualized end-to-end connections

powered by Nexus switches from Cisco, connecting to the IBM

System Storage TS3500 tape library. This arrangement offers

very fast system backup and restore processes while ensuring

that the data traffic does not impact production systems.

The storage landscape is completed with two IBM System

Storage TS3500 tape library systems, each with five LTO-4

drives, 260 slots and about 200 cartridges. These systems

provide a data transfer speed of 350 GB per hour.

Database architecture

For CIBER, DB2 for Linux, UNIX and Windows is the most

appropriate database platform to support the SAP applications.

Small and medium-sized businesses tend to underestimate their

data growth, and easily reach sizes where DB2 offers significant

advantages, such as Deep Compression that helps to reduce

the database size and data growth.

DB2 also offers close integration with Tivoli Storage Manager,

providing a comprehensive backup solution that protects DB2

and related data reliably and cost-effectively.

The new index compression feature within DB2 is the next step to

improve database performance and reduce storage costs. In

addition, compression for temporary tables for SAP systems is

activated ‘under the cover’ to reduce disk costs. DB2, compared

with other database compression algorithms, can handle data

and indexing more efficiently. CIBER experienced a

compression rate in the high double-digit area, and

performance improvements of 60 percent with DB2.

Besides performance, DB2 also makes administration tasks

easier and faster by offering a high level of automation with Self

Tuning Memory Management, Automatic Storage, automatic

runstats, and Real Time Statistics.

For example, CIBER uses the autoresize functionality so that

tablespaces can enlarge themselves if needed, while the

Container Management is completed manually to preserve a

certain degree of control over the data management by the

database administrators.

14

Automatic reorganization offers time-saving advantages for

database administrators, because it checks if a reorganization of

data and indices is needed and if so, the Reorg utility is

automatically called and reorganizes the data and indices in the

background. CIBER also plans to use the reclaimable storage

feature for its SAP NetWeaver Business Warehouse to reduce

the tablespaces from about 1 TB down to 300 MB

With the new tablespace option of reclaimable storage,

introduced in DB2 version 9.7, it is now much easier to reduce

the size of the tablespace and return unused space to the file

system, including unused space below the High-Water Mark

(HWM).

DB2 Migration: Opportunities for optimization

It is possible to activate compression features during a SAP

migration project to DB2, which eliminates the need for

subsequent compression tasks.

To do so, CIBER used the “Sampled” option, which comes with

the R3Load Version 7. This option causes every ‘N’th row of the

target table to be loaded first, followed by a reorg to create the

Compression Dictionary based on the data loaded. The count of

data used for the sampling process is provided by the

environment variables DB6LOAD_SAMPLING_FREQUENCY

and DB6LOAD_MAX_SAMPLE_SIZE. Subsequently, R3load is

restarted to load the rest of the table in a compressed format.

Transition to Unicode

If Unicode is required on the target solution, then recommended

practice is to perform the DB2 migration in conjunction with an

SAP software upgrade and/or Unicode conversion in a single

project. This combination eliminates redundant project tasks,

keeping the cost of the migration as low as possible. Around 90

percent of the Unicode conversion steps are also part of the DB2

migration process. By combining the two processes, both costs

and workload can be reduced, eliminating duplication of effort.

Automatic provisioning of SAP with Cloud services

CIBER’s ultimate goal is to deliver Cloud services to its

customers, which means being able to provision, upgrade and

clone SAP instances very quickly using automated systems.

Defining customer requirements will become an interactive, face

to face activity, and once requirements have been defined, the

SAP landscapes will be provisioned automatically.

The Service Request Tool is the “CIBER Solution Suite”, which

CIBER developed in-house, based on SAP Solution Manager.

The backend (workflows) were developed in ABAP, while CIBER

uses Adobe Flex™ (Fx) as GUI front end. This tool enables the

CIBER SAP Consultant simply to select the correct SAP instance

template, and then kick off the automated provisioning.

CIBER used the standard OS tools, such as NIM & NIS (Network

Installation Manager & Services), to install the operating system.

To clone SAP instances, CIBER uses XIV point-in-time copy.

CIBER clones (snapshots) the entire SAP system, including EXE,

DATA, and LOG files, while keeping the SID. The new clone is

started in its own VLAN segment and will be automatically

reconfigured and customized. This ensures that the new clone

with the same SID does not interfere with the existing SAP

landscape.

Because CIBER does not need to install the SAP instance, there

are no pre- or post-processing steps, and a new clone can

therefore be provisioned and made available within 30 minutes.

“IBM has brought in major contribution to the quick success of CIBER – to develop such a high customer rate within such a short timeframe.”

Stefan Rother Lead Infrastructure Services CIBER Managed Services GmbH

15

Performance improvements

The new SAP infrastructure allows CIBER to improve efficiency

for its SAP managed services business significantly by offering a

solution that scales in performance and capacity, while

providing a very cost-effective model.

The virtualization capabilities of the storage and server

infrastructure allow CIBER to offer customers a very flexible

service with real end-to-end virtualization, able to scale up

quickly in times of peak workload and back again as required.

For example, being able to clone a SAP system inside 30

minutes, or to activate another 1 TB of storage on the XIV system

within an hour, mean that CIBER can cope with almost any

customer demand. The result is that CIBER can offer “pay as you

grow” SAP hosting as software as a service (SaaS) backed by a

powerful cloud infrastructure that encourages customers to

make the leap without having to predict workload or commit to

specific infrastructure spending.

The combined performance improvements available from the

IBM Power servers, IBM System x BladeServers, and IBM

storage infrastructure have led to increased user satisfaction.

Many customers have noticed greatly improved response times

for online operation of the DB2 database – in some cases

recording an eight-fold improvement. The enhanced response

times contribute to increased user productivity, and help build

the case for using hosted SAP applications rather than in-house,

as CIBER is able to make the kind of technology investment that

might be out of reach of a single customer.

CIBER plans to take advantage of Active Memory Expansion,

available on Power servers with POWER7 processors and

upwards, which employs memory compression technology to

transparently compress in-memory data, allowing more data to

be placed into memory and thus expanding the memory

capacity of POWER7 systems. Utilizing Active Memory

Expansion can improve system utilization and increase a

system’s throughput.

CIBER plans to roll out IBM DB2 Deep Compression and Row

Compression features on all of its systems, and deploy IBM Tivoli

Provisioning Manager and IBM Tivoli FlashCopy to enhance

data security and performance.

For improved data center management and new services,

CIBER plans to apply VMware vCloud Director in 2011. vCloud

Director is a data center management software, which pools

virtual infrastructure resources and delivers them to users as a

catalog-based service. For cost models that are customized for

accounting of resource requests and consumption, CIBER will

implement VMware vCenter Chargeback.

Project achievements

“We recommend the DB2 database for new customers who start with SAP, because DB2 is very cost-effective and brings performance benefits and DB2 and SAP simply fit together.”

Holger Stasch SAP Senior Consultant CIBER Managed Services GmbH

Authors

Markus Fehling, IBM, markus.fehling@de.ibm.com

Elke Hartmann-Bakan, IBM, elke.hartmann-bakan@de.ibm.com

Frank Neumann, IBM, frank.m.neumann@de.ibm.com

Carsten Siegler, IBM, csiegler@de.ibm.com

Hans-Achim Muscate, VMware, hamuscate@vmware.com

Markus Peltz, CIBER, markus.peltz@ciber.com

For more information

To learn more about the solutions from IBM and SAP, visit:

ibm-sap.com

For more information about SAP products and services, contact

an SAP representative or visit: sap.com

For more information about IBM products and services, contact

an IBM representative or visit: ibm.com

For more information about CIBER products and services,

contact a CIBER representative or visit: ciber.com

Contacts

Markus Fehling, markus.fehling@de.ibm.com

Markus Peltz, markus.peltz@ciber.com

For feedback and further questions please contact the IBM SAP

International Competency Center via isicc@de.ibm.com

SPC03307-DEEN-00 (December 2010)

© Copyright IBM Corp. 2010 All Rights Reserved.

IBM Deutschland GmbH D – 71137 Ehningen ibm.com

Produced in Germany IBM, the IBM logo, ibm.com, i5/OS, DB2, Domino, FlashCopy, Lotus, Notes, POWER, POWER4, POWER5, POWER6, System i, System x, and Tivoli are trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of other IBM trademarks is available on the Web at: http://www.ibm.com/legal/copytrade.shtml

UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product or service names may be trademarks, or service marks of others.

This brochure illustrates how IBM customers may be using IBM and/or IBM Business Partner technologies/services. Many factors have contributed to the results and benefits described. IBM does not guarantee comparable results. All information contained herein was provided by the featured customer/s and/or IBM Business Partner/s. IBM does not attest to its accuracy. All customer examples cited represent how some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.

This publication is for general guidance only. Photographs may show design models.