emc cloud-enabled infrastructure for · pdf fileemc cloud-enabled infrastructure for...

White Paper

EMC Solutions

Abstract

This white paper focuses on the disaster recovery add-on bundle of the Cloud-enabled Infrastructure for SAP. It explains the transformation of a single data center solution into a mission-critical business continuity solution, which protects data from full site disaster and increases resiliency of operations. The solution is enabled by EMC RecoverPoint® , EMC® next-generation VNX®, VMware vCloud Suite, VMware vCenter Site Recovery Manager, and VMware PowerCLI.

December 2013

EMC CLOUD-ENABLED INFRASTRUCTURE FOR SAP—BUSINESS CONTINUITY SERIES:

DISASTER RECOVERY BUNDLE—VNX EMC RecoverPoint, EMC next-generation VNX, VMware vCenter Site Recovery Manager, VMware vCloud Suite, VMware PowerCLI

Protection of mission-critical SAP landscapes in the cloud Cloud infrastructure resiliency Automated cloud infrastructure disaster recovery

EMC Cloud-Enabled Infrastructure for SAP—Business Continuity Series: Disaster Recovery Bundle—VNX

EMC RecoverPoint, EMC next-generation VNX, VMware SRM, VMware vCloud Suite, VMware PowerCLI

2

Copyright © 2013 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

All trademarks used herein are the property of their respective owners.

Part Number H12594

3 EMC Cloud-Enabled Infrastructure for SAP—Business Continuity Series: Disaster Recovery Bundle—VNX


Table of contents

Executive summary ............................................................................................................................... 6

Business case .................................................................................................................................. 6

Solution overview ............................................................................................................................ 7

Key benefits ..................................................................................................................................... 7

Introduction.......................................................................................................................................... 8

Purpose ........................................................................................................................................... 8

Scope .............................................................................................................................................. 8

Audience ......................................................................................................................................... 8

Terminology ..................................................................................................................................... 8

Technology overview .......................................................................................................................... 10

Introduction ................................................................................................................................... 10

Physical architecture ...................................................................................................................... 12

Hardware resources ....................................................................................................................... 13

Software resources ........................................................................................................................ 14

Disaster recovery for Cloud-Enabled Infrastructure ............................................................................ 16

Introduction ................................................................................................................................... 16

Disaster recovery logical architecture ............................................................................................. 16

Recovery modes ............................................................................................................................. 18

DR operational cycle ...................................................................................................................... 19

Failover process ............................................................................................................................. 21

Workflow ................................................................................................................................... 21

Failback process ............................................................................................................................ 23

Workflow ................................................................................................................................... 23

Setting up disaster recovery for CEI .................................................................................................... 25

Virtual Data Center ......................................................................................................................... 25

Introduction .............................................................................................................................. 25

Configuring VMware vCloud Director .......................................................................................... 25

Configuring the network ............................................................................................................ 28

Configuring infrastructure Chargeback Manager ........................................................................ 29

EMC storage infrastructure ............................................................................................................. 30

Introduction .............................................................................................................................. 30

Configuring VNX storage ............................................................................................................ 31

EMC RecoverPoint .......................................................................................................................... 31

Introduction .............................................................................................................................. 31

Configuring RecoverPoint consistency groups............................................................................ 33

VMware vCenter Site Recovery Manager ......................................................................................... 35



4

Introduction .............................................................................................................................. 35

Integrating SRM with RecoverPoint ............................................................................................ 35

Configuring VMware SRM .......................................................................................................... 35

SAP system architecture..................................................................................................................... 37

Introduction ................................................................................................................................... 37

SAP ERP 6.0 ................................................................................................................................... 37

SAP system architecture ................................................................................................................. 38

Key SAP design considerations ...................................................................................................... 38

Design best practices ......................................................................................................................... 39

Data center .................................................................................................................................... 39

RecoverPoint .................................................................................................................................. 39

VNX storage ................................................................................................................................... 39

Cloud ............................................................................................................................................. 40

SAP ................................................................................................................................................ 40

Test and validation ............................................................................................................................. 41

Overview ........................................................................................................................................ 41

Notes ............................................................................................................................................. 42

Test Scenario 1: Failover after disaster ........................................................................................... 42

Test objectives .......................................................................................................................... 42

Test procedure .......................................................................................................................... 42

Results and analysis.................................................................................................................. 43

Test Scenario 2: Resynchronizing to Data Center A ......................................................................... 44

Test objectives .......................................................................................................................... 44

Test procedure .......................................................................................................................... 44


Test Scenario 3: Planned migration ................................................................................................ 45

Test objectives .......................................................................................................................... 45

Test procedure .......................................................................................................................... 46


Test Scenario 4: Reprotection for Data Center A ............................................................................. 46

Test objectives .......................................................................................................................... 46


Conclusion ......................................................................................................................................... 48

Summary ....................................................................................................................................... 48

Findings ......................................................................................................................................... 48

References.......................................................................................................................................... 49

EMC documents ............................................................................................................................. 49



VMware documents ....................................................................................................................... 49

SAP documents .............................................................................................................................. 49

Appendix ............................................................................................................................................ 50

Configuring consistency group ....................................................................................................... 50

Configuring VMware SRM ............................................................................................................... 51

Site pairing ................................................................................................................................ 52

Array-based replication ............................................................................................................. 53

Resource mapping..................................................................................................................... 53

Protection groups ...................................................................................................................... 54

Recovery plans .......................................................................................................................... 55

DR design for three data centers .................................................................................................... 56



6

Executive summary

For most businesses, the focus on the customer’s experience has led to more opportunities and competition. Businesses have expanded beyond the traditional geographical boundaries and markets, interacting directly with consumers. Capitalizing on this opportunity requires:

Responsiveness—Consumers have choices. If they are not satisfied with the service level they receive from one source, they go elsewhere to find what they need.

Market and consumer intelligence—Availability of the most current data and information is critical to understanding customer behavior and preference, which in turn helps to maintain a competitive advantage.

Supply, demand, and order fulfillment management—What may seem to be a simple transaction to an eCommerce consumer in reality necessitates the seamless availability of a federated landscape of interdependent business applications.

As IT systems embark on their journey to the cloud, the importance of business continuity has become increasingly critical to the operations of an enterprise. Only one extended downtime may cause irrecoverable loss for many businesses. However, proper preparation typically involves a significant investment of time and cost to keep up with a company’s recovery time objectives (RTOs) and recovery point objectives (RPOs) during a disaster. Designing disaster recovery (DR) solutions poses many challenges, including:

Ensuring RTO and RPO for varying SAP applications in different organizations in a cloud environment

Testing the recovery plan periodically

Reconfiguration effort needed to prepare another site for DR

Human intervention effort needed to carry out DR for hundreds of systems

Data protection by using backup and restore is designed to protect the business data from isolated system failure. However, in a full data center disaster, backup strategies no longer meet these challenges. Besides, backup and restore strategies for restoring hundreds of SAP systems to DR within the restriction of RTOs expose system administrators to human error.

This white paper, specifically designed to protect SAP mission-critical applications in a cloud environment, introduces an EMC solution that addresses all these challenges.

Business case



This solution provides simplified and reliable DR for SAP systems and management infrastructures. We1 built the solution on top of the Cloud-enabled Infrastructure (CEI) for SAP and used EMC RecoverPoint® and VMware vCenter Site Recovery Manager (SRM) as the primary enabling technologies.

EMC RecoverPoint is a high-performing, reliable, and flexible replication solution designed for enterprise-class DR.

VMware SRM, when integrated with RecoverPoint, can fully orchestrate the entire recovery cycle, from the failover until the SAP systems are ready for user access.

This white paper demonstrates how the following technologies support the solution:

The EMC® next-generation VNX® flash-optimized unified storage platform delivers innovation and enterprise capabilities for file, block, and object storage in a single, scalable, and easy-to-use solution. Ideal for mixed workloads in physical or virtual environments, VNX combines powerful and flexible hardware with advanced efficiency, management, and protection software to meet the demanding needs of today’s virtualized application environments.

EMC RecoverPoint provides a robust replication mechanism between the VNX storage arrays located in production and at the recovery data centers. Its data compression technology reduces bandwidth requirements to minimize overhead.

VMware SRM integrates seamlessly with the RecoverPoint system and VNX storage to orchestrate the DR process.

VMware vCloud Director applies the principles of pooling, abstraction, and automation to all data center services, such as compute, storage, networking, and security. It enables the multitenancy business models for IT operation and provides the portal for self-service provisioning.

This solution adds DR functionality to the CEI for the SAP foundation bundle by providing:

Increased resiliency during disasters by automating the failover and failback sequences for both management applications and SAP systems in the cloud environment

Well-organized and highly coordinated recovery plans for a trouble-free failover

The convenience of triggering the failover from any surviving site by a few clicks of a button

Minimized network overhead during replication

1 In this white paper, "we" and “our” refers to the EMC Solutions engineering team that validated the solution

Solution overview

Key benefits



8

Introduction

This white paper describes a solution that provides DR for SAP systems running on the EMC CEI for the SAP foundation bundle. This solution provides automated DR with seamless integration of RecoverPoint and VMware SRM in a vCloud environment.

This white paper:

Introduces the key enabling technologies

Describes the solution architecture and design

Describes how the key components are configured

Presents the test results demonstrating the increased availability of the environment

Identifies the key business benefits of the solution

The solution discussed in this paper focuses mainly on its integration with a stand-alone foundation bundle on a VNX platform. However, the solution is fully compatible with any other CEI add-on bundles that may already be in place, such as the high availability (HA) and application mobility bundle and data protection bundle. This paper does not discuss additional configurations required to work with the HA bundle.

This white paper is intended for SAP BASIS administrators, storage administrators, IT architects, and DR managers responsible for designing, creating, and managing business continuity, especially for customers who have implemented CEI for SAP.

Understanding this paper requires a basic understanding of the concepts and workflows used by the CEI for SAP foundation bundle. We highly encourage you to read the EMC Cloud-Enabled Infrastructure for SAP Foundation Bundle – VNX White Paper first before continuing.

This white paper includes the terminology in Table 1.

Table 1. Terminology

Term Description

AAS SAP Additional Application Server (previously known as Dialog instances).

ABAP SAP Advanced Business Application Programming.

ASCS ABAP System Central Services.

Asynchronous replication The process of committing data writes when the data has been sent out for replication without waiting for acknowledgement from the receiving side. Asynchronous replication provides crash-consistent protection and recovery to specific points-in-time, with a small RPO.

Purpose

Scope

Audience

Terminology



Term Description

Consistency group RecoverPoint configuration method that logically groups data sets so that they are consistent with one another. The consistency groups ensure that the updates to the production volumes are written to the replicas in a consistent write order and that the replicas can always be used to continue working or to restore the production source.

Copy A logical RecoverPoint entity that constitutes all volumes of a consistency group.

Deduplication A specialized data reduction technique that eliminates duplicate copies of repeating data.

ERP Enterprise Resource Planning.

IDES SAP Internet Demonstration and Evaluation System. An SAP system with preconfigured settings and sample data.

Journal volume A RecoverPoint volume that stores time-stamped application writes for later recovery to selected points-in-time.

OvDC Organization Virtual Data Center. A vDC abstraction, which is partitioned from a PvDC that maps resources to an organization.

PAS SAP Primary Application Server.

Protection Group A collection of virtual machines and templates that you protect together by using VMware SRM.

PvDC Provider Virtual Data Center. A vDC abstraction that combines the computer and memory resources of a single vCenter Server resource pool with the storage resources of one or more datastores connected to that resource pool.

Repository volume A volume that stores configuration information about RPAs and consistency groups. This volume enables a surviving RPA to take over the replication activities of a failed RPA from the same cluster.

RPO Recovery point objective. The maximum tolerable period in which data might be lost from an IT service.

RTO Recovery time objective. The maximum amount of time allowed to resolve an issue.



10

Technology overview

EMC, VMware, and SAP have joined forces to design a complete hardware and software solution that supports an on-premises private cloud. This initiative is now known as the CEI. Its architecture is divided into six different functionalities, delivered as separate bundles, as illustrated in Figure 1. Within the CEI, customers can choose the cloud functionalities they want to enable first without losing sight of their end goal—an efficient and reliable private cloud.

Figure 1. Cloud-enabled Infrastructure for SAP

The CEI is based on a mandatory component called the foundation bundle. Table 2 outlines the functions and benefits of the CEI foundation bundle. EMC Cloud-Enabled Infrastructure for SAP Foundation Bundle – VNX White Paper provides more information about the foundation bundle. Customers can then develop their own roadmaps for what to build next.

Table 2. On-premises EMC Cloud-enabled Infrastructure for SAP solution: Foundation bundle

Function Benefits Technology

Virtual data center for SAP

Autonomy of business units and application operations

Service catalogs

Service level agreements (SLAs)

Management of vCloud tenants

Resource pooling

VMware vCloud Suite Enterprise

Infrastructure chargeback

Cost measurement, analysis, and reporting of the use of compute, network, storage, and backup resources (in combination with data protection add-on bundle)

VMware vCenter Chargeback

Introduction



Function Benefits Technology

Integrated cloud management and performance analysis

Manage availability, capacity, performance, and health in the SAP landscape

VMware vCenter Operations Manager

EMC Storage Resource Management

EMC Storage Analytics (ESA)

EMC Virtual Storage Integrator (VSI)

Storage tiering Automatically get the right data to the right place at the right time

EMC VNX

EMC Fully Automated Storage Tiering (FAST™) Suite

Cloud networking and security for SAP

Cloud-enabled Infrastructure security framework

Authorization concepts

Compliance and non-compliance tracking

VMware vCloud networking and security

Table 3 describes the add-on bundles that enable additional cloud functionality and data center enhancements.

Table 3. EMC Cloud-enabled Infrastructure for SAP: Add-on bundles

IT strategy Bundle Benefits Technology

Business continuity

HA and application mobility for SAP

High availability within one data center and across two data centers with application awareness for the complete SAP landscape

Nondisruptive movement of applications from one data center to another data center

Improved resource utilization across data centers

Minimize downtime for infrastructure maintenance

EMC VPLEX Metro

vSphere HA

VMware vSphere Distributed Resource Scheduler (DRS)

Symantec ApplicationHA

DR for SAP Provide disaster recovery protection for cloud management applications and SAP systems

VMware SRM

EMC RecoverPoint

Data protection for SAP Provide data protection to cover the cloud management applications and SAP systems

Backup and recovery at all data centers with remote replication of backup sets for offsite protection

EMC Avamar®

EMC Data Domain®

EMC Data Protection Advisor



12

IT strategy Bundle Benefits Technology

Service support/ provisioning

Automation and operations for SAP

SAP application virtualization enables any service any time on any server

Provision SAP systems on demand with automated end-to-end process

Reduce downtime window during maintenance leveraging mass operations

SAP NetWeaver Landscape Virtualization Management

Enhanced security

Enhanced security and compliance for SAP

Efficient, collaborative enterprise governance, risk and compliance (eGRC) program across IT, finance, operations, and legal domains

Data loss prevention

Secure user to network authentication

RSA Archer eGRC

RSA Data Loss Prevention

RSA SecurID

The stand-alone foundation bundle combined with all five add-on bundles form the fully realized CEI for SAP.

The full physical architecture of the CEI for SAP consists of three data centers. Production Data Center A and Data Center B are co-located within approximately 100 kilometers (km), with cross-site IP/SAN connectivity and Layer 2 network. Data Center A and Data Center B together provide active/active access, cross-data-center high availability, application mobility, and maximum resource utilization. In addition, Data Center A and Data Center B are equipped with RecoverPoint to provide DR, while Data Center B is also equipped with Avamar and Data Domain to provide data protection.

A third data center (Data Center C) is situated remotely in a different failure domain, serving as a standby recovery data center that contains replicated datastores and backup copies. Data Center C also houses a VPLEX witness cluster to monitor the status of the two production sites. EMC Cloud-Enabled Infrastructure for SAP – VNX Reference Architecture provides more details.

In this white paper, we focus on a two-site scenario (Data Center A and Data Center C), where the DR bundle is implemented on a foundation bundle without a dedicated high-availability data center (Data Center B). The DR configuration for Data Center B is similar. See DR design for three data centers in the Appendix for the design overview for three data centers.

In this solution, we built Data Center A for the cloud services upon which all the management and SAP systems run. We built Data Center C as a remote standby data center to add DR capabilities. Data Center C is located far enough from Data Center A to avoid a natural disaster from affecting both sites. The hardware resource and network topology in Data Center C and Data Center A are quite similar, as shown in Figure 2.

Physical architecture



Figure 2. Physical architecture

The ESXi servers and storage are totally reserved for DR. Proper sizing should be taken into account before providing DR capability. For example, if the total resource for all tenants subscribing to DR capability in Data Center A is 200 GHZ CPU, 2048 GB memory, and 50 TB storage, the resource reserved in Data Center C should be no less than this capacity.

Table 4 lists the hardware resources for this solution and includes the minimum requirements for validating the solution.

Table 4. Solution hardware resources

Purpose Quantity Configuration

Storage (Data Center A) 1 EMC VNX8000

Storage (Data Center C) 1 EMC VNX5600

Storage replication 4 Two RecoverPoint Appliance (RPA) clusters:

• Two RPAs in Data Center A

• Two RPAs in Data Center C

VMware ESXi servers for the management cluster

4 Two eight-core CPUs, 256 GB RAM:

• Two ESXi servers in Data Center A

• Two ESXi servers in Data Center C

Hardware resources



14

Purpose Quantity Configuration

VMware ESXi servers for the resource cluster

4 Two eight-core CPUs, 256 GB RAM:

• Two ESXi servers in Data Center A

• Two ESXi servers in Data Center C

SAN-switching and network-switching/routing platform

4 Backbone SAN switches

2 Backbone Layer 3 IP switches

2 Routers

4 10 GbE Layer 2 IP switches

Table 5 details the software resources for the solution.

Note: Other operating system and database combinations supported by the EMC Solutions Support Matrix (SSM) can also be used.

Table 5. Solution software resources

Software Version Purpose

EMC Block OE 05.33 Operating environment for VNX

Active Directory Windows 2008 Central hostname and user management

EMC PowerPath® /VE 5.8 Multipathing software

EMC Unisphere® 1.3.0 VNX management software

EMC RecoverPoint 4.0 P1 RecoverPoint software

EMC RecoverPoint Storage Replication Adaptor

2.1 EMC RecoverPoint integration with VMware SRM

SAP ERP 6.0 EHP 5 IDES/Unicode Reference system

Microsoft SQL Server 2012 Used for all SAP systems

Microsoft Windows Server 2008 R2 SP1 Operating system (OS) for SAP

VMware vCenter Chargeback Manager

2.5.0 Chargeback software

VMware vCenter Server 5.1 Management and resource clusters

Microsoft SQL Server 2008 R2 Used by VMware SRM, VMware vCenter, VMware vCenter Chargeback Manager, and vCloud Director

VMware vCloud Networking and Security

5.1 Networking and security software for vCloud Director

VMware vCloud Director 5.1 Cloud management tool for provisioning virtual data center

VMware vSphere 5.1 Hypervisor hosting all virtual machines

Software resources



Software Version Purpose

VMware vSphere PowerCLI 5.1 Release 2 Windows PowerShell plug-in for managing vSphere

VMware vSphere PowerCLI for Tenants

5.1 Release 2 Windows PowerShell plug-in for managing vCloud Director



16

Disaster recovery for Cloud-Enabled Infrastructure

A DR solution is typically complex because a cloud landscape often involves multitenancy and large-scale environments. This solution integrates products from VMware and EMC to automate the DR process and reduce the complexity of the recovery plan. This section includes the following topics:

DR logical architecture

Recovery modes

Operational cycle

Failover process

Failback process

Figure 3 shows the logical architecture for DR.

Figure 3. Disaster recovery logical architecture

Introduction

Disaster recovery logical architecture



In this solution, Data Center A is the production site while Data Center C is the remote recovery site where hardware resources are reserved for DR. We focus on an active/standby DR scenario where hosts at the recovery site are unused under normal conditions.

We separated the DR process into two phases:

Management group DR—The virtual machines in the management group are not managed by vCloud Director. You can fully automate the DR process with native support of VMware SRM.

VMware vCenter servers are deployed on both data centers. They manage their own ESXi hosts individually. VMware SRM servers are also deployed on both data centers. They continuously synchronize data with each other so that the failover and failback operations can be executed on any of the available data centers.

The DNS server in Data Center C is configured to operate in slave mode. All DNS records updated in the master DNS server in Data Center A are replicated to the slave DNS server in Data Center C to ensure consistent DNS records across the entire infrastructure.

Because of their unique replication technique, the DNS, VMware vCenter, and VMware SRM servers no longer need to be replicated at the storage level by EMC RecoverPoint.

We highly recommend deploying a backup and restore strategy using the data protection bundle to protect these management applications against OS corruption or other logical errors.

Resource group DR—vCloud Director manages the SAP systems in the resource group. We used the RecoverPoint CLI and VMware PowerCLI to automate DR for this phase2. The VMware SRM server executes the scripts after successful recovery of the management group.

ESXi hosts in both Data Center A and Data Center C form a cross-data-center DRS cluster called the advanced cluster. This cluster is managed by a single vCenter server, as shown in Figure 3.

ESXi hosts in Data Center C remain in maintenance mode so that the workload does not run in the recovery site under normal circumstances. When a failover occurs on virtual machines belonging to the resource group, the VMware PowerCLI script we developed shifts the ESXi servers from maintenance mode to active mode, ready for failover.

We used the same VMware SRM to orchestrate both sequences.

2 vCenter Site Recovery Manager 5.1 (or earlier) does not support the protection of SAP workloads in vCloud Director (resource group) at this time. VMware vCloud Director Infrastructure Resiliency Case Study provides details about the integration between VMware SRM and vCloud Director.



18

This solution, through VMware SRM, allows two modes of recovery: Planned Migration and Disaster Recovery, as shown in Figure 4.

Figure 4. Types of recovery in VMware SRM

Both types attempt to replicate recent changes to the recovery site, but they differ in error tolerance.

The Disaster Recovery option is used during the failover process. Real-life disasters often present uncontrollable factors and unforeseen consequences. However, the prime objective of DR is system availability; that is, to reinstate as many services as possible within the RTO and RPO. If the outage breaches the maximum tolerable downtime (MTD), the business is at critical risk of irrecoverable damage. In this case, the organization would rather resume the service first to relieve the operations from immediate danger, and deal with the smaller problems later. This is why the Disaster Recovery option is intended for actual site-wide impact.

The Planned Migration option is used as a step in the failback process. Planned migrations only succeed when no errors are encountered during their execution. This guarantees a trouble-free failback normal operations are finally resumed.

The following section demonstrates how to apply the recovery options in the operational cycle of DR.

Recovery modes



All protected systems, from failover to failback, undergo a series of six states. Figure 5 shows transitions from one state to another. DC stands for data center and S1 through S6 represent the protection states at the data centers.

Figure 5. Operational states transitions

DR operational cycle



20

Table 6 describes the states during the failover and failback processes.

Table 6. Failover and failback transition states

State Description

S1 Both resource group and management group workloads run in Data Center A. Data Center C serves as the recovery site where data are replicated.

S2 A disaster causes Data Center A to fail. The entire infrastructure becomes unavailable, including the ESXi servers, network, storage, and RPA.

S3 IT administrator triggers a DR in VMware SRM at Data Center C. Data Center C becomes the production site and all virtual machines in both the resource group and the management group become active on Data Center C. Service is restored at Data Center C while Data Center A is being returned to an operational state. Until IT restores the original data center (Data Center A), remote protection is no longer possible.

S4 Data Center A recovers and becomes functional. The VMware SRM server and vCenter server in Data Center A are back online and resynchronized with Data Center C. Storage failback has not yet commenced.

The effort required to rebuild and configure Data Center A depends on the disaster’s impact on the site. At worst case, where all hardware devices are irrecoverably damaged and a complete data center rebuild is necessary, you need to reconfigure VNX, RecoverPoint, VMware SRM, and VMware vCloud Suite. The detailed steps are described in the following sections:

Virtual Data Center

EMC storage infrastructure

EMC RecoverPoint

VMware vCenter Site Recovery Manager

S5 IT administrator triggers a reprotect operation in VMware SRM to reverse the original replication direction (that is, from C to A). After reprotection, RPA protects Data Center C and Data Center A becomes the DR data center.

S6 A downtime maintenance is scheduled. IT administrator triggers a planned migration. After migration, Data Center A becomes the production data center once again, but is yet to reestablish replication as a final post procedure.

Back to S1 Finally, reprotection is triggered again to restore the replication from Data Center A to Data Center C. Afterward, Data Center A becomes protected once again and the environment is now considered to be fully restored to its normal state.



Table 7 summarizes the major elements of the six states during the failover and failback processes.

Table 7. Failover and failback process states

States Activity Role of Data Center A

Role of Data Center C

Replication Workloads run on

S1 Normal activity Production Recovery A -> C Data Center A

S2 A full data center outage occurs after a disaster

Outage Standby None None

S3 Workloads fail over to Data Center C

Outage Production None Data Center C

S4 IT rebuilds Data Center A

Standby Production None Data Center C

S5 Resynchronize replication

Recovery Production C -> A Data Center C

S6 Migrate workload back to Data Center A

Production Standby None Data Center A

S1 Resynchronize replication

Production Recovery A > C Data Center A

The failover operation consists of states S1 through S3 in Figure 5. The entire failover process includes:

1. Preparing the storage in Data Center C which consists of the RPA operations.

2. Failing over the management and resource groups:

VMware SRM automates the management group failover.

Custom PowerCLI scripts automate the resource group failover.

Workflow

After a successful management group failover, you can initiate the resource group failover. Figure 6 shows the key steps involved in failing over the resource group.

Failover process



22

Figure 6. Resource group failover workflow

To execute the failover process, perform the following steps, as shown in Figure 6:

1. Enable image access for remote copy in Data Center C. This makes the volumes in Data Center C accessible to the ESXi hosts.

2. Fail over the volume to the remote copy. This turns the remote copy into the production copy while the original copy is not available.

3. Disable VMware High Availability (VMHA) for the DRS cluster. This ensures that VMHA does not restart the SAP virtual machines. We started vApps in Step 8 by using a script so that the correct startup sequence within the vApp will be followed.

4. Rescan the host bus adapter (HBA) and virtual machine file system (VMFS) volumes on the ESXi hosts in Data Center C. All virtual machines in the resource group appear Inactive in the vCenter server inventory after rescanning.

Note: Resolving the unresolved volume issue arising from a universally unique identifier (UUID) conflict is essential. You must perform this process with caution to ensure that the mount process does not result in resignaturing volumes. Critical to the success of the approach described in this solution is that no MoRef or UUID changes occur.

5. Perform Exit maintenance mode for ESXi servers after you mount the datastores.

6. Register the virtual machines to the ESXi hosts because VMHA is disabled to prevent the virtual machines from automatically restarting.

7. Start the vShield Edge gateways before starting the SAP systems to ensure that the networking, security settings, and rules are ready.

8. Start all the vApps in vCloud Director using vCloud PowerCLI scripts. This ensures proper start order within the vApp. In this solution, ABAP System Central Services (ASCS) instance is started first, followed in order by the database instance and then the application instances.



The failback operation includes states S4 through S6 in Figure 5. The failback process requires the following VMware SRM operations:

Execute the reprotect process of Data Center C. This operation reestablishes replication with the direction reversed, that is, from Data Center C to Data Center A.

Perform a planned migration from Data Center C to Data Center A. This operation requires downtime for both the management group and resource group. The required downtime depends on the number of ESXi servers, datastores, and virtual machines. Test Scenario 3: Planned migration demonstrates an estimation of downtime based on our environment. During a planned migration, all applications and storage data are reinstated to Data Center A. After this state, the business operations can be resumed, but Data Center A remains unprotected until storage replication is reestablished.

Execute the reprotect process of Data Center A. This reestablishes the RecoverPoint replication from Data Center A to Data Center C, returning both data centers to their normal operating state. Data Center A then becomes protected against another disaster.

Workflow

Figure 7 shows the failback workflow.

Figure 7. Resource group failback workflow

Failback process



24

To execute the failback process, perform the following steps, as shown in Figure 7:

1. Restart replication in RecoverPoint once Data Center A is fully functional. VMware SRM can restart the replication for the datastores used by the management group in the reprotect operation. You must execute the RPA script for the resource group on VMware SRM server.

A successful reprotection ensures replication of datastores from Data Center C to Data Center A. Thus, the later planned migration can be executed with consistent data.

2. Shut down all the workloads including vShield Edge gateways resource group before planned migration. In this solution, the shutdown operations are included in the scripts.

3. Unmount the datastores from all ESXi hosts in Data Center C.

4. Enable image access for remote copy in Data Center A. This allows ESXi hosts access to volumes in Data Center A.

5. Fail over the volume to the remote copy. This turns the remote copy into the production copy while the original copy becomes replica.

6. Disable VMHA for the DRS cluster. This ensures that VMHA does not restart the SAP virtual machines. We started the vApps in Step 10 by using a script so that the correct startup sequence within the vApp will be followed.

7. Rescan the HBA and VMFS volumes on the ESXi hosts in Data Center A.

8. Enable Exit maintenance mode for the ESXi servers in Data Center A after mounting the datastores.

9. Register the virtual machines to the ESXi hosts, because VMHA is disabled to prevent the virtual machines from automatically restarting.

10. Start the vShield Edge gateways before the SAP systems to ensure that the networking, security settings, and rules are ready.

11. Start all the vApps in vCloud Director using vCloud PowerCLI scripts. This ensures the proper start order within the vApp. In this solution, we started the ASCS instance first, the database instance second, and the application instances last.

12. Enable replication from Data Center A to Data Center C again.



Setting up disaster recovery for CEI

Introduction

Virtual Data Center is the key enabling technology introduced in the EMC Cloud-Enabled Infrastructure for SAP Foundation Bundle – VNX White Paper. This solution reuses the existing architecture in the foundation bundle and configures the DR capability as an add-on. This section describes how to configure the following components:

VMware vCloud Director

Networking

VMware vCenter Chargeback

Configuring VMware vCloud Director

This section details the vCloud Director settings needed to fully integrate the DR bundle with the CEI for SAP.

Figure 8 illustrates the cloud architecture with the DR add-on.

Figure 8. Cloud architecture with the DR bundle

Virtual Data Center



26

You must configure the following components to facilitate the DR capability:

Storage

Provider vDC

Organization vDC

Configuring storage We created new datastores together with storage DRS clusters and storage profiles to enable DR. Table 8 and Table 9 list the storage settings needed in vCloud Director, configured in both data centers.

Table 8. Storage configuration in vCloud Director for Data Center A

Consistency group

Storage DRS cluster

Storage profile

Datastore Replicated

Management group

N/A N/A DS_MGMT_DR_Protected (2 TB) Yes

DR MGMT SRM Infrastructure

N/A DS_MGMT_DR_SRM (2 TB) No

N/A N/A DS_MGMT_Placeholder (10 GB) No

Resource group

DR Gold Tier DR Gold DS_DR_Gold_001 (4 TB) DS_DR_Gold_002 (4 TB)

Yes

DR Silver Tier DR Silver

DS_DR_Silver_001 (4 TB) Yes

Table 9. Storage configuration in vCloud Director for Data Center C

Consistency group

Storage DRS cluster

Storage profile

Datastore Replicated

Management group

DR MGMT SRM Infrastructure

N/A DS_MGMT_DR_SRM (2 TB) No

N/A N/A DS_MGMT_Placeholder (10 GB) No

For the management group, we created a 2 TB datastore DS_MGMT_DR_Protected in Data Center A to serve as the RecoverPoint replication source for the management virtual machines. We then placed the following virtual machines on this datastore:

Active Directory (AD) server

Microsoft SQL Server for vCenter, vCloud Director, and vCenter Chargeback

vCenter server for the resource group

vCloud Director

vShield Manager

vCenter Chargeback



For the resource group, we created two storage DRS clusters in Data Center A to group datastores with different performance levels. RecoverPoint replicates the following datastores:

DR Gold Tier—A storage DRS cluster for placing datastores set to the highest performance

DR Silver Tier—A storage DRS cluster for placing datastores set to high performance

We assigned two 4 TB datastores to the DR Gold Tier and one 4 TB datastore to the DR Silver Tier, and then created two new storage profiles to characterize the datastores with DR capabilities:

DR Gold

DR Silver

We then assigned storage profiles to the corresponding datastores in the storage DRS clusters and configured datastores in the DR Gold Tier with storage profile DR Gold and datastores in the Silver Tier with storage profile DR Silver.

As listed in Table 8 and Table 9, two datastores are marked as not replicated by RecoverPoint in both data centers as follows:

VMware SRM infrastructure—DNS, VMware vCenter server for management group, VMware SRM server, and their Microsoft SQL Server database server are deployed on this datastore. Since an independent VMware SRM server is already deployed in Data Center C, if a disaster occurs in Data Center A, we can trigger a failover from the VMware SRM server directly in Data Center C instead. Therefore, the containing datastore does not need to be replicated anymore.

Placeholder—Required for VMware SRM setup. Placeholder virtual machines consist of a few configuration files deployed on these datastores, including a VMX file. However virtual machine disk (VMDK) files are not placed on these datastores. Therefore, these datastores can be configured with a smaller size.

Configuring the provider vDC In the EMC Cloud-Enabled Infrastructure for SAP Foundation Bundle – VNX White Paper, we defined two provider vDCs (PvDCs): Advanced and Standard. In this solution, we configured only the Advanced PvDC with the DR capability. We based this decision on the design consideration that the Standard PvDC is intended for development, test, sandbox, and POC systems. These may not require a fast and automated DR solution. Instead, we can recover these systems in Data Center C with the backup/restore solution.

To add the DR capability to the Advanced PvDC, add the storage profiles DR Gold and DR Silver to the PvDC storage profile tab.

Configuring the organization vDC After adding the DR storage profiles to PvDC, the DR storage profiles are also available in each existing organization vDC (OvDC) built on the Advanced PvDC. However, EMC recommends creating new OvDCs dedicated for DR storage profiles to group together the vApps with DR capabilities. This simplifies the management efforts during a failover or failback operation.



28

In this solution, we created new OvDCs in two organizations: OrgA_Dedicated_DR in OrgA and OrgB_Committed_DR in OrgB.

Configuring the network

The IP network environment varies in every cloud provider’s data center. This solution introduces a network design that requires minimum change efforts during the failover and failback processes. Figure 9 illustrates the network design.

Figure 9. Network design

For the management group, VLAN 10 in Data Center A and VLAN 100 in Data Center C are designed to handle network activities of management virtual machines needed for the VMware SRM server. VLAN 11 is designed to handle the activities of management virtual machines needed to be protected. In case of a disaster, the protected virtual machines will be restarted in Data Center C on the same VLAN tag (VLAN 11). All the IP addresses of protected virtual machines remain unchanged so that no DNS records updates are required.

For the resource group, VLAN 20 is designed to be an external network for tenants’ front-end access to SAP systems. Within vCloud Director, an organization network is created for each organization. A vShield Edge gateway is also created for each organization network to act as router and firewall.



We recommend establishing a secure WAN connection between two data centers. In this solution we used a site-to-site VPN to secure the connection. This solution has three types of cross-data center communications:

Communication between the vCenter servers for the resource group in Data Center A and ESXi servers in Data Center C.

Communication between the VMware SRM servers in both data centers.

Communication between RPA clusters in both data centers. RecoverPoint replicates data over WAN.

During the failover and failback processes, management virtual machines connected to VLAN 11 and SAP systems connected to VLAN 20 need to be transferred to another data center. Thus, the route table must be updated prior to a failover or failback action. This is a mandatory manual step and prerequisite on routes or any other network devices for a successful failover or failback.

Table 10 summarizes the networks required in this solution.

Table 10. Network configurations

Network Purpose IP address

VLAN 10

(Data Center A only)

For DNS, VMware SRM and vCenter for management with their SQL Server

192.168.10.0/24

VLAN 11 For management virtual machines to be protected

192.168.11.0/24

VLAN 20 External network in vCloud 192.168.20.0/24

VLAN 100

(Data Center C only)

For DNS, VMware SRM and vCenter for management with their SQL Server

192.168.100.0/24

OrgNet_OrgA Organization network for OrgA in vCloud 192.168.30.0/24

OrgNet_OrgB Organization network for OrgB in vCloud 192.168.40.0/24

Configuring infrastructure Chargeback Manager

vCenter Chargeback Manager provides the virtualized infrastructure metering functionality. It integrates with Cloud Director seamlessly, providing the service provider with the ability to chargeback the resource consumed by tenants, and generates cost and utilization reports periodically or on demand.



30

The EMC Cloud-Enabled Infrastructure for SAP Foundation Bundle – VNX White Paper defines several pricing models to differentiate OvDC with different allocation models and storage profiles with different storage performance levels. Several approaches are available to reflect the chargeback of DR capability. You can create and bind a pricing model to a new OvDC with DR capability, with reasonable charges for infrastructure including CPU, memory, and storage. In this way, you can accurately charge the infrastructure and account for the resource reserved in Data Center C. Another simple approach is to set a rate factor to the storage profiles DR Gold and DR Silver to reflect a charge for the DR capability. You only need to set it once and all the vApps deployed on the storage profiles DR Gold and DR Silver will be charged accordingly with base rate times rate factor. That simplifies the management efforts needed to create pricing models for each OvDC compared to the previously described approach. Figure 10 shows how to set the rate factor in vCenter Chargeback Manager.

Figure 10. Setting the factor rate for DR storage profiles

Introduction

The EMC VNX flash-optimized unified storage platform delivers innovation and enterprise capabilities for file, block, and object storage in a single, scalable, and easy-to-use solution.

VNX includes many features and enhancements designed and built on the success of the first generation. From the performance and scalability perspective, these features and enhancements include:

More capacity with optimized multicore optimization (MCx) with Multicore Cache, Multicore RAID, and Multicore FAST Cache

Greater efficiency with a flash-optimized hybrid array

The VNX array-based splitter seamlessly integrates with EMC RecoverPoint, providing reliable replication.

EMC storage infrastructure



Configuring VNX storage

For this solution, we configured all storage into VNX Fully Automated Storage Tiering for Virtual Pools (FAST VP) storage pools. We bound thin LUNs to the storage pools and present thin LUNs to the ESXi servers using masking views. Standard thin LUN sizes of 2 TB and 4 TB are presented from the VNX storage array to the ESXi servers. Both VNX systems at Data Centers A and C have similar configurations. Table 11 details the storage layout for the solution.

Table 11. VNX storage layout

Items Data Center A Data Center C

Management group

2 x 2 TB thin LUNs:

1 thin LUN for management virtual machines that are not replicated to Data Center C, including VMware vCenter, VMware SRM, and its Microsoft SQL Server database server

1 thin LUN for management virtual machines that are replicated to Data Center C

2 x 2 TB thin LUNs:

1 thin LUN for management virtual machines including VMware vCenter, VMware SRM, and its Microsoft SQL Server database server

1 thin LUN as the remote copy

Resource group

3 x 4 thin LUNs:

2 thin LUNs associated with FAST VP policy Gold

1 thin LUN associated with FAST VP policy Silver

3 x 4 TB thin LUNs:

2 thin LUNs associated with FAST VP policy Gold as the remote copy

1 thin LUN associated with FAST VP policy Silver as the remote copy

Journal volume 3 x 800 GB thin LUNs 3 x 800 GB thin LUNs

Repository volume

1 x 8 GB thin LUN 1 x 8 GB thin LUN

Placeholder volume

1 x 10 GB thin LUN 1 x 10 GB thin LUN

Introduction

EMC RecoverPoint is an advanced enterprise-class DR solution designed with the performance, reliability, and flexibility required for enterprise applications. It provides integrated continuous data protection and continuous remote replication to recover applications to any point-in-time (PiT).

RecoverPoint systems enable reliable replication locally with one RPA cluster to up to four remote RPA clusters, and over any distance. RecoverPoint systems support data replication for applications writing to SAN-attached storage using existing Fibre Channel (FC) and Internet SCSI (iSCSI) infrastructures to integrate seamlessly with existing host applications and data storage subsystems. For remote replication, the systems use existing connections to send the replicated data over FC and WAN, and use FC and IP infrastructures to replicate data asynchronously or synchronously. The

EMC RecoverPoint



32

systems provide failover of operations to another cluster in the event of a disaster at the production cluster.

RecoverPoint variants The RecoverPoint family consists of three variants:

RecoverPoint/CL for EMC and non-EMC storage platforms

RecoverPoint/EX for EMC storage platforms (Block), licensed by storage array

RecoverPoint/SE for VNX series, EMC CLARiiON® CX3 and CX4 series, and EMC Celerra® unified storage environments

In this solution, we used array-based RecoverPoint/SE for VNX.

RecoverPoint 4.0 features RecoverPoint 4.0 offers the following new features and enhancements to significantly increase capacity and improve performance, helping to build a more flexible and reliable DR solution:

Support for multisite replication

RecoverPoint can replicate a primary site to up to four remote sites (1:4) for increased data availability. It can also protect data across branch offices with multisite replication from the branches to a central site (4:1).

Any PiT recovery to VMware SRM operations

Asynchronous throughput up to 400 MB/s using distributed consistency groups

Replicated capacity from up to 2 PB per RecoverPoint cluster

Support for synchronous replication over IP (versus FC-only in previous versions)

A new, easy-to-use Unisphere interface for RecoverPoint management

RecoverPoint Appliance The RPA is the RecoverPoint system's intelligent data protection appliance. RPAs manage all aspects of reliable data replication including physical hardware or virtual machines. RPAs are used to attain availability. If an RPA fails, another RPA assumes the workload.

A group of two to eight physical (or virtual) RPAs at the same geographic location working together to replicate and protect data is called an RPA cluster. An RPA cluster supports immediate switchover to another appliance if one of the RPA nodes in the cluster fails. There can be up to five RPA clusters in a RecoverPoint system. To scale up and support a higher throughput rate, you can add up to eight RPAs to an RPA cluster.

RPAs use powerful bandwidth reduction technologies, including deduplication and compression, to minimize the use of bandwidth and dramatically reduce the time lag between the source and target sites when writing data to storage.

RecoverPoint splitter The RecoverPoint splitter is proprietary software that you can install on storage subsystems, and is built into the VNX, Symmetrix VMAX, and VPLEX storage systems.



The RecoverPoint splitter is used to split the application writes so that they are sent first to the RPA and then to their normally designated storage volumes.

Splitters perform this activity efficiently, with little impact on host performance because the RPA performs all CPU-intensive processing necessary for replication. This feature can also be upgraded without disrupting storage access.

Unisphere for RecoverPoint The RecoverPoint 4.0 user interface was developed using Adobe Flex technology and promotes ease of use and a common user experience between the RecoverPoint, VNX, VPLEX, and Symmetrix products.

RecoverPoint includes a dashboard that provides an overview of the system’s health, and all of the information displayed through Unisphere for RecoverPoint is consistently auto-refreshed.

RecoverPoint replication options RecoverPoint supports three modes of replication:

Continuous Local Replication (CLR)—RecoverPoint continuously captures and stores data modifications locally, enabling local recovery from any PiT, with no data loss. CLR supports both synchronous and asynchronous replications.

Continuous Remote Replication (CRR)—RecoverPoint captures local data modifications and replicates data to remote sites through IP or FC connections. CRR supports both synchronous and asynchronous replications.

Continuous Local and Remote Replication (CLRR)—A combination of CLR and CRR that provides both local and remote data replication.

CLR is normally used for PiT recovery when business data or application logic has been severely compromised. CRR is normally used for DR. CLRR thus implies that two replicas are made for every protected volume (local and remote) to protect the business from both scenarios.

This solution demonstrates a CRR with asynchronous replication.

Configuring RecoverPoint consistency groups

RecoverPoint protects volumes using consistency groups. If two data sets are dependent on one another (such as a database and a database log), they should be part of the same consistency group. When creating a consistency group, define the following elements:

Production copy—The production copy consists of all source volumes for a consistency group replication, as well as the production journal volumes.

Local copy—A local copy consists of all target volumes for the replication in local storage for a specific consistency group and the copy journal volumes.

Remote copy—A remote copy consists of all target volumes of the replication in the remote storage for a specific consistency group and the copy journal volumes.



34

Defining consistency groups A consistency group can contain a local copy, remote copy, or both, which facilitates CLR, CRR, and CLRR respectively. Besides containing a mandatory production copy, the consistency group can also contain a local copy, a remote copy, or both copies, depending on the replicating mode it facilitates.

In this solution, we created two consistency groups with remote copy:

The management group, which consists of all the volumes for managing the virtual machines

The resource group, which consists of all the volumes for SAP systems from different tenants

Table 12 details the consistency groups in this solution.

Table 12. RecoverPoint consistency groups

Consistency group Data Center A Data Center C

Management group Production copy volumes: DR_Management_1 (2 TB)

Journal volumes: Journal_1 (400 GB)

Remote copy volumes: DR_Management_1 (2 TB)

Journal volumes: Journal_1 (400 GB)

Resource group Production copy volumes:

DR_Resource_1

DR_Resource_2

DR_Resource_3 (4 TB each)

Journal volumes:

Journal_2

Journal_3

Journal_4 (800 GB each)

Remote copy volumes:

DR_Resource_1

DR_Resource_2

DR_Resource_3 (4 TB each)

Journal volumes:

Journal_2

Journal_3

Journal_4 (800 GB each)

Configuring consistency groups For detailed configuration steps for defining consistency groups, see Configuring consistency group in Appendix.

After the steps to configure consistency groups are completed, RecoverPoint initiates the replication. The first initialization run may take a long time if the volumes to be replicated are very large.

For the management group, VMware SRM takes over the operation of RecoverPoint by using RecoverPoint Storage Replication Adaptor (SRA). Therefore, you need to configure the consistency group for the management group to be externally managed by VMware SRM. Figure 11 highlights the additional configuration required.



Figure 11. Configuring a consistency group to be managed by VMware SRM

Introduction

VMware SRM is the market-leading DR management product. It provides the simplest and most reliable disaster protection for all virtualized applications. VMware SRM leverages a broad set of high-performance storage-replication products to replicate virtual machines to a secondary site.

VMware SRM provides a simple interface for setting up recovery plans that are coordinated across all infrastructure layers, replacing traditional error-prone human processes. You can test recovery plans nondisruptively as frequently as required to ensure that they meet your business objectives. At the time of a site failover or migration, VMware SRM automates both failover and failback processes, ensuring fast and highly predictable RPOs and RTOs.

Integrating SRM with RecoverPoint

VMware SRM integrates with RecoverPoint using SRA. RecoverPoint SRA supports the discovery of arrays attached to RecoverPoint and of consistency groups for the management group. It also supports VMware SRM functions such as recovery and reprotect by mapping VMware SRM recovery plans to the appropriate RecoverPoint actions.

Configuring VMware SRM

Most of the configuration of VMware SRM in this solution is set up for the management group.

For detailed configuration steps, see Configuring VMware SRM in the Appendix.

As mentioned in Disaster recovery logical architecture, we used VMware SRM to orchestrate both the management group and the resource group. We added steps in the recovery plan that invoke our customized VMware PowerCLI and RPA CLI scripts to automate the recovery process of the resource group. Figure 12 shows the configuration needed.

VMware vCenter Site Recovery Manager



36

Figure 12. Automating recovery of resource group

We added steps in different modes—recovery and planned migration—to perform relevant operations in RPA and vCloud Director.



SAP system architecture

This section describes the SAP system architecture deployed for the solution in the two data centers. The SAP application layer uses the following SAP components:

SAP ERP 6.0 Core IDES Enhancement Package 5

SAP NetWeaver 7.02

SAP ASCS

SAP Database Service (database)

SAP Primary Application Server (PAS)

SAP Additional Application Servers (AAS)

All SAP instances were deployed on vCloud environment with the Windows Server 2008 R2 SP1 operating system.

Note: Other operating systems and database combinations supported by the SAP Product Availability Matrix (PAM) and the EMC Solutions Support Matrix (SSM) can also be used with this solution.

SAP ERP 6.0, powered by the SAP NetWeaver technology platform, is a world-class, fully integrated enterprise resource planning (ERP) application that fulfills the core business needs of midsize companies and large enterprises across all industries and market sectors. SAP ERP 6.0 delivers a comprehensive set of integrated, cross-functional business processes and can serve as a solid business process platform that supports continued growth, innovation, and operational excellence.

SAP Internet Demonstration and Evaluation System (IDES) supports demos, testing, and functional evaluation based on preconfigured data and clients. IDES contains application data for various business scenarios, with business processes that are designed to reflect real-life business requirements and have access to many realistic characteristics. This solution uses IDES to represent a model company for testing purposes.

Introduction

SAP ERP 6.0



38

This solution implements a distributed SAP system architecture, as shown in Figure 13.

Figure 13. SAP system architecture

We decoupled the enqueue server and message server from the central instance and implemented them as services within the ASCS instance. We installed two AASs to provide redundant work processes such as dialog (DIA), background (BGD), update (UPD), spool (SPO), and gateway.

The SAP system deployed for the solution implements these key design features:

The ASCS instance is installed with a virtual hostname (SAPVIPE) to decouple it from the virtual machine hostname.

We installed and configured the SAP patches, parameters, basis settings, and load balancing settings according to the SAP Installation Guide and the SAP Notes listed in the References section.

We used VMware best practices for SAP in this solution.

SAP update processes (UPD/UP2) are configured on the AAS instances.

SAP shared file systems, including //SAPVIPE/sapmnt/SID (available to all SAP instances), are stored on the ASCS server and mounted as share folders on the SAP virtual machines.

SAP system architecture

Key SAP design considerations



Design best practices

The hardware resources in Data Center C should be sufficient to hold virtual machines that failover from Data Center A. Do not allocate CPU and memory resources used for DR in Data Center A more than what Data Center C can provide; otherwise, failover may not be successful due to resource shortage, which may result in failure to provide service-level compliance.

We recommend the following design best practices for configuring RecoverPoint:

Ensure that the number of RPAs in an RPA cluster is the same with all RPA clusters in a RecoverPoint system. You can manage all RPAs in an RPA cluster through a floating cluster management IP address. In other words, after you configure RecoverPoint, you can manage the entire installation from a single IP address.

Enable compression on RPA Gen5 because there is almost no performance penalty when using compression.

Enable deduplication for remote replications in asynchronous mode to save bandwidth on RPA Gen4 or Gen5.

Use a distributed consistency group in a large-scale environment if the maximum throughput rate of a single RPA does not sufficiently sustain the write-rate or peaks of the consistency group, or if you expect that a future consistency group will require a higher throughput rate than that of a single RPA.

Running a distributed consistency group on an RPA cluster with three RPAs provides only a small improvement in performance. However, a huge improvement in performance is achieved when a distributed consistency group is run on an RPA cluster with four RPAs.

Ensure that the size of the journal volumes reflect the RPO. You need to determine the expected peak change in the environment to calculate the minimum size of the journal volume. The formula is:

( ) ( )

( )

However, in a multitenancy cloud environment, estimating parameters in the formula is difficult. We recommend that you reserve at least 20 percent of the production volume capacity for the journal volume.

We recommend the following design best practices for configuring VNX storage:

The majority I/O pattern of the journal volumes is write I/O. Therefore, I/O may not benefit from FAST VP technology. We recommend placing journal volumes on thin LUNs of a SAS storage pool.

Avoid using FAST Cache on journal LUNs or pools. FAST Cache is suitable for workloads of high locality, small blocks, and random I/O. The sequential large block I/Os of journals benefit very little from FAST Cache.

Data center

RecoverPoint

VNX storage



40

EMC recommends the following design best practices for configuring a vCloud:

Do not include non-DR storage profiles in an OvDC that contains DR storage profiles. Always create new OvDCs to include storage profiles with DR capability only, or migrate all vApps to the DR storage profile in an existing OvDC and remove the non-DR storage profile. This simplifies management and script development to find vApps that are deployed on DR storage profiles.

Set the Stop Action to guest-shutdown in the vApp properties so that whenever a stop action is executed (whether in the web GUI or by a script) vCloud Director shuts down all the virtual machines in the vApp instead of powering them off.

Set a proper start and stop order in the vApp properties to ensure that the SAP system can be started and stopped correctly. We recommend setting the following order for a SAP system with distributed installation:

First: ASCS

Second: database

Third: PAS and AAS instances

EMC recommends creating OS scripts for starting and stopping SAP application and database instances along with the starting and stopping of the OS. This ensures minimum downtime and reduces the manual efforts needed to start or stop SAP and databases during failover and planned migration.

Cloud

SAP



Test and validation

We initially installed and validated the environment without any preexisting DR or business-continuity protection schemes. We then transformed the environment to a mission-critical business continuity solution described in this white paper. We simulated a disaster that caused a full-production data center to become unavailable, and validated the outcome. The use cases were designed with the following key business considerations:

Operations: 24 hours a day, 7 days a week

RTO: 1 hour

RPO: 30 minutes

RTO and RPO in this paper are based on the current lab environment. The RTO in a customer’s environment depends on the number of ESXi hosts, datastores, and virtual machines on DR datastores. You can calculate your RTO based on the test result. RTO and RPO are configured in the RecoverPoint Link settings. The link bandwidth is limited to 100 Mb/s to better simulate the real situation. Table 13 lists the SAP systems deployed in our environment.

Table 13. SAP systems deployed in our environment

Organization OvDC vApp name Virtual machines

OrgA OrgA_Dedicated_DR SAP ERP 6.05 PRD ASCS: saperpascs

Database: saperpdb

PAS: saperppas

AAS: saperpaas1

SAP ERP 6.05 DEV Centralized: saperpvm6

OrgB OrgB_Committed_DR SAP HR 6.05 PRD ASCS: saphrascs

Database: saphrdb

PAS: saphrpas

AAS: saphraas1

SAP HR 6.05 QAS Centralized: saphrvm6

T-code SGEN is executed on SAP ERP DEV and SAP HR QAS to generate write inputs/outputs (I/Os).

We ran the following tests to validate the solution and demonstrate the full cycle of DR operations:

Test scenario 1: Failover after disaster

Test scenario 2: Resynchronizing to Data Center A

Test scenario 3: Planned migration

Test scenario 4: Reprotection for Data Center A

Overview



42

Benchmark results are highly dependent upon workload, specific application requirements, and system design and implementation. Relative system performance will vary as a result of these and other factors. Therefore, this workload should not be used as a substitute for a specific customer application benchmark when critical capacity planning and/or product evaluation decisions are contemplated.

All performance data contained in this report was obtained in a rigorously controlled environment. Results obtained in other operating environments may vary significantly.

EMC Corporation does not warrant or represent that a user can or will achieve similar performance expressed in transactions per minute.

Virtual machines in the management group and SAP systems in the resource group are running in Data Center A. A disaster occurred and resulted in a site outage at Data Center A. This outage could not be recovered in Data Center A within the RTO. Therefore, a failover to Data Center C was performed to bring services back online.

Test objectives

The test objectives include:

Failover of virtual machines in the management group must be automated by the solution in the correct sequence.

Failover of the SAP systems must be triggered after a successful failover of the management group.

Virtual machines such as database, ASCS, PAS, and AAS within a vApp must be started automatically after failover, following the startup order defined in the vApp.

All of the SAP systems must be restored and back in service within the RTO and RPO.

No errors should appear in the SAP systems.

Test procedure

We used the following test procedure:

1. Run the workload on the SAP system in Data Center A. Run SD benchmarks to fill VBAK with new data at a constant rate (approximately 15 seconds per entry).

2. Take a screenshot of the latest entries in the VBAK table.

3. Power off all ESXi servers in the management group and the resource group.

4. Disable the network ports of the RPAs in Data Center A.

5. Log in to a vSphere client in Data Center C. Navigate to the VMware SRM plug-in and execute the recovery operation. Note the failover procedure and timestamp.

Notes

Test Scenario 1: Failover after disaster



6. Log in to the SAP system—SAP ERP 6.05 PRD—in Org A vCenter server for the resource group. Check the VBAK table to determine data loss and note the data loss time period.

7. Log in to all SAP systems using SAP GUI and check the transactions for errors including SICK, SM51, SM50, SM21, ST22, and DB02.

Results and analysis

Table 14 outlines the failover test results.

Table 14. Failover test results

Test item Success criteria Results

Total failover duration (minutes)

< 60 34

Data loss in resource (minutes)

< 30 3

Failover sequence VMware SRM automates failover of the virtual machines in the management group in the correct sequence.

Successful

The failover script for the resource group must be invoked by VMware SRM as part of the failover process.

Successful

Startup sequence

The SAP virtual machines such as database, ASCS, PAS, and AAS within a vApp that are started by scripts must be started automatically after failover in the startup order defined in the vApp.

Successful

Database and SAP startup scripts are executed along with the OS boot.

Successful

Errors in the SAP systems

0 0

Final evaluation PASSED

As shown in Table 14, the total recovery time is 34 minutes and the total data loss happens within three minutes, with limited bandwidth of 100 Mb/s, which successfully meets the RTO and RPO. There are no errors in the SAP system, which means the data is consistent. Figure 14 shows the data before disaster and the data after failover in table VBAK of the SAP system.



44

Figure 14. RPO in the SAP system

After Data Center A is rebuilt, the customer plans to resynchronize the virtual machines in the management group and resource group that are running in Data Center C. The replication direction is reversed. Data Center C becomes the production site and Data Center A becomes the recovery site.

Test objectives


Resynchronization of virtual machines in the management group must be automated.

The replication direction of the virtual machines in the resource group must be reversed.

No service interruption of SAP systems should occur.

Test procedure


1. Check connectivity of RPA and VMware SRM between Data Center A and Data Center C.

2. Run the workload on the SAP system in Data Center C. Run the SD benchmark to fill VBAK with the new data at a constant rate (approximately 15 seconds per entry).

3. Log in to the vSphere client in Data Center C. Navigate to the VMware SRM plug-in and execute the reprotect operation. If errors occurred, fix the errors and rerun reprotect.

Test Scenario 2: Resynchronizing to Data Center A



4. Log in to the VMware SRM server OS and execute the RecoverPoint CLI script to reverse the replication for the resource group.3

5. During the resynchronization process, continuously check to see if the SAP GUI is terminated.


Table 15 outlines the resynchronization test results.

Table 15. Resynchronization test results for Data Center C


Duration of resynchronization for management (minutes)

Direction of replication must be reversed completely.

Successful (33 minutes)

Duration of resynchronization for resource (minutes)



Service interruption in SAP system

0 0

Final evaluation PASSED

The total duration of resynchronization depends on the total capacity of the volumes in the consistency groups and WAN bandwidth between two data centers. RecoverPoint initiated a full sweep because the link between RPA clusters in Data Center A and Data Center C was broken for a long time during the outage of Data Center A. The services of SAP systems are not interrupted during the resynchronization process.

After Data Center C is resynchronized, the customer plans to migrate virtual machines running in the management group and SAP systems running in the resource group from Data Center C back to Data Center A. A planned downtime is required.

Test objectives


Migration of virtual machines in the management group must be automated by VMware SRM in a predefined order.

Migration of SAP systems in the resource group must be triggered automatically after successful migration of the management group.

All vShield Edge gateways must be gracefully shut down by scripts.

Virtual machines such as database, ASCS, PAS, and AAS within a vApp must be gracefully shut down by scripts. The stop order defined in vApp must be followed.

3 Reprotection of virtual machines in the resource group is a manual process in RPA and can be automated by an RPA script. However, the VMware SRM server does not support adding a step in the reprotect process. A manual execution of the RPA scripts is required.

Test Scenario 3: Planned migration



46

All the SAP systems should be migrated and brought back to service within the specified RTO and RPO.

No errors should be shown in SAP systems.

Test procedure


1. Check the connectivity of RPA and SRM between Data Center A and Data Center C.

2. Stop the SD benchmark and take a screenshot of the latest entries in the VBAK table.

3. Log in to the vSphere client in Data Center A. Navigate to the VMware SRM plug-in and execute the planned migration operation. Note the migration procedure and timestamp.

4. Log in to the SAP system using the SAP GUI and check the transactions for any errors including SICK, SM51, SM50, SM21, ST22, and DB02.

5. Take a screenshot of the VBAK table and compare it with the screenshot taken before the disaster. Note the period of data loss.


Table 16 details the planned migration test results.

Table 16. Planned migration test results


Total failover duration (minutes) < 60 40

Data loss in resource (minutes) < 30 0

Automation The script must be invoked by VMware SRM as part of the entire failover process for the resource group.

Successful

Errors in SAP systems 0 0

Final evaluation Passed

As shown in Table 16, the total migration time is 40 minutes and the total data loss is zero, which successfully meets the RTO and RPO. There are no errors in the SAP system, which means the data is consistent.

The virtual machines migrated back to Data Center A must now be reprotected using VMware SRM and RPA.

Test objectives

The test objectives and procedure are the same as those in Test Scenario 2.

Test Scenario 4: Reprotection for Data Center A




Table 17 details the reprotection test results for Data Center A.

Table 17. Reprotection test results for Data Center A


Duration of reprotect for management (minutes)



Duration of Reprotect for resource (minutes)



Service interruption in SAP system 0 0

Final evaluation Passed

As listed in Table 17, the reprotection duration for both the management group and the resource group are shorter than that of Scenario 2. The full sweep is not performed because the replication between the RPAs at both sites is active before reprotection. We recommend triggering reprotection right after a successful planned migration.



48

Conclusion

This solution offers effective, affordable, and efficient DR protection for CEIs running critical business functions. This strategy significantly improves business agility and scales out for SAP production systems with varying needs. This solution demonstrates the DR bundle that is added on top of the foundation bundle solution by combining products and components from EMC and VMware to:

Build active/standby data centers for SAP environments by using the integrated RecoverPoint splitters for VNX arrays across the two data centers.

Integrate RecoverPoint and VMware SRM to automate the entire DR process.

Develop customized VMware PowerCLI and RecoverPoint CLI script to minimize human intervention during DR for SAP systems managed by vCloud Director.

The EMC CEI DR bundle provides:

Business continuity solution for mission-critical SAP applications in a cloud environment.

Cloud tenants have the option to easily deploy their most critical SAP systems on the DR infrastructures described in this solution. The expense need for DR will be reflected correctly in the chargeback report. The failover and failback processes require no additional intervention from the tenants’ side.

Rapid and automated DR in a cloud environment powered by VMware PowerCLI and RecoverPoint CLI.

The DR administrator can easily complete the entire DR process with a few clicks. This is critical to ensure RTO in a large-scale cloud environment.

In addition to the conclusions drawn from the test results, we identified the following points of interest during our analysis:

Test Scenario 1 (failover after disaster)—The failover process can be executed on Data Center C without first bringing up components like vCenter or VMware SRM in Data Center A. The start script defined in the OS level to start DB and SAP further reduces human intervention, thus ensuring the compliance of RTO.

Test Scenario 2 (resynchronizing to Data Center A)—The services in both the management group and the resource group are not interrupted during the reprotection process.

Test Scenario 3 (planned migration)—The operation cannot be successful unless all errors in the process are eliminated, ensuring the data integrity and zero RPO.

Test Scenario 4 (reprotection for Data Center A)—The duration of reprotection is shorter than in Test Scenario 2 because the replication link was not broken and the reprotection was triggered after a successful planned migration.

Summary

Findings



References

For additional information, see the following EMC documents (available on EMC.com and EMC Online Support):

EMC Cloud-Enabled Infrastructure for SAP Foundation Bundle—VNX White Paper

EMC Cloud-Enabled Infrastructure for SAP—VNX Reference Architecture

EMC RecoverPoint Installation and Deployment Guide

EMC RecoverPoint 4.0 Administrator’s Guide

EMC RecoverPoint Adapter for VMware vCenter Site Recovery Manager Release Notes (version 2.x)

EMC RecoverPoint 4.0 CLI Reference Guide

EMC RecoverPoint Deploying with VNX/CLARiiON Arrays and Splitter Technical Notes

EMC Transforms IT for SAP: Fully Automated Disaster Recovery White Paper

For additional information, see the following VMware documents:

VMware vSphere 5.1 Command-Line Documentation

VMware vCenter Site Recovery Manager 5.1 Installation and Configuration

VMware vCenter Site Recovery Manager 5.1 Administration

VMware vCloud Director Infrastructure Resiliency Case Study

SAP Solutions on VMware: Best Practices Guide

For additional information, see the following SAP documents:

SAP Note 1056052—Windows: VMware vSphere Configuration Guidelines

Installation Guide: SAP Enhancement Package 5 for SAP ERP 6.0—Based on SAP NetWeaver 7.0 including Enhancement Package 2

EMC documents

VMware documents

SAP documents



50

Appendix

For the management group, we created a consistency group named Management_Group, which required the following steps:

1. Define the consistency group and select the production volumes. Figure 15 shows the parameter required for continuous configuration.

Figure 15. Defining consistency group and select production volumes

2. Define a production journal as shown in Figure 16.

Figure 16. Defining production journal

3. Add a remote copy. We selected a volume from Data Center C as a remote copy. The RPO parameter is defined in this step and can be changed later in the consistency group settings. We selected asynchronous replication mode because Data Center C is located in a remote site connected by a WAN. Figure 17 shows the parameters required for adding a remote copy.

Configuring consistency group



Figure 17. Adding a remote copy

4. Define a copy journal as shown in Figure 18.

Figure 18. Defining copy journal

The same configuration steps also apply to creating the consistency group for the resource group.

The following configuration requirements are discussed in this section:

Site pairing

Array-based replication

Resource mapping

Configuring VMware SRM



52

Protection groups

Recovery plans

Site pairing

Site pairing establishes a connection for the VMware SRM server between the production site (Data Center A) and recovery site (Data Center C). Figure 19 shows the configuration steps.

Figure 19. Site pairing

We recommend installing the VMware SRM server certificate in Data Center A to the remote vCenter server in Data Center C so that the communication between two data centers is secured.



Array-based replication

Configure the array-based replication after site pairing. We used SRA to establish a connection between VMware SRM and RecoverPoint. Figure 20 shows the configuration required in Data Center A.

Figure 20. Configuration of array-based replication

Follow the same steps to add Array Manager in Data Center C. After configuring Array Manager in both data centers, we enable array pairs in both VMware SRM servers. As a result, VMware SRM can automatically retrieve the consistency group information in RecoverPoint.

Resource mapping

Resource mapping is critical to ensure that the virtual machine settings, especially the network configuration, are correct and usable after failover and resynchronization. Three types of mappings must be configured: Resource pool, network, and folder. During failover and resynchronization, VMware SRM automatically changes the corresponding settings to the mapped resource. Table 18 lists the mappings between our two data centers.



54

Table 18. Resource mapping

Mapping Data Center A Data Center C

Resource pool MGMT Protected MGMT Recovery

Network dvPG_VLAN11 dvPG_VLAN11

Folder MGMT Protected MGMT Recovery

Protection groups

A protection group is a collection of virtual machines and templates that you protect together using VMware SRM. In this solution, we use array-based replication; therefore, a protection group consists of one or several consistency groups defined in RecoverPoint. VMware computes the set of virtual machines in the consistency groups. Figure 21 shows the configuration steps for creating a protection group.

Figure 21. Configuring a protection group

All the management virtual machines that need to be protected by VMware SRM and RecoverPoint are automatically computed and included in the protection group named Management_Group.



Recovery plans

A recovery plan specifies how VMware SRM recovers the virtual machines in the protection groups contained in the recovery plan. One or more protection groups can be included in each recovery plan. Figure 22 shows the configuration steps needed to create a recovery plan.

Figure 22. Configuring a recovery plan

After creating the recovery plan, we specify the restart priority and dependency of each management virtual machine. The priorities depend on the business requirements of each customer. In this solution, we defined the priorities as follows:

First: AD

Second: Microsoft SQL Server

Third: vCenter Server, vShield Manager

Fourth: vCloud Director, vCenter Chargeback



56

Figure 23 shows the DR design overview on top of the HA bundle, which involves two production data centers as a cluster enabled by VPLEX Metro.

Figure 23. Configuring a recovery plan

Customers who have already implemented the HA bundle can chose to deploy RPA clusters in all production data centers. VLPEX splitter is used instead of VNX splitter, enabling storages in the Metro cluster in production data center (Data Center A and Data Center B) to be replicated to Data Center C.

If one of the production data centers has an outage, HA ensures management virtual machines and SAP systems are restarted in another production data center. The RPA cluster in another data center continues to protect the storage. The recovery operation is executed only when both production data centers are unavailable due to the disaster.

DR design for three data centers

emc cloud-enabled infrastructure for · pdf fileemc cloud-enabled infrastructure for...

Documents