emc virtual infrastructure for microsoft exchange 2007 · emc virtual infrastructure for microsoft...

EMC Global Solutions42 South Street

Hopkinton, MA 01748-91031-508-435-1000www.EMC.com

EMC Virtual Infrastructure for Microsoft Exchange 2007

Enabled by EMC Replication Manager, CLARiiON AX4-5i,and VMware ESX Server

Integration Guide

EMC Virtual Infrastructure for Microsoft Exchange 2007 Enabled by EMC Replication Manager, CLARiiON AX4-5i, and VMware ESX Server Integration Guide

Copyright and Trademark Information

2

Copyright © 2009 EMC Corporation. All rights reserved.

Published February 2009

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

Benchmark results are highly dependent upon workload, specific application requirements, and system design and implementation. Relative system performance will vary as a result of these and other factors. Therefore, this workload should not be used as a substitute for a specific customer application benchmark when critical capacity planning and/or product evaluation decisions are contemplated.

All performance data contained in this report was obtained in a rigorously controlled environment. Results obtained in other operating environments may vary significantly.

EMC Corporation does not warrant or represent that a user can or will achieve similar performance expressed in transactions per minute.

No warranty of system performance or price/performance is expressed or implied in this document. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

All other trademarks used herein are the property of their respective owners.


P/N H5982

About this DocumentPurpose................................................................................................... 5Audience ................................................................................................ 6Scope...................................................................................................... 6Related documents ................................................................................. 7

Chapter 1 Use Case OverviewUse case objectives ................................................................................9Environment profile .............................................................................10Architectural overview .........................................................................11Resources .............................................................................................12

Hardware....................................................................................... 12Software........................................................................................ 13

Chapter 2 Use Case DetailsServer and network design ...................................................................15iSCSI best practices .............................................................................17CLARiiON AX4-5i storage design ......................................................19Creating the metaLUNs .......................................................................21

Chapter 3 Test ResultsJetstress testing .....................................................................................23LoadGen testing ...................................................................................32Replication Manager testing ................................................................40Workload characterization ...................................................................49Testing processes ..................................................................................50

Contents

3EMC Virtual Infrastructure for Microsoft Exchange 2007 Enabled by EMC Replication Manager, CLARiiON AX4-5i, andVMware ESX Server Integration Guide

4

Contents

Chapter 4 ConclusionConclusion ........................................................................................... 51Additional information ........................................................................ 52


EMC's commitment to consistently maintain and improve quality is led by the Total Customer Experience (TCE) program, which is driven by Six Sigma methodologies. As a result, EMC has built Customer Integration Labs in its Global Solutions Centers to reflect real-world deployments in which TCE use cases are developed and executed. These use cases provide EMC with an insight into the challenges currently facing its customers.

This document provides an overview of the architecture of an EMC Industry Solution developed by EMC Global Solutions.

Purpose The purpose of this use case is to provide a consolidated, virtualized, out of the box solution for Microsoft Exchange Server 2007 using EMC CLARiiON AX4-5i as the back-end storage array and VMware ESX Server to host the virtual machines (VM). This use case includes all the requirements needed to run this environment from hardware to software, including Active Directory and the required Exchange Server roles. Disaster Recovery (DR) is also included in this solution using EMC Replication Manager.

Information in this document can be used as the basis for a solution build, white paper, best practices document, or training.

Information in this document can also be used by other EMC organizations (for example, the technical services or sales organization) as the basis for producing documentation for a technical services or sales kit.

About thisDocument


About this Document

Audience It is assumed that the reader of this document understands the principles of the following technologies:

• Microsoft Exchange Server 2007

• EMC CLARiiON AX4-5

• EMC Replication Manager

• VMware

This document is intended for:

• Account personnel involved in pre-sales activities relating to Microsoft Exchange 2007, CLARiiON, Replication Manager, and VMware.

• Field personnel who are tasked with implementing virtualized solutions involving Exchange 2007 and VMware.

• Engineering and product development groups, for example, CLARiiON Solution Engineering and Replication Manager development.

Scope This document describes the architecture of an EMC solution built and tested at the EMC Global Solutions lab in Cork, Ireland.

The scope of this solution includes the following:

• Design of the EMC CLARiiON AX4-5i storage array to support up to 2000 heavy Microsoft Exchange Server mailboxes

• Design of the CLARiiON AX4-5i storage array to support Replication Manager clones for Exchange databases and logs on a two-day rotation

• Storage design validation with Jetstress and LoadGen testing

• Replication Manager performance testing and analysis

Implementation instructions and sizing guidelines are beyond the scope of this document. Guidelines on how to install and configure Microsoft Exchange Server 2007 and required EMC products are outside the scope of this document. however, links to the location of the required software are provided in “Related documents” on page 7.

Note: The steps outlined during each stage are high-level in nature and should be read in conjunction with the documentation referenced throughout the guide. The setup of this use case models one example of a Microsoft Exchange 2007 environment. Actual production environments will almost certainly have different behavior in terms of database load (IOPS, read/write ratio, and concurrency). The information contained in this integration guide should therefore be used only as part of a rigorous analysis and design process that takes into account the specifics of each customer environment.

6 EMC Virtual Infrastructure for Microsoft Exchange 2007 Enabled by EMC Replication Manager, CLARiiON AX4-5i, and VMware ESX Server Integration Guide

Related documents

Related documents

The following documents published on the Microsoft website provide additional information on the tools used to validate and simulate Microsoft Exchange Server loads:

• Microsoft Exchange Server Jetstress

• Exchange Server 2007 Load Generator

• Installing Microsoft Exchange Server 2007

The following documents provide additional, relevant information about the required EMC products. Access to these documents is based on your login credentials. If you do not have access to the content listed below, contact your EMC representative.

• Replication Manager Version 5.0.6 Product Guide

• CLARiiON Integration with VMware ESX Server - Applied Technology

• EMC Solutions for Microsoft Exchange Server 2007 CLARiiON CX3 Series iSCSI - Best Practices Planning


http://technet.microsoft.com/en-us/library/bb643093.aspx

http://www.microsoft.com/downloads/details.aspx?FamilyId=0FDB6F14-1E42-4165-BB17-96C83916C3EC&displaylang=en

http://technet.microsoft.com/en-us/library/aa998619.aspx

About this Document


1

This chapter provides an overview of the EMC® virtual infrastructure for Microsoft Exchange 2007 enabled by Replication Manager, CLARiiON® AX4-5, and iSCSI use case. The chapter contains the following sections:

• Use case objectives

• Environment profile

• Architectural overview

• Resources

Use case objectivesThe purpose of this use case is to build a virtual infrastructure for Microsoft Exchange 2007 using the EMC CLARiiON AX4 platform, with VMware ESX Server as the virtual platform, and integrate all the components required to run a complete messaging system.This use case validates the performance of all aspects of the solution and also provides guidelines and best practices for building similar solutions.

Use CaseOverview


Use Case Overview

Table 1details the main objectives of this use case.

Environment profileThe environment profile that was used to configure the storage array optimally for these specific requirements is outlined below. Table 2 presents environment parameters, showing the various options available based on this configuration.

Table 1 Objectives

Objective Details

Define the storage layout • Identify the process used to determine the layout (RAID type, RAID groups, number of spindles, and LUN size) required to support the environment.

Validate the storage layout for Microsoft Exchange Server 2007

• Validate the layout by using the Microsoft Jetstress tool to run performance testing.

• Validate the application performance using the Microsoft Load Generator tool.

Validate the functionality and performance of Replication Manager

• Validate the functionality of Replication Manager with the VMware environment.• Validate the performance of Replication Manager without LoadGen running.• Validate the performance of Replication Manager with LoadGen running.

Table 2 Environment profile

Parameter Value

Number of Exchange 2007 users 2000

Exchange 2007 mailbox servers 1

Exchange 2007 Dual Role Hub/CAS servers 1

Number of Exchange 2007 users per server 2000

Number of Exchange 2007 storage groups per server 8

Number of Exchange 2007 mail databases per storage group 1

Number of Exchange 2007 users per mail database 250

Size of Exchange 2007 user mailbox 300 MB

Storage group database LUN size 90 GB

Storage group log LUN size 25 GB

Local replicas 2


Architectural overview

For more details on how this configuration was achieved, refer to “CLARiiON AX4-5i storage design” on page 19.

Architectural overviewFigure 1 illustrates the use case environment, showing all the physical and virtual components included in this use case and how they interconnect with each other.

Figure 1 EMC Virtual Infrastructure for Microsoft Exchange 2007 enabled by Replication Manager, CLARiiON AX4-5, and ISCSI

SP-A management

Mount host

Production NetworkiSCSI Network 1iSCSI Network 2

VMware ESX Server

Physical Server

Virtual Exchange Servers

EMC CLARiiON AX4-5i storage array

EMC CLARiiON AX4 management ports - 201.0.x.101 SPA iSCSI 0- 202.0.x.102 SPA iSCSI 1- 201.0.x.103 SPB iSCSI 0- 202.0.x.104 SPB iSCSI 1

ESX iSCSI configured for datastore access to CLARiiON AX4-5i storage srray - Two NICs dedicated

One switch with two VLANs (201/202)Two switches are better for fault tolerance, but not required

Two production NICs in fault tolerance on VMware ESX server- Each virtual machine uses this production network

CL4243

Active directory Production

VMware Virtual InfrastructureServer/Physical Domain Controller

Hub/CAS

MBX

201.0.X.25 202.0.X.25

Dedicated NICs and VLANs on ESX server for Exchange iSCSIto CLARiiON AX4-5i

ESX

EMC Replication Manager

Virtual active directory andReplication Manager Servers

SP-B management

Active directory


Use Case Overview

ResourcesThis section describes the hardware and software resources of the use case.

HardwareTable 3 lists the hardware used

Table 3 Hardware resources

Equipment Quantity Configuration

42 rack 1

EMC CLARiiON AX4-5i array 1 2 storage processors2 disk array enclosures24 x 300 GB 15k hard disk drives

Dell R900 ESX Server 1 4 quad-core E7310 Xeon, 1.6 GHz, 4M cache, 80W, 1066 MHz FSB, 32 GB RAM 4 fully integrated Broadcom 5708 Gigabit NICs, TOE capable

Dell SC1435 Domain Controller 1 6 GB RAM Quad-core AMD Opteron 2352 2.1 GHz, 4 X 512K cache1 Ghz HyperTransport

Dell Powerconnect 6224 Ethernet switch

2 24-port copper Gigabit Ethernet Layer 3 switch with 4 combo ports (SFP or 10/100/1000) with resilient stacking and 10 Gigabit Ethernet capabilities


Resources

SoftwareTable 4 lists the software used.

Table 4 Software resources

Title Version

Microsoft Windows Server 2008 x32 Enterprise edition

Microsoft Windows Server 2008 x64 Enterprise edition

Microsoft Windows Server 2003 Standard edition

Microsoft Exchange Server 2007 Service Pack 1

Microsoft iSCSI Initiator 6.0 (Build 6000)

EMC PowerPath® x64 5.1 (Service Pack 2)

EMC Replication Manager 5.2

VMware ESX Server 3.5

VMware vCenter Server 2.5

Jetstress 08.02.0060

LoadGen 08.02.0045


Use Case Overview


2

This chapter provides an overview of the EMC Virtual Infrastructure for Microsoft Exchange 2007 enabled by EMC Replication Manager, CLARiiON AX4-5, and iSCSI use case. This chapter contains the following sections:

• Server and network design

• iSCSI best practices

• CLARiiON AX4-5i storage design

• Creating the metaLUNs

Server and network designOne additional physical server was included to function as a Domain Controller (DC) and a VMware vCenter Server. While there is no problem having a VMware vCenter Server on a virtual host, the decision was made to have a physical host available also in the event of any problems occurring when accessing the VirtualCenter host. This ensures the ESX server can still be managed by the VMware vCenter Server. This physical machine was running on Microsoft Windows Server 2003 to support running VMware Virtual Infrastructure Client (VIC).

Note: VIC is not currently supported to run on Microsoft Windows Server 2008.

The ESX server has 32 GB RAM and four quad-core Xeon processors to ensure that the required performance was achieved for all virtual machines. Running this solution on a server with four dual-core processors instead of four quad-core processors would be sufficient. Forty-five VMs were required to complete this solution. Each VM was configured as shown in Table 5.

Use Case Details


Use Case Details

EMC’s Replication Manager Server must run on Windows 2003 as it is not currently supported to run on Microsoft Windows Server 2008. Once this is supported, it will be possible to have the RM server and the mount host as part of the same virtual server. For this solution, the mount host had to be separated as the mount host must run the same operating system as the production Exchange server.

The solution in total requires three licenses for Microsoft Windows Server:

• Microsoft Windows Server 2003 Standard for the physical DC.

• Microsoft Windows 2003 Server Standard for the virtual RM server.

• Microsoft Windows Server 2008 Enterprise for the remaining four VMs.

The Microsoft Windows Server 2008 Enterprise License allows the use of up to four instances of server software in a virtual environment.

For more information about Microsoft Windows Server 2008 licensing, refer to: http://www.microsoft.com/windowsserver2008/en/us/licensing-faq.aspx.

Table 5 VM configuration

1. Microsoft Domain Controller(Microsoft Windows Server 2008 Enterprise)

• 2 processors• 1 network card• 4 GB RAM

2. Microsoft Exchange Hub/CAS(Microsoft Windows Server 2008 Enterprise)


3. Microsoft Exchange Mailbox(Microsoft Windows Server 2008 Enterprise)

• 2 processors• 3 network cards (1 for

production, 1 for each iSCSI VLAN)

• 12 GB RAM

4. EMC Replication Manager(Microsoft Windows Server 2003 Standard)


5. EMC Replication Manager mount host(Microsoft Windows Server 2008 Enterprise)



http://www.microsoft.com/windowsserver2008/en/us/licensing-faq.aspx

http://www.microsoft.com/windowsserver2008/en/us/licensing-faq.aspx

iSCSI best practices

To spread the load of these VMs, it is recommended to create two separate datastores and load balance these datastores across both storage processors (SP) on the CLARiiON array. Place the VMs as follows:

• SPA

• Active Directory Domain Controller• Exchange Mailbox Servers

• SPB

• Exchange Hub/CAS Server• Replication Manager Server• Replication Manager mount host

Two network switches were used in this solution. These switches offer advanced core switching capabilities for small to medium-sized enterprises.

While two switches are recommended for this solution, this is not an absolute requirement. In this solution, one switch was used for a production network, for all hosts, and the management of the CLARiiON AX4-5i array. The second switch was used solely for the iSCSI communication between the hosts and the array. Having two switches provided for additional fault tolerance in the event of a problem occurring with one switch.

iSCSI best practices For iSCSI, the following best practices are recommended:

1. Use dedicated switches for production and iSCSI networks. If this is not possible, ensure that VLANs can be created on the switch being used so as to guarantee the iSCSI traffic from the server to the array is separated from all other network traffic.

2. Ensure all ports on the switch are set to auto-negotiate to 1000.

Note: It is not possible to disable auto-negotiation on ports on Dell Powerconnect 6224 switches. While the GUI/CMD line will allow the change, the port will become disabled and non-functional.

3. Disable spanning tree on the network switch or switches.

4. Disable power management on the NICs on the server.


Use Case Details

5. Disable each of the following advanced settings on the NICs:

• Jumbo Packet• Pv4 Checksum off load • TCP Checksum Offload (IPv4)• TCP Large send Offload (IPv4)• UDP Checksum Offload (IPv4)

6. On the iSCSI NICs on the Exchange servers, clear the boxes for Client for Microsoft Networks and File and Printer Sharing for Microsoft Networks.

7. Modify the registry with the following changes on the Exchange servers to optimize iSCSI performance:

a. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TCPIP\Parametersi.KeepAliveTime=Dword:300000 (Decimal)

For more details, visit http://support.microsoft.com/kb/324270.

b. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TCPIP\Parameters\Interfaces\GUID

An example of a GUID is 1CDF670F-6155-4652-A317-118489577A22.

Note: This just an example and the text will be different on each interface on each server.

For more details, visit http://support.microsoft.com/kb/328890.

For more details on these best practices, refer to EMC Solutions for Microsoft Exchange Server 2007 CLARiiON CX3 Series iSCSI - Best Practices Planning. This document is available on EMC’s Powerlink® website. If you do not have access to the content, contact your EMC representative.

Note: The AX4-5i is a CLARiiON array. This document contains generic iSCSI best practices that should be applied, regardless of the array.


http://support.microsoft.com/kb/328890

http://support.microsoft.com/kb/324270

CLARiiON AX4-5i storage design

CLARiiON AX4-5i storage designWhen designing a configuration for the storage layout on CLARiiON AX4-5i, the key objectives are that:

• Performance requirements can accommodate the required I/O based on the user profile.

• The design is easy to understand and build on.

• The design helps to minimize the amount of time and complexity involved in configuring the storage layout.

A building block approach was used to determine the number of physical spindles required for this solution. No specific mathematical formula was used to calculate this number.

Jetstress was used to validate the backend performance on the CLARiiON AX4-5i.

The solution was designed for up to maximum of 2,000 Exchange users with a requirement of .5 IOPS per user. Therefore, the configuration needed to achieve 1,000 IOPS to satisfy these requirements. This configuration produced 1,073 IOPS when Jetstress was run. Eight physical disks were used for the Exchange databases. The decision to use eight disks was based on previous Jetstress testing completed for ESRP documents where it was determined that approximately 1,000 IOPS could be achieved with this configuration. This figure was again confirmed with the Jetstress testing for this solution. Further detailed information on this testing is provided in Chapter 3, “Test Results”.

Four physical disks were used for the Exchange log files. The number of disks was based on the numerous tests previously run that determined this number was sufficient to meet the requirements of this solution. This number was validated with Jetstress and LoadGen testing.

Using this building block approach, we know that eight disks provide approximately 1000 IOPS. This allows for a variation in the configuration of this solution. Refer to Chapter 3, “Test Results” for the multiple configurations that can be set up with this storage layout.

Once the number of physical disks has been determined, RAID groups must be created and LUNs bound.

The configuration used for this solution is shown in Figure 2.


Use Case Details

Figure 2 Configuration diagram

RAID 1/0 is the optimal RAID type for Exchange data.The mirrored relationships have been explicitly set during the creation of the RAID group for databases. Each mirrored pair of disks for the database contains one disk from DAE 0_0 and one disk from DAE 1_0.

Exchange logs were placed in a RAID group containing the first four disks and two from DAE 1_0. The first four disks in the array contain the FLARE® operating system. Testing has determined that this does not cause any negative performance impact or cause any problems on the array. Two additional disks were added to this RAID group as the VMDK files for the virtual machines are also on these disks.

Eight physical disks were required to support a two-day rotation of EMC Replication Manager clones.


Creating the metaLUNs

Interleaved, concatenated metaLUNs were used in this solution. MetaLUNs provide for an increase in performance over regular LUNs. During the testing phase metaLUNs created within the same RAID group were tested and metaLUNs spread across RAID groups were tested. There was no significant difference in terms of throughput and response times. For more information, refer to Chapter 3, “Test Results”.

Creating the metaLUNsTesting was completed on another CLARiiON AX4 system to determine the fastest part of the physical disk. The storage array on which this test was run is not relevant as only the physical disk was tested.

Eight regular LUNs were bound in the order from SGDBLUN1 up to SGDBLUN8. Figure 3 provides a graphical representation of the throughput and response times. As can be seen, the middle LUNs have the best response times with the first and last LUNs being the slowest. This confirms that the middle sectors of the physical disk are the fastest.

Figure 3 Throughput and response times

This data was used to determine the components to place together when creating the metaLUNs.


Use Case Details

For example, using LUNs 1-8 to create four metaLUNs bound in the following order:

1 (slower)

2 (slow)

3 (fast)

4 (fastest)

5 (fastest)

6 (fast)

7 (slow)

8 (slower)

and applying the knowledge that the middle LUNs will be fastest, the first component of the metaLUN should be the fastest component. Therefore, the optimal metaLUN configuration is as follows:

4-1

3-2

5-8

6-7

This method was applied to the database metaLUNs in this solution.


3

The chapter contains the following sections:

• Jetstress testing

• LoadGen testing

• Replication Manager testing

• Workload characterization

• Testing processes

Jetstress testingTwo categories of test scenarios are available with Jetstress:

• Exchange mailbox profile

Using this method a specific I/O requirement is specified and the system is then tested to see if the required throughput is achieved while maintaining Microsoft recommended thresholds.

• Disk subsystem throughput

Using this method maximum throughput can be determined and the breaking point for response times can also be determined.

In this solution the disk subsystem throughput was used to determine the maximum throughput that can be achieved with this configuration. This allowed us to determine the breaking point of this configuration and also allowed for multiple configurations to be achieved with the storage layout. Refer to Table 6 for the various configuration options available.

Test Results


Test Results

Jetstress testing was run for two hours as previous testing has shown that this provides similar performance results to running the test for anything up to eight hours.

Two metaLUN configurations were compared during Jetstress testing to determine if there was any significant difference between them. This included creating metaLUNs within the same RAID group and creating metaLUNs that spanned across the RAID groups.

The first configuration of metaLUNs within the RAID group produced a maximum I/O per second of 1073.094. The breakdown, including response times, is represented in Figure 4 and Table 7.

Table 6 Configuration options

Number of user mailboxes Profile User mailbox size

2000 User Profile .5 IOPS 300 MB

1500 User Profile .6 IOPS 400 MB

1000 User Profile 1 IOP 600 MB

500 User Profile 2 IOPS 1.2 GB

250 User Profile 4 IOPS 2.4 GB

125 User Profile 8 IOPS 5 GB


Jetstress testing

Figure 4 Performance results - first configuration

Table 7 Disk subsystem performance

LogicalDiskAvg. Disk sec/Read

Avg. Disk sec/Write

Disk Reads/sec

Disk Writes/sec

Avg. Disk Bytes/Write

Database (S:\T1SG1DB) 0.019 0.015 62.107 72.341 (n/a)

Database (T:\T1SG2DB) 0.018 0.014 62.647 72.474 (n/a)







Log (T:\T1SG1LOGS) 0 0.006 0 43.813 4477.224

Log (S:\T1SG2LOGS) 0 0.006 0 43.756 4479.906


Test Results

The second configuration of metaLUNs spanning multiple RAID groups produced a maximum I/O per second of 1098.552. The breakdown of this test, including response times, is represented in Figure 5 and Table 8.

Figure 5 Performance results - second configuration

Log (T:\T1SG3LOGS) 0 0.006 0 44.376 4519.053

Log (S:\T1SG4LOGS) 0 0.006 0 42.414 4538.495

Log (T:\T1SG5LOGS) 0 0.006 0 43.543 4510.555

Log (S:\T1SG6LOGS) 0 0.006 0 42.482 4593.585

Log (T:\T1SG7LOGS) 0 0.006 0 42.968 4583.994

Log (S:\T1SG8LOGS) 0 0.006 0 43.355 4569.706



Avg. Disk sec/Write

Disk Reads/sec

Disk Writes/sec



Jetstress testing


As can be seen from these two tests, the difference between the I/O achieved and response times is negligible. To reduce the complexity of creating metaLUNs it was decided to create metaLUNs within the RAID group. This is a simpler option than creating metaLUNs that span multiple RAID groups.

Table 9 shows the host details from the Exchange Server mailbox during the Jetstress test.


Avg. Disk sec/Write

Disk Reads/sec

Disk Writes/sec


Database (Q:\T2SG1DB) 0.018 0.015 64.174 72.917 (n/a)

Database (R:\T2SG2DB) 0.017 0.014 63.787 72.976 (n/a)







Log (R:\T2SG1LOGS) 0 0.006 0 43.327 4694.731

Log (Q:\T2SG2LOGS) 0 0.006 0 43.18 4872.508

Log (R:\T2SG3LOGS) 0 0.006 0 42.785 4823.193

Log (Q:\T2SG4LOGS) 0 0.006 0 43.195 4771.258

Log (R:\T2SG5LOGS) 0 0.006 0 43.855 4673.748

Log (Q:\T2SG6LOGS) 0 0.006 0 44.346 4773.595

Log (R:\T2SG7LOGS) 0 0.006 0 43.284 4752.859

Log (Q:\T2SG8LOGS) 0 0.006 0 44.084 4717.307


Test Results

During the two-hour tests, the average SP utilization of the CLARiiON array was approximately 17 percent. Maximum SP utilization never exceeded 25 percent, indicating that the CLARiiON AX4-5i array is more than capable of sustaining the required I/O.

SP utilization during the Jetstress testing is detailed from Figure 6 through Figure 10.

Table 9 Host details from the Exchange Server mailbox

Performance Counter Value

Microsoft Recommended Value Details of Counter

Processor

% Processor Time 18% n/a

Memory

Available Mbytes 8659 >100 Shows the amount of physical memory, in megabytes (MB), immediately available for allocation to a process or for system use.

Free System Page Table Entries 33558322 n/a Free System Page Table Entries is the number of page table entries not currently in used by the system.

Pool Nonpaged Bytes 55174548 n/a Consists of system virtual addresses that are guaranteed to be resident in physical memory at all times and can thus be accessed from any address space without incurring paging input/output (I/O).

Pool Paged Bytes 106547102 n/a Shows the portion of shared system memory that can be paged to the disk paging file.


Jetstress testing

Figure 6 SP utilization summary

Figure 7 SP utilization detail


Test Results

Figure 8 Response times


Jetstress testing

Figure 9 Total throughput


Test Results

Figure 10 Queue length

LoadGen testingBecause the Jetstress tool only allows for disk subsystem analysis and does not test client response times, it was also necessary to run LoadGen to verify that Exchange performance statistics were sufficient to ensure acceptable client response times.

Although the required I/O throughput per user was .5 IOPS, it was not possible to achieve this with LoadGen. The LoadGen user profile used in this testing was Microsoft Outlook Online and Very heavy User. The LoadGen test was run twice for eight hours. Databases were restored between tests.

Both LoadGen tests ran successfully and performance results were within the Microsoft recommended thresholds.


LoadGen testing

Note: For more details on monitoring mailbox servers, visit http://technet.microsoft.com/en-us/library/bb201689.aspx

Table 10 and Table 11 detail some of the performance results from the mailbox server and the CLARiiON array. The numbers were averaged across both test runs.

Table 10 LoadGen tests - performance results[



Disk Transfers 616 n/a The rate of read and write operations on the disk.

Database Page Fault Stalls/Sec 0 0 Shows rate that database file page requests require of the database cache manager to allocate a new page from the database cache.

Log Record Stalls/sec 0 < 10 (max <100) Shows number of log records that cannot be added to the log buffers per second because the log buffers are full or log buffer is a bottleneck.

Log Threads Waiting 0 < 10 on average Regular spikes concurrent with log record stall spikes indicate the log disks are a bottleneck.

MSExchangeIS

Client:RPCs failed:Server Too Busy 0 0 Shows client-reported rate of failed RPCs since the store was started.

RPC Averaged Latency 8.12 Should not be higher than 25ms on average

For information about how clients are affected when overall server RPC averaged latencies increase, see RPC Client Throttling.

RPC Num. of slow packets 0.025 < 1 on average< 3 at all times

Shows number of RPC packets in past 1024 packets that have latencies longer than 2 seconds.

RPC Requests 4.79 < 70 at all times Indicates overall requests that are currently executing within the information store process.

MSExchangeIS Mailbox

Messages Delivered/sec 9 n/a




Test Results

* 50 percent of this processor usage is from msftesql, which is responsible for indexing and searching on Exchange 2007

During this test the physical ESX Server displayed approximately 10 percent of the CPU utilization and had 16 GB RAM available.

Messages Submitted/sec 2 n/a

Messages Queued for Submission 1.85 < 50 at all times and not sustained for more than 15 minutes

Shows the current number of messages that are not yet processed by the transport layer.

Processor

% Processor Time 80% * n/a

Memory





Table 10 LoadGen tests - performance results[




LoadGen testing

Figure 11 through Figure 15 detail the CLARiiON SP utilization during one of the LoadGen tests.



Recommended Microsoft Value

Avg. Disk sec/Write


Database (S:SG1DB) 0.010 < 0.020 0.013 <0.050

Database (T:SG2DB) 0.012 < 0.020 0.014 <0.050

Database (S:SG3DB) 0.012 < 0.020 0.015 <0.050

Database (T:SG4DB) 0.009 < 0.020 0.013 <0.050

Database (S:SG5DB) 0.011 < 0.020 0.014 <0.050

Database (T:SG6DB) 0.010 < 0.020 0.013 <0.050

Database (S:SG7DB) 0.011 < 0.020 0.013 <0.020

Database (T:SG8DB) 0.011 < 0.020 0.013 <0.010

Log (T:SG1LOGS) 0.007 n/a 0.006 <0.010

Log (S:SG2LOGS) 0.005 n/a 0.006 <0.010

Log (T:SG3LOGS) 0.007 n/a 0.006 <0.010

Log (S:SG4LOGS) 0.008 n/a 0.006 <0.010

Log (T:SG5LOGS) 0.006 n/a 0.006 <0.010

Log (S:SG6LOGS) 0.006 n/a 0.006 <0.010

Log (T:SG7LOGS) 0.008 n/a 0.006 <0.010

Log (S:SG8LOGS) 0.009 n/a 0.006 <0.010


Test Results

Figure 11 SP utilization summary


LoadGen testing

Figure 12 SP utilization detail


Test Results



LoadGen testing



Test Results


Replication Manager testingAs expected, there were no problems with the functionality of Replication Manager (RM) within a VMware environment. For details on how to set up RM, refer to “Related documents” on page 7 that lists the RM product guide that can be used for setup.

Performance testing was completed with EMC's Replication Manager to determine the impact and timing of creating clones as follows:

• Testing with no production workload

• Testing with production workload


Replication Manager testing

Testing with no production workloadThe first test run was to complete a full synchronization of the database and log volumes without putting any load on the production Exchange server. RM was configured with two jobs, each of which contained four databases and log volumes to be replicated. Table 12 lists the results of this initial synchronization.

SP utilization on the CLARiiON during the synchronization of all LUNs is shown in Figure 16. Maximum utilization never exceeds 50 percent. The response times are displayed in Figure 17.

Figure 16 SP utilization on CLARiiON during synchronization

Table 12 Initial synchronization results

RM job Exchange SGs Time to Complete Time to Sync Time for consistency check

1 SG1-SG4 4 hours 30 minutes 3 hours 40 minutes 44 minutes

2 SG5-SG8 4 hours 25 minutes 3 hours 25 minutes 41minutes


Test Results


Performance data was also gathered on the mount host during the creation of the replica. The impact on this host was expected to occur during the consistency check. Each RM job contained four databases and RM runs the consistency check on these databases in a serial manner. (It is possible to run them in parallel if all databases are part of the same storage group, but this is not the case in this solution).

Because two of the RM jobs overlapped, there was a maximum of two consistency checks running at the same time. The percentage of processor time on the mount host during the checks averaged at 45 percent. There was no negative impact to the mount host, which was more than capable of handling this load.

Testing with the production workloadReplication Manager testing was also completed with LoadGen running to determine the impact of creating replicas while a load is running against the Exchange Server. This is similar to creating replicas during the production day.



Note: LoadGen was running with 100 percent concurrency and this is peak load.

LoadGen was run for eight hours and the creation of replicas was scheduled to occur three hours into this day. The purpose was to determine the impact, if any, to the CLARiiON system and the Microsoft Exchange Server.

The results of this test are show in Table 13 through Table 15.

Table 13 Replication Manager with LoadGen test results

RM job Exchange SGs Time to Complete Time to Sync Time for consistency check

1 SG1-SG4 2 hours 22 minutes 61 minutes 72 minutes

2 SG5-SG8 2 hours 11 minutes 55 minutes 69 minutes

Table 14 Performance counter results



Processor

% Processor Time 76% * n/a

Memory






Test Results

* 40 percent of this processor usage is from msftesql, which is responsible for indexing and searching on Exchange 2007.

During this test the physical ESX Server displayed approximately 32 percent of CPU utilization and had 13 GB RAM available.

Graphical representations of performance on the array are shown in Figure 18 through Figure 20




Avg. Disk sec/Write


Database (S:SG1DB) 0.015 < 0.020 0.022 <0.050

Database (T:SG2DB) 0.013 < 0.020 0.017 <0.050

Database (S:SG3DB) 0.017 < 0.020 0.024 <0.050

Database (T:SG4DB) 0.014 < 0.020 0.016 <0.050

Database (S:SG5DB) 0.016 < 0.020 0.021 <0.050

Database (T:SG6DB) 0.014 < 0.020 0.019 <0.050

Database (S:SG7DB) 0.017 < 0.020 0.019 <0.050

Database (T:SG8DB) 0.014 < 0.020 0.019 <0.010

Log (T:SG1LOGS) 0.007 n/a 0.009 <0.010

Log (S:SG2LOGS) 0.007 n/a 0.011 <0.010

Log (T:SG3LOGS) 0.007 n/a 0.008 <0.010

Log (S:SG4LOGS) 0.009 n/a 0.011 <0.010

Log (T:SG5LOGS) 0.007 n/a 0.009 <0.010

Log (S:SG6LOGS) 0.010 n/a 0.011 <0.010

Log (T:SG7LOGS) 0.006 n/a 0.009 <0.010

Log (S:SG8LOGS) 0.009 n/a 0.011 <0.010



.



Test Results





Performance results from testing with the production workload are shown in Table 16.

Table 16 Performance results

Performance Counter ValueMicrosoft Recommended Value Details of Counter

Disk Transfers 568 n/a The rate of read and write operations on the disk.

Database Page Fault Stalls/Sec

0 0 Shows rate that database file page requests require of the database cache manager to allocate a new page from the database cache.

Log Record Stalls/sec 0 < 10 (max <100) Shows number of log records that cannot be added to the log buffers per second because the log buffers are full or log buffer is a bottleneck.

Log Threads Waiting 0 < 10 on average Regular spikes concurrent with log record stall spikes indicate the log disks are a bottleneck.


Test Results

SP utilization on the array also increases during this time frame as shown in Figure 21. Average utilization increases approximately 10 percent and the maximum increases significantly. Again, this is to be expected.

MSExchangeIS

Client:RPCs failed:Server Too Busy

0 0 Shows client-reported rate of failed RPCs since the store was started.

RPC Averaged Latency 12.48 Should not be higher than 25ms on average

For information about how clients are affected when overall server RPC averaged latencies increase, see RPC Client Throttling.

RPC Num. of slow packets

.022 < 1 on average< 3 at all times

Shows number of RPC packets in past 1024 packets that have latencies longer than 2 seconds.

RPC Requests 6.315 < 70 at all times Indicates overall requests that are currently executing within the information store process.

MSExchangeIS Mailbox

Messages Delivered/sec

1.127 n/a

Messages Submitted/sec

0.278 n/a

Messages Queued for Submission

10.586 < 50 at all times and not sustained for more than 15 minutes

Shows the current number of messages that are not yet processed by the transport layer.

Table 16 Performance results

Performance Counter ValueMicrosoft Recommended Value Details of Counter


Workload characterization

Figure 21 SP utilization on the array

Workload characterizationTwo workloads were used in the validation of this solution. The first was using Microsoft's Jetstress tool to test the disk subsystem. Jetstress is designed to match the Exchange 2007 server disk I/O ratios and patterns of burstiness to ensure the disk subsystem is capable of producing the required throughput. Jetstress has various thresholds that must be met to ensure a successful pass of a test. A summary report is produced once the test has completed with details of these performance thresholds. While Jetstress simulates Exchange I/O, Exchange is not installed on the host during testing and no clients are used in the testing process. The only thing being tested is the storage itself.

Once the storage performance has been confirmed as acceptable another tool can be used to determine the actual impact to clients and the Exchange application. This is the Microsoft LoadGen tool. LoadGen was used in the second phase of testing to ensure client, server, and application performance is acceptable. LoadGen also simulates Exchange I/O but does so using real mailboxes, databases, user accounts, and distribution lists. Client machines are used to


Test Results

simulate the I/O mimicking a real-world scenario where a client makes a remote connection to the Exchange Server. LoadGen is configured to run for a certain number of mailboxes for a recommended period of eight hours. During this eight-hour period, LoadGen simulates random regular Exchange activities such as sending and receiving e-mails, appointments, browsing calendar entries, creating and deleting tasks, and so on. More information on LoadGen can be found in the LoadGen Help file.

Testing processesThis section describes in detail the steps that were taken to ensure consistency with all the tests that were run on the environment.

• The physical domain controller was set up and the VMware vCenter Server was installed and configured on this.

• VMware ESX Server was installed and configured.

• Virtual servers were set up and configured with the base operating system using VMware vCenter Server.

• Servers were set up and configured with the base operating system, required EMC software, and Microsoft Jetstress.

• Storage LUNs were assigned and configured on the virtual Exchange server.

• For each configuration, Jetstress was run twice for two hours to ensure consistency with results.

• Microsoft Exchange Server 2007 was set up on all required servers and blank databases were created and mounted.

• LoadGen was installed on all required servers.

• LoadGen clients were set up.

• LoadGen databases were initialized.

• LoadGen was run twice, for eight hours at a time, to ensure consistency with the results.

• EMC Replication Manager and mount host servers were set up and configured.

• EMC Replication Manager was run with performance logging to determine the impact and time for the initial synchronization without any workload.

• LoadGen was run while EMC's Replication Manager was also creating clones of production databases to determine what impact, if any, this had on response times and so on.


4

The chapter contains the following sections:

• Conclusion• Additional information

ConclusionThis solution contains all components required to run a messaging infrastructure, which includes an option for protection at a relatively low cost.

The solution provides many benefits to customers on a number of levels:

• Cost-effectiveness

• Simple scalability

• Protection

• Simplified mailbox design

• Virtualization

Cost-effectivenessCost effectiveness is a key driver for leveraging server virtualization in the first place, that is, being able to consolidate many servers into a few. With CLARiiON AX4, EMC extends this value by providing a cost-effective platform with technologies such as iSCSI, where you can leverage an existing Ethernet infrastructure and still maintain superior performance.

Virtualizing the environment provides additional cost benefits in reduced hardware and maintenance costs, reduced rack and floor space, reduced data center power and cooling, and reduced infrastructure costs (cables, switch ports, and so on).

Conclusion


Conclusion

Simple scalabilityPerformance and scalability are also key differentiators for CLARiiON AX4. While this particular design is applicable for up to 2,000 users, the building block allows for other solutions to be scaled up to higher numbers based on that specific approach. The scalability of CLARiiON AX4 is not only evident in terms of capacity and connectivity but its advanced software capabilities are also available to support current or future requirements.

EMC's CLARiiON arrays will not degrade over time. The flat responses from the storage array that can be seen in the results from testing will continue to remain consistently flat as time goes on.

ProtectionReplication Manager is ideal for backup acceleration in creating a “Gold Copy” of production data for instant restore should corruption occur. RM is an easy to use point-and-click application to manage and automate data protection at the array level.

Simplified mailbox designBy using the building block method, Exchange deployments result in predictable performance for all mailbox servers. The building block method removes any assumptions/guesswork when sizing the servers and storage.

VirtualizationWith EMC’s continued partnership with VMware, this solution illustrates the many benefits of a virtualized platform for Exchange 2007.

EMC can help accelerate the assessment, design, implementation, and management of creating a virtualized Exchange 2007 environment while lowering the implementation risk and cost.

Additional informationTo learn more about this, and other solutions, contact an EMC representative or visit www.EMC.com/solutions/microsoft.


www.EMC.com/solutions/microsoft

www.EMC.com/solutions/microsoft

emc virtual infrastructure for microsoft exchange 2007 · emc virtual infrastructure for microsoft...

Documents