1© copyright 2013 emc corporation. all rights reserved. accelerating microsoft exchange performance...

1© Copyright 2013 EMC Corporation. All rights reserved.

ACCELERATING MICROSOFT EXCHANGE PERFORMANCE WITH EMC XtremSW Cache

EMC VNX Storage and VMware vSphere

EMC Solutions GroupSSE Midrange Apps Engineering

March 2013


About this solution

This proven solution demonstrates how EMC XtremSW Cache can accelerate Exchange 2010 performance on EMC VNX storage and VMware virtual infrastructure.

Solution describes the XtremSW Cache design, deployment, best practices, and performance results.

The solution’s detailed whitepaper is published on www.emc.com.

http://www.emc.com/


Solution scope

Evaluate advantages of using XtremSW Cache with Exchange 2010

Evaluate if XtremSW Cache improves Exchange 2010 performance

Evaluate performance advantages of XtremSW Cache with data deduplication option

Provide XtremSW Cache design and configuration best practices for Exchange 2010 on vSphere


Solution Overview

15,000 users in an Exchange 2010 environment deployed in a DAG with two copies on VMware vSphere

VNX storage configured for Exchange database and logs (NL-SAS storage pools)

XtremSW Cache configured to accelerate performance for database volumesMultiple user workload profiles tested (150-300 msgs/user/day)


Solution Components

VNX 5700 Block OE 5.32 (5.32.000.5.015)

– Storage pools with 2 TB NL-SAS (7.2k rpm) disks for Exchange database and logs

– Think LUNs

vSphere 5.0 Update 1– Two servers with Intel Xeon X7560

CPU @ 2.27 GHz, 32 (4x8) CPUs and 192 GB RAM hosting Exchange Mailbox server virtual machines

– Hyperthreading disabled

EMC XtremSW Cache – XtremSF 300 GB PCIe card per ESXi 5.0

server– XtremSW Cache version 1.5.1 SP1 (build

224)

Exchange 2010– 15,000 users– DAG with 2 copies, 6 servers– 5,000 users per Mailbox server virtual

machine– Designed for 150 messages/user/day

profile


About XtremSW Cache

• XtremSW Cache is a server Flash caching solution that reduces latency and increases throughput to improve application performance by leveraging intelligent software and PCIe Flash technology.

– XtremSW Cache accelerates block I/O reads for those applications that require the highest IOPS and/or the lowest response time.

– XtremSW Cache accelerates reads and protects data by using a write-through cache to the networked storage to deliver persistent high availability and disaster recovery.

– Works with array-based EMC Fully Automated Storage Tiering (FAST) software and FAST Cache

– Optimized for both physical and virtual environments


Server

Application CPU PCIe Flash Card

Reads serviced by XtremSW Cache for

Network Storage

Switch RAID Controller VNX Pools

PERFORMANCE

PROTECTION

XtremSW Cache Advanced Architecture

Writes passed through to The storage array for

Low read latencies from PCI Flash

2

8

Pass-through writes for protection by VNX storage

4

6

3

7

5

1

9 Exchange Data


EMC XtremSW Cache components


Solution architecture


Exchange 2010 building block detailsItem ValueTotal number of mailboxes per server building block 5,000 mailboxes/serverMailbox size 1.5 GB per userUser profile 150 messages/user/day (0.150 IOPS)Target average message size 75 KB

Database design

• 6 databases/server

• 833 user per DB

• DB size ~1300 GB

• LUN size 1650 GB

Log design 6 Log LUNs (90 GB LUN size)

Number of Exchange Mailbox virtual machines per ESX 3

Disk configuration per server 18 (16 DB+2 Logs), 2TB NL-SAS drives

Memory/CPU per virtual machine recommended 32 GB RAM, 29040 CPU megacycles

Deleted items retention window (“dumpster”) 14 days

Logs protection buffer 3 days

24 x 7 BDM configuration Enabled

Database read/write ratio 3:2 (in mailbox resiliency configuration)


Storage design with XtremSW Cache Two storage pools created for

databases– 48 x 2 TB 7.2k rpm NL-SAS drives per pool, RAID 1/0

Each pool contains multiple copies from different virtual machines

– 3 Building blocks (3 virtual machines)– 18 x 1.6 TB LUNs (6 LUNs per virtual machine)

326 GB VFMS datastore is created from the XtremFS card on each vSphere server

– 50 GB cache devices created for each Exchange virtual machine from the VMFS cache datastore

– Remaining capacity reserved for virtual machines that can be migrated from the other vSphere server


XtremSW Cache configuration details• Each Exchange virtual machines is configured with 50 GB Cache

device

• 6 source devices - 6 database LUNs (RDMs) 1.6 TB each are added to XtremSW Cache device

• XtremSW Cache page size and max IO block size are set to 64KBCLI command used to add cache devices to virtual machines:

vfcmt add -cache_dev <harddisk13> -set_page 64 max_io_size 64


Why change XtremSW Cache Max I/O size for Exchange?

Exchange DB page size is 32 KB

Exchange BDM is 256 KB read

When both I/O are mixed and coalesced on the host, they produce an average I/O of ~48 KB

When DB read IO is mixed with BDM I/O, the average read I/O size is ~128+ KB

Exchange 2010 DB I/O size on the VNX storage


How to create cache device

1. Using vCenter VSI Plug-in select Create XtremSW Cache device

2. Select vSphere server with XtremSF PCIe flash card

3. Select cache device size and virtual machines, and click OK to finish.


XtremSW Cache configuration for Exchange 2010 Mailbox server virtual machine


XtremSW Cache disk on the Exchange virtual machine


Virtual machine migration with the XtremSW cache device

• Use vCenter XtremSW Cache VSI Plug-in

• Ensure enough cache capacity on the destination server

• Cache will start cold on the destination server for migrated virtual machine Cache effectiveness will be reduced until cache is re-warmed.


Cache device with data deduplication You can enable data deduplication

using the XtremSW Cache VSI Plug-in or CLI on the Exchange virtual machine by executing the following command:

vfcmt add -cache_dev harddisk13 –set_page_size 64 –set_max_io_size 64 –enable_ddup –ddup_gain 20

Where:– harddisk13 is a configured operating-system

cache device for the virtual machine.– ddup_gain 20 is the deduplication gain

percentage for the system cache device on the virtual machine.


Monitoring statistics via the VSI Plug-in


Monitoring statistics via the CLI


vMotion with the XtremSW Cache device Ensure that the following requirements

are met before performing vMotion with XtremSW Cache device attached to the Exchange virtual machines:

– The target datastore has enough available capacity for the new device

– There are no additional DAS flash-based devices for the host virtual machine

– Only one XtremSW Cache device is configured on the host virtual machine

– The virtual machine you want to migrate is not currently being migrated

– The source host and the target host must be able to communicate with each other (ensure that the IP address and DNS have been properly configured)


Solution Validation


Performance validation with Jetstress

Configuration– 3 servers (building-block) in a single VNX storage pool

(48x2 TB NL-SAS drives)– 50 GB XtremSW Cache device created for each Exchange

virtual machine– Cache acceleration enabled only on database LUNs– 7.8 TB dataset per server (23.4 per 3 servers)

Simulated workload– JetStress total throughput test (2 hours)


Exchange performance with Jetstress (IOPS)

IOPS aggregate from three Mailbox servers improved by 26% from 2,812 IOPS to 3,545 IOPS

34% increase in read IOPS from 1,388 IOPS to 1,862 IOPS

33% increase in write IOPS from 851 IOPS to 1,118 IOPS


Exchange performance with Jetstress (latencies) Read latencies

decreased by 3.2 ms.

Slight increase to write latencies


VNX Performance (storage pool) 16.5% decrease in read IOPS to the

back-end storage array because XtremSW Cache offloads the reads from the array to the server

50% increase in write IOPS to the back-end storage due to XtremSW Cache reads offload from the array to the server, allowing more write activity to be processed by write-through cache

15% increase in disk utilization due to the array processing more writes

6.5% increase in bandwidth (MB/s) due to the increased write activity processed by the array


Performance validation with LoadGen

Configuration– 3 servers in a single VNX storage pool (48x2 TB NL-SAS drives)– 50Gb XtremSW Cache device created for each Exchange virtual

machine– Cache acceleration enabled only on database LUNs– Users 1.5 GB mailboxes initialized at 60% (5.4 TB dataset per

server (16.2TB per 3 servers)

• Workload details• Exchange 2010 Loadgen with Outlook cached mode user profile

of 150, 250 and 300 messages/user/day (3:2 read/write ratio)


Performance results with LoadGen 150-msgs/user/day workload

– 51% reduction in read latencies (by6.4 ms)

– 14.6% increase in user IOPS (by 224 IOPS)

250-msgs/user/day workload– 69.3% reduction in read latencies

(by 11.1 ms)– 12.8% increase in user IOPS (by

275 IOPS)

300-msgs/user/day workload– 56.8% reduction in read latencies

(by 12.5 ms)– 12 percent increase in user IOPS

(by 346 IOPS)


Performance with XtremSW Cache data deduplication

Decreased Exchange server CPU utilization with each workload

Slightly increased write latencies due to XtremSW Cache analysis and processing of duplicate data


Performance with XtremSW Cache data deduplication

27.7% decrease in write IOPs to the VNX storage pool


BDM impact on XtremSW cache effectiveness Max I/O is set at 64 k (default)

XtremSW Cache skips fewer read I/O when BDM is not running on the Exchange databases

Read latencies are higher (by 3-4 ms) on databases without BDM compare to databases that have BDM running


Recommendations and Best Practices based on validation


When to consider XtremSW Cache for Exchange workload

Consider XtremSW Cache for Exchange if:– You have an I/O bound Exchange solution– You are not sure about your anticipated workload– You need to guarantee high performance and low latency for

specific users (VIP servers, databases, and so on)


Benefits with XtremSW Cache for Exchange workload

Based on EMC validation, XtremSW Cache improves Exchange 2010 performance by:

– Reduced database read latencies– Increased I/O throughput– Elimination of high latency spikes– Providing more improvements as workload increases– Reduction in RPC latencies– Reduction in reads to the back-end storage– Reduction in writes to the back-end storage with XtremSW Cache

deduplication


Best Practices based on solution validation

• When implementing XtremSW Cache into an existing Exchange 2010 deployment on VMware vSphere, consider the following:

– Size of the XtremSF PCIe card to deploy– Number of Exchange virtual machines deployed on each vSphere

host that will be using XtremSW Cache– Exchange workload characteristics (read:write ratio, user profile

type)– The most benefits will be achieved when all reads from a working

dataset are cached


Best practices based on solution validation

You can install a XtremSF Cache PCIe Flash card on a physical Exchange Mailbox server or hypervisor server hosting Exchange Mailbox virtual machines (VMware or Hyper-V)

Enable XtremSW Cache acceleration only on database volumes

XtremSW Cache sizing guidance:For a 1,000 GB working dataset, configure 10 GB of XtremSW Cache device


Configuration recommendations

• When adding an XtremSW Cache device to an Exchange virtual machine:

– Set the cache page and Max I/O size to 64 KB (BDM I/O will not be cached)Use the XtremSW Cache CLI to add a cache device to a virtual machine with XtremSW Cache release prior to 2.0. With release 2.0 you should be able to configure the page size using the VSI Plug-in.

– Use the following CLI command to set the cache page and max I/O size when adding the cache device to a virtual machine:vfcmt add -cache <cache_device> -set_page_size 64 -set_max_io_size 64


XtremSW Cache with deduplication

Evaluate your workload before considering enabling deduplication for accelerated Exchange LUNs

Consider CPU overhead when enabling deduplication

Set the deduplication ratio based on workload characteristics:− If the observed dedup ratio is less than 10%, turn off deduplication, or

reconfigure the deduplication gain to 0If the observed ratio is over 35%, raise the deduplication gain to match the observed deduplication.

− If the observed ratio is between 10% - 35%, leave the deduplication gain as it is.


Deduplication Configuration Recommendations

Recommendations:− If the observed dedup ratio is less than 10%, turn off deduplication, or reconfigure the

deduplication gain to 0.− if the observed ratio is over 35%, raise the deduplication gain to match the observed

deduplication.− If the observed ratio is between 10% - 35%, leave the deduplication gain as it is.


How to monitor deduplication statistics

Dedup statistics command:vfcmt display -ddup -cache_dev harddisk13

Where: <harddisk13> is a cache disk device on the virtual machine

To calculate the observed deduplication hit ratio (amount of duplicate data in the cache) manually, use this formula:(Write Hits ÷ Writes received) *100

Example: 44414733 ÷ 125477788 = 35.4%

To change the configured ratio, remove the XtremSW Cache device, and recreate a new one using the vfcmt add -cache_dev command


Summary

• XtremSW Cache improves Exchange 2010 I/O and read/ write latenciesWith XtremSW Cache high latency spikes are almost not existent

• XtremSW Cache improves Exchange user experienceRPC latencies are reduced

• Based on observations from testing, we recommend that you set the cache page and max I/O size to 64 KB

• XtremSW Cache deduplication works very well with Exchange and does not affect server performance or read/write latencies

1© copyright 2013 emc corporation. all rights reserved. accelerating microsoft exchange performance...

Documents

xtremsw cachextremsw

emc corporation

serverxtremsw cache

exchange database

networked storage

application performance

arraybased emc

automated storage