giantloop testing and certification (gtac) lab

GiantLoop Testing and Certification (GTAC) Lab

Benchmark Test Results: VERITAS and EMC Replication

Technologies

Prepared For:

March 2003

Project Lead: Mike Schwarm, Director, GiantLoop Testing and Certification Lab

Lead Test Engineer:

Sean Rinas, Principal Engineer

Contributions by: Richard Aseltine, Principal UNIX Administrator

William Roberts III, Senior Engineer Solomon Murungu, Oracle DBA

Copyright GiantLoop Network, Inc. ii March 2003

Table of Contents

Introduction.................................................................................................................................... 1 Approaches to Data Replication .................................................................................................. 1

Host-Based Replication............................................................................................................... 1 Array-Based Replication ............................................................................................................. 1

Comparison Objectives ................................................................................................................ 2 Product Descriptions .................................................................................................................... 2 Replication Modes......................................................................................................................... 2

Synchronous Replication ............................................................................................................ 2 Asynchronous Replication........................................................................................................... 3

Test Environment .......................................................................................................................... 4 Test Results Detail ........................................................................................................................ 5

Manageability .............................................................................................................................. 5 Availability and Robustness ........................................................................................................ 7 Scalability .................................................................................................................................... 9 Performance.............................................................................................................................. 10 Price/Performance .................................................................................................................... 13

Summary ...................................................................................................................................... 15 About the GTAC Lab ................................................................................................................... 16 Appendix I .................................................................................................................................... 17

Figure 1: Sample Disaster Recovery Configuration for VERITAS Volume Manager with Volume Replicator Option (SONET Transport)......................................................................... 17 Figure 2: Sample Disaster Recovery Configuration for VERITAS Volume Manager with Remote Mirroring (DWDM Transport) ....................................................................................... 17 Figure 3: Sample Disaster Recovery Configuration for EMC SRDF (DWDM Transport) ........ 18

Appendix II ................................................................................................................................... 19 Table 1: Synchronous Replication Throughput Over Distance................................................ 19 Figure 4: Synchronous Replication Throughput Over MAN Distance 8KB Block Size............ 20 Figure 5: Synchronous Replication Throughput Over WAN Distance 8KB Block Size ........... 20 Figure 6: Synchronous Replication Throughput Over MAN Distance 128KB Block Size........ 21 Figure 7: Synchronous Replication Throughput Over WAN Distance 128KB Block Size ....... 21 Figure 8: Synchronous vs. Asynchronous Replication ............................................................ 22

Appendix III .................................................................................................................................. 23 Setup and Configuration Details ............................................................................................... 23 TCP and UDP Options over IP.................................................................................................. 24 Fibre Channel Design Considerations ...................................................................................... 24

Copyright GiantLoop Network, Inc. 1 March 2003

Introduction Competitive pressures, new government regulations, and changing security concerns are causing large firms to reevaluate their disaster recovery infrastructure and processes. No longer is it sufficient to simply back up data on a nightly basis and ship physical tapes to an offsite storage facility. An enterprise must examine the requirements of the business and determine how much data loss and how much downtime can be tolerated without affecting the business itself. This approach is causing the organization to not only have a solid backup plan and set up local availability clusters, but also to replicate critical information to remote sites as part of a comprehensive disaster recovery solution. Replication allows either a real-time or near real-time replica or copy of data to exist at an alternate location which can reduce or eliminate potential data loss and dramatically speed application and data recovery in a disaster scenario. With the high (and ever increasing) costs associated with data loss and downtime, more and more enterprises rely on data replication technologies as the foundation for disaster recovery planning. This paper compares the two leading approaches to storage replication and provides insight to the relative strengths and weaknesses of each approach. After reading this report, the IT professional interested in disaster recovery planning will have a better understanding of available data replication technologies and can make a more informed decision on which approach may best suit their organizational requirements. Approaches to Data Replication The two most common types of data/storage replication architectures are host-based replication and array-based replication. Host-Based Replication Host-based replication, also called volume-based replication, is managed from within the host or server. As writes occur to the local disk volumes, the replication process sends those write requests to disk at the remote location as well. This can occur over a variety of network protocols and across various transports. With volume-based replication, the host is responsible for the replication links, making this approach independent of the storage hardware. An example of a leading volume-based replication technology would be VERITAS Volume Manager and VERITAS Volume Manager with the Volume Replicator option. Array-Based Replication Array-based replication manages the replication process from within the storage hardware array. As disk I/O is written from the local host to local disk, those updates are sent to an identically configured hardware array at the remote location. Array-based replication is independent of anything occurring on the host and requires the exact same storage array layout at both the primary and secondary locations. An example of a leading hardware-based replication technology would be EMC Corporation’s Symmetrix Remote Data Facility (SRDF ).


Comparison Objectives The GiantLoop Testing and Certification (GTAC) Lab was contracted by VERITAS Software Corporation to evaluate the usability and performance of enterprise data replication solutions based on VERITAS and EMC data replication technologies. GiantLoop evaluated VERITAS volume replication technologies both on their own merits as well as in direct comparison to EMC’s Symmetrix Remote Data Facility (SRDF). All products were tested in multi-site, remote data replication configurations with approximately one terabyte of data storage per site. The objectives of the comparison were to test and evaluate the performance, stability, and reliability of VERITAS and EMC data replication solutions over various network configurations, traffic loads, and distances. Test configurations were selected that represent typical configurations commonly found in IT organizations. Specifically, VERITAS and EMC technologies were tested with respect to: • Manageability • Availability and robustness • Scalability • Performance • Price/performance Product Descriptions VERITAS Volume Manager is an easy-to-use, online storage management tool for heterogeneous enterprise environments. Volume Manager’s remote mirror capability can speed recovery time and eliminate data loss after a disaster by making current data available immediately at an alternate location. VERITAS Volume Manager can mirror data natively over storage protocols such as Fibre Channel. Also, VERITAS Volume Manager has an optional capability called Volume Replicator, which allows writes to be replicated natively over a standard IP network. VERITAS Volume Manager and the Volume Replicator option are core components in the popular storage foundation products from VERITAS such as VERITAS Foundation Suite or the VERITAS Database Edition. (Source: VERITAS Software Corporation) EMC Symmetrix Remote Data Facility (SRDF) protects information and provides comprehensive business continuity in the face of both planned and unplanned outages. This online, host-independent, mirrored data solution duplicates production site data on one or more physically separate target Symmetrix systems. Systems can be across the room, across the globe, or anywhere in between. (Source: EMC Web site) Replication Modes There are two main modes of replication: synchronous and asynchronous. An IT professional must firmly grasp the differences between these two when implementing a replication-based disaster recovery strategy. Distance between the primary and secondary locations, network bandwidth, latency, network type, and the amount of data loss an organization can tolerate will all affect which mode will be appropriate for any given environment. Both modes have their advantages and disadvantages for the IT professional. Synchronous Replication Synchronous replication is essentially mirroring data over any network. Writes are not committed on the primary system until they are successfully replicated to the remote location. Synchronous replication ensures that the stored information at both the local and remote sites is always identical and up to date. For maximum availability and protection, the primary and secondary data sets should be in separate facilities located far enough apart that they are not subject to the same set of environmental risks (e.g.


major power failure, catastrophic flood, fire, etc.). This involves extending the replication solution over a metropolitan area network (MAN) or wide area network (WAN) infrastructure using either Fibre Channel or IP. Since the performance of synchronous replication technologies degrades as distance increases (due to latency), most synchronous solutions are currently deployed between facilities in and around a metropolitan area network. Both VERITAS and EMC replication technologies offer synchronous replication options which ensure that both primary and secondary storage are accurate, fully up to date, and available for immediate use. Asynchronous Replication For organizations that cannot tolerate the performance impact imposed by network latency or those wanting to replicate over longer distances, asynchronous replication is an ideal choice. Asynchronous replication minimizes latency-related performance problems, which may accompany some synchronous implementations. With asynchronous replication, the secondary site may lag behind the primary site. In a properly configured implementation with sufficient bandwidth between sites, this lag is typically measured in milliseconds. With this in mind, asynchronous replication offers near real-time replication without the application performance impact normally associated with synchronous replication over longer distances. For any asynchronous replication solution to guarantee data integrity at the remote site, it must preserve write order fidelity. This essentially means that writes applied to the secondary storage must occur in the exact same order they were applied at the primary. Asynchronous replication without this capability compromises data consistency at the disaster recovery site and may jeopardize the recoverability of the data VERITAS Volume Manager supports write ordered asynchronous replication over IP networks with the Volume Replicator option. Writes received by the local Volume Manager are written to the primary volumes as well as written to a log or queue volume which is used as a cache while the ordered writes are transmitted to the secondary volumes. This log or queue asynchronously sends writes to the secondary volumes where the data is applied in the exact same order as at the primary. Only when the data has been acknowledged to have safely arrived and placed on persistent storage will the data get purged from the primary log. EMC SRDF does not support a true asynchronous mode option. Instead, EMC offers a semi-synchronous mode that is similar to their synchronous mode. EMC’s semi-synchronous mode allows a single I/O to complete before synchronization completes with the secondary array. This allows for a read to be performed, but a second write will not be performed until synchronization with the secondary has been confirmed. For planning purposes, this mode should be considered in the exact same way that one would plan for a synchronous implementation, as the performance and latency impacts appear to be similar. In addition to semi-synchronous, EMC also provides a periodic replication mode called Multi-hop, based on EMC’s TimeFinder and Adaptive Copy functions. This option sends data periodically and thus provides less data currency when used in a disaster recovery context. In addition, Adaptive Copy does not preserve write order fidelity. Adaptive Copy produces a secondary that is behind the primary and does not maintain write order fidelity.


Test Environment To perform the tests described in this paper, GiantLoop constructed test environments to emulate typical enterprise-class data center networking configurations. These configurations included several components such as high-end fault tolerant servers, enterprise-class storage arrays, director-class Fibre Channel SAN switches, DWDM and SONET transport devices (Nortel OPTera Metro 5200, Cisco Systems ONS 15540, and Cisco Systems ONS 15454), and several hundred kilometers of Corning SMF-28 optical fiber. For the primary site host, a Sun Microsystems Sun Fire 4810 server with 8 CPUs and 8 GB of memory was used for all of the tests. The secondary site consisted of a Sun E450 server with 4 processors and 4 GB of memory. Two EMC Symmetrix 8730 storage arrays were used for the tests. Each array was configured with 4 GB of buffer cache and approximately 1 TB of storage. Primary test volumes used in performance testing were built with 26 columns configured across 13 physical disks. Other volumes were configured across 8 and 4 physical disks. The EMC Symmetrix 8730 had Fibre Channel adapter ports with 2 HBAs connected to separate FA ports on the Symmetrix through a McDATA switch fabric. The software parameters of interest are listed below. Operating system: Solaris 9 Database: Oracle9i, Release 2 DataTest Version 12.0 Benchmark Factory Version 3.3, Patch 021311 EMC SRDF Microcode version 5566 VERITAS Foundation Suite: Release 3.5 (VERITAS Foundation Suite included VERITAS Volume Manager with the Volume Replicator option) It must be noted that the GiantLoop tests primarily consisted of comparisons between the VERITAS and EMC replication solutions in an extended SAN. While both solutions support replication over IP, the VERITAS solution was tested over native IP by enabling VERITAS Volume Manager with the Volume Replicator option with a license key. Although EMC Fibre Channel based SRDF can also replicate over IP, it can only accomplish this by converting Fibre Channel into IP with specialized protocol conversion equipment. Due to the significant performance degradation of the Fibre Channel protocol over extended distances and latencies typically introduced by conversion devices, testing synchronous FC SRDF over a WAN network using FC to WAN or IP conversion was not included in the scope of this testing. Refer to Figures 1-3 in Appendix I for a detailed diagram of the hardware architectures used in the testing.


Test Results Detail Manageability The GiantLoop Testing and Certification (GTAC) Lab conducted configuration baseline tests to capture all relevant installation and configuration information. As a result of these tests, certain conclusions were reached regarding initial setup, manageability options, initial synchronization, and monitoring to provide an overview of available options and general product functionality. The detailed results and analysis are listed in the following matrix.

Item Description VERITAS EMC Analysis

Initial Setup

This test measures the ease of installation and amount of effort required in the initial configuration and setup of the replication.

VERITAS replication was performed with only the documentation as a guide. Assistance from VERITAS Professional Services was not required, even for a first time user. All CLI commands were intuitive, one-line commands. Volume creation was accomplished quickly. Steps to achieve first time synchronization include: Create a volume, add disks to the volume, create the SRL volume, assign VVR memory tunables, setup the replication link at both primary and secondary sites, and start replication.

EMC replication assignments required professional services assistance. All CLI commands were intuitive, one-line commands. First time users perform starting and stopping replication and determining replication mode. Steps to achieve first time synchronization include: LUN discovery, SRDF local group creation, assigns devices to the group, and start replication.

Both VERITAS and EMC provide adequate initial setup capabilities to quickly initialize and set up replication. While both VERITAS and EMC provide professional services, EMC requires these services before any replication can be initialized and if any additional setup parameters require adjustment. (Refer to setup details in Appendix III)

Adding Volumes to Replication

This test evaluates the level of complexity in adding additional data volumes and disks to an existing replication set.

From within the same Volume Manager GUI or CLI, volumes are easily added to (or deleted from) the replication group. Depending on the network type used, this task takes place at either the primary host only or at both the primary and secondary hosts. A system or storage administrator can complete this task.

Storage can be added to the replication set, but this requires EMC Professional Services to modify SRDF disk mapping at the primary and secondary array.

Adding volumes and/or disks to a replication set is a common task that should be easy to accomplish. Both VERITAS and EMC allow for this capability while applications are online. EMC requires additional professional services to accomplish this, including tasks such as changing the bin file, etc. Requiring professional services will typically add additional time and cost to every modification.

CLI

Tests the capabilities and ease of use of the command line interface (CLI).

All replication functions or tasks can be configured using the CLI provided by VERITAS.

Once the disk mapping is configured, EMC’s CLI is simple to use.

Both vendors offer an acceptable CLI to accomplish common configuration, monitoring and management tasks.

GUI

Tests the capabilities and ease of use of the graphical user interface (GUI).

The VERITAS solution has a GUI with full functionality that is accessed from any web-based browser. This functionality is included with the product.

EMC offers a GUI through a control center component called SRDF Manager. This functionality can be purchased for additional cost.

A GUI can simplify management for the administrator as it can be used to provide replication status information and performance statistics, or add additional volumes for replication to an existing configuration.



Initial

Synchronization

Examines the options for performing the initial synchronization between the primary and secondary sites.

Multiple options exist for initial synchronization. Initial synchronization can occur (1) over the replication link, (2) by leveraging a tape-based restore, or (3) by using a restore from disk.

Initial synchronization can occur by physically moving disk arrays for restore or by initializing directly over the SRDF links. EMC does not provide a restore from tape capability.

The restore from tape initialization option is useful, especially when trying to initialize large replication sets or in environments with limited bandwidth.

Monitoring

Tests the monitoring capability of the replication solution

Both the CLI and GUI can be used to display the current status of replication for both primary and secondary locations.

Some limited statistics are available through the CLI and through SRDF Manager.

The ability to monitor replication status provides the IT professional with the necessary tools to ensure their recovery site is available and up to date.

Switch Between

Replication Modes

Tests the ability of the solution to switch between synchronous and asynchronous modes of replication.

User initiated “on-the-fly” switch between synchronous and asynchronous while replication is active. Also, by default, the switch is completely automated and dynamic in the event of replication link failure.

It is possible to switch between synchronous and semi-synchronous modes, however, this requires manual intervention.

The ability to dynamically switch from synchronous to asynchronous mode (and vice versa) is useful if network links become unavailable. The VERITAS solution can switch to asynchronous mode and start logging writes in a queue. Once links have been re-established, the replication log will drain and the product will snap back into synchronous mode when it catches up. This can prevent downtime at the primary if and when replication links go down. EMC has no similar capability.

Protocol Support

Ability to utilize different transport protocol options.

Supports Fibre Channel as well as IP. IP support includes Ethernet, ATM, VPN, SONET, FDDI, token ring, frame relay, leased circuit, packet, and other network layer technologies.

Support for Fibre Channel. IP support for Fibre Channel SRDF may be added with special converters and at additional cost.

Native IP support is useful when replicating over very long distances or where dark fiber for extended SAN connectivity is cost prohibitive. Native IP support can reduce the cost of replicating over WAN.

Dissimilar Storage Between Primary and Secondary

Ability to replicate between different types of storage arrays at the primary and secondary.

Able to utilize any type of storage, regardless of storage vendor, at primary and secondary. The only requirement is that sufficient disk capacity exists at the secondary to store the replica.

Can only replicate if primary and secondary have identical EMC storage.

The VERITAS volume-based approach is managed by the host/server and therefore offers flexibility to replicate across similar or dissimilar storage arrays. In some environments the cost benefit of this capability can be a key differentiating feature.

Operating System Support

Looks at the solution’s ability to replicate data from a variety of operating systems.

VERITAS currently supports Solaris, HP-UX, AIX, Linux, and Windows server environments.

Because EMC SRDF is hardware based, it can support virtually any server operating system.

For the enterprise looking for a single solution for everything from the smallest Windows server to the largest mainframe, a storage array-based approach offers advantages over a host-based approach. For environments searching for solutions exclusively for Unix and Windows operating systems, VERITAS offers a broad range of support.


Availability and Robustness The GiantLoop Testing and Certification (GTAC) Lab conducted robustness and fault tolerance tests to measure how well each solution responded to various failures and protection switches (both manual and forced) in the data path. Failure tests occurred at all points of the solution: Between local server and local storage, between local server and remote server, and between remote server and remote storage. Since the characteristics of most transport failures are well understood by the GTAC Lab, these failures are generated to understand how the products being tested will react to and recover from standard fail-over conditions in the transport network. Complete failures, automatic protection switches (APS), and manual switches were all performed with several transport solutions and with several different fiber distances and delays. To ensure that enterprise customers never experience “production” surprises when deploying this type of data replication solution, actual fiber spools were used by the GTAC Lab to insert, not simulate, real distance between the transport nodes.


Replication Link Failure

Tests the ability to handle Fibre Channel and/or IP replication link failure between primary and secondary. Both manual and forced transport failures and protection switches were tested with VERITAS and EMC.

VERITAS Volume Manager is dependent on fiber and will be affected by a fiber failure. Therefore, full redundancy is recommended. Fibre Channel replication link failures can be mitigated by installing secondary transport links. VERITAS Volume Manager with the Volume Replicator option has the added capability to replicate over IP. In an IP network failure, VERITAS can handle IP network outages and transient failures of the replication link. While full redundancy can help, it is not required.

EMC is dependent on fiber and will be affected by a fiber failure. Therefore, full redundancy is recommended. Fibre Channel replication link failures can be mitigated by installing secondary transport links (i.e. more ISLs), however this will increase the cost of the solution. EMC does not support replication over native IP.

Failure of the replication link will adversely affect replication. It is recommended that full redundancy for fiber implementation be in place to minimize the threat of a replication stoppage. Over IP, VERITAS has the capability to switch dynamically between different replication modes and log changes. GiantLoop’s testing revealed that the VERITAS solution reacts transparently to any transport failures, offering increased reliability and ease of design and installation.

Fabric Failure

Ability to handle component failure in the Storage Area Network (SAN)

Requires segregated fabrics (between host and storage) at local and remote sites.

Requires segregated fabrics (host-to-storage and storage-to-storage) at local and remote sites.

Both synchronous replication solutions over Fibre Channel are susceptible to fabric failures. (Refer to Appendix III for a detailed discussion on network/fabric design considerations)

HBA Failure

Ability to handle failure of the Host Bus Adapter (HBA)

HBA failure is transparent with Dynamic Multipathing (DMP), a VERITAS feature available for no additional cost with the purchase of VERITAS Volume Manager. DMP is a host-based software technology that integrates multiple I/O capabilities, automatic load balancing, and path failover functions into one feature set for use on open server platforms connected to any storage supported by VERITAS Volume Manager.

PowerPath can provide protection against HBA failures but requires additional configuration and cost.

EMC PowerPath is a host based software solution. It integrates multiple path I/O capabilities, automatic load balancing, and path failover functions into one package for use on open server platforms connected to Symmetrix and EMC CLARiiON storage systems. (Source: EMC Web site)

The replication software itself does not provide protection against HBA failures. However, both EMC and VERITAS offer additional technologies to address HBA failures. For VERITAS, DMP is included as a component of VERITAS Volume Manager. For EMC, PowerPath is an add-on product that can be purchased at additional cost.



NIC Failure with replication over IP

Ability to handle failure of the Network Interface Card (NIC).

When using the VERITAS Volume Manager with the Volume Replicator option NIC failures may impact replication. NIC failures would be handled similarly to any other IP network failure. (See replication link failure). Also, NIC failures may be protected using VERITAS Cluster Server or Sun IPMP.

EMC has no native IP replication capability.

Various component failures in the transport path can be transparent to both solutions.

Host Failure

Ability to handle failure of the server host.

The VERITAS product line includes clustering technology that can guard against any host failure. If a host fails, replication services can move to any clustered node to maintain highly available replication services.

Without clustering technology, the application halts upon host failure and replication can continue when host and application returns.

Replication has little to do with host failures. Clustering technology should be considered to guard against any failure at the host. Both replication solutions recovered with the recovery of the host.

Failover from

primary to secondary

Ability to switch from primary to secondary.

Secondary can become primary with a single command.

Secondary can become primary with a single command.

Promoting the secondary to primary is easily accomplished with both EMC and VERITAS solutions. However, it is important to look at application availability as well. Just having storage mounted at a secondary is not a solution. Applications must be restarted and users must be re-directed to the remote site for a complete disaster recovery solution.

Resynchronization

after primary outage

(Failback)

Ability to switch back after recovery of primary.

Secondary automatically synchronizes only the changes that occurred at the secondary when recovering the primary.

Manual intervention is required to resynchronize.

VERITAS FastFailback capability, a feature of VERITAS Volume Manager with the Volume Replicator option, minimizes the amount of resynchronization from secondary to primary to facilitate the “return home” scenario.

Data Write Order

Fidelity In Synchronous

Mode

Evaluates data consistency between primary and secondary in synchronous replication.

Data write order fidelity is maintained in both modes of replication, including synchronous.

Write order integrity is maintained in synchronous replication.

Both VERITAS and EMC maintain data write order fidelity in synchronous mode.

Data Write Order

Fidelity In Asynchronous

Mode

Evaluates data consistency between primary and secondary in asynchronous replication.

Write order fidelity is maintained in both modes of replication, including asynchronous replication.

Write order is not maintained in non-synchronous replication.

Without write order fidelity there is no guarantee that the current data existing at the secondary will be recoverable. The VERITAS volume-based approach has this capability.


Scalability The GiantLoop Testing and Certification (GTAC) Lab examined the relative scalability of the VERITAS and EMC data replication solutions. The table below illustrates some of the items to take into consideration when implementing various data replication scenarios using VERITAS or EMC.


Host

Host configuration included determining the ease of setup for physical replication links, setup of storage connections and the need for hosts with respect to applications and management of the replication solution.

VERITAS Volume Manager mirroring over Fibre Channel requires only one local host to generate load and run the application. While no secondary host is required for replication, an additional host is required to utilize the secondary storage, to run any application in a disaster recovery scenario. VERITAS Volume Manager with the Volume Replicator option requires an additional host at the target to ensure uncompromised data integrity. In either case, a host is obviously needed at the secondary to recover and utilize the up-to-date storage in a disaster recovery scenario. Using VERITAS, there are no limits on the number of hosts that can act as a primary/secondary.

A host is obviously needed at primary to generate load and run the application. While no secondary host is required for replication, an additional host is required to utilize the secondary storage, to run any application in a disaster recovery scenario.

Both solutions, for the practical purpose of disaster recovery, require a host for running applications at the primary and secondary site. This means that the VERITAS solution is merely a software solution with no additional host requirements above and beyond the practical implementation of the SRDF solution.

Replication Links

This tests the ability of the solution to scale to accommodate enterprise environments.

VERITAS allows one replication link per application. Scales to as many applications as needed. Can replicate to multiple sites simultaneously (up to 32 sites per application), with any mix of synchronous or asynchronous replication

Each additional replication link requires more cards in the Symmetrix – there is a physical limitation here depending on the hardware ports available.

VERITAS scales by adding NICs (which can also mean adding hosts). This compares to SRDF scaling by adding cards to the Symmetrix. Both approaches have no theoretical limit – however hardware must be added and total costs must be weighed for each solution.

Storage

Maximum limits to the amount of data replicated.

No limitation on amount of replicated data. With adequately provisioned bandwidth, the VERITAS solution appears to scale linearly as the volume sizes and number of volumes increases. It is also able to span arrays with no negative impact on replication.

Limited to the maximum storage capacity of each Symmetrix array. Cannot span arrays with replication. All primary storage for any given application must be housed entirely within a single array.

Host based storage is easier to upgrade in that you can just add any type of hardware. With the Symmetrix array, you are limited to the chassis.


Performance To evaluate the performance characteristics of VERITAS and EMC replication solutions, the GiantLoop Testing and Certification (GTAC) Lab measured the read and write throughput of applications and raw test data. This data was generated at high throughput rates characteristic of an enterprise customer environment. Replication throughput, synchronization time, and round trip delay were measured and compared for both EMC and VERITAS replication technologies. These tests were conducted with both solutions over various distances. The GTAC Lab used both an OLTP (on-line transaction processing) application simulator as well as a raw I/O generation utility to generate traffic for these tests. Raw I/O Performance The raw I/O generation tool, DataTest, was used to generate multiple types of load with a representative mixture of both reads and writes and at various block sizes. This is useful, as different customer applications will write to data replication products in varying block sizes. The different blocks are broken down into specific block sizes for transport depending on the replication solution. How efficiently the replication software performs this activity can have a large impact on the throughput rate of data.


Synchronous I/O @

0 Kilometer

At the MAN distance of zero kilometers, VERITAS Volume Manager

remote mirroring is up to 54% faster than EMC SRDF.

Synchronous I/O @

100 Kilometers

At the MAN distance of 100 kilometers, VERITAS Volume Manager remote mirroring is up to 54% faster than EMC SRDF.

Synchronous I/O @

200 Kilometers

These tests measure the overall random write I/O throughput in megabytes per second with replication enabled at the various MAN distances.

For each distance, the tests were run at the write block sizes of 2k, 4k, 8k, 16k, 32k, 64k, 128k, and 256k

At the MAN distance of 200 kilometers VERITAS Volume Manager remote mirroring is up to 70% faster than EMC SRDF.

GiantLoop testing of performance over distance illustrate that the VERITAS solution outperformed EMC SRDF. (Refer to table 1.) For synchronous native Fibre Channel replication over MAN distances (typically less than 200km) VERITAS Volume Manager’s remote mirroring outperformed SRDF by as much as 70%. (Refer to figures 4 and 5)

Synchronous I/O @

300 Kilometers

At the WAN distance of 300 kilometers, VERITAS Volume Manager with the Volume Replicator option is up to 32% faster than EMC SRDF.

Synchronous I/O @

400 Kilometers


Synchronous I/O @

500 Kilometers

These tests measure the overall random write I/O throughput in megabytes per second with replication enabled at the various WAN distances. For each distance, the tests were run at the write block sizes of 2k, 4k, 8k, 16k, 32k, 64k, 128k, and 256k


For distances greater than 200km, which are typical of WAN environments, VERITAS Volume Manager with its Volume Replicator option outperformed EMC SRDF by as much as 72%. (Refer to table 1.) Both solutions displayed performance degradation over distance. The VERITAS solution with the Volume Replicator option is able to minimize the performance degradation at longer distances. At 500km, the maximum performance degradation with VERITAS was less than 20%, compared to EMC, which had a maximum of 80% degradation at the same distance. (Refer to figures 6 and 7).


Application Performance To assist in the execution of the Oracle database benchmark testing, GiantLoop utilized an industry standard OLTP (on-line transaction processing) benchmark from Quest Software called Benchmark Factory. This tool is capable of generating both online and deferred warehouse type transactions, non-uniform in nature, applied to an Oracle database. BenchMark Factory simulated an enterprise database environment. Initially the database was set up, which included creating tables, creating indexes, loading tables, and assigning indexes. Multiple simultaneous users create transactions on the database, including new orders, make payments, check order status, check stock level, and deliveries. Replication software throughput was captured and used to compare the VERITAS and EMC solutions. CPU usage was also captured during this testing. Replication performance tests were conducted in both synchronous and asynchronous modes. Tests were also conducted in various MAN and WAN configurations using various enterprise transport solutions. For MAN distances, real fiber was deployed between DWDM nodes. For WAN testing, delay was inserted into the SONET network (using an Adtech AX/4000 by Spirent Communications) to simulate up to 50ms (or approximately 10,000 km distance) of network latency in each direction.


Synchronous @ 0 km (0 ms)

This tests the database performance, measured in transactions per second, with synchronous replication over a zero km distance. The equivalent latency is presented in milliseconds.

With VERITAS Volume Manager with the Volume Replicator option in synchronous replication at zero kilometers, the database performance is roughly equal compared to the baseline without replication performance results.

With EMC SRDF synchronous replication at zero kilometers over FC, the database performance was good but showed a more significant drop compared to the VERITAS and the baseline without replication performance results.

Synchronous replication of mission critical data should not impose a significant performance impact on the applications that rely on that data. The tests show that the VERITAS solution had less impact on application performance.

Synchronous @ 130 km (1 ms)

The same database performance test was conducted with synchronous replication over a distance of 130 km.

With VERITAS Volume Manager with the Volume Replicator option, synchronous replication at 130 kilometers, the database performance showed a degradation of less than 3% compared to the baseline without replication.

With EMC SRDF synchronous replication at 130 kilometers, the database performance showed a degradation of over 13% when compared to the baseline without replication.

VERITAS synchronous replication at a distance of 130 km has minimal impact to the application performance.



Asynchronous @

0 km (0 ms)

The same database performance test was conducted with asynchronous replication over a distance of zero km.

With VERITAS asynchronous replication at zero km, the database performance is roughly equal compared to the baseline without replication.

Refer to Synchronous test results. As previously explained, EMC was not tested with its non-synchronous Adaptive Copy mode since it does not maintain write order fidelity.

Asynchronous @ 10,000 km (50 ms)

The same database performance test was conducted with asynchronous replication over a distance of 10,000 km. This distance equates to a latency of 50ms.

With VERITAS asynchronous replication the database performance, even at 10,000 km, is still almost equal compared to the baseline without replication.

Refer to Synchronous test results. As previously explained, EMC was not tested with its non-synchronous Adaptive Copy mode since it does not maintain write order fidelity.

GiantLoop test results show that asynchronous data replication using the VERITAS solution is an attractive choice when replicating over any substantial distance. The tests show that asynchronous replication outperforms synchronous with respect to local write throughput. The VERITAS solution provides this asynchronous replication while maintaining the data write order fidelity. (Refer to figure 8 in Appendix II).

Host CPU Utilization

This test measures the increase in server CPU utilization when enabling replication during the OLTP test loads.

VERITAS host based replication increased the server CPU utilization by 4.3%.

EMC’s storage based replication had zero impact to the server CPU utilization.

Host Memory

Utilization

This test measures the increase in server memory utilization when enabling replication.

VERITAS’ host based replication nominally increased the server memory utilization by approximately 5%.

EMC’s storage based replication had zero impact to the server memory utilization.

The VERITAS solution showed a slight increase in the server CPU and memory utilization. EMC’s storage based replication had zero impact on these resources.


Price/Performance This section provides a qualitative analysis of various cost factors that should be considered when making an assessment of different replication technologies.


Storage Storage needed to store replicated data

Replication configuration requires as least one storage device at both the primary and secondary locations. VERITAS technologies allow replication between any storage arrays, regardless of vendor. For asynchronous replication, a small amount of disk space is required for a log volume. For determination of the log volume size, VERITAS provides an equation with inputs including: network capacity over a given interval; log volume growth and the excess of application write rate over network bandwidth. The SRL should be large enough to hold the volume of data generated during the busiest interval. This information is discussed in the Configuration Notes document for VERITAS Volume Manager with the Volume Replicator option, written by VERITAS.

Replication configuration requires at least one EMC Symmetrix storage array at both the primary and secondary locations. EMC does not have a capability for real-time asynchronous replication. Adaptive copy allows for a lag between the primary and secondary volumes, increasing the need for additional disk space.

For EMC customers wanting to replicate between Symmetrix arrays, the cost component is the same. For customers who do not have EMC Symmetrix at both locations, the VERITAS solution would be a more cost effective approach by allowing replication to/from any vendor’s storage devices. The VERITAS solution offers the additional advantage of flexibility now and in the future. Customers can choose their storage array vendor(s) independent of replication.

Hosts Hosts needed for operating applications and accessing data

For remote mirroring over Fibre Channel one host is required at the primary location. For replication over IP VERITAS requires a host at each location although they need not be identical.

EMC requires one host at the primary location.

Regardless of whether a host is required for replication at the secondary, a secondary host is absolutely required to run applications or access data in a disaster recovery scenario. Therefore, from a disaster recovery best practices standpoint, the host requirement should be the same for both VERITAS and EMC.

Network Requirements

Infrastructure necessary for replication

VERITAS supports both Fibre Channel and IP natively. Therefore, organizations can choose the appropriate protocol based on requirements. For Fibre Channel mirroring, one Fibre Channel controller (two for high availability) are needed on the host. For IP replication, one NIC is required (two for high availability).

EMC supports Fibre Channel natively. Two Fibre Channel ports are required on Symmetrix to carry SRDF traffic. For WAN transport, IP is not supported natively. This would require additional devices on each side of the network to convert Fibre Channel to SONET or IP.

Network requirements for VERITAS and EMC over Fibre Channel are the same. For current EMC Symmetrix customers, additional costs may be incurred if two ports are not available for replication. For replication over IP, VERITAS costs are lower since additional IP converter hardware is not required.



Multi-site Replication

Cost aspects of replicating to more than one location.

VERITAS Volume Manager with the Volume Replicator option can support this functionality natively with up to 32 secondary systems, which can be located at up to 32 sites.

Requires additional software to support SRDF to two sites. Beyond that, a multi-hop configuration would be required. This would not offer the same functionality and would require double the disk space for each copy of data.

Organizations looking to replicate the same data set to more than one location should see lower costs with the VERITAS solution.

GUI

Cost associated with the graphical user interface (GUI).

A Java-based GUI is available from VERITAS at no additional cost.

EMC offers a GUI through a control center component called SRDF Manager, which is available at additional cost.

Customers that want the added capability to manage their systems through a graphical interface can support this using VERITAS at no additional cost.


Summary The GiantLoop Testing and Certification (GTAC) Lab evaluated the usability and performance of enterprise data replication solutions based on VERITAS and EMC data replication technologies. The basic requirements for a data replication solution to be considered “enterprise-class” are that the solution must be stable, scalable, high-performance, include complete fail-over protection, and have no single point-of-failure. Based on the testing described in this paper, both VERITAS and EMC provide enterprise-class data replication solutions. In environments where the requirement is to replicate data within a SAN over a short distance, both VERITAS and EMC can provide synchronous replication over Fibre Channel. In these types of environments, using VERITAS Volume Manager for remote mirroring provides a more cost effective and higher performance solution over EMC’s SRDF. Increasingly, IT organizations have more challenging environments that dictate replicating data over WAN spanning great distances. In these cases, synchronous replication imposes unacceptable latency to the applications. With these strict requirements, VERITAS Volume Manager with the Volume Replicator option has the unique ability to replicate data asynchronously over native IP with high performance, which makes this solution a more compelling choice. VERITAS Volume Manager with the Volume Replicator option is able to replicate data synchronously or asynchronously over any standard IP network. The asynchronous replication mode eliminates the latency associated with synchronous replication, while maintaining the write order fidelity between the primary and secondary. In asynchronous mode, the write I/O performance is consistent regardless of the distance between the primary and secondary. Also, in asynchronous mode, performance degradation is negligible even at the maximum tested distance of 50 ms (approximately 10,000 km).


About the GTAC Lab The GiantLoop Testing and Certification (GTAC) Lab, located in Waltham, MA, is an independent, vendor-neutral testing facility that provides unbiased evaluations of data movement technologies, products, and solutions. The GTAC lab focuses primarily on both product evaluation and performance and interoperability testing for multi-vendor, multi-domain, and data movement configurations. The lab is staffed by an experienced engineering team with extensive experience in:

• DWDM, SONET and IP networking technologies • Storage, SAN, and SAN extension products such as iSCSI, FCIP, and Fibre channel-over-SONET • Benchmarking and characterization of complete inter-data-center storage replication solutions

based on leading hardware and software • Benchmarking and characterization of host and file system technologies for Microsoft, Sun Solaris,

HP-UX, and IBM AIX environments The GTAC Lab offers services for both enterprise customers and technology vendors. For more information, please visit www.giantloop.com/solutions_gtaclab.shtml


Appendix I Figures 1-3 illustrate some of the network configurations used to test and verify the relative capabilities, integration, and interoperability of VERITAS Volume Manager, VERITAS Volume Manager with the Volume Replicator option, and EMC SRDF with various transport network options. Note that over IP, the VERITAS Volume Replicator option was tested with Gigabit Ethernet, while VERITAS Volume Manager remote mirroring and EMC SRDF used Fibre Channel.

Figure 1: Sample Disaster Recovery Configuration for VERITAS Volume Manager with Volume Replicator Option (SONET Transport)

Figure 2: Sample Disaster Recovery Configuration for VERITAS Volume Manager with Remote Mirroring (DWDM Transport)


Figure 3: Sample Disaster Recovery Configuration for EMC SRDF (DWDM Transport)


Appendix II Table 1 illustrates that VERITAS Volume Manager and VERITAS Volume Manager with the Volume Replicator option outperform EMC SRDF in their respective target applications. For synchronous native Fibre Channel replication over metro distances (typically less than 200km), VERITAS Volume Manager outperformed SRDF by as much as 70%. For distances greater than 200km, which are typical of WAN environments, VERITAS Volume Manager with the Volume Replicator option outperformed EMC SRDF by as much as 72% in GiantLoop’s performance/distance tests. For example, synchronous random writes using VERITAS Volume Manager with the Volume Replicator option are shown to be of a higher throughput and are less susceptible to changes in throughput over distance than similar writes using EMC SRDF. This is clearly demonstrated from a graphical point-of-view in Figures 4, 5, 6 and 7. The increase in spacing between the performance trend lines at different distances for SRDF illustrates how SRDF throughput degrades significantly as distance increases. With companies increasingly looking to extend their data replication technologies over greater distances, it is extremely important that users have a solid understanding of this relationship between distance and replication performance. Table 1: Synchronous Replication Throughput Over Distance Using VERITAS Volume Manager, VERITAS Volume Manager with the Volume Replicator option, and EMC SRDF (Mbytes/Second)

Block Size: 2k 4k 8k 16k 32k 64k 128k 256k

No Replication Baseline 1.1 2.2 4.3 8.3 15.0 23.2 36.0 45.2

MAN Distances VERITAS over FC 0km 1.1 2.3 4.3 8.2 14.9 21.8 33.9 44.9 EMC over FC 0km 1.1 2.0 3.8 6.7 11.4 16.0 22.0 30.7 VERITAS over FC 100km 1.1 2.1 3.9 6.9 11.9 17.5 25.2 34.6 EMC over FC 100km 0.9 1.8 3.1 5.3 8.7 12.0 16.4 22.5 VERITAS over FC 200km 1.0 2.0 3.4 5.7 9.6 13.4 19.4 27.4 EMC over FC 200km 0.9 1.5 2.6 4.3 7.1 9.4 12.5 16.1

WAN Distances VERITAS over IP 300km 0.9 1.5 2.7 4.4 7.3 9.5 12.8 17.5 EMC over FC 300km 0.8 1.3 2.2 3.7 6.0 7.6 10.0 13.3 VERITAS over IP 400km 0.8 1.4 2.5 4.4 6.6 9.0 12.2 15.4 EMC over FC 400km 0.8 1.3 2.0 3.2 5.0 6.4 8.2 11.3 VERITAS over IP 500km 0.8 1.6 2.6 3.8 6.1 8.4 12.0 15.8 EMC over FC 500km 0.8 1.2 1.9 2.8 4.4 5.2 7.0 9.5


Figure 4: Synchronous Replication Throughput Over MAN Distance 8KB Block Size

00.5

11.5

22.5

33.5

44.5

5

0km 100km 200km

Distance

MB

/sec VERITAS over Fibre Channel

EMC over Fibre Channel

Figure 5: Synchronous Replication Throughput Over WAN Distance 8KB Block Size

0

0.5

1

1.5

2

2.5

3

300km 400km 500km

Distance

MB

/sec VERITAS over IP



Figure 6: Synchronous Replication Throughput Over MAN Distance 128KB Block Size

0

5

10

15

20

25

30

35

40

0km 100km 200km

Distance

MB

/sec VERITAS over Fibre Channel


Figure 7: Synchronous Replication Throughput Over WAN Distance 128KB Block Size

0

2

4

6

8

10

12

14

300km 400km 500km

Distance

MB

/sec VERITAS over IP



Figure 8: Synchronous vs. Asynchronous Replication

GiantLoop test results show that asynchronous data replication using VERITAS Volume Manager with the Volume Replicator option is an attractive choice when replicating over any substantial distance. The chart above shows VERITAS Volume Manger with the Volume Replicator option, in asynchronous mode at a 50ms delay (simulating the latency introduced into the network by increased distance), compared with EMC synchronous replication at 0km. In this comparison it is easy to see that asynchronous outperforms synchronous with respect to local write throughput.

VERITAS Asynchronous @ 10,000km vs.

EMC Synchronous @ 0km

0

5

10

15

20

25

30

35

2k 4k 8k 16k 32k 64k 128k 256kWrite Block Size

MB

/sec VERITAS

EMC


Appendix III Setup and Configuration Details VERITAS Once volumes are created there are no additional setup requirements when mirroring over Fibre Channel. Options including striping, stripe width, and number of columns were available from the VERITAS Volume Manager. The disk arrangement is simple to setup and create. For replication over IP, the Volume Replicator option must be enabled within VERITAS Volume Manager. This IP option adds three additional objects: replication log, replication links and replicated volume groups. The replication log size and configuration (striping and mirroring) were determined before the other volumes were created. Space for the SRL was allocated as a meta-volume on the EMC hardware. For this testing, the metavolume was created from 4 x 36 GB disks (Raid 0), which were hardware striped. This gave testers a useable SRL space of just over 67 GB. The VERITAS replication log acts as a form of cache for replication. It can reside on the data storage array or can be located separately on a different array such as an inexpensive JBOD array. When replicating over IP, this log is an essential part of VERITAS Volume Replicator for both synchronous and asynchronous replication. Synchronous replication uses the log as cache for overhead. In the event of a replication link failure, the log acts as the secondary until the link is repaired and replication switches autonomously to asynchronous mode. As this point, the log takes advantage of times when the application is writing slower, to drain to the secondary. When the log is completely drained, VERITAS replication automatically switches back to synchronous mode. Another feature, DCM (Data Change Map), is similar to EMC’s Dirty Region Logging (DRL) functionality. Both of these features are bit maps that track changed sections of volumes when the link to the secondary goes down. The difference is that the DCM is active only when the replication log overflows. Assuming the log was sized properly (based on maximum write rates, average write rates, and link throughput), this should never happen with VERITAS volume-based replication. For EMC SRDF, DRL becomes active as soon as the secondary site fails. When the secondary site recovers, DRL is used to point to the individual tracks on the primary that were altered while the secondary was down, so that these changed tracks can now be replicated. This is an extra step not needed with the VERITAS Volume Replicator SRL feature. Since all changes are entered into the SRL, as soon as the secondary recovers, the SRL automatically replicates its entire contents to the secondary, which is then up to date. Replication links were configured between primary and secondary hosts and replication mode (either synchronous or asynchronous) was specified. The replicated volume group ensures that data integrity is maintained within each volume and among associated volumes. Replicated volumes were placed inside a logical container to ensure this consistency. This easy and straightforward since it generally maps directly to the disk groups already set up within VERITAS Volume Manager. EMC No commands are required to perform specific actions on the secondary. All commands entered on the primary are propagated automatically to the secondary. However, this does require that both sites are physically configured exactly the same. Once LUNs have been discovered, only three commands are needed for the user to begin replication. This allows for ease of use but also limits the ability for user modification.


Replication is physically performed over the SRDF Fibre Channel links. Since both ends of the replication solution must be Symmetrix, the setup is simple but inflexible. Options tested for replication included synchronous and semi-synchronous. EMC does not have the added advantage of SRL protection. Instead it relies on DRL (Dirty Region Logging) as protection for when the synchronous replication breaks down. EMC relies on the cache of the Symmetrix devices for speeding up the replication process. This allows for a separate de-staging of cached data to take place at a lower priority. TCP and UDP Options over IP The VERITAS Volume Manager with Volume Replicator option allows the administrator to select either TCP or UDP as the protocol to use when replicating over IP. GiantLoop tested VERITAS Volume Replicator with both TCP and UDP protocols. Throughput was largely unaffected by the type of protocol used in transporting the replication data. Customers may choose to use TCP as it is a connection-based protocol and offers the added protection of retransmission. However, if the network is a dedicated, private infrastructure, there is no reason why the faster, connectionless UDP option should not be used when the replication distance is reasonable (i.e., under 65km). Fibre Channel Design Considerations Both synchronous replication solutions over Fibre Channel are susceptible to component failures in the fabric. Failures introduced into the transport network can impact both EMC and VERITAS replication. For example, during the EMC SRDF tests when two inter-switch links (ISLs) were being used for protection, a single ISL failure caused a fabric rebuild, resulting in an application interruption. If all ISLs momentarily failed, manual intervention would be required. The switchover is long enough to cause SRDF to go into split mode and require manual recovery. If the ISL is being protected by an optical transport solution and the ISL circuits’ active paths are over the same physical fiber, and the fault occurs on this fiber, both ISL circuits will perform a protection switch. This is a major factor when considering the design of the transport network for the SRDF ISL links. If protection is used in the transport layer, the ISLs must have active paths traveling diverse routes. If this is not the case, the above manual intervention may be required when recovering. If diverse routes are used, throughput can be affected as one route is usually much longer than the other, and Fibre Channel based data replication is susceptible to throughput degradation over increased distance. Either way, both the VERITAS and EMC solutions require a careful design that takes all of these factors into account.


About GiantLoop GiantLoop is a leading provider of data movement solutions for Global 2,000 businesses. GiantLoop offers a suite of services that help companies assess, design, test, implement, and manage data movement infrastructures encompassing enterprise storage networks, public and private data transport networks, and data mirroring, replication, and backup technologies. To ensure high-performance solutions, GiantLoop tests and qualifies complex infrastructure configurations in its multi-vendor testing facility, the GiantLoop Testing and Certification (GTAC) Lab. GiantLoop solutions enable companies to ensure business continuity and disaster recovery readiness, consolidate technology resources for increased efficiencies, and deploy new productivity-enhancing applications. GiantLoop is based in Waltham, Mass., and has additional offices in the United States and the United Kingdom. For additional information, please visit www.giantloop.com.

GiantLoop Network, Inc. 265 Winter Street Waltham, MA 02451 !!!! 781-902-5100 """" [email protected] !!!! www.giantloop.com

Copyright 2002, GiantLoop Network, Inc. Trademark Notice: GiantLoop and the GiantLoop logo are trademarks of GiantLoop Network, Inc. VERITAS, VERITAS Volume Replicator, VERITAS Volume Manager, VERITAS File System, VERITAS Foundation Suite, and VERITAS Cluster Server are trademarks of VERITAS Software Corporation. EMC and Symmetrix are registered trademarks and SRDF is a trademark of EMC Corporation. Sun, Sun Microsystems, Solaris, and Sun Fire are trademarks of Sun Microsystems, Inc. Benchmark Factory is a registered trademark and Quest is a trademark of Quest Software, Inc. Oracle is a registered trademark and Oracle9i is a trademark of Oracle Corporation. Cisco and Cisco Systems are registered trademarks of Cisco Systems, Inc. Nortel Networks and OPTera are trademarks of Nortel Networks Limited. McDATA is a registered trademark of McDATA Corporation. Corning is a registered trademark and SMF-28 is a trademark of Corning Incorporated.

giantloop testing and certification (gtac) lab

Documents