hitachi content platform for vmware vsphere esxi …...1 1 hitachi content platform for vmware...

September 12, 2014

By

Reference Architecture Guide

Hitachi Content Platform for VMware vSphere ESXi using Hitachi NAS Platform as an NFS Datastore

Federick Brillantes and Dave Brandman

FeedbackHitachi Data Systems welcomes your feedback. Please share your thoughts by sending an email message to [email protected]. To assist the routing of this message, use the paper number in the subject and the title of this white paper in the text.

mailto:[email protected]?subject=Document%20AS-327-00%20

Table of ContentsSolution Overview........................ .......................................................................2

Key Solution Components..................................................................................5

Hitachi Compute Rack 210H......................... ...........................................6Hitachi NAS Platform......................... .......................................................7Hitachi Unified Storage......................... ....................................................7LAN Switches......................... ..................................................................7Hitachi Content Platform...........................................................................8Software Components......................... .....................................................9Hitachi Storage Navigator Modular 2......................... ...............................9VMware vSphere 5......................... ........................................................10Microsoft Windows Server 2008 R2 Enterprise......................... .............10Red Hat Enterprise Linux 6.3..................................................................11

Solution Design........................ .........................................................................12

High Level Infrastructure.........................................................................12Storage Architecture......................... ......................................................13Hitachi NAS Platform Architecture..........................................................15VMware Architecture ......................... ....................................................17Hitachi Content Platform Architecture.....................................................19Compute Architecture......................... ....................................................21Network Architecture ......................... ....................................................22

Engineering Validation......................................................................................25

Test Methodology......................... ..........................................................25Test Results............................................................................................25Test Analysis......................... .................................................................37VMware Validation..................................................................................37

Conclusion........................ .................................................................................43

1

1

Hitachi Content Platform for VMware vSphere ESXi using Hitachi NAS Platform as an NFS DatastoreReference Architecture Guide

This reference architecture guide is used in deploying Hitachi Content Platform (HCP) for VMware vSphere ESXi in Hitachi NAS Platform with NFS as a datastore using Hitachi Compute Rack 210H and Hitachi Unified Storage 130. It provides a solution that is stable, highly available, and scalable utilizing the VMware platform.

Hitachi Content Platform is a distributed storage system designed to support large, growing repositories of fixed-content data. Hitachi Content Platform systems consist of both hardware (physical or virtual) and software. Hitachi Content Platform stores objects that include both data and metadata that describe the data. It distributes these objects across the storage space. Hitachi Content Platform represents objects either as URLs or as files in a standard file system.

You need familiarity with the use and configuration of the following to use this reference architecture guide:

Hitachi Unified Storage 100 family

Hitachi Compute Rack 210H server

Brocade switches

Hitachi NAS Platform 4000 series

VMware vSphere 5

Note — Testing of this configuration was in a lab environment. Many things affect production environments beyond prediction or duplication in a lab environment. Follow the recommended practice of conducting proof-of-concept testing for acceptable results in a non-production, isolated test environment that otherwise matches your production environment before your production implementation of this solution.

2

2

Solution OverviewThe Hitachi Content Platform in Hitachi NAS Platform (HNAS) solution described in this reference architecture uses Hitachi NAS Platform as an NFS datastore in a VMware environment with Hitachi Compute Rack 210H (CR 210H) servers.

Hitachi Content Platform in Hitachi NAS Platform has the following components:

Storage Infrastructure, a Hitachi Unified Storage 130 storage system

Hitachi Unified Storage 130 is a symmetric active/active midrange storage platform for all data and was the storage behind Hitachi NAS Platform 4060.

Dual controllers

42.7 TB raw capacity, 4.8 TB capacity of all volumes

32 GB cache

Hitachi NAS Platform Infrastructure

Two-node Hitachi NAS Platform 4060 Cluster used as an NFS datastore for Hitachi Content Platform. Hitachi NAS Platform is also used for file sharing, data protection, and critical application workloads, and as a file server

VMware VSphere 5

Virtualization technology providing infrastructure for Hitachi Content Platform

Hitachi Content Platform Infrastructure

Hitachi Content Platform provides an active archive environment for fixed-content with a single online repository that enables protection, search, and retrieval across all content types.

3

3

Compute Infrastructure

Hitachi Compute Rack 210H is a midrange rack mountable server platform providing advanced system management and redundancy options

4 Hitachi Compute Rack 210H servers are used for Hitachi Content Platform

Dual Fibre Channel HBAs for redundant connectivity to SAN storage infrastructure

8 Processor Cores and 12 GB memory per Hitachi Content Platform node.

1 Hitachi Compute Rack 210H server is used for VMware vCenter

Network Infrastructure

Dell 16-Port 1 GigE switch for Hitachi Content Platform front-end and back-end switch connectivity

Brocade VDX6720 24-Port 10 GigE data path connectivity

Hitachi Content Platform Front-End Switch - 2 Dell 16-Port 1 GigE switches

Hitachi Content Platform Back End Switch - 2 Dell 16-Port 1 GigE switches

Data Switch - 2 Brocade VDX6720 24-Port 10 GigE switches

4

4

Figure 1 shows a basic overview of the solution stack.

Figure 1

5

5

T

Key Solution Components The Hitachi Content Platform includes key hardware and software components in deploying this solution.

Table 1 lists the detailed information about the hardware components used in this solution.

able 1. Key Hardware Components

Component Description Firmware Quantity

Server (Hitachi Content Platform)

Hitachi Compute Rack 210H

Form Factor 1U (rack mountable)

2 × Intel Xeon E5-2620 6 Core, 2.0 GHz CPU

32 GB, 1333MHz DDR3 LV RDIMM

PCIe, 1000Base-T, Quad port Broadcom NetXtreme BCM5719

Onboard Broadcom 1000Base-T NetXtreme 57218 (dual port)

Intel 82599EB-10 GigE SFI/SFP+

BIOS: MH.1.06 4

VMware vCenter Server Hitachi Compute Rack 210H

Form Factor 1U (rack mountable)



Onboard Broadcom 1000Base-T NetXtreme 5721C1 (dual port)

Intel Ethernet X520-2 10 GigE

Broadcom NetXtreme 1 GigE

LSI MegaRAID SAS 9267-8i

BIOS: MH.1.06 1

6

6

T

Hitachi Compute Rack 210HHitachi Compute Rack 210H is a dual-socket Intel Xeon processor-based, midrange rack-mountable server platform. It provides advanced systems management and redundancy options. It is data center friendly, with a 1U footprint, while delivering the performance that is required to meet enterprise-level challenges.

It provides advanced systems management and redundancy options, such as the following:

Onboard dual 1 Gb/sec network adapters

Optional mezzanine dual 1 Gb/sec network adapters

Optional Quad port PCIe 1 Gb/sec network adapters

Web-based management interface

RAID level configuration, with up to six 2.5 inch internal drives

Sustainable power-saving capabilities

Storage System Hitachi Unified Storage 130

32 GB Cache

24 × 4 TB SAS 7.2K RPM HDD

0975/A-S 1

LAN Switch Brocade VDX6720 24-Port 10 GigE Data Switch

Dell 16-Port 1 GigE Front-End Switch

Dell 16-Port 1 GigE Backend-End Switch

3.0.1b 2

2

2

Hitachi NAS Platform Hitachi NAS Platform 4060 Two-Node Cluster

11.2.3319.16 1

able 1. Key Hardware Components (Continued)

Component Description Firmware Quantity

7

7

Hitachi NAS PlatformHitachi NAS Platform is an advanced and integrated network attached storage (NAS) solution. It provides a powerful tool for file sharing, file server consolidation, data protection, and business-critical NAS workloads.

Powerful hardware-accelerated file system with multi-protocol file services, dynamic provisioning, intelligent tiering, virtualization and cloud infrastructure

Seamless integration with Hitachi SAN storage, Hitachi Command Suite, and Hitachi Data Discovery Suite for advanced search and index

Integration with Hitachi Content Platform for active archiving, regulatory compliance, and large object storage for cloud infrastructure

Hitachi Unified StorageHitachi Unified Storage 130 is a midrange storage platform for all data. It helps businesses meet their service level agreements for availability, performance, and data protection.

The performance provided by Hitachi Unified Storage is reliable, scalable, and available for block and file data. Unified Storage is simple to manage, optimized for critical business applications, and more efficient.

Using Unified Storage requires a lower investment in storage. Deploy this storage, which grows to meet expanding requirements and service level agreements, for critical business applications. Simplify your operations with integrated set-up and management for a quicker time to value.

Unified Storage enables extensive cost savings through file and block consolidation. Build a cloud infrastructure at your own pace to deliver your services.

LAN SwitchesFor this solution, Brocade VDX6720 10 GigE 24-Port switches were used for the data path while Dell 16-Port 1 GigE switches were used for the front-end and back-end network for Hitachi Content Platform.

Brocade and Hitachi Data Systems partner to deliver storage networking and data center solutions. These solutions reduce complexity and cost, as well as enable virtualization and cloud computing to increase business agility

http://www.hds.com/products/file-and-content/network-attached-storage/

http://www.hds.com/products/storage-software/hitachi-command-suite.html

http://www.hds.com/products/storage-software/hitachi-command-suite.html

http://www.hds.com/products/storage-software/data-discovery-suite.html

http://www.hds.com/products/file-and-content/content-platform/

http://www.hds.com/products/networking/fibrechannel/brocade.html

8

8

Hitachi Content PlatformHitachi Content Platform provides an active archive environment for fixed-content with a single online repository that enables protection, search, and retrieval across all content types. Hitachi Content Platform is based on a unique SAN-based architecture that leverages Hitachi storage capabilities for availability, performance, and multi-petabyte scalability. Hitachi Content Platform provides authentication and secure retention of content to ensure its long-term preservation and accessibility in an active fixed-content archive storage environment.

Hitachi Content Platform is designed to enable archiving of fixed content in a manner that:

Ensures content integrity, authenticity, security, completeness and accessibility over the long term, in accordance with relevant laws and regulations

Offers fast, online access to content

Allows integrated searching and indexing of the archive, including search of file contents

Supports business continuity, data recovery, compliance search and retention needs

Scales horizontally to support multiple applications and content types; and vertically to support continued data growth

Allows for sub-second search response

9

9

T

Software ComponentsTable 2 lists the detailed information about the software components used in this solution.

Hitachi Storage Navigator Modular 2Hitachi Storage Navigator Modular 2 enables essential management and optimization of storage system functions. Using Java agents, Storage Navigator Modular 2 runs on most browsers. A command line interface is available.

Use Storage Navigator Modular 2 for the following:

RAID-level configurations

LUN creation and expansion

Online microcode updates and other system maintenance functions

Performance metrics

able 2. Software Components

Component Description Version

Hitachi Content Platform Digital archive and object store 6.1.1.33

VMware vSphere VMware virtualization platform that provides datacenter infrastructure

5.1

VMware vCenter Management of VCenter via a single user interface

5.1

Hitachi Storage Navigator Modular 2 (SNM2) Storage array management 25.50

Microsoft® Windows Server® VMware vCenter Operating System

2008 R2 Enterprise

Red Hat Enterprise Linux Linux client 6.3

10

10

VMware vSphere 5VMware vSphere 5 is a virtualization platform that provides a datacenter infrastructure. It features vSphere Distributed Resource Scheduler (DRS), high availability, and fault tolerance.

VMware vSphere 5 has the following components:

ESXi 5 — this is a hypervisor that loads directly on a physical server. It partitions one physical machine into many virtual machines that share hardware resources.

vCenter Server 5 — this allows management of the vSphere environment through a single user interface. With vCenter, there are features available such as vMotion, Storage vMotion, Storage Distributed Resource Scheduler, High Availability, and Fault Tolerance.

ESXi 5VMware ESX servers provide the foundation for building a reliable and dynamic IT infrastructure. These market leading production proven hypervisors abstract processor, memory, storage, and networking resources into multiple virtual machines that can each run an unmodified operating system and applications. VMware ESX servers are the most widely deployed hypervisors delivering the highest levels of reliability and performance to companies of all sizes.

VMware vCenter ServerUse VMware vCenter Server to manage VMware vSphere. It provides advanced data management that improves storage operations, provisioning, optimization, and resilience for Hitachi storage environments. There is unified management of all hosts and virtual machines from a single console, aggregating performance monitoring of clusters, hosts, and virtual machines.

VMware vCenter Server gives you insight into the status and configuration of clusters, hosts, virtual machines, storage, guest operating systems, and other critical components of a virtual infrastructure. Using VMware vCenter Server, you can manage 100 or more virtualization environment workloads, more than doubling typical productivity in managing physical infrastructure.

Microsoft Windows Server 2008 R2 EnterpriseMicrosoft Windows Server 2008 R2 is an operating system developed by Microsoft. It builds on the award-winning foundation of Windows Server 2008 expanding existing technology and adding new features to enable IT professionals to increase reliability, flexibility, and scalability of their server infrastructure.

https://www.vmware.com/products/vsphere/

http://www.vmware.com/products/vcenter-server/overview.html

http://technet.microsoft.com/en-us/library/dd349801%28v=ws.10%29.aspx

11

11

Red Hat Enterprise Linux 6.3Red Hat Enterprise Linux is the world's leading enterprise Linux platform that fulfills core operating system functions and includes additional capabilities that provide a firm foundation for application structure.

http://www.redhat.com/products/enterprise-linux/server/

12

12

Solution DesignThis is a detailed description of a reference architecture for Hitachi Content Platform with Hitachi NAS Platform used as an NFS datastore with Hitachi Unified Storage 130 storage system and Hitachi Compute Rack 210H server. It includes software and hardware design information required to build the basic infrastructure for the Hitachi Content Platform with the Hitachi NAS Platform solution.

High Level InfrastructureFigure 2 shows the high-level infrastructure of the solution.

Figure 2

13

13

T

Storage ArchitectureFor this reference architecture, HUS 130 was dual connected to Hitachi NAS Platform 4060. RAID-6 was set up for the two-node Hitachi NAS Platform cluster and an exported NFS was mounted as an Hitachi Content Platform datastore. The use of RAID-6 lowers the risk of data loss or pool failure. The different drive and volume configurations as well as host group mapping is shown below:

Table 3 shows the RAID configuration for Hitachi Content Platform with the Hitachi NAS Platform solution.

able 3. RAID Configuration

RAID # RAID Level RAID Configuration

Drive Size Drive Type Drive RPM

000 RAID-6 4D+2P 2 TB SAS7K 7.2K




14

14

Figure 3 shows HUS 130 storage configuration allocated for Hitachi NAS Platform 4060.

Figure 3

Two 8 Gb/sec Fibre Channel ports of HUS 130 controller 0 were connected directly to Hitachi NAS Platform 4060 Fibre Channel ports. Port 0A connected to node 1 of Hitachi NAS Platform 4060 cluster, while port 0B connected to node 2 of Hitachi NAS Platform 4060 Fibre Channel ports. On the other hand, in controller 1 of HUS 130, two 8 Gb/sec Fibre Channel ports were used with direct connection to the Hitachi NAS Platform 4060 cluster. Port 1A connected to Hitachi NAS Platform 4060 node 1, while port 1B was connected to Hitachi NAS Platform 4060 node 2. In addition, two Brocade VDX6720 switches were used to provide access to file storage resources.

15

15

T

T

T

Table 4 shows the volume configuration used for this solution.

Table 5 shows the storage Fibre Channel settings used for this solution.

Hitachi NAS Platform ArchitectureFor this solution, an Hitachi NAS Platform 4060 two-node cluster provided high-availability in the event of a node failure. This cluster provides highly available, high performance NAS storage to ESXi servers. Each Hitachi NAS Platform 4060 head is configured with two 10 GigE link aggregation interfaces. The LAGs provide high availability and load balancing on the network connection between the redundant 10 GigE interfaces.

The Hitachi NAS Platform 4060 cluster is configured in a single drive group with a single storage pool, with four enterprise virtual servers (EVS) that have one filesystem each.

Table 6 shows the Hitachi NAS Platform configuration for this solution.

able 4. Volume Configuration

Volume # RAID Group # RAID Level Capacity Stripe Size

0000 000 RAID-6 (4D+2P) 1.2 TB 64 KB

0001 001 RAID-6 (4D+2P) 1.2 TB 64 KB

0002 002 RAID-6 (4D+2P) 1.2 TB 64 KB

0003 003 RAID-6 (4D+2P) 1.2 TB 64 KB

able 5. Fibre Channel Settings

FC Port Node Name Port Name Port Address Topology Transfer Rate

0A 50060E80103B6D00

50060E80103B6D00

0000EF Loop 8 Gbps

0B 50060E80103B6D01

50060E80103B6D01

0000EF Loop 8 Gbps

1A 50060E80103B6D08

50060E80103B6D08

0000EF Loop 8 Gbps

1B 50060E80103B6D09

50060E80103B6D09

0000EF Loop 8 Gbps

able 6. Hitachi NAS Platform Configuration

SD # STORAGE POOLNAME

STORAGE POOL SIZE

FILESYSTEM NAME EVS

000 HCPVMPOOL 1.2 TB ESX1FS HCPVM-ESX1




16

16

T

T

Table 7 shows the filesystem configuration for this solution.

Table 8 shows the system drive configuration for this solution.

When creating a new storage pool, use the same size system drives within the storage pool. It is strongly advised to use the same drive technology (FC/SAS/SATA) at the same spindle speed (7.2K/10K/15K RPM) within a storage pool.

able 7. Filesystem Configuration

Configuration Value

Deduplication Not Converted

Thin Provisioning Disabled

EVS HCPVM-ESX1

Security Mode Mixed (Windows and Unix)

WORM Not WORM

Block Size 4 KB

Read Cache No

WFS Version WFS-2

able 8. System Drive Configuration

System Drive ID Host Port

RAID Controller Port Name Capacity

0 1 03/A/92142336-2 50060E80103B6D08 1.2 TB

1 4 03/A/92142336-2 50060E80103B6D0B 1.2 TB

2 2 03/A/92142336-2 50060E80103B6D09 1.2 TB

3 3 03/A/92142336-2 50060E80103B6D0A 1.2 TB

17

17

Figure 4 shows the backend storage and file system mapping.

Figure 4

VMware Architecture Four VMware ESX 5.1 servers were used with each hosting a single VM per Hitachi Content Platform node. Each ESX server has 2 processor sockets with 6 cores per socket at 1.999GHz. Each ESX server has one 10 GigE two-port interface for data paths, one 1 GigE two-port for front-end connection, and one 1 GigE four-port for back-end connection. Each ESX host containing Hitachi Content Platform has the Network Time Protocol (NTP) enabled.

18

18

Each Hitachi Content Platform node has at least 8 vCPUs and 12 GB RAM to enable the system to maintain performance for most client workloads.

Figure 5 shows the VMware ESX server with Hitachi NAS Platform NFS datastore.

Figure 5

For this solution, a single VMware cluster was set up with vSphere HA enabled to address automatic failover in the event of an ESXi server failure with vSphere DRS off.

19

19

T

T

Table 9 describes the vSphere HA configuration.

It is recommended that only one LUN from a RAID group be allocated for use in Hitachi Content Platform. Adding multiple LUNS from the same RAID group increases risk to data in the event of failure. Moreover, each datastore can only be allocated for one Hitachi Content Platform node.

For more details of recommended settings, please refer to the Hitachi Content Platform Deployment Guide 6.1. http://www.hds.com/assets/pdf/hitachi-content-platform-deploying-an-hcp-vm-system.pdf

Hitachi Content Platform ArchitectureA four-node Hitachi Content Platform cluster for VMware vSphere ESXi was used for this reference architecture. Each CR 210H server has two 500 GB SATA2 7200 rpm, 2.5-inch internal drives that are set in a RAID-1 configuration dedicated to run the VMware ESXi software The four-node Hitachi Content Platform system for VMware vSphere ESXi has each node running in a virtual machine on an ESXi host while data is stored on Hitachi NAS Platform via an NFS datastore. Each Hitachi Content Platform VM is configured with a 32 GB OS partition and two 500 GB data partitions.

Table 10 describes the tenant settings.

able 9. vSphere HA Configuration

vSphere HA Configuration Settings

Host Monitoring Status Enable

Admission Control Enable: Disallow VM power on operations that violate availability constraints

Admission Control Policy Host failure that cluster tolerates = 1e

Cluster default settings - VM Restart Priority Medium

Cluster default settings - Host Isolation Response Leave powered on

VM Monitoring Status Disabled

VM Monitoring Sensitivity High

Datastore Heartbeating Select any of the cluster datastores taking into account my preferences

able 10. Tenant Settings

Tenant Option Setting

Tenant Name hcpvm3

Hard Quota 1 TB

Soft Quota 85%

Namespace Quota 1

Retention Mode Selection Enable

Versioning Enable

MAPI Enable

20

20

T

T

T

Table 11 shows the namespace settings.

A tenant admin user was created and assigned the appropriate role permissions as described in Table 12.

A data access account with admin permissions was also created for the namespace used as shown in Table 13.

able 11. Namespace Settings

Namespace Option Setting

Namespace Name ns2

Hard Quota None

Soft Quota 85%

Retention Mode Enterprise

Versioning Off

Indexing Policy Enabled

Metadata Policy Check on ingestion that XML custom metadata is well-formed

Protocols Enable HTTP, HTTPS, and REST API with authenticated access only

able 12. Tenant Admin User Roles

Hitachi Content Platform Tenant Admin User Roles Setting

Monitor Enable

Administrator Enable

Security Enable

Service Enable

Compliance Enable

able 13. Data Access with Admin Permissions

Data Access Permissions for Admin User Settings

Browse Enable

Read Enable

Write Enable

Delete Enable

Purge Enable

Privileged Enable

Search Enable

Read ACL Enable

Write ACL Enable

Change Owner Enable

21

21

T

Compute ArchitectureThe compute architecture utilized in this solution are four Hitachi Compute Rack 210H each running VMware ESX server with Hitachi Content Platform being hosted on top. One CR 210H was used as VMware vCenter Server for management.

Table 14 provides the details of the compute server used for this solution.

able 14. Compute Architecture

Hardware Description Firmware Quantity

CR 210H Form Factor 1U (rack mountable)



PCIe, 1000Base-T, Quad port Broadcom NetXtreme BCM5719

Onboard Broadcom 1000Base-T NetXtreme 57218 (dual port)

Intel 82599EB-10 GigE SFI/SFP+

BIOS MH.1.06 5

Hard Drives 500 GB SATA Drives N/A 10

22

22

Figure 6 shows the 4-node Hitachi Content Platform compute architecture.

Figure 6

Network Architecture The network component for this solution used two Dell 16-port 1 GigE front-end switches, two Dell 16-port 1 GigE back-end switches and two Brocade VDX6720 24-Port 10 GigE data path switches.

23

23

Figure 7 shows the physical network configuration for this architecture.

Figure 7

Separate networks are set up for the front-end, back-end, and data. The front-end uses two Dell 16-port 1 GigE switches interconnected to provide link availability. This is also the case for the back-end network. A separate data network is configured with two Brocade VDX6720 24-Port 10 GigE switches.

The network design for Hitachi NAS Platform cluster in this solution uses the following:

Two 10 Gb/sec Ethernet ports on each node connect to two Brocade VDX6720 switches for high availability in case of a failure

Separate NFS traffic from VMware vSphere Management

Two 10 Gb/sec Ethernet ports are combined into a single logical link to provide increased bandwidth, load balancing, and high-availability

24

24

Note — The Hitachi Content Platform cluster network configuration in this reference architecture is a 1 GigE network for both front-end and back-end, limiting the throughput. If available, and for best results, the larger 10 GigE network for the front-end and back-end of Hitachi Content Platform deployment is recommended. This requires switching the compute server from CR 210H to CR 210S or CR 220 to accommodate 10 GigE cards. Furthermore, the 1 GigE Dell switches must be swapped out with 10 GigE Brocade switches.

25

25

T

T

Engineering ValidationThis describes the methodology and test results used to validate this reference architecture.

To validate this reference architecture, Rise and VMBench tools were used to simulate the generation of files ingested to Hitachi Content Platform. Rise is a C++ based tool to drive workloads, and it is used as a testing, diagnostic, and exercise tool to perform a variety of file operations.

Test MethodologyThe Rise tool was used from a client to generate a workload and ingest or read files to/from Hitachi Content Platform with specific options defined in Table 15, and distribute data load to multiple Hitachi Content Platform nodes via DNS.

WorkloadTable 15 shows how the workload for this reference architecture was configured using the Rise tool.

Test ResultsTable 16 shows the test results of HTTP ingest of single files of various sizes.

able 15. Workload Configuration for Rise Testing

Option / Configuration Value

Thread ( -t ) 50

File Size ( -fs ) 1 KB - 500 MB

Directory Depth 1 - 15

Directory Width 1 - 15

able 16. HTTP Single File Ingest Test Results

File Size I/O Rate Throughput

1 K 0.07 operations/sec 0.07 KB/sec


1 MB 0.19 operations/sec 194.25 KB/sec



26

26

T

T

T

Table 17 shows the test results of HTTP ingest of 1000 files of various sizes with Thread=50.

Table 18 shows test results of CIFS ingest of single files of various sizes.

Table 19 shows the test results of HTTP ingest of 1000 files of various sizes with Thread=50.

A test load of 944,801 files that are 1 MB in size generates 944,801 GB with 50 threads over a period of 428,976 sec (119.16 hours).

able 17. HTTP Ingest of 1000 Files Test Results

File Size Number of Files I/O Rate Throughput

1 K 1000 5.51 operations/sec 5.51 KB/sec


1 MB 1000 0.61 operations/sec 609.53 KB/sec


able 18. CIFS Single File Ingest Test Results

File Size I/O Rate Throughput





able 19. HTTP Ingest of 1000 Files with Thread=50 Test Results

File Size Number of Files I/O Rate Throughput





27

27

Hitachi NAS Platform ValidationTo validate the Hitachi NAS Platform configuration as an NFS datastore for Hitachi Content Platform, Hitachi NAS Platform is set up taking into consideration general best practices for NFS datastores with VMware as outlined below:

vSphere host root access to the datastore was established using the "no_root_squash" option in the /etc/exports file

A separate network for Hitachi NAS Platform cluster data path should be used, preferably a 10 GigE network if available

Setup the file system block size to 4 KB

Set cache-bias to large (cache-bias --large-files)

Disable shortname generation and access time maintenance (shortname -g off, fs-accessed-time --file-system<file_system>off)

Disable quick start option for Hitachi NAS Platform read ahead when VM I/O profiles are primarily random

Do not export the root of the file system (NFS exports)

Do not mix disk types in the same storage pool

Limit ownership of all file systems that are created on a storage pool to one EVS

Configure a minimum of four System Drives (SD) in a storage pool

Configure one LU\LDEV per RAID group consuming all space (if possible)

Do not have multiple datastores on the same file system or same underlying disk due to performance and availability considerations

28

28

Test Results:Figure 8 shows the Hitachi NAS Platform Cluster without any failure for 1 KB file (READ).

Figure 8

The system load result is 3.84% average with a peak of 5%.

Figure 9 shows the Hitachi NAS Platform cluster without any failure for 1 MB files (READ).

Figure 9


29

29

Figure 10 shows the Hitachi NAS Platform cluster without any failure for 1 MB files (WRITE).

Figure 10

Figure 11 shows the Hitachi NAS Platform cluster when one node failure is encountered during 1 KB file WRITE operations.

Figure 11


30

30

Figure 12 shows the 2-node Hitachi NAS Platform cluster when one node failure is encountered during READ operations on 1 KB files.

Figure 12

The system load result was 3.45% average with a peak of 5%.

During read operations of 1 KB files, when the system encountered a node failure, disk latency increased from 2 ms to 5 ms. This is about a 25% drop for network throughput and a node ops/sec fall from the average of 43 ops/sec to 11 ops/sec.

During read operations of 1 MB files, when a node fails, the network throughput was reduced to about half, with a node ops/sec drop to almost 60% from the average of 600 ops/sec to 248 ops/sec, while the system load remained unchanged. For write operations of 1 MB files, when a node failure occurs, the network throughput drops from an average of 90 MB/sec to 70 Mb/sec - about 22%. Node ops/sec also drops about 23%.

Additional Hitachi NAS Platform Recommendations:

Use smaller chunk sizes of 64 KB and a smaller RAID group size (RAID-6 4D+2P)

Create one LUN per RAID group when possible

For a file protection mechanism, use a minimum of 2 system drives when configuring Hitachi NAS Platform storage pools

For more information about general guidelines for Hitachi NAS Platform with VMware, please refer to Hitachi NAS Platform Best Practices Guide for NFS with VMware vSphere - http://loop.hds.com/docs/DOC-31768

31

31

T

1

6

1

T

1

6

1

T

1

6

1

T

1

6

1

Hitachi Content Platform ValidationTo validate that files were ingested to Hitachi Content Platform, the Rise tool was used to write files to Hitachi Content Platform.

The test was configured with the option of 50 threads (t), one directory width (dw), and one directory depth (dd) with various file sizes from 1 KB, 64 KB, 1 MB, 10 MB, and 500 MB.

The test is divided into two parts. The first part is to capture numbers without any failure and the second part is designed to determine the impact when a failure is encountered during read or write operations.

Table 20 and Table 21 show the results of write and read operations.

The test also includes simulating failure on a 4-node Hitachi Content Platform system, particularly shutting down one of the Hitachi Content Platform nodes, with the results listed in Table 22 and Table 23.

able 20. Write Operations

Data Size I/O Rate ( Op/sec ) Throughput (KB/sec) Duration (sec)

KB 19.24 19.24 512

4 KB 03.15 201.53 2,739

MB 09.54 9,539.64 1,024

able 21. Read Operations


KB 303.37 303.37 33

4 KB 100.90 6,457.80 85

MB 23.66 23,655.83 423

able 22. Write Operation with One Node Shut Down


KB 15.47 15.47 647

4 KB 05.00 320.00 1,717

MB 04.39 4,100.44 2,439

able 23. Read Operation with One Node Shut Down


KB 303.66 303.66 33

4 KB 028.69 1,835.87 301

MB 27.39 27,392.29 365

32

32

Figure 13 shows a 4 Node Hitachi Content Platform system CPU Usage, Front-End and Back-End Network for 1 KB File (WRITE).

Figure 13

33

33

Figure 14 shows the number of ingested objects.

Figure 14

Figure 15 shows the 4-node Hitachi Content Platform system when a node failure is encountered during WRITE operations.

Figure 15

34

34

Figure 16 shows the 4-node Hitachi Content Platform system when a node failure is encountered during READ operations for 1 KB files.

Figure 16

35

35

Figure 17 shows a 4-node Hitachi Content Platform system without failure on 1 MB files (WRITE).

Figure 17

36

36

Figure 18 shows a 4-node Hitachi Content Platform system when a node failure is encountered during WRITE operations on 1 MB files.

Figure 18

Figure 19 shows the 4-node Hitachi Content Platform system when a node failure is encountered during READ operations on 1 MB files.

Figure 19

37

37

Test AnalysisFor write operations, the results show that a node failure in Hitachi Content Platform causes a drop of 20% in I/O rate and throughput in 1 KB files, while a significant impact of a 53% drop in I/O rate and 57% drop in throughput for 1 MB files.

VMware ValidationTo validate the VMware component for this solution, VMware was set up taking into consideration general guidelines in Deploying Hitachi Content Platform for vSphere ESXi. http://www.hds.com/assets/pdf/hitachi-content-platform-deploying-an-hcp-vm-system.pdf

Figure 20 shows the 4-node VMware ESX servers without any failure for 1 KB files (WRITE).

Figure 20

38

38

Figure 21 shows the VMware ESX2 server without any failure for 1 KB files (WRITE). It also shows that the network rate is 4Mb/sec average with a peak of 88 Mb/sec.

Figure 21

39

39

Figure 22 shows the VMware ESX4 server without any failure for 1 KB files (WRITE). It also shows that the network rate is 3 Mb/sec average with a peak of 18 Mb/sec.

Figure 22

40

40

Figure 23 shows Hitachi Content Platform for vSphere ESX1 component for 1 KB File READ Operation.

Figure 23

41

41

Figure 24 shows Hitachi Content Platform for vSphere ESX2 compo-

nent for 1 KB File READ Operation

Figure 24

Table 24 shows the 4-node VMware ESX servers without any failure for 1 KB files (READ).

The initial run resulted in the network (Mb/sec) climbing from 0Mbps to 24 Mb/sec (duration = 8 minutes) and going down gradually (duration = 7 Minutes), while a second run resulted in the network climbing from 28 Mb/sec to 43 Mb/sec (duration= 50 Minutes), 24.72 Mb/sec (Avg).

Table 24. VMware ESX Servers Without Failure

ESX1 ESX2 ESX3 ESX4

Memory% (Avg) 6.2 7.75 2.79 2.64

CPU% (Avg) 1.04 1.54 1.00 1.00

Disk (Kb/sec) 3.09 26.91 28.04 28.72

42

42

Figure 25 shows the 4-node VMware ESX servers when one node encountered failure during WRITE operations on 1 MB files.

Figure 25

Table 25 shows 4-node VMware ESX server resources during node failure (WRITE-1 MB file).

Table 26 shows 4-node VMware ESX server resources during node failure (READ-1 MB file).

Table 25. VMware ESX Servers During Node Failure (WRITE)

ESX1 ESX2 ESX3 ESX4

Memory% (Avg) 1.20 18.00 Down 15%

Disk (Kb/sec) 35.00 25.00 Down 25.00

Network (Mb/sec) 40.00 30.00 Down 35.00

Table 26. VMware ESX Servers During Node Failure (READ)

ESX1 ESX2 ESX3 ESX4

Memory% (Avg) 49.40 43.00 Down 44.00%

CPU% (Avg) 2.00 5.00 Down 2.50

Disk Rate (Kb/sec) 29.00 23.00 Down 15.00

43

43

ConclusionThis reference architecture documents a tested Hitachi Content Platform in Hitachi NAS Platform with NFS datastore solution using Hitachi Compute Rack 210H servers and Hitachi Unified Storage 130. The solution runs in VMware ESX5.0 Servers and vSphere Server for management.

For more information on Hitachi Content Platform and related references, please visit the following links:

Hitachi Content Platform Deploying an HCP VM System

http://www.hds.com/assets/pdf/hitachi-content-platform-deploying-an-hcp-vm-system.pdf

Hitachi Content Platform Installing a Hitachi Content Platform System

http://www.hds.com/assets/pdf/hcp-installation-guide.pdf

Performance Best Practices for VMware vSphere 5.1

http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.1.pdf

vSphere Storage ( ESXi 5.1 / vCenter Server 5.1 )

http://pubs.vmware.com/vsphere-51/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-51-storage-guide.pdf

Hitachi Content Platform Installing Hitachi Content Platform System

http://www.hds.com/assets/pdf/hcp-installation-guide.pdf

Hitachi NAS Platform Best Practices Guide for NFS with VMware vSphere

http://loop.hds.com/docs/DOC-31768



For More InformationHitachi Data Systems Global Services offers experienced storage consultants, proven methodologies and a comprehensive services portfolio to assist you in implementing Hitachi products and solutions in your environment. For more information, see the Hitachi Data Systems Global Services website.

Live and recorded product demonstrations are available for many Hitachi products. To schedule a live demonstration, contact a sales representative. To view a recorded demonstration, see the Hitachi Data Systems Corporate Resources website. Click the Product Demos tab for a list of available recorded demonstrations.

Hitachi Data Systems Academy provides best-in-class training on Hitachi products, technology, solutions and certifications. Hitachi Data Systems Academy delivers on-demand web-based training (WBT), classroom-based instructor-led training (ILT) and virtual instructor-led training (vILT) courses. For more information, see the Hitachi Data Systems Services Education website.

For more information about Hitachi products and services, contact your sales representative or channel partner or visit the Hitachi Data Systems website.

http://www.hds.com/services/

http://www.hds.com/services/

http://www.hds.com/corporate/resources/

http://www.hds.com/services/education/

http://www.hds.com/

http://www.hds.com/corporate/resources/

http://www.hds.com/services/education/

http://www.hds.com/

Corporate Headquarters2845 Lafayette Street, Santa Clara, California 95050-2627 USAwww.HDS.com

Regional Contact InformationAmericas: +1 408 970 1000 or [email protected], Middle East and Africa: +44 (0) 1753 618000 or [email protected] Asia-Pacific: +852 3189 7900 or [email protected]

© Hitachi Data Systems Corporation 2014. All rights reserved. HITACHI is a trademark or registered trademark of Hitachi, Ltd. “Innovate with Information” is a trademark or registered trademark of Hitachi Data Systems Corporation. Microsoft and Windows Server are trademarks or registered trademarks of Microsoft Corporation. All other trademarks, service marks, and company names are properties of their respective owners.

Notice: This document is for informational purposes only, and does not set forth any warranty, expressed or implied, concerning any equipment or service offered or to be offered by Hitachi Data Systems Corporation.

AS-327-00, September 2014

www.HDS.com

www.HDS.com

hitachi content platform for vmware vsphere esxi …...1 1 hitachi content platform for vmware...

Documents