Technical Report
FlexPod Express with VMware vSphere 5.5 Update 1: Small and Medium Configurations Implementation Guide
Arvind Ramakrishnan, Karthick Radhakrishnan, NetApp
Jeffrey Fultz, Cisco
September 2014 | TR-4328
2 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
TABLE OF CONTENTS
1 Overview ................................................................................................................................................ 6
2 Audience ................................................................................................................................................ 6
3 Architecture ........................................................................................................................................... 6
3.1 Small Configuration .........................................................................................................................................6
3.2 Medium Configuration .....................................................................................................................................7
4 Hardware Details ................................................................................................................................... 8
4.1 Small Configuration .........................................................................................................................................8
4.2 Medium Configuration .....................................................................................................................................8
5 Software Details .................................................................................................................................... 9
6 Configuration Guidelines ..................................................................................................................... 9
7 FlexPod Express Cabling Information .............................................................................................. 15
7.1 FlexPod Express Small Configuration ........................................................................................................... 15
7.2 FlexPod Express Medium Configuration ....................................................................................................... 18
8 Cisco Nexus 3048 Deployment Procedure ...................................................................................... 22
8.1 Performing Initial Cisco Nexus 3048 Switch Setup ....................................................................................... 22
8.2 Upgrading the Software (Optional) ................................................................................................................ 23
8.3 Enabling Advanced Features ........................................................................................................................ 24
8.4 Performing Global PortChannel Configuration .............................................................................................. 24
8.5 Performing Global Spanning-Tree Configuration .......................................................................................... 24
8.6 Configuring Jumbo Frames ........................................................................................................................... 25
8.7 Defining VLANs ............................................................................................................................................. 25
8.8 Configuring Access and Management Port Descriptions .............................................................................. 26
8.9 Configuring Server and Storage Management Interfaces ............................................................................. 27
8.10 Performing Virtual PortChannel Global Configuration ................................................................................... 27
8.11 Configuring Storage PortChannels ............................................................................................................... 28
8.12 Configuring Server Connections ................................................................................................................... 28
8.13 Performing In-Band Management SVI Configuration .................................................................................... 29
8.14 Saving the Configuration ............................................................................................................................... 29
8.15 Uplinking to Existing Network Infrastructure.................................................................................................. 29
9 NetApp FAS Storage Deployment Procedure .................................................................................. 30
9.1 Prerequisites for NetApp FAS2500 Series Controller Installation ................................................................. 30
9.2 Configuring Clustered Data ONTAP 8.2.2 .................................................................................................... 30
3 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
9.3 Creating Clusters in Clustered Data ONTAP ................................................................................................ 34
9.4 Joining Clusters in Clustered Data ONTAP .................................................................................................. 36
9.5 Changing the RAID Configuration on Root Aggregates ................................................................................ 37
9.6 Zeroing All Spare Disks ................................................................................................................................ 37
9.7 Configuring IFGRP LACP in Clustered Data ONTAP ................................................................................... 37
9.8 Configuring VLANs in Clustered Data ONTAP .............................................................................................. 38
9.9 Migrating the Cluster Management Interface on Controller 01 ...................................................................... 38
9.10 Logging in to the Cluster ............................................................................................................................... 38
9.11 Setting Auto-Revert on Cluster Management................................................................................................ 38
9.12 Configuring Failover Group Management in Clustered Data ONTAP ........................................................... 38
9.13 Assigning the Management Failover Group to the Cluster Management LIF ................................................ 39
9.14 Configuring Failover Group Node Management in Clustered Data ONTAP .................................................. 39
9.15 Assigning Node Management Failover Groups to Node Management LIFs ................................................. 39
9.16 Creating Aggregates ..................................................................................................................................... 39
9.17 Configuring the Service Processor ................................................................................................................ 39
9.18 Configuring Jumbo Frames in Clustered Data ONTAP ................................................................................. 40
9.19 Configuring NTP in Clustered Data ONTAP.................................................................................................. 41
9.20 Configuring SNMP in Clustered Data ONTAP .............................................................................................. 41
9.21 Configuring SNMPv1 in Clustered Data ONTAP ........................................................................................... 41
9.22 Configuring SNMPv3 in Clustered Data ONTAP ........................................................................................... 41
9.23 Configuring AutoSupport HTTPS in Clustered Data ONTAP ........................................................................ 42
9.24 Configuring Cisco Discovery Protocol in Clustered Data ONTAP ................................................................. 42
9.25 Setting Up Vserver ........................................................................................................................................ 42
9.26 Configuring HTTPS Access in Clustered Data ONTAP ................................................................................ 43
9.27 Configuring NFSv3 in Clustered Data ONTAP .............................................................................................. 44
9.28 Configuring FlexVol in Clustered Data ONTAP ............................................................................................. 44
9.29 Configuring Deduplication in Clustered Data ONTAP ................................................................................... 44
9.30 Configuring Failover Group NAS in Clustered Data ONTAP ......................................................................... 45
9.31 Configuring NFS LIF in Clustered Data ONTAP ........................................................................................... 45
9.32 Configuring Failover Group Vserver Management in Clustered Data ONTAP .............................................. 45
9.33 Adding Infrastructure Vserver Administrator ................................................................................................. 45
10 Cisco UCS C-Series Rack Server Deployment Procedure ............................................................. 45
10.1 Performing Initial Cisco UCS C-Series Standalone Server Setup for Cisco IMC .......................................... 45
10.2 Configuring Cisco UCS C-Series Servers for ESXi Installation on FlexFlash Cards ..................................... 48
11 VMware ESXi Deployment Procedure .............................................................................................. 51
4 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
11.1 Logging in to Cisco IMC Interface for Cisco UCS C-Series Standalone Servers .......................................... 51
11.2 Setting Up the VMware ESXi Installation ...................................................................................................... 52
11.3 Installing VMware ESXi ................................................................................................................................. 52
11.4 Setting Up VMware ESXi Host Management Networking ............................................................................. 52
11.5 Downloading VMware vSphere Client and vSphere Remote Command Line ............................................... 53
11.6 Logging in to VMware ESXi Hosts Using the VMware vSphere Client .......................................................... 54
11.7 Setting Up VMkernel Ports and the Virtual Switch ........................................................................................ 54
11.8 Mounting Required Datastores ..................................................................................................................... 56
11.9 Moving the Virtual Machine Swap-File Location ........................................................................................... 56
12 VMware vCenter 5.5 Update 1 Deployment Procedure ................................................................... 57
12.1 Building the VMware vCenter Virtual Machine .............................................................................................. 57
12.2 Changing the Root Password for VMware vCenter Server Appliance Web Client ........................................ 70
12.3 Configuring Single Sign-On Using VMware vSphere Web Client .................................................................. 73
12.4 Setting Up VMware vCenter Server .............................................................................................................. 76
12.5 Setting Up a Microsoft Windows Template ................................................................................................... 83
13 NetApp Virtual Storage Console 5.0 Deployment Procedure ........................................................ 85
13.1 NetApp VSC 5.0 Preinstallation Considerations ........................................................................................... 85
13.2 Installing NetApp VSC 5.0............................................................................................................................. 85
13.3 Registering NetApp VSC with VMware vCenter Server ................................................................................ 88
13.4 Updating Credentials for Storage Resources ................................................................................................ 89
13.5 Using Optimal Storage Settings for VMware ESXi Hosts .............................................................................. 92
13.6 Performing NetApp VSC 5.0 Backup and Recovery ..................................................................................... 94
14 Bill of Materials ................................................................................................................................... 98
14.1 Small Configuration ....................................................................................................................................... 98
14.2 Medium Configuration ................................................................................................................................. 100
15 FlexPod Express with Cisco UCS Director .................................................................................... 102
16 Conclusion ........................................................................................................................................ 102
References ............................................................................................................................................... 102
LIST OF TABLES
Table 1) Small configuration details. ..............................................................................................................................8
Table 2) Medium configuration details. ...........................................................................................................................8
Table 3) Software details. ...............................................................................................................................................9
5 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Table 4) Required VLANs. ............................................................................................................................................ 10
Table 5) VMware virtual machines created. ................................................................................................................. 10
Table 6) Deployment variables. .................................................................................................................................... 10
Table 7) Cabling information. ....................................................................................................................................... 16
Table 8) Cabling information. ....................................................................................................................................... 19
Table 9) NetApp FAS2500 series controller prerequisites. ........................................................................................... 30
Table 10) Small configuration components. ................................................................................................................. 98
Table 11) Medium configuration components. ............................................................................................................ 100
LIST OF FIGURES
Figure 1) Physical topology of FlexPod Express small configuration. ............................................................................7
Figure 2) Physical topology of FlexPod Express medium configuration. ........................................................................8
Figure 3) FlexPod Express small configuration cabling diagram. ................................................................................. 16
Figure 4) FlexPod Express medium configuration cabling diagram. ............................................................................. 19
6 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
1 Overview
The small and medium FlexPod® Express configurations are low-cost, standardized infrastructure
solutions developed to meet the needs of small and midsize businesses. The configurations have been
built and tested to deliver a cost-effective, high-value, best practice architecture. Each configuration
provides a standardized base platform capable of running a number of business-critical applications while
providing scalability options to enable the infrastructure to grow with the demands of the business.
2 Audience
This document describes the architecture and deployment procedures for both small and medium
FlexPod Express configurations with NetApp® clustered Data ONTAP
®. The intended audience for this
document includes, but is not limited to, sales engineers, field consultants, professional services, IT
managers, partner engineers, and customers who want to deploy FlexPod Express.
3 Architecture
Both the small and medium FlexPod Express configurations use Cisco UCS® C-Series rack servers,
Cisco Nexus® switches, and NetApp FAS storage (NetApp clustered Data ONTAP: switchless). Although
FlexPod Express supports an open ecosystem of virtualization and management software solutions, the
architecture described in this document specifically includes VMware vSphere® virtualization and Cisco
UCS Director software. NetApp strongly recommends virtualization software and infrastructure
management software as part of every FlexPod Express deployment. Each configuration uses the best
practices for each component to enable a reliable, enterprise-class infrastructure.
3.1 Small Configuration
The small configuration consists of the following components (Figure 1):
Cisco Nexus 3048 switches
Cisco UCS C220 M3 rack servers
NetApp FAS2520 storage controllers
VMware vSphere 5.5 Update 1 virtualization hypervisor
Cisco UCS Director 4.1 infrastructure management software
7 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Figure 1) Physical topology of FlexPod Express small configuration.
3.2 Medium Configuration
The medium configuration consists of the following components (Figure 2):
Cisco Nexus 3048 switches
Cisco UCS C220 M3 rack servers
NetApp FAS2520 storage controllers
VMware vSphere 5.5 Update 1 virtualization hypervisor
Cisco UCS Director 4.1 infrastructure management software
8 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Figure 2) Physical topology of FlexPod Express medium configuration.
4 Hardware Details
4.1 Small Configuration
Table 1 details the hardware and software configuration of a small FlexPod Express configuration.
Table 1) Small configuration details.
Layer Component Quantity
Computing Cisco UCS C220 M3 rack servers (standalone) 2
Network Cisco Nexus 3048 switches 2
Storage NetApp FAS2520 (high-availability pair) 1
Disks 900GB 10,000-rpm SAS 12
4.2 Medium Configuration
Table 2 details the hardware and software configuration of a medium FlexPod Express configuration.
Table 2) Medium configuration details.
Layer Component Quantity
Computing Cisco UCS C220 M3 rack servers (standalone) 4
Network Cisco Nexus 3048 switches 2
Storage NetApp FAS2520 (high-availability pair) 1
9 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Layer Component Quantity
Disks 900GB 10,000-rpm SAS 12
5 Software Details
It is important to note the software versions used in this document. Table 3 details the software revisions
used throughout this document.
Table 3) Software details.
Layer Component Version or Release Details Computing Cisco UCS C220 M3 rack
servers 2.0(1a) Cisco
® Integrated Management
Controller (IMC) software
Network Cisco Nexus 3048 Gigabit Ethernet switches
6.0(2)U3(2) Cisco NX-OS Software
Storage NetApp FAS2520 high-availability storage
8.2.2 NetApp Data ONTAP software
Software VMware vSphere 5.5 Update 1 with P02
VMware® Virtualization
Hypervisor Suite
NetApp Virtual Storage Console (VSC)
5.0 NetApp Plug-In for VMware vCenter
™
Cisco UCS Director 4.1 Infrastructure management software
6 Configuration Guidelines
This document provides details for configuring a fully redundant, highly available FlexPod Express
system. To reflect this redundancy, the component being configured in each step is referred to as either
Component 01 or Component 02. For example, Controller 01 and Controller 02 identify the two NetApp
storage controllers that are provisioned with this document, and Switch A and Switch B identify the pair of
Cisco Nexus switches that are configured.
Additionally, this document details steps for provisioning multiple Cisco UCS hosts, and these are
identified sequentially: Server-1, Server-2, and so on.
To indicate that you should include information pertinent to your environment in a given step, <<text>>
appears as part of the command structure. See the following example for the vlan create command:
Controller01>vlan create vif0 <<ib_mgmt_vlan_id>>
This document is intended to enable you to fully configure the FlexPod Express environment. In this
process, various steps require you to insert customer-specific naming conventions, IP addresses, and
VLAN schemes. Table 4 describes the VLANs necessary for deployment as outlined in this guide. This
table can be completed based on the specific site variables and used in implementing the document
configuration steps.
Note: If you use separate in-band and out-of-band management VLANs, you must create a Layer 3 route between these VLANs. For this validation, a common management VLAN was used.
10 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Table 4) Required VLANs.
VLAN Name VLAN Purpose ID Used in Validating This Document
Management VLAN VLAN for management interfaces 186
Native VLAN VLAN to which untagged frames are assigned
2
Network File System (NFS) VLAN VLAN for NFS traffic 3011
VMware vMotion® VLAN
VLAN designated for the movement of virtual machines from one physical host to another
3012
Virtual machine traffic VLAN VLAN for virtual machine application traffic
3013
Table 5 lists VMware virtual machines (VMs) created.
Table 5) VMware virtual machines created.
Virtual Machine Description Host Name
VMware vCenter Server
Cisco UCS Director
NetApp Virtual Storage Console
Use Table 6 to gather all the necessary information required during the deployment.
Table 6) Deployment variables.
Variable Description Customer Implementation Value
<<admin_password>> Global default administrative password
<<switch_A_hostname>> Cisco Nexus A host name
<<switch_A_mgmt0_ip_addr
>> Cisco Nexus A management IP address
<<switch_A_mgmt0_netmask
>> Cisco Nexus A netmask
<<switch_B_hostname>> Cisco Nexus B host name
<<switch_B_mgmt0_ip_addr
>> Cisco Nexus B management IP address
<<switch_B_mgmt0_netmask
>> Cisco Nexus B netmask
<<nfs_vlan_id>> NFS VLAN ID
<<vmotion_vlan_id>> VMware vMotion VLAN ID
<<vmtraffic_vlan_id>> Virtual machine traffic VLAN ID
11 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Variable Description Customer Implementation Value
<<ib_mgmt_vlan_id>> Management VLAN ID
<<native_vlan_id>> Native VLAN ID
<<inband_mgmt_ip_address
>>
In-band management IP address for switch virtual interface (SVI)
<<inband_mgmt_netmask>> In-band management netmask for SVI
<<inband_mgmt_gateway>> In-band management gateway for SVI
<<controller01_mgmt_ip>> Management IP address for Controller 01
<<controller01_mgmt_netm
ask>> Controller 01 management netmask
<<controller01_mgmt_gate
way>> Controller 01 management gateway
<<controller02_mgmt_ip>> Management IP address for Controller 02
<<controller02_mgmt_netm
ask>> Controller 02 management netmask
<<controller02_mgmt_gate
way>> Controller 02 management gateway
<<url_boot_software>> NetApp Data ONTAP 8.2 URL; format: http://
<<dns_domain_name>> Domain Name System (DNS) domain name
<<nameserver_ip>> DNS server IP addresses
<<controller_location>> Physical location for each controller device
<<controller01>> Controller 01 host name
<<#_of_disks>> Number of disks to assign to each storage controller
<<controller02>> Controller 02 host name
<<num_disks>> Number of disks to assign to storage data aggregate
<<controller01_sp_ip>> Controller 01 service processor IP address
<<controller01_sp_netmas
k>> Controller 01 service processor netmask
12 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Variable Description Customer Implementation Value
<<controller01_sp_gatewa
y>> Controller 01 service processor gateway
<<controller02_sp_ip>> Controller 02 service processor IP address
<<controller02_sp_netmas
k>> Controller 02 service processor netmask
<<controller02_sp_gatewa
y>> Controller 02 service processor gateway
<<timezone>> FlexPod Express time zone
<<global_ntp_server_ip>> Network Time Protocol (NTP) server IP address
<<snmp_contact>> Storage administrator e-mail address
<<snmp_location>> Storage location string
<<snmp_trap_server_fqdn>
>
Fully qualified domain name (FQDN) of fault management system or NetApp DataFabric
®
Manager
<<snmp_community>> Simple Network Management Protocol Version 1 and 2 (SNMP v1 and v2) community name
<<mailhost>> Mail server host name
<<storage_admin_email>> Storage administrator e-mail address
<<country_code>> Two-letter country code
<<state>> State or province name
<<city>> City name
<<org>> Organization or company name
<<unit>> Organizational unit name
<<esxi_host1_nfs_ip>> NFS VLAN IP address for VMware ESXi™
host 1
<<esxi_host2_nfs_ip>> NFS VLAN IP address for VMware ESXi host 2
<<esxi_host3_nfs_ip>> NFS VLAN IP address for VMware ESXi host 3
13 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Variable Description Customer Implementation Value
<<esxi_host4_nfs_ip>> NFS VLAN IP address for VMware ESXi host 4
<<esxi_host1_nfs_netmask
>> NFS VLAN netmask for VMware ESXi host 1
<<esxi_host2_nfs_netmask
>> NFS VLAN netmask for VMware ESXi host 2
<<esxi_host3_nfs_netmask
>> NFS VLAN netmask for VMware ESXi host 3
<<esxi_host4_nfs_netmask
>> NFS VLAN netmask for VMware ESXi host 4
<<esxi_host1_vmotion_ip>
>
VMware vMotion VLAN IP address for VMware ESXi host 1
<<esxi_host2_vmotion_ip>
>
VMware vMotion VLAN IP address for VMware ESXi host 2
<<esxi_host3_vmotion_ip>
>
VMware vMotion VLAN IP address for VMware ESXi host 3
<<esxi_host4_vmotion_ip>
>
VMware vMotion VLAN IP address for VMware ESXi host 4
<<esxi_host1_vmotion_net
mask>>
VMware vMotion VLAN netmask for VMware ESXi host 1
<<esxi_host2_vmotion_net
mask>>
VMware vMotion VLAN netmask for VMware ESXi host 2
<<esxi_host3_vmotion_net
mask>>
VMware vMotion VLAN netmask for VMware ESXi host 3
<<esxi_host4_vmotion_net
mask>>
VMware vMotion VLAN netmask for VMware ESXi host 4
<<cimc_server1_ip>> Cisco IMC IP address for Cisco UCS C220 M3 Server-1
<<cimc_server2_ip>> Cisco IMC IP address for Cisco UCS C220 M3 Server-2
<<cimc_server3_ip>> Cisco IMC IP address for Cisco UCS C220 M3 Server-3
<<cimc_server4_ip>> Cisco IMC IP address for Cisco UCS C220 M3 Server-4
<<cimc_netmask>> Cisco IMC netmask for Cisco UCS C220 M3 servers
<<cimc_gateway>> Cisco IMC gateway for Cisco UCS C220 M3 servers
<<esxi_host1_mgmt_ip>> Management IP address for VMware ESXi host 1
<<esxi_host1_mgmt_netmas
k>> Management netmask for VMware ESXi host 1
14 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Variable Description Customer Implementation Value
<<esxi_host1_mgmt_gatewa
y>> Management gateway for VMware ESXi host 1
<<esxi_host1_fqdn>> FQDN for VMware ESXi host 1
<<esxi_host2_mgmt_ip>> Management IP address for VMware ESXi host 2
<<esxi_host2_mgmt_netmas
k>> Management netmask for VMware ESXi host 2
<<esxi_host2_mgmt_gatewa
y>> Management gateway for VMware ESXi host 2
<<esxi_host2_fqdn>> FQDN for VMware ESXi host 2
<<esxi_host3_mgmt_ip>> Management IP address for VMware ESXi host 3
<<esxi_host3_mgmt_netmas
k>> Management netmask for VMware ESXi host 3
<<esxi_host3_mgmt_gatewa
y>> Management gateway for VMware ESXi host 3
<<esxi_host3_fqdn>> FQDN for VMware ESXi host 3
<<esxi_host4_mgmt_ip>> Management IP address for VMware ESXi host 4
<<esxi_host4_mgmt_netmas
k>> Management netmask for VMware ESXi host 4
<<esxi_host4_mgmt_gatewa
y>> Management gateway for VMware ESXi host 4
<<esxi_host4_fqdn>> FQDN for VMware ESXi host 4
<<clustername>> Storage cluster host name
<<cluster_base_license_k
ey>> Cluster base license key
<<clustermgmt_ip>> Cluster management IP address for storage cluster
<<clustermgmt_netmask>> Cluster management netmask for storage cluster
<<clustermgmt_gateway>> Cluster management gateway for storage cluster
<<controller01_rootaggrn
ame>> Root aggregate name of Controller 01
<<security_cert_vserver_
common_name>> Infrastructure virtual server (Vserver) FQDN
15 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Variable Description Customer Implementation Value
<<security_cert_cluster_
common_name>> Storage cluster FQDN
<<security_cert_controll
er01_common_name>> Controller 01 FQDN
<<security_cert_controll
er02_common_name>> Controller 02 FQDN
<<security_certificate_v
servser_authority>> Infrastructure Vserver security certificate authority
<<security_certificate_v
server_serial_no>>
Infrastructure Vserver security certificate serial number
<<security_certificate_c
luster_authority>> Storage cluster security certificate authority
<<security_certificate_c
luster_serial_no>> Storage cluster security certificate serial number
<<security_certificate_c
ontroller01_authority>> Controller 01 security certificate authority
<<security_certificate_c
ontroller01_serial_no>> Controller 01 security certificate serial
<<security_certificate_c
ontroller02_authority>> Controller 02 security certificate authority
<<security_certificate_c
ontroller02_serial_no>> Controller 02 security certificate serial
<<controller01_nfs_lif_i
p>> Controller 01 NFS logical interface (LIF) IP address
<<controller02_nfs_lif_i
p>> Controller 02 NFS LIF IP address
<<controller01_nfs_lif_n
etmask>> Controller 01 NFS LIF netmask
<<controller02_nfs_lif_n
etmask>> Controller 02 NFS LIF netmask
<<vserver_mgmt_ip>> Management IP address for Vserver
<<vserver_mgmt_netmask>> Subnet mask for Vserver
<<vsadmin_password>> Password for Vserver administrator account
7 FlexPod Express Cabling Information
7.1 FlexPod Express Small Configuration
Figure 3 provides a cabling diagram for the FlexPod Express small configuration, and Table 7 provides
cabling information.
16 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Figure 3) FlexPod Express small configuration cabling diagram.
Table 7) Cabling information.
Local Device Local Port Remote Device Remote
Port
Cabling
Code
Cisco Nexus
3048 Switch A
Eth1/1 NetApp FAS2520 Storage Controller 01 e0a
Eth1/2 NetApp FAS2520 Storage Controller 02 e0a
Eth1/3 NetApp FAS2520 Storage Controller 01 e0b
Eth1/4 NetApp FAS2520 Storage Controller 02 e0b
Eth1/13 Cisco UCS C220 M3 Standalone Server-1 Port 1/1
Eth1/14 Cisco UCS C220 M3 Standalone Server-1 Port 1/2
Eth1/15 Cisco UCS C220 M3 Standalone Server-1 Port 0/1
Eth1/16 Cisco UCS C220 M3 Standalone Server-2 Port 1/1
Eth1/17 Cisco UCS C220 M3 Standalone Server-2 Port 1/2
Eth1/18 Cisco UCS C220 M3 Standalone Server-2 Port 0/1
Eth1/25 Cisco Nexus 3048 Switch B Eth1/25
1
2
3
4
5
6
7
8
11
9
10
17 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Local Device Local Port Remote Device Remote
Port
Cabling
Code
Eth1/26 Cisco Nexus 3048 Switch B Eth1/26
Eth1/37 Cisco UCS C220 M3 Standalone Server-1 Cisco IMC
Eth1/39 NetApp FAS2520 Storage Controller 01 e0M
Local Device Local Port Remote Device Remote
Port
Cabling
Code
Cisco Nexus
3048 Switch B
Eth1/1 NetApp FAS2520 Storage Controller 01 e0c
Eth1/2 NetApp FAS2520 Storage Controller 02 e0c
Eth1/3 NetApp FAS2520 Storage Controller 01 e0e
Eth1/4 NetApp FAS2520 Storage Controller 02 e0e
Eth1/13 Cisco UCS C220 M3 Standalone Server-1 Port1/3
Eth1/14 Cisco UCS C220 M3 Standalone Server-1 Port1/4
Eth1/15 Cisco UCS C220 M3 Standalone Server-1 Port0/2
Eth1/16 Cisco UCS C220 M3 Standalone Server-2 Port1/3
Eth1/17 Cisco UCS C220 M3 Standalone Server-2 Port1/4
Eth1/18 Cisco UCS C220 M3 Standalone Server-2 Port0/2
Eth1/25 Cisco Nexus 3048 Switch A Eth1/25
Eth1/26 Cisco Nexus 3048 Switch A Eth1/26
Eth1/37 Cisco UCS C220 M3 Standalone Server-2 Cisco IMC
Eth1/39 NetApp FAS2520 Storage Controller 02 e0M
12
13
14
15
16
17
18
19
20
21
22
11
12
25
26
23
24
18 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Local Device Local Port Remote Device Remote
Port
Cabling
Code
NetApp
FAS2520
Storage
Controller 01
e0d NetApp FAS2520 Storage Controller 02 e0d
e0f NetApp FAS2520 Storage Controller 02 e0f
ACP NetApp FAS2520 Storage Controller 02 ACP
SAS 0b NetApp FAS2520 Storage Controller 01 SAS 0a
SAS 0a NetApp FAS2520 Storage Controller 01 SAS 0b
Local Device Local Port Remote Device Remote
Port
Cabling
Code
NetApp
FAS2520
Storage
Controller 02
e0d NetApp FAS2520 Storage Controller 01 e0d
e0f NetApp FAS2520 Storage Controller 01 e0f
ACP NetApp FAS2520 Storage Controller 01 ACP
SAS 0b NetApp FAS2520 Storage Controller 01 SAS 0a
SAS 0a NetApp FAS2520 Storage Controller 01 SAS 0b
7.2 FlexPod Express Medium Configuration
Figure 4 provides a cabling diagram for the FlexPod Express medium configuration, and Table 8 provides
cabling information.
27
28
29
30
31
27
28
28
28
28
19 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Figure 4) FlexPod Express medium configuration cabling diagram.
Table 8) Cabling information.
Local Device Local Port Remote Device Remote
Port
Cabling
Code
Cisco Nexus
3048 Switch A
Eth1/1 NetApp FAS2520 Storage Controller 01 e0a
Eth1/2 NetApp FAS2520 Storage Controller 02 e0a
Eth1/3 NetApp FAS2520 Storage Controller 01 e0b
Eth1/4 NetApp FAS2520 Storage Controller 02 e0b
Eth1/13 Cisco UCS C220 M3 Standalone Server-1 Port1/1
Eth1/14 Cisco UCS C220 M3 Standalone Server-1 Port1/2
Eth1/15 Cisco UCS C220 M3 Standalone Server-1 Port0/1
Eth1/16 Cisco UCS C220 M3 Standalone Server-2 Port1/1
1
2
3
4
5
6
7
8
20 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Local Device Local Port Remote Device Remote
Port
Cabling
Code
Eth1/17 Cisco UCS C220 M3 Standalone Server-2 Port1/2
Eth1/18 Cisco UCS C220 M3 Standalone Server-2 Port0/1
Eth1/19 Cisco UCS C220 M3 Standalone Server-3 Port1/1
Eth1/20 Cisco UCS C220 M3 Standalone Server-3 Port1/2
Eth1/21 Cisco UCS C220 M3 Standalone Server-3 Port0/1
Eth1/22 Cisco UCS C220 M3 Standalone Server-4 Port1/1
Eth1/23 Cisco UCS C220 M3 Standalone Server-4 Port1/2
Eth1/24 Cisco UCS C220 M3 Standalone Server-4 Port0/1
Eth1/25 Cisco Nexus 3048 Switch B Eth1/25
Eth1/26 Cisco Nexus 3048 Switch B Eth1/26
Eth1/37 Cisco UCS C220 M3 Standalone Server-1 Cisco IMC
Eth1/38 Cisco UCS C220 M3 Standalone Server-3 Cisco IMC
Eth1/39 NetApp FAS2520 Storage Controller 01 e0M
Local Device Local Port Remote Device Remote
Port
Cabling
Code
Cisco Nexus
3048 Switch B
Eth1/1 NetApp FAS2520 Storage Controller 01 e0c
Eth1/2 NetApp FAS2520 Storage Controller 02 e0c
Eth1/3 NetApp FAS2520 Storage Controller 01 e0e
Eth1/4 NetApp FAS2520 Storage Controller 02 e0e
Eth1/13 Cisco UCS C220 M3 Standalone Server-1 Port1/3
17
18
19
21
22
23
24
25
26
9
10
20
11
12
13
14
15
16
21 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Local Device Local Port Remote Device Remote
Port
Cabling
Code
Eth1/14 Cisco UCS C220 M3 Standalone Server-1 Port1/4
Eth1/15 Cisco UCS C220 M3 Standalone Server-1 Port0/2
Eth1/16 Cisco UCS C220 M3 Standalone Server-2 Port1/3
Eth1/17 Cisco UCS C220 M3 Standalone Server-2 Port1/4
Eth1/18 Cisco UCS C220 M3 Standalone Server-2 Port0/2
Eth1/19 Cisco UCS C220 M3 Standalone Server-3 Port1/3
Eth1/20 Cisco UCS C220 M3 Standalone Server-3 Port1/4
Eth1/21 Cisco UCS C220 M3 Standalone Server-3 Port0/2
Eth1/22 Cisco UCS C220 M3 Standalone Server-4 Port1/3
Eth1/23 Cisco UCS C220 M3 Standalone Server-4 Port1/4
Eth1/24 Cisco UCS C220 M3 Standalone Server-4 Port0/2
Eth1/25 Cisco Nexus 3048 Switch A Eth1/25
Eth1/26 Cisco Nexus 3048 Switch A Eth1/26
Eth1/37 Cisco UCS C220 C-Series Standalone Server-2 Cisco IMC
Eth1/38 Cisco UCS C220 M3 Standalone Server-4 Cisco IMC
Eth1/39 NetApp FAS2520 Storage Controller 02 e0M
Local Device Local Port Remote Device Remote
Port
Cabling
Code
NetApp
FAS2520
Storage
e0d NetApp FAS2520 Storage Controller 02 e0d
e0f NetApp FAS2520 Storage Controller 02 e0f
27
28
29
17
18
38
40
30
31
32
33
39
34
35
36
37
41
42
22 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Local Device Local Port Remote Device Remote
Port
Cabling
Code
Controller 01 ACP NetApp FAS2520 Storage Controller 02 ACP
SAS 0b NetApp FAS2520 Storage Controller 02 SAS 0a
SAS 0a NetApp FAS2520 Storage Controller 02 SAS 0b
Local Device Local Port Remote Device Remote
Port
Cabling
Code
NetApp
FAS2520
Storage
Controller 02
e0d NetApp FAS2520 Storage Controller 01 e0d
e0f NetApp FAS2520 Storage Controller 01 e0f
ACP NetApp FAS2520 Storage Controller 01 ACP
SAS 0b NetApp FAS2520 Storage Controller 01 SAS 0a
SAS 0a NetApp FAS2520 Storage Controller 01 SAS 0b
8 Cisco Nexus 3048 Deployment Procedure
The following section details the Cisco Nexus 3048 switch configuration for use in a FlexPod Express
environment.
8.1 Performing Initial Cisco Nexus 3048 Switch Setup
Upon initial boot and connection to the console port of the switch, the Cisco NX-OS setup automatically
starts. This initial configuration addresses basic settings, such as the switch name, the mgmt0 interface
configuration, and Secure Shell (SSH) setup, and defines the control-plane policing policy.
The first major decision involves the configuration of the management network for the switches. For
FlexPod Express, there are two main options for configuring the mgmt0 interfaces. The first involves
configuring and cabling the mgmt0 interfaces into an existing out-of-band network. In this instance, when
a management network already exists, all you need are valid IP addresses and the netmask configuration
for this network and a connection from the mgmt0 interfaces to this network.
The other option, for installations without a dedicated management network, involves cabling the mgmt0
interfaces of each Cisco Nexus 3048 switch together in a back-to-back configuration. Any valid IP
address and netmask can be configured on each mgmt0 interface as long as they are in the same
network. Because they are configured back to back with no switch or other device in between, no default
gateway configuration is needed, and they should be able to communicate with each other. This link
cannot be used for external management access such as SSH access, but it will be used for the virtual
PortChannel (vPC) peer keepalive traffic. To enable SSH management access to the switch, you need to
configure the in-band interface VLAN IP address on an SVI, as discussed later in this document.
43
44
45
41
42
43
44
45
23 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Power on the switch and follow the onscreen prompts as illustrated here for the initial setup of both 1.switches, substituting the appropriate values for the switch-specific information.
Switches A and B
Abort Power On Auto Provisioning and continue with normal setup ?(yes/no)[n]: yes
---- System Admin Account Setup ----
Do you want to enforce secure password standard (yes/no): yes
Enter the password for "admin":<<admin_password>>
Confirm the password for "admin":<<admin_password>>
---- Basic System Configuration Dialog ----
This setup utility will guide you through the basic configuration of
the system. Setup configures only enough connectivity for management
of the system.
Please register Cisco Nexus 3000 Family devices promptly with your
supplier. Failure to register may affect response times for initial
service calls. Nexus devices must be registered to receive entitled
support services.
Press Enter at anytime to skip a dialog. Use ctrl-c at anytime
to skip the remaining dialogs.
Would you like to enter the basic configuration dialog (yes/no): yes
Create another login account (yes/no) [n]: Enter
Configure read-only SNMP community string (yes/no) [n]:Enter
Configure read-write SNMP community string (yes/no) [n]:Enter
Enter the switch name : <<switch_A/B_hostname>>
Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]:Enter
Mgmt0 IPv4 address : <<switch_A/B_mgmt0_ip_addr>>
Mgmt0 IPv4 netmask : <<switch_A/B_mgmt0_netmask>>
Configure the default gateway? (yes/no) [y]:n
Enable the telnet service? (yes/no) [n]:Enter
Enable the ssh service? (yes/no) [y]:Enter
Type of ssh key you would like to generate (dsa/rsa) : rsa
Number of key bits <768-2048> : 1024
Configure the ntp server? (yes/no) [n]:Enter
Configure default interface layer (L3/L2) [L2]:Enter
Configure default switchport interface state (shut/noshut) [noshut]:Enter
Configure CoPP System Policy Profile ( default / l2 / l3 ) [default]:Enter
The following configuration will be applied:
switchname <<switch_A/B_hostname>>
interface mgmt0
ip address <<switch_A/B_mgmt0_ip_addr>> <<switch_A/B_mgmt0_netmask>>
no shutdown
no telnet server enable
ssh key rsa 1024 force
ssh server enable
system default switchport
no system default switchport shutdown
policy-map type control-plane copp-system-policy ( default )
Would you like to edit the configuration? (yes/no) [n]:Enter
Use this configuration and save it? (yes/no) [y]:Enter
8.2 Upgrading the Software (Optional)
You should perform any required software upgrades on the switch at this point in the configuration
process. Download and install the latest available Cisco NX-OS software for the Cisco Nexus 3048 switch
from the Cisco software download site. There are multiple ways to transfer both the kickstart and system
images for Cisco NX-OS to the switch. The most straightforward procedure uses the onboard USB port
24 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
on the switch. Download the Cisco NX-OS kickstart and system files to a USB drive and plug the USB
drive into the external USB port on the Cisco Nexus 3048 switch.
Note: Cisco NX-OS software release 6.0(2)U3(2) is used in this solution.
Copy the files to the local bootflash memory and update the switch by following the procedure shown 1.here.
Switches A and B
copy usb1:<<kickstart_image_file>> bootflash:
copy usb1:<<system_image_file>> bootflash:
install all kickstart bootflash:<<kickstart_image_file>> system bootflash:<<system_image_file>>
Note: The switch will install the updated Cisco NX-OS files and reboot.
8.3 Enabling Advanced Features
Certain advanced features need to be enabled in Cisco NX-OS to provide additional configuration
options.
Note: The interface-vlan feature is required only if you are using the back-to-back mgmt0 option described throughout this document. This feature allows an IP address to be assigned to the interface VLAN (SVI), which enables in-band management communication to the switch, such as through SSH.
Enter configuration mode using the (config t) command and type the following commands to enable
the appropriate features on each switch.
Switches A and B
feature interface-vlan
feature lacp
feature vpc
8.4 Performing Global PortChannel Configuration
The default PortChannel load-balancing hash uses the source and destination IP addresses to determine
the load-balancing algorithm across the interfaces in the PortChannel. Better distribution across the
members of the PortChannels can be achieved by providing more inputs to the hash algorithm beyond
the source and destination IP addresses. For that reason, adding the source and destination TCP ports to
the hash algorithm is highly recommended.
From configuration mode (config t) type the following commands to configure the global PortChannel
load-balancing configuration on each switch.
Switches A and B
port-channel load-balance ethernet source-dest-port
8.5 Performing Global Spanning-Tree Configuration
The Cisco Nexus platform uses a new protection feature called bridge assurance. Bridge assurance helps
protect against a unidirectional link or other software failure and a device that continues to forward data
traffic when it is no longer running the spanning-tree algorithm. Ports can be placed in one of several
states, including network and edge, depending on the platform.
The recommended setting for bridge assurance is to consider all ports to be network ports by default.
This setting will force the network administrator to review the configuration of each port and will help
reveal the most common configuration errors, such as unidentified edge ports or a neighbor that does not
25 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
have bridge assurance enabled. Also, it is safer to have spanning tree block too many ports than not
enough, allowing the default port state to enhance the overall stability of the network.
Pay close attention to the spanning-tree state when adding servers, storage, and uplink switches,
especially if they do not support bridge assurance. In those cases, you might need to change the port
type to make the ports active.
Bridge Protocol Data Unit (BPDU) guard is enabled on edge ports by default as another layer of
protection. To prevent loops in the network, this feature will shut down the port if BPDUs from another
switch are seen on this interface.
From configuration mode (config t) type the following commands to configure the default spanning-
tree options, including the default port type and BPDU guard on each switch.
Switches A and B
spanning-tree port type network default
spanning-tree port type edge bpduguard default
8.6 Configuring Jumbo Frames
Jumbo frames should be configured throughout the network to allow any applications and operating
systems to transmit these larger frames without fragmentation. Note that both endpoints and all interfaces
between the endpoints (Layer 2 and Layer 3) must support and be configured for jumbo frames to
achieve the benefits and to prevent performance problems by fragmenting frames.
From configuration mode (config t) type the following commands to enable jumbo frames on each
switch.
Switches A and B
policy-map type network-qos jumbo
class type network-qos class-default
mtu 9000
system qos
service-policy type network-qos jumbo
exit
8.7 Defining VLANs Before configuring individual ports with different VLANs, those Layer 2 VLANs must be defined on the switch. It’s also good practice to name the VLANs to help with any troubleshooting in the future.
From configuration mode (config t) type the following commands to define and give descriptions to
the Layer 2 VLANs.
Switches A and B
vlan <<nfs_vlan_id>>
name NFS-VLAN
vlan <<vmotion_vlan_id>>
name vMotion-VLAN
vlan <<vmtraffic_vlan_id>>
name VM-Traffic-VLAN
vlan <<ib_mgmt_vlan_id>>
name IB-MGMT-VLAN
vlan <<native_vlan_id>>
name NATIVE-VLAN
exit
26 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
8.8 Configuring Access and Management Port Descriptions
As with the assignment of names to the Layer 2 VLANs, setting descriptions for all the interfaces can help
with both provisioning and troubleshooting.
For the small configuration, the descriptions for the management ports and data ports associated with
Server-3 and Server-4 are not required because the FlexPod Express small configuration contains only
two servers.
From configuration mode (config t) in each switch, type the following commands to set up the port
descriptions.
FlexPod Express Small Configuration
Enter the following port descriptions for the FlexPod Express small configuration.
Switch A Switch B
int eth1/1
description Controller-01:e0a
int eth1/2
description Controller-02:e0a
int eth1/3
description Controller-01:e0b
int eth1/4
description Controller-02:e0b
int eth1/13
description Server-1:Port1/1
int eth1/14
description Server-1:Port1/2
int eth1/15
description Server-1:Port-0/1
int eth1/16
description Server-2:Port-1/1
int eth1/17
description Server-2:Port-1/2
int eth1/18
description Server-2:Port-0/1
int eth1/25
description vPC peer-link NX3048-B:1/25
int eth1/26
description vPC peer-link NX3048-B:1/26
int eth1/37
description Server-1:mgmt
int eth1/39
description Controller-01:mgmt
int eth1/1
description Controller-01:e0c
int eth1/2
description Controller-02:e0c
int eth1/3
description Controller-01:e0e
int eth1/4
description Controller-02:e0e
int eth1/13
description Server-1:Port-1/3
int eth1/14
description Server-1:Port-1/4
int eth1/15
description Server-1:Port-0/2
int eth1/16
description Server-2:Port-1/3
int eth1/17
description Server-2:Port-1/4
int eth1/18
description Server-2:Port-0/2
int eth1/25
description vPC peer-link NX3048-A:1/25
int eth1/26
description vPC peer-link NX3048-A:1/26
int eth1/37
description Server-2:mgmt
int eth1/39
description Controller-02:mgmt
FlexPod Express Medium Configuration
For the FlexPod Express medium configuration, provide the following port descriptions in addition to the
port descriptions for the FlexPod Express Small Configuration.
Switch A Switch B
int eth1/19
description Server-3:Port-1/1
int eth1/20
description Server-3:Port-1/2
int eth1/21
description Server-3:Port-0/1
int eth1/22
description Server-4:Port-1/1
int eth1/23
description Server-4:Port-1/2
int eth1/24
description Server-4:Port-0/1
int eth1/19
description Server-3:Port-1/3
int eth1/20
description Server-3:Port-1/4
int eth1/21
description Server-3:Port-0/2
int eth1/22
description Server-4:Port-1/3
int eth1/23
description Server-4:Port-1/4
int eth1/24
description Server-4:Port-0/2
27 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
int eth1/38
description Server-3:mgmt
int eth1/38
description Server-4:mgmt
8.9 Configuring Server and Storage Management Interfaces
The management interfaces for both the server and storage typically use only a single VLAN. Therefore,
you should configure the management interface ports as access ports. Define the management VLAN for
each switch and change the spanning-tree port type to edge.
From configuration mode (config t) type the following commands to configure the port settings for the
management interfaces of both the servers and storage.
Switches A and B
int eth1/37-39
switchport access vlan <<ib_mgmt_vlan_id>>
spanning-tree port type edge
exit
8.10 Performing Virtual PortChannel Global Configuration
The vPC feature requires some initial setup between the two Cisco Nexus switches to function properly. If
you are using the back-to-back mgmt0 configuration, be sure to use the addresses defined on the
interfaces and verify that they can communicate by using the ping
<<switch_A/B_mgmt0_ip_addr>>vrf management command.
From configuration mode (config t) type the following commands to configure the vPC global
configuration for switch 1.
Switch A
vpc domain 1
role priority 10
peer-keepalive destination <<switch_B_mgmt0_ip_addr>> source <<switch_A_mgmt0_ip_addr>> vrf management
int eth1/25-26
channel-group 10 mode active
int Po10
description vPC peer-link
switchport
switchport mode trunk
switchport trunk native vlan <<native_vlan_id>>
switchport trunk allowed vlan <<nfs_vlan_id>>,<<vmotion_vlan_id>>, <<vmtraffic_vlan_id>>,
<<ib_mgmt_vlan_id>>
spanning-tree port type network
vpc peer-link
no shut
exit
copy run start
From configuration mode (config t), type the following commands to configure the vPC global
configuration for switch 2.
Switch B
vpc domain 1
role priority 20
peer-keepalive destination <<switch_A_mgmt0_ip_addr>> source <<switch_B_mgmt0_ip_addr>> vrf
management
int eth1/25-26
channel-group 10 mode active
28 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
int Po10
description vPC peer-link
switchport
switchport mode trunk
switchport trunk native vlan <<native_vlan_id>>
switchport trunk allowed vlan <<nfs_vlan_id>>,<<vmotion_vlan_id>>, <<vmtraffic_vlan_id>>,
<<ib_mgmt_vlan_id>>
spanning-tree port type network
vpc peer-link
no shut
exit
copy run start
8.11 Configuring Storage PortChannels
The NetApp storage controllers allow an active-active connection to the network using Link Aggregation
Control Protocol (LACP). The use of LACP is preferred because it adds both negotiation and logging
between the switches. Because the network is set up for vPC, this approach allows you to have active-
active connections from the storage to completely separate physical switches. Each controller will have
two links to each switch, but all four are part of the same vPC and interface group (IFGRP).
From configuration mode (config t) type the following commands on each switch to configure the
individual interfaces and the resulting PortChannel configuration for the ports connected to the NetApp FAS controller.
Switches A and B and Controller-01 Configuration
int eth1/1, eth1/3
channel-group 11 mode active
int Po11
description vPC to Controller-01
switchport
switchport mode trunk
switchport trunk native vlan <<native_vlan_id>>
switchport trunk allowed vlan <<nfs_vlan_id>>,<<ib_mgmt_vlan_id>>
spanning-tree port type edge trunk
vpc 11
no shut
Switches A and B and Controller-02 Configuration
int eth1/2, eth1/4
channel-group 12 mode active
int Po12
description vPC to Controller-02
switchport
switchport mode trunk
switchport trunk native vlan <<native_vlan_id>>
switchport trunk allowed vlan <<nfs_vlan_id>>,<<ib_mgmt_vlan_id>>
spanning-tree port type edge trunk
vpc 12
no shut
exit
copy run start
8.12 Configuring Server Connections
The Cisco UCS servers have multiple Ethernet interfaces that can be configured to fail over to one
another, providing additional redundancy beyond a single link. Spreading these links across multiple
switches enables the server to survive even a complete switch failure.
29 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
For the small configuration, you need to configure only Server-1 and Server-2 because only two servers
are used in the small FlexPod Express configuration.
From configuration mode (config t) type the following commands to configure the port settings for the
interfaces connected to each server.
FlexPod Express Small Configuration
Switches A and B and Server-1 and Server-2 Configuration
int eth1/13-18
switchport mode trunk
switchport trunk native vlan <<native_vlan_id>>
switchport trunk allowed vlan <<nfs_vlan_id>>,<<vmotion_vlan_id>>,<<vmtraffic_vlan_id>>,
<<ib_mgmt_vlan_id>>
spanning-tree port type edge trunk
no shut
exit
copy run start
FlexPod Express Medium Configuration
Switches A and B and Server-1, Server-2, Server-3, and Server-4 Configuration
int eth1/13-24
switchport mode trunk
switchport trunk allowed vlan
<<native_vlan_id>>,<<nfs_vlan_id>>,<<vmotion_vlan_id>>,<<vmtraffic_vlan_id>>, <<ib_mgmt_vlan_id>>
spanning-tree port type edge trunk
no shut
8.13 Performing In-Band Management SVI Configuration
In-band management using SSH in the FlexPod Express environment is handled by an SVI. To configure
the in-band management on each switch, you must configure an IP address on the interface VLAN and
set up a default gateway.
From configuration mode (config t) type the following commands to configure the Layer 3 SVI for
management purposes.
Switches A and B
int Vlan <<ib_mgmt_vlan_id>>
ip address <<inband_mgmt_ip_address>>/<<inband_mgmt_netmask>>
no shut
ip route 0.0.0.0/0 <<inband_mgmt_gateway>>
8.14 Saving the Configuration
Save the configuration on both switches for configuration persistence.
Switches A and B
copy run start
8.15 Uplinking to Existing Network Infrastructure
Depending on the available network infrastructure, several methods and features can be used to uplink
the FlexPod Express environment. If an existing Cisco Nexus environment is present, you should use
vPC to uplink the Cisco Nexus 3048 switches included in the FlexPod Express environment to the
30 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
infrastructure. Be sure to type copy run start to save the configuration on each switch after
configuration is complete.
9 NetApp FAS Storage Deployment Procedure
This section describes the NetApp FAS storage deployment procedure.
9.1 Prerequisites for NetApp FAS2500 Series Controller Installation
Table 9 lists the prerequisites for installing the NetApp FAS2500 series controller.
Table 9) NetApp FAS2500 series controller prerequisites.
Requirement Reference Comments
Physical site where storage system needs to be installed must be ready.
Site Requirements Guide Refer to the “Site Preparation” section.
Storage system connectivity requirements must be met.
Site Requirements Guide Refer to the “System Connectivity Requirements” section.
Storage system general power requirements must be met.
Site Requirements Guide Refer to the “Circuit Breaker, Power Outlet Balancing, System Cabinet Power Cord Plugs, and Console Pinout Requirements” section.
Storage system model-specific requirements must be met.
Site Requirements Guide Refer to the “NetApp FAS2500 Series Systems” section.
NetApp Hardware Universe
The NetApp Hardware Universe provides supported hardware and software components for the specific
Data ONTAP version. It provides configuration information for all NetApp storage appliances currently
supported by the Data ONTAP software. It also provides a table of component compatibilities.
Make sure that the hardware and software components are supported with the version of Data 1.ONTAP that you plan to install by checking the NetApp Hardware Universe at the NetApp Support site.
Access the NetApp Hardware Universe application to view the system configuration guides. Click the 2.Controllers tab to view the compatibility between Data ONTAP software versions and storage appliances with the desired specifications.
Alternatively, to compare components by storage appliance, click Compare Storage Systems. 3.
Storage Controllers
Follow the physical installation procedures for the controllers in the NetApp FAS2500 documentation
available at the NetApp Support site.
9.2 Configuring Clustered Data ONTAP 8.2.2
These steps demonstrate how to assign disk ownership and perform disk initialization and verification.
Controller 01
Connect to the storage system console port. You should see a Loader-A prompt. However, if the 1.
storage system is in a reboot loop, press Ctrl-C to exit the autoboot loop when you see this message:
Starting AUTOBOOT press Ctrl-C to abort…
31 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
From the Loader-A prompt, enter: 2.
printenv
If the last-OS-booted-ver parameter is not set to 8.2.2, proceed to step 4 to load Data ONTAP 3.
8.2.2 software. If Data ONTAP 8.2.2 is already loaded, proceed to step 16.
Allow the system to boot. 4.
boot_ontap
Press Ctrl-C when the Press Ctrl-C for Boot Menu message appears. 5.
Note: If Data ONTAP 8.2.2 is not the version of software being booted, proceed with the following steps to install new software. If Data ONTAP 8.2.2 is the version being booted, then select option 8 and yes to reboot the node. Then proceed with step 15.
To install new software, first select option 7. 6.
7
Answer yes to perform a nondisruptive upgrade. 7.
y
Select e0M as the network port you want to use for the download. 8.
e0M
Select yes to reboot now. 9.
y
Enter the IP address, netmask, and default gateway for e0M in their respective places. 10.
<<controller01_mgmt_ip>> <<controller01_mgmt_netmask>> <<controller01_mgmt_gateway>>
Enter the URL where the software can be found. 11.
<<url_boot_software>>
Press Enter for the user name, indicating no user name. 12.
Enter
Enter yes to set the newly installed software as the default to be used for subsequent reboots. 13.
y
Enter yes to reboot the node. 14.
y
Note: When installing new software, the system might perform firmware upgrades to the BIOS and adapter cards, causing reboots and possible stops at the Loader-A prompt. If these actions occur, the system might deviate from this procedure.
Press Ctrl-C to exit autoboot when you see this message: 15.
Starting AUTOBOOT press Ctrl-C to abort…
From the Loader-A prompt, enter: 16.
printenv
Note: If bootarg.init.boot_clustered true is not listed, the system is not set to boot in clustered Data ONTAP.
If the system is not set to boot in clustered Data ONTAP, at the Loader-A prompt, enter the following 17.
command to make sure the system boots in clustered Data ONTAP:
setenv bootarg.init.boot_clustered true
32 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
setenv bootarg.bsdportname e0M
At the Loader-A prompt, enter: 18.
autoboot
When you see Press Ctrl-C for Boot Menu, press: 19.
Ctrl - C
Select option 5 to enter Maintenance mode. 20.
5
When prompted Continue with boot?, enter y. 21.
Use the disk show –n command to view how many disks are unowned. 22.
Use the disk assign –n <<#_of_disks>> command to assign disks to controller 01. 23.
Note: For the small and medium FlexPod Express configuration, <<#_of_disks>> should equal 9 for controller 01.
To verify the HA status of your environment, run the following command: 24.
ha-config show
Note: If either component is not in HA mode, use the ha-config modify command to put the components in HA mode.
Reboot the controller by using the halt command. 25.
At the LOADER-A prompt, enter: 26.
autoboot
When you see Press Ctrl-C for Boot Menu: 27.
Ctrl - C
Select option 4 for clean configuration and initialize all disks. 28.
4
Answer yes to Zero disks, reset config and install a new file system. 29.
y
Enter yes to erase all the data on the disks. 30.
y
Note: The initialization and creation of the root volume can take 75 minutes or more to complete, depending on the number of disks attached. After initialization is complete, the storage system reboots. You can continue to controller 02 configuration while the disks for controller 01 are zeroing.
Controller 02
Connect to the storage system console port. You should see a Loader-A prompt. However, if the 1.
storage system is in a reboot loop, press Ctrl-C to exit the autoboot loop when you see this message:
Starting AUTOBOOT press Ctrl-C to abort…
From the Loader-A prompt, enter: 2.
printenv
If the last-OS-booted-ver parameter is not set to 8.2.2, proceed to step 4 to load Data ONTAP 3.
8.2.2 software. If Data ONTAP 8.2.2 is already loaded, proceed to step 16.
33 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Allow the system to boot. 4.
boot_ontap
Press Ctrl-C when Press Ctrl-C for Boot Menu is displayed. 5.
Ctrl-C
Note: If Data ONTAP 8.2.2 is not the version of software being booted, proceed with the following steps to install new software. If Data ONTAP 8.2.2 is the version being booted, then select option 8 and yes to reboot the node. Then proceed with step 15.
To install new software, first select option 7. 6.
7
Answer yes to perform a nondisruptive upgrade. 7.
y
Select e0M as the network port you want to use for the download. 8.
e0M
Select yes to reboot now. 9.
y
Enter the IP address, netmask, and default gateway for e0M in their respective places. 10.
<<controller02_mgmt_ip>> <<controller02_mgmt_netmask>> <<controller02_mgmt_gateway>>
Enter the URL where the software can be found. 11.
Note: This web server must be pingable.
<<url_boot_software>>
Press Enter for the user name, indicating no user name. 12.
Enter
Select yes to set the newly installed software as the default to be used for subsequent reboots. 13.
y
Select yes to reboot the node. 14.
y
Note: When installing new software, the system might perform firmware upgrades to the BIOS and adapter cards, causing reboots and possible stops at the Loader-A prompt. If these actions occur, the system might deviate from this procedure.
Press Ctrl-C to exit autoboot when you see this message: 15.
Starting AUTOBOOT press Ctrl-C to abort…
From the Loader-A prompt, enter: 16.
printenv
Note: If bootarg.init.boot_clustered true is not listed, the system is not set to boot in clustered Data ONTAP.
If the system is not set to boot in clustered Data ONTAP, at the Loader-A prompt enter the following 17.
command to make sure that the system boots in clustered Data ONTAP:
setenv bootarg.init.boot_clustered true
setenv bootarg.bsdportname e0M
34 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
At the Loader-A prompt, enter: 18.
autoboot
When you see Press Ctrl-C for Boot Menu, enter: 19.
Ctrl – C
Select option 5 to enter Maintenance mode. 20.
5
When prompted Continue with boot?, enter y. 21.
Use the disk show –n command to view how many disks are unowned. 22.
Use the disk assign –n <<#_of_disks>> command to assign disks to controller 02. 23.
Note: For the small and medium FlexPod Express configuration, <<#_of_disks>> should equal 3 for controller 02.
To verify the HA status of your environment, run the following command: 24.
ha-config show
If either component is not in HA mode, use the ha-config modify command to put the 25.
components in HA mode
Reboot the controller by using the halt command. 26.
At the Loader-A prompt, enter: 27.
autoboot
When you see Press Ctrl-C for Boot Menu, enter: 28.
Ctrl - C
Select option 4 for clean configuration and initialize all disks. 29.
4
Answer yes to Zero disks, reset config and install a new file system. 30.
y
Enter yes to erase all the data on the disks. 31.
y
Note: The initialization and creation of the root volume can take 75 minutes or more to complete, depending on the number of disks attached. When initialization is complete, the storage system reboots.
9.3 Creating Clusters in Clustered Data ONTAP
The first node in the cluster performs the cluster create operation. All other nodes perform a cluster join
operation. The first node in the cluster is considered Controller 01.
Controller 01
During the first node boot, the Cluster Setup wizard starts running on the console. 1.
Welcome to the cluster setup wizard.
You can enter the following commands at any time:
"help" or "?" - if you want to have a question clarified,
"back" - if you want to change previously answered questions, and
"exit" or "quit" - if you want to quit the cluster setup wizard.
Any changes you made before quitting will be saved.
35 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
You can return to cluster setup at any time by typing "cluster setup".
To accept a default or omit a question, do not enter a value.
Do you want to create a new cluster or join an existing cluster? {create, join}:
Note: If a login prompt appears instead of the Cluster Setup wizard, start the wizard by logging in using the factory default settings and then enter the cluster setup command.
Enter the following command to create a new cluster: 2.
create
Follow these steps to activate high availability and set storage failover: 3.
Do you intend for this node to be used as a single node cluster? {yes, no} [no]: Enter
Will the cluster network be configured to use network switches? [yes]:no
Non-HA mode, Reboot node to activate HA
Do you want to reboot now to set storage failover (SFO) to HA mode? {yes, no}
[yes]: Enter
After the reboot, continue with the cluster create operation. 4.
The existing cluster interface configuration is displayed: 5.
Existing cluster interface configuration found:
Port MTU IP Netmask
e0d 9000 169.254.250.41 255.255.0.0
e0f 9000 169.254.175.136 255.255.0.0
Do you want to use this configuration? {yes, no} [yes]:
Note: Before accepting this configuration, make sure that the correct ports are being listed for the cluster interfaces. If the correct interfaces are not listed, enter no and accept the system defaults.
Accept the configuration by pressing the Enter key. The steps to create a cluster are displayed: 6.
Enter the cluster name: <<clustername>>
Enter the cluster base license key: <<cluster_base_license_key>>
Creating cluster <<clustername>>
Note: The cluster is created; this can take a minute or two.
Enter an additional license key []:
Note: For this validated architecture, you should install license keys for NetApp SnapRestore®, NFS,
FlexClone®, and SnapManager
® suite.
After you finish entering the license keys, press Enter. 7.
Enter the cluster administrator’s (username “admin”) password: <<admin_password>>
Retype the password: <<admin_password>>
Enter the cluster management interface port [e0a]: Enter
Enter the cluster management interface IP address: <<clustermgmt_ip>>
Enter the cluster management interface netmask: <<clustermgmt_netmask>>
Enter the cluster management interface default gateway: <<clustermgmt_gateway>>
Enter the DNS domain name. 8.
Enter the DNS domain names:<<dns_domain_name>>
Enter the name server IP addresses:<<nameserver_ip>>
Note: If you have more than one name server IP address, separate them with commas.
Set up the node. 9.
Where is the controller located []:<<controller_location>>
Enter the node management interface port [e0M]: Enter
36 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Enter the node management interface IP address: <<controller01_mgmt_ip>>
Enter the node management interface netmask:<<controller01_mgmt_netmask>>
Enter the node management interface default gateway:<<controller01_mgmt_gateway>>
Enable IPv4 DHCP on the service processor interface [no]:Enter
Enter the service processor interface IP address: <<controller01_sp_ip>>
Enter the service processor interface netmask: <<controller01_sp_netmask>>
Enter the service processor interface default gateway: <<controller01_sp_gateway>>
Press Enter to accept the NetApp AutoSupport™
message. 10.
Log in to the cluster. 11.
Disable the disk autoassign option by entering the following command: 12.
storage disk option modify -autoassign off
9.4 Joining Clusters in Clustered Data ONTAP
The first node in the cluster performs the cluster create operation. All other nodes perform a cluster join
operation. The first node in the cluster is considered Controller 01, and the node joining the cluster in this
example is Controller 02.
Controller 02
During the node boot, the Cluster Setup wizard starts running on the console. 1.
Welcome to the cluster setup wizard.
You can enter the following commands at any time:
"help" or "?" - if you want to have a question clarified,
"back" - if you want to change previously answered questions, and
"exit" or "quit" - if you want to quit the cluster setup wizard.
Any changes you made before quitting will be saved.
You can return to cluster setup at any time by typing "cluster setup".
To accept a default or omit a question, do not enter a value.
Do you want to create a new cluster or join an existing cluster?{create, join}:
Note: If a login prompt appears instead of the Cluster Setup wizard, start the wizard by logging in using the factory default settings, and then enter the cluster setup command.
Enter the following command to join a cluster: 2.
join
Follow these steps to activate high availability and set storage failover: 3.
Non-HA mode, Reboot node to activate HA
Do you want to reboot now to set storage failover (SFO) to HA mode? {yes, no}
[yes]: Enter
After the reboot, continue with the cluster join operation. The existing cluster interface configuration is 4.displayed:
Existing cluster interface configuration found:
Port MTU IP Netmask
e0d 9000 169.254.49.199 255.255.0.0
e0f 9000 169.254.132.123 255.255.0.0
Do you want to use this configuration? {yes, no} [yes]:
Note: Before accepting this configuration, make sure that the correct ports are listed for the cluster interfaces.
Accept the configuration by pressing the Enter key. The steps to create a cluster are displayed. 5.
37 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Enter the name of the cluster you would like to join [<<clustername>>]:Enter
Note: The node should find the cluster name automatically.
Note: The cluster join operation can take a minute or two.
Set up the node. 6.
Enter the node management interface port [e0M]: Enter
Enter the node management interface IP address: <<controller02_mgmt_ip>>
Enter the node management interface netmask: Enter
Enter the node management interface default gateway: Enter
Enable IPv4 DHCP on the service processor interface [no]:Enter
Enter the service processor interface IP address: <<controller02_sp_ip>>
Enter the service processor interface netmask: <<controller02_sp_netmask>>
Enter the service processor interface default gateway: <<controller02_sp_gateway>>
Press Enter to accept the AutoSupport message. 7.
Log in to the cluster interface with the admin user ID and <<admin_password>>. 8.
Disable the disk autoassign option by entering the following command: 9.
storage disk option modify -node <<controller02>> -autoassign off
9.5 Changing the RAID Configuration on Root Aggregates
The RAID configuration on the root aggregates of Controller 01 and 02 need to be changed from
raid_dp to raid4.
Controller 01
Connect to the storage system console port to change the RAID configuration.
aggr modify -aggregate aggr0 -raidtype raid4
aggr modify -aggregate aggr0_<<controller02>>_0 -raidtype raid4
9.6 Zeroing All Spare Disks
Controller 01
Connect to the storage system console port to zero all spare disks in the cluster.
disk zerospares
9.7 Configuring IFGRP LACP in Clustered Data ONTAP
This type of interface group requires two or more Ethernet interfaces and a switch that supports LACP.
Therefore, make sure that the switch is configured properly.
Connect to the storage system console port of Controller 01 and run the following commands on the
command line to create an interface group.
Controller 01
ifgrp create -node <<controller01>> -ifgrp a0a -distr-func port -mode multimode_lacp
network port ifgrp add-port -node <<controller01>> -ifgrp a0a -port e0b
network port ifgrp add-port -node <<controller01>> -ifgrp a0a -port e0c
network port ifgrp add-port -node <<controller01>> -ifgrp a0a -port e0e
Controller 02
ifgrp create -node <<controller02>> -ifgrp a0a -distr-func port -mode multimode_lacp
network port ifgrp add-port -node <<controller02>> -ifgrp a0a -port e0a
38 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
network port ifgrp add-port -node <<controller02>> -ifgrp a0a -port e0b
network port ifgrp add-port -node <<controller02>> -ifgrp a0a -port e0c
network port ifgrp add-port -node <<controller02>> -ifgrp a0a -port e0e
Note: All interfaces must be in the down status before being added to an interface group.
Note: The interface group name must follow the standard naming convention of <number><letter>, where <number> is an integer in the range 0 to 999 without leading zeros, and <letter> is a lowercase letter.
9.8 Configuring VLANs in Clustered Data ONTAP
Connect to the storage system console port of Controller 01 and run the following commands on the
command line to create VLANs.
Controller 01 and Controller 02
Follow these steps to create a VLAN interface for NFS data traffic: 1.
network port vlan create –node <<controller01>> -vlan-name a0a-<<nfs_vlan_id>>
network port vlan create –node <<controller02>> -vlan-name a0a-<<nfs_vlan_id>>
Follow these steps to create a VLAN interface for node management failover groups: 2.
network port vlan create –node <<controller01>> -vlan-name a0a-<<ib_mgmt_vlan_id>>
network port vlan create –node <<controller02>> -vlan-name a0a-<<ib_mgmt_vlan_id>>
9.9 Migrating the Cluster Management Interface on Controller 01
Connect to the storage system console port of Controller 01 and run the following commands on the
command line.
Migrate the cluster_mgmt logical interface from e0a to IFGRP a0a-<<ib_mgmt_vlan_id>> on 1.
Controller 01.
network interface migrate -vserver <<clustername>> -lif cluster_mgmt -destination-node
<<controller01>> -destination-port a0a-<<ib_mgmt_vlan_id>>
Set the home port of the cluster_mgmt logical interface to the new destination port. 2.
network interface modify -vserver <<clustername>> -lif cluster_mgmt -home-node <<controller01>> -
home-port a0a-<<ib_mgmt_vlan_id>>
Add the physical port e0a to the IFGRP a0a. 3.
network port ifgrp add-port -node <<controller01>> -ifgrp a0a -port e0a
9.10 Logging in to the Cluster
Open an SSH connection to cluster IP or host name and log in to the admin user with the password you
provided earlier.
9.11 Setting Auto-Revert on Cluster Management
To set the auto-revert parameter on the cluster management interface, enter:
network interface modify –vserver <<clustername>> -lif cluster_mgmt –auto-revert true
9.12 Configuring Failover Group Management in Clustered Data ONTAP
Create a cluster management port failover group.
network interface failover-groups create -failover-group fg-clus-mgmt -node <<controller01>> -
port a0a-<<ib_mgmt_vlan_id>>
39 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
network interface failover-groups create -failover-group fg-clus-mgmt -node <<controller02>> -
port a0a-<<ib_mgmt_vlan_id>>
9.13 Assigning the Management Failover Group to the Cluster Management LIF
Assign the management port failover group to the cluster management LIF.
network interface modify –vserver <<clustername>> -lif cluster_mgmt –failover-group fg-clus-mgmt
9.14 Configuring Failover Group Node Management in Clustered Data ONTAP
Create node management port failover groups.
network interface failover-groups create -failover-group fg-node-mgmt01 -node <<controller01>> -
port a0a-<ib_mgmt_vlan_id>
network interface failover-groups create -failover-group fg-node-mgmt01 -node <<controller01>> -
port e0M
network interface failover-groups create -failover-group fg-node-mgmt02 -node <<controller02>> -
port a0a-<ib_mgmt_vlan_id>
network interface failover-groups create -failover-group fg-node-mgmt02 -node <<controller02>> -
port e0M
9.15 Assigning Node Management Failover Groups to Node Management LIFs
Assign the management port failover group to the cluster management LIF.
network interface modify -vserver <<controller01>> -lif mgmt1 -auto-revert true -failover-group
fg-node-mgmt01
network interface modify –vserver <<controller02>> -lif mgmt1 –auto-revert true –failover-group
fg-node-mgmt02
9.16 Creating Aggregates
A 64-bit aggregate containing the root volume is created during the Data ONTAP setup process. To
create additional 64-bit aggregates, determine the aggregate name, the node on which to create it, and
the number of disks that it will contain.
Run the following command to create new aggregates: 1.
aggr create -aggregate aggr1_controller01 -nodes <<controller01>> -diskcount <<num_disks>> -
raidtype raid4
Note: For the small and medium FlexPod Express configuration, <<num_disks>> should equal 6.
The aggregate cannot be created until disk zeroing completes. Use the aggr show command to 2.
display aggregate creation status. Do not proceed until aggr1_controller01 is online.
Disable Snapshot™
copies for the two data aggregates just created. 3.
node run <<controller01>> aggr options aggr1_controller01 nosnap on
Delete any existing Snapshot copies for the two data aggregates. 4.
node run <<controller01>> snap delete –A –a –f aggr1_controller01
Rename the root aggregate on Controller 01 to match the naming convention for this aggregate on 5.Controller 02.
aggr show
aggr rename –aggregate aggr0 –newname aggr0_controller01
9.17 Configuring the Service Processor
Gather information about the network and the NetApp AutoSupport settings before configuring the service
processor.
40 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Configure the service processor using Dynamic Host Configuration Protocol (DHCP) or static addressing.
If the service processor uses a static IP address, verify that the following service processor prerequisites
have been met:
A static IP address is available.
The network netmask is set up.
The network gateway IP address is known.
NetApp AutoSupport information is available.
A best practice is to configure the AutoSupport recipients and mail host before configuring the SP. Data
ONTAP automatically sends the AutoSupport configuration to the SP, allowing the SP to send alerts and
notifications through an AutoSupport message to the system administrative recipients specified in
AutoSupport. When configuring the SP, enter the name or the IP address of the AutoSupport mail host,
when prompted.
A service processor needs to be set up on each node.
Upgrading the Service Processor on Each Node to the Latest Release
With Data ONTAP 8.2, you must upgrade to the latest service processor firmware to take advantage of
the latest updates available for the remote management device.
Enter this command to obtain the version of the service processor firmware that is currently running 1.on your storage system:
system node service-processor show
Using a web browser, connect to http://support.netapp.com/NOW/cgi-bin/fw. 2.
Select the storage platform. 3.
Select Service Process Image for installation from the Data ONTAP prompt. 4.
Check the latest firmware version that is available for your storage platform. If your storage system is 5.not running the latest version, proceed to the download page for the latest release of the service processor firmware for your storage platform.
Using the instructions on this page, update the service processors on both nodes in your cluster. You 6.will need to download the .zip file to a web server that can be reached from the cluster management interface. In step 2 of the instructions, substitute the following command:
system node image get –node * -package http://web_server_name/path/SP_FW.zip -replace-package
true
Run step 3 on each node if service processor automatic updating is not enabled. 7.
View the status of the service processor upgrade using steps 4 and 5. 8.
9.18 Configuring Jumbo Frames in Clustered Data ONTAP
To configure a clustered Data ONTAP network port to use jumbo frames (which usually have a 1.maximum transmission unit [MTU] of 9,000 bytes), run the following command from the cluster shell:
network port modify -node <<controller01>> -port a0a -mtu 9000
WARNING: Changing the network port settings will cause a serveral second interruption in carrier.
Do you want to continue? {y|n}: y
network port modify –node <<controller01>> -port a0a-<<nfs_vlan_id>> -mtu 9000
WARNING: Changing the network port settings will cause a serveral second interruption in carrier.
Do you want to continue? {y|n}: y
network port modify –node <<controller02>> -port a0a -mtu 9000
WARNING: Changing the network port settings will cause a serveral second interruption in carrier.
41 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Do you want to continue? {y|n}: y
network port modify –node <<controller02>> -port a0a-<<nfs_vlan_id>> -mtu 9000
WARNING: Changing the network port settings will cause a serveral second interruption in carrier.
Do you want to continue? {y|n}: y
9.19 Configuring NTP in Clustered Data ONTAP
To configure time synchronization on the cluster, complete the following steps:
Set the time zone for the cluster. 1.
timezone <<timezone>>
Note: For example, in the eastern United States, the time zone is America/New_York.
Set the date for the cluster. 2.
date <ccyymmddhhmm.ss>
Note: The format for the date is:
<[Century][Year][Month][Day][Hour][Minute].[Second]>; for example, 201309231128.50
Configure NTP for each node in the cluster. 3.
system services ntp server create -node <<controller01>> -server <<global_ntp_server_ip>>
system services ntp server create -node <<controller02>> -server <<global_ntp_server_ip>>
9.20 Configuring SNMP in Clustered Data ONTAP
Configure SNMP basic information, such as the location and contact. When the system is polled, this 1.information is visible as the sysLocation and sysContact variables in SNMP.
snmp contact <<snmp_contact>>
snmp location “<<snmp_location>>”
snmp init 1
options snmp.enable on
Configure SNMP traps to send to remote hosts, such as a NetApp DFM server or another fault 2.management system.
snmp traphost add <<snmp_trap_server_fqdn>>
9.21 Configuring SNMPv1 in Clustered Data ONTAP
Set the shared secret plain-text password, which is called a community. 1.
snmp community delete all
snmp community add ro <<snmp_community>>
Use the delete all command with caution. If community strings are used for other monitoring 2.
products, the delete all command will remove them.
9.22 Configuring SNMPv3 in Clustered Data ONTAP
SNMPv3 requires that a user be defined and configured for authentication.
Create a user called snmpv3user. 1.
security login create -username snmpv3user -authmethod usm -application snmp
Enter the authoritative entity's engine ID and select MD5 as the authentication protocol. Use the 2.
command security snmpusers to view the engine ID.
42 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Enter a password with a minimum length of eight characters for the authentication protocol when 3.prompted.
Confirm the authentication protocol password. 4.
Select DES as the privacy protocol. 5.
Enter a password with a minimum length of eight characters for the privacy protocol when prompted. 6.
Confirm the privacy protocol password. 7.
9.23 Configuring AutoSupport HTTPS in Clustered Data ONTAP
NetApp AutoSupport sends support summary information to NetApp through HTTPS.
Run the following command to configure AutoSupport: 1.
system node autosupport modify -node * –mail-hosts <<mailhost>> -noteto <<storage_admin_email>>
Warning: Do you want to continue running this command? {y|n}:y
9.24 Configuring Cisco Discovery Protocol in Clustered Data ONTAP
Enable Cisco Discovery Protocol (CDP) on the NetApp storage controllers by using the following
procedure.
To be effective, CDP must also be enabled on directly connected networking equipment such as switches
and routers.
To enable CDP on the NetApp storage controllers, complete the following step:
Enable CDP on Data ONTAP. 1.
node run -node * options cdpd.enable on
9.25 Setting Up Vserver
To create an infrastructure Vserver, complete the following steps:
Run the Vserver setup wizard. 1.
vserver setup
Welcome to the Vserver Setup Wizard, which will lead you through
the steps to create a virtual storage server that serves data to clients.
You can enter the following commands at any time:
"help" or "?" if you want to have a question clarified,
"back" if you want to change your answers to previous questions, and
"exit" if you want to quit the Vserver Setup Wizard. Any changes
you made before typing "exit" will be applied.
You can restart the Vserver Setup Wizard by typing "vserver setup". To accept a default
or omit a question, do not enter a value.
Vserver Setup wizard creates and configures only data Vservers.
If you want to create a Vserver with Infinite Volume use the vserver create command.
Step 1. Create a Vserver.
You can type "back", "exit", or "help" at any question.
Enter the Vserver name. 2.
Enter the Vserver name:Infra_Vserver
Select the Vserver data protocols to configure. 3.
Choose the Vserver data protocols to be configured {nfs, cifs, fcp, iscsi, ndmp}: nfs
43 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Select the Vserver client services to configure. 4.
Choose the Vserver client services to configure {ldap, nis, dns}:Enter
Enter the Vserver’s root volume aggregate. 5.
Enter the Vserver's root volume aggregate [aggr1_controller01]: aggr1_controller01
Enter the Vserver language setting. English is the default [C]. 6.
Enter the Vserver language setting, or enter help to see all languages. 7.
Enter the Vserver’s security style. 8.
Enter the Vserver root volume's security style {mixed, ntfs, unix} [unix]: Enter
Do not create the data volume. 9.
Do you want to create a data volume? {yes, no} [yes]: no
Do not create the logical interface. 10.
Do you want to create a logical interface? {yes, no} [yes]: no
Add the data aggregate to the Infra_Vserver aggregate list for NetApp Virtual Storage Console. 11.
vserver modify -vserver Infra_Vserver -aggr-list aggr1_controller01
9.26 Configuring HTTPS Access in Clustered Data ONTAP
Secure access to the storage controller must be configured. 1.
Increase the privilege level to access the certificate commands. 2.
set -privilege diag
Do you want to continue? {y|n}: y
Generally, a self-signed certificate is already in place. Check it with the following command: 3.
security certificate show
Run the following commands as one-time commands to generate and install self-signed certificates. 4.
You can also use the security certificate delete command to delete expired certificates.
security certificate create -vserver Infra_Vserver -common-name
<<security_cert_vserver_common_name>> -type server -size 2048 -country <<country_code>> -state
<<state>> -locality <<city>> -organization <<org>> -unit <<unit>> -email-addr
<<storage_admin_email>>
security certificate create -vserver <<clustername>> -common-name
<<security_cert_cluster_common_name>> -type server -size 2048 -country <<country_code>> -state
<<state>> -locality <<city>> -organization <<org>> -unit <<unit>> -email-addr
<<storage_admin_email>>
security certificate create -vserver <<controller01>> -common-name
<<security_cert_controller01_common_name>> -type server -size 2048 -country <<country_code>> -
state <<state>> -locality <<city>> -organization <<org>> -unit <<unit>> -email-addr
<<storage_admin_email>>
security certificate create -vserver <<controller02>> -common-name
<<security_cert_controller02_common_name>> -type server -size 2048 -country <<country_code>> -
state <<state>> -locality <<city>> -organization <<org>> -unit <<unit>> -email-addr
<<storage_admin_email>>
Configure and enable SSL and HTTPS access and disable telnet access. 5.
system services web modify -external true -sslv3-enabled true
Do you want to continue {y|n}: y
system services firewall policy delete -policy mgmt -service http -action allow
system services firewall policy create -policy mgmt -service http -action deny -ip-list 0.0.0.0/0
44 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
security ssl modify -vserver Infra_Vserver -common-name <<security_cert_vserver_common_name>> -
server-enabled true -client-enabled false -ca <<security_certificate_vservser_authority>> -serial
<<security_certificate_vserver_serial_no>>
security ssl modify -vserver <<clustername>> -common-name <<security_cert_cluster_common_name>> -
server-enabled true -client-enabled false -ca <<security_certificate_cluster_authority>> -serial
<<security_certificate_cluster_serial_no>>
security ssl modify -vserver <<controller01>> -common-name
<<security_cert_controller01_common_name>> -server-enabled true -client-enabled false -ca
<<security_certificate_controller01_authority>> -serial
<<security_certificate_controller01_serial_no>>
security ssl modify -vserver <<controller02>>-common-name
<<security_cert_controller02_common_name>> -server-enabled true -client-enabled false -ca
<<security_certificate_controller02_authority>> -serial
<<security_certificate_controller02_serial_no>>
set –privilege admin
It is normal for some of these commands to return error messages stating that the entry does not 6.exist.
9.27 Configuring NFSv3 in Clustered Data ONTAP
Run all commands to configure NFS on the Vserver.
Secure the default rule for the default export policy and create the FlexPod_Express export policy. 1.
vserver export-policy rule modify -vserver Infra_Vserver -policyname default -ruleindex 1 -rorule
never -rwrule never -superuser none
vserver export-policy create –vserver Infra_Vserver FlexPod_Express
Create a new rule for the FlexPod_Express export policy. 2.
For each ESXi host being created, create a rule. Each host will have its own rule index. Your first 3.ESXi host will have rule index 1, your second ESXi host will have rule index 2, and so on.
vserver export-policy rule create –vserver Infra_Vserver -policyname FlexPod_Express –ruleindex 1
–protocol nfs -clientmatch <<esxi_host_nfs_ip>> -rorule sys –rwrule sys -superuser sys –allow-
suid false
Assign the FlexPod Express export policy to the infrastructure Vserver root volume. 4.
volume modify –vserver Infra_Vserver –volume rootvol –policy FlexPod_Express
9.28 Configuring FlexVol in Clustered Data ONTAP
The following information is required to create a NetApp FlexVol® volume: the volume’s name and 1.
size and the aggregate on which the volume will exist.
volume create –vserver Infra_Vserver –volume infra_datastore_1 –aggregate aggr1_controller01 -
size 500g -state online –policy FlexPod_Express –junction-path /infra_datastore_1 –space-
guarantee none –percent-snapshot-space 0
volume create –vserver Infra_Vserver –volume infra_swap –aggregate aggr1_controller01 -size 100g
-state online –policy FlexPod_Express –junction-path /infra_swap –space-guarantee none –percent-
snapshot-space 0 –snapshot-policy none
9.29 Configuring Deduplication in Clustered Data ONTAP
Enable deduplication on appropriate volumes. 1.
volume efficiency on –vserver Infra_Vserver -volume infra_datastore_1
Note: Deduplication can also be scheduled to run at desired intervals so that the operation does not consume resources during production hours.
45 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
9.30 Configuring Failover Group NAS in Clustered Data ONTAP
Create an NFS port failover group. 1.
network interface failover-groups create -failover-group nfs -node <<controller01>> -port a0a-
<<nfs_vlan_id>>
network interface failover-groups create -failover-group nfs -node <<controller02>> -port i0a-
<<nfs_vlan_id>>
9.31 Configuring NFS LIF in Clustered Data ONTAP
Create an NFS LIF. 1.
network interface create -vserver Infra_Vserver -lif nfs_lif01 -role data -data-protocol nfs -
home-node <<controller01>> -home-port a0a-<<nfs_vlan_id>> –address <<controller01_nfs_lif_ip>> -
netmask <<controller01_nfs_lif_netmask>> -status-admin up –failover-policy nextavail –firewall-
policy data –auto-revert true –failover-group nfs
network interface create -vserver Infra_Vserver -lif nfs_lif02 -role data -data-protocol nfs -
home-node <<controller02>> -home-port a0a-<<nfs_vlan_id>> –address <<controller02_nfs_lif_ip>> -
netmask <<controller02_nfs_lif_netmask>> -status-admin up –failover-policy nextavail –firewall-
policy data –auto-revert true –failover-group nfs
9.32 Configuring Failover Group Vserver Management in Clustered Data ONTAP
Create a management port failover group. 1.
network interface failover-groups create -failover-group vs_mgmt01 -node <<controller01>> -port
a0a-<<ib_mgmt_vlan_id>>
network interface failover-groups create -failover-group vs_mgmt01 -node <<controller02>> -port
a0a-<<ib_mgmt_vlan_id>>
9.33 Adding Infrastructure Vserver Administrator
Add the infrastructure Vserver administrator and Vserver administration LIF in the out-of-band 1.management network with the following commands:
network interface create –vserver Infra_Vserver –lif vsmgmt –role data –data-protocol none –home-
node <<controller02>> -home-port a0a-<<ib_mgmt_vlan_id>> –address <<vserver_mgmt_ip>> -netmask
<<vserver_mgmt_netmask>> -status-admin up –failover-policy nextavail –firewall-policy mgmt –auto-
revert true –failover-group vs_mgmt01
network routing-groups route create –vserver Infra_Vserver –routing-group
d<<clustermgmt_ip>>/<<vserver_mgmt_netmask>> -destination 0.0.0.0/0 –gateway
<<clustermgmt_gateway>>
security login password –username vsadmin –vserver Infra_Vserver
Enter a new password: <<vsadmin_password>>
Enter it again: <<vsadmin_password>>
security login unlock –username vsadmin –vserver Infra_Vserver
10 Cisco UCS C-Series Rack Server Deployment Procedure
The following section provides a detailed procedure for configuring a Cisco UCS C-Series standalone
rack server for use in either the small or medium FlexPod Express configuration.
10.1 Performing Initial Cisco UCS C-Series Standalone Server Setup for Cisco IMC
These steps provide details for the initial setup of the Cisco IMC interface for Cisco UCS C-Series
standalone servers.
46 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
All Servers
Attach the Cisco keyboard, video, and mouse (KVM) dongle (provided with the server) to the KVM 1.port on the front of the server. Plug a VGA monitor and USB keyboard into the appropriate KVM dongle ports.
Power on the server and press F8 when prompted to enter the Cisco IMC configuration. 2.
In the Cisco IMC configuration utility, set the following options: 3.
Network Interface Card (NIC) Mode:
Dedicated [X]
IP (Basic):
IPV4: [X]
DHCP enabled: [ ]
CIMC IP:<<cimc_ip>>
Prefix/Subnet:<<cimc_netmask>>
Gateway: <<cimc_gateway>>
VLAN (Advanced): Leave unchecked to disable VLAN tagging.
NIC Redundancy:
None: [X]
47 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Press F1 to see additional settings.
Common Properties:
Hostname: <<esxi_host_name>>
Dynamic DNS: [ ]
Factory Defaults: Leave unchecked.
Default User (Basic):
Default password: <<admin_password>>
Reenter password: <<admin_password>>
Port Properties: Use default values.
Port Profiles: Leave unchecked.
48 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Press F10 to save the Cisco IMC interface configuration. 4.
After the configuration is saved, press Esc to exit. 5.
10.2 Configuring Cisco UCS C-Series Servers for ESXi Installation on FlexFlash Cards
Some Cisco UCS C-Series rack servers support an internal secure digital (SD) memory card for storage
of server software tools and utilities. The SD card is hosted by the Cisco flexible flash (FlexFlash) storage
adapter. Users can also install operating systems that have small storage footprints on these cards.
This section describes the steps for installing the VMware ESXi operating system on these cards.
Boot Order Configuration
From the Cisco IMC interface browser window (do not close the virtual KVM window), click the Server 1.tab and choose BIOS.
Choose Configure Boot Order and click OK. 2.
In the Boot Order section, remove all the entries and configure the following: 3.
Add Virtual Media
Name: KVM-CD-DVD
Sub Type: KVM MAPPED DVD
State: Enabled
Click Add Device.
Add SD Card
Name: FlexFlash
49 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
State: Enabled
Order: 2
Click Add Device. 4.
Click Save. Click Close. 5.
Click Save Changes. 6.
RAID Configuration
Cisco UCS C-Series rack servers that support Cisco FlexFlash cards come with an internal Cisco
FlexFlash controller that is responsible for RAID functions.
When a single card is installed on the server, the RAID configuration will be in a degraded mode.
RAID 1 (mirror) can be set up by adding a card (if an additional card is not already present) to the server.
The two cards will need to be synchronized for RAID 1 to become operational.
Follow these steps to synchronize RAID on the two cards:
In the virtual KVM window, choose Virtual Media at the top of the window. 1.
Select Activate Virtual Devices from the drop-down menu. 2.
Select Accept this session to accept an unencrypted virtual media session. Click Apply. 3.
Click the Virtual Media tab at the top of the window and select Map CD/DVD. 4.
Browse to the location of the server configuration utility ISO image and select it. Click Open. 5.
Click Map Device. 6.
Click the Server tab and then choose Summary. Choose Power On Server. Click OK. 7.
Return to the virtual KVM window. The server should now boot into the server configuration utility. 8.
Select the checkbox to accept the license agreement. Click Next. 9.
Click the Hypervisor Sync icon on the toolbar. 10.
50 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click Yes to start the synchronization process. 11.
Note: You will not be able to perform any updates to the partition while the synchronization is taking place.
Click OK to confirm the synchronization. 12.
Note: Do not reboot the system during this process.
Click OK when the synchronization is completed. 13.
From the Cisco IMC window, click the Refresh icon on top of the toolbar to view the changes. 14.
51 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
11 VMware ESXi Deployment Procedure
This section provides detailed procedures for installing VMware ESXi 5.5 Update 1 in a FlexPod Express
configuration. The deployment procedures that follow are customized to include the environment
variables described in previous sections.
Multiple methods exist for installing VMware ESXi in such an environment. This procedure uses the virtual
KVM console and virtual media features of the Cisco IMC interface for Cisco UCS C-Series servers to
map remote installation media to each individual server.
11.1 Logging in to Cisco IMC Interface for Cisco UCS C-Series Standalone Servers
The following steps detail the method for logging in to the Cisco IMC interface for Cisco UCS C-Series
standalone servers. You must log in to the Cisco IMC interface to run the virtual KVM, which enables the
administrator to begin installation of the operating system through remote media.
All Hosts
Navigate to a web browser and enter the IP address for the Cisco IMC interface for the Cisco UCS C-1.Series. This step launches the Cisco IMC GUI application.
Log in to the Cisco IMC GUI with the admin user name and credentials. 2.
In the main menu, select the Server tab. 3.
Click Launch KVM Console. 4.
52 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
11.2 Setting Up the VMware ESXi Installation
This section details the steps required to prepare the server for OS installation.
All Hosts
From the virtual KVM console, select the Virtual Media tab. 1.
Select Map CD/DVD. 2.
Browse to the VMware ESXi 5.5 Update 1 installer ISO image file and click Open. 3.
Select the Power tab and choose Power Cycle System (cold boot). Click Yes. 4.
11.3 Installing VMware ESXi
The following steps describe how to install VMware ESXi on each host’s Cisco FlexFlash card.
All Hosts
When the system boots, the machine detects the presence of the VMware ESXi installation media. 1.
Select the VMware ESXi installer from the menu that appears. 2.
After the installer is finished loading, press Enter to continue with the installation. 3.
After reading the end-user license agreement (EULA), accept it and continue with the installation by 4.pressing F11.
Select the HV partition on the Cisco FlexFlash card that was set up previously as the installation 5.location for VMware ESXi and press Enter to continue with the installation.
Select the appropriate keyboard layout and press Enter to continue. 6.
Enter and confirm the root password and press Enter to continue. 7.
The installer will warn you that existing partitions will be removed on the volume. Continue with the 8.installation by pressing F11.
After the installation is complete, be sure to unmap the VMware ESXi installation image on the Virtual 9.Media tab of the KVM console to help make sure that the server reboots into VMware ESXi and not the installer.
The Virtual Media window might warn you that it is preferable to eject the media from the guest. 10.Because you cannot do this in this example (and the media is read-only), unmap the image anyway by selecting Yes.
On in the KVM tab, press Enter to reboot the server. 11.
11.4 Setting Up VMware ESXi Host Management Networking
The following steps describe how to add the management network for each VMware ESXi host.
All Hosts
After the server has finished rebooting, enter the option to customize the system by pressing F2. 1.
Log in with root as the login name and the root password previously entered during the installation 2.process.
Select the Configure Management Network option. 3.
Select Network Adapters and press Enter. 4.
Six ports should be listed as Connected in the Status column that is displayed. These ports should 5.correspond to ports 1, 2, 3, and 4 of the quad-port Broadcom PCI Express (PCIe) adapter and the two onboard LAN-on-motherboard (LOM) ports. Select all six ports and press Enter.
53 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Select VLAN (optional) and press Enter. 6.
Enter the VLAN ID: <<ib_mgmt_vlan_id>>. Press Enter. 7.
From the Configure Management Network menu, configure the IP address of the management 8.interface by selecting the IP Configuration option. Then press Enter.
Use the space bar to select set static IP address and network configuration. 9.
Enter the IP address for managing the VMware ESXi host: <<esxi_host_mgmt_ip>>. 10.
Enter the subnet mask for the VMware ESXi host: <<esxi_host_mgmt_netmask>>. 11.
Enter the default gateway for the VMware ESXi host: <<esxi_host_mgmt_gateway>>. 12.
Press Enter to accept the changes to the IP configuration. 13.
Enter the IPv6 configuration menu. 14.
Use the space bar to disable IPv6 by unselecting the Enable IPv6 (restart required) option. Press 15.Enter.
Enter the menu to configure the DNS settings. 16.
Because the IP address is assigned manually, the DNS information must also be entered manually. 17.
Enter the primary DNS server’s IP address: <<nameserver_ip>>. 18.
(Optional) Enter the secondary DNS server’s IP address. 19.
Enter the FQDN for the VMware ESXi host: <<esxi_host_fqdn>>. 20.
Press Enter to accept the changes to the DNS configuration. 21.
Exit the Configure Management Network submenu by pressing Esc. 22.
Press Y to confirm the changes and reboot the server. 23.
Log out of the VMware Console by pressing Esc. 24.
11.5 Downloading VMware vSphere Client and vSphere Remote Command Line
The following steps provide details for downloading the VMware vSphere Client and installing the remote
command line.
54 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Open a web browser on a management workstation and navigate to the management IP address of 1.one of the VMware ESXi hosts.
Download and install both the VMware vSphere Client and the Microsoft® Windows
® version of the 2.
VMware vSphere remote command line.
11.6 Logging in to VMware ESXi Hosts Using the VMware vSphere Client
This step provides details for logging into each VMware ESXi host using the VMware vSphere Client.
All Hosts
Open the recently downloaded VMware vSphere Client and enter the IP address of the host to which 1.
you want to connect: <<esxi_host_mgmt_ip>>.
Enter root for the user name. 2.
Enter the root password. 3.
Click the Login button to connect. 4.
Install VMware ESXi Patches
Follow these steps to install VMware ESXi patches on all the VMware ESXi hosts.
Download the following VMware ESXi patch from the VMware Patch Download website by selecting 1.VMware ESXi 5.5.0:
Patch 02 - ESXi550-201407001 (Build number 1892794)
From each VMware vSphere Client, select the host in the inventory. 2.
Click the Summary tab to view the environment summary. 3.
Choose Resources > Storage, right-click datastore1, and choose Browse Datastore. 4.
Click the fourth button and select Upload File. 5.
Navigate to the saved location for the downloaded patch and select ESXi550-201407001.zip. 6.
Click Open to upload the file to datastore1. 7.
Right-click the VMware ESXi host and select Enter Maintenance Mode. Click Yes. 8.
From the management workstation, open the VMware vSphere remote command-line interface (CLI) 9.that was previously installed.
At the command prompt, run the following commands to account for each host: 10.
esxcli –s <<esxi_host_mgmt_ip>> -u root –p <<password>> software vib update -d
/vmfs/volumes/datastore1/ESXi550-201407001.zip
From the VMware vSphere Client, right-click the VMware ESXi host and select Reboot. 11.
After the VMware ESXi host has rebooted, right-click the VMware ESXi host and choose Exit 12.Maintenance Mode.
11.7 Setting Up VMkernel Ports and the Virtual Switch
The following steps provide details for setting up VMkernel ports and virtual switches.
All Hosts
In the VMware vSphere Client, select the host on the left pane. 1.
Select the Configuration tab. 2.
Select the Networking link in the Hardware box. 3.
Select the Properties link in the right field on vSwitch0. 4.
55 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Select the vSwitch configuration and click Edit. 5.
On the General tab, change the MTU to 9000. 6.
On the NIC Teaming tab, change all adapters so that they are active adapters by clicking each 7.individual adapter and using the Move Up button to the right.
Close the properties for vSwitch0 by clicking OK. 8.
Select the Management Network configuration and click Edit. 9.
Verify that the Management Traffic checkbox is checked. 10.
Finalize the edits for the management network by clicking OK. 11.
Select the VM Network configuration and click Edit. 12.
Change the Network label to MGMT-Network and enter the VLAN ID (optional): 13.<<ib_mgmt_vlan_id>>.
Finalize the edits for the VM network by clicking OK. 14.
Click Add to add a network element. 15.
Select Virtual Machine. 16.
Enter NFS-Network for the network label and enter the VLAN ID: <<nfs_vlan_id>>. 17.
Click Next. 18.
Click Finish. 19.
Click Add to add a network element. 20.
Select the VMkernel button and click Next. 21.
Change the network label to VMkernel-NFS and enter the VLAN ID (optional) : 22.<<nfs_vlan_id>>.
Continue with the NFS VMkernel creation by clicking Next. 23.
For the NFS VLAN interface for the host, enter <<esxi_host_nfs_ip>> 24.<<esxi_host_nfs_netmask>>.
Continue with the NFS VMkernel creation by clicking Next. 25.
Finalize the creation of the NFS VMkernel interface by clicking Finish. 26.
Select the VMkernel-NFS configuration and click Edit. 27.
Change the MTU to 9000. 28.
Finalize the edits for the VMkernel NFS network by clicking OK. 29.
Click Add to add a network element. 30.
Select the VMkernel button and click Next. 31.
Change the network label to VMkernel-vMotion and enter the VLAN ID (optional): 32.<<vmotion_vlan_id>>.
Select the checkbox to use this port group for VMware vMotion. 33.
Continue with the VMware vMotion VMkernel creation by clicking Next. 34.
For the VMware vMotion VLAN interface for the host, enter: <<esxi_host_vmotion_ip>> 35.<<esxi_host_vmotion_netmask>>.
Continue with the VMware vMotion VMkernel creation by clicking Next. 36.
Finalize the creation of the VMware vMotion VMkernel interface by clicking Finish. 37.
Select the VMkernel vMotion configuration and click Edit. 38.
Change the MTU to 9000. 39.
Finalize the edits for the VMware vMotion VMkernel network by clicking OK. 40.
Click Add to add a network element. 41.
56 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Leave the virtual machine connection type selected and click Next. 42.
Change the network label to VM-Network and enter the VLAN ID (optional) : << 43.
vmtraffic_vlan_id >>.
Click Next. 44.
Click Finish. 45.
Close the dialog box to finalize the VMware ESXi host networking setup. 46.
11.8 Mounting Required Datastores
This step provides details for mounting the required datastores.
All Hosts
In each VMware vSphere Client, select the host on the left pane. 1.
Go to the Configuration tab to enable configurations. 2.
Click the Storage link in the Hardware box. 3.
In the right pane, in the Datastore section, click Add Storage. 4.
The Add Storage wizard appears. Select the Network File System button and click Next. 5.
Enter the server IP address: <<controller01_nfs_lif_ip>>. 6.
Enter the path for the NFS export: /infra_datastore_1. 7.
Make sure that the Mount NFS read only checkbox is left unchecked. 8.
Enter the datastore name: infra_datastore_1. 9.
Continue with the NFS datastore creation by clicking Next. 10.
Finalize the creation of the NFS datastore by clicking Finish. 11.
In the right pane, in the Datastore section, click Add Storage. 12.
The Add Storage wizard appears. Select the Network File System button and click Next. 13.
Enter the server IP address: <<controller01_nfs_lif_ip>>. 14.
Enter the path for the NFS export: /infra_swap. 15.
Make sure the Mount NFS read only checkbox is left unchecked. 16.
Enter the datastore name: infra_swap. 17.
Continue with the NFS datastore creation by clicking Next. 18.
Finalize the creation of the NFS datastore by clicking Finish. 19.
11.9 Moving the Virtual Machine Swap-File Location
These steps provide details for moving the virtual machine swap-file location.
All Hosts
Select the host in the left pane in the VMware vSphere Client. 1.
Go to the Configuration tab to enable configuration. 2.
Click the Virtual Machine Swapfile Location link in the Software box. 3.
In the right pane, click Edit. 4.
Select the Store the swap file in a swap file datastore selected below button. 5.
Select the infra_swap datastore. 6.
Finalize the movement of the swap-file location by clicking OK. 7.
57 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
12 VMware vCenter 5.5 Update 1 Deployment Procedure
The procedures in the following subsections provide detailed instructions for installing VMware vCenter
5.5 Update 1 in a FlexPod environment. After the procedures are completed, a VMware vCenter Server
will be configured.
12.1 Building the VMware vCenter Virtual Machine
To build the VMware vCenter virtual machine, complete the following steps:
From the VMware vSphere 5 download page on the VMware website, download the .OVA file for the 1.VMware vCenter Server appliance onto your system.
Log in to a VMware ESXi host by using the VMware vSphere Client. 2.
From the VMware vSphere Client interface, choose File > Deploy OVF Template. 3.
Browse to the location of the OVF downloaded in step 1. 4.
58 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click Next to continue the installation. 5.
59 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
After reviewing the OVF Template Details window, click Next. 6.
60 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Provide a name for the VMware vCenter virtual machine. Then click Next to continue. 7.
61 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Select infra_datastore_1 as the location for the VMware vCenter virtual machine virtual disks. 8.
Then click Next to continue.
Review the disk format selection and click Next to continue. 9.
Select MGMT-Network as the network to be mapped for Network 1. Click Next. 10.
62 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Review the installation details. Then click Finish to continue. 11.
After the installation is complete, click the plus symbol to the left of the host IP address in the left 12.pane of the VMware vSphere Client window.
Right-click the VMware vCenter virtual machine and choose Power > Power On. 13.
Right-click the VMware vCenter virtual machine and click Open Console to open a console view. 14.
The following screen will be displayed if the VMware vCenter appliance does not receive an IP 15.address through DHCP. Press Enter to log in.
63 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Note: If the in-band management network provides a DHCP server and provided an IP address to the vCenter server, proceed to step 30.
From the login screen, type root as the login name and press Enter. 16.
Type vmware as the password and press Enter. 17.
From the prompt, type /opt/vmware/share/vami/vami_config_net and press Enter. 18.
To configure the IP address for the VMware vCenter Server, type 6 and press Enter. 19.
64 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
To disable IPv6, type n and press Enter. 20.
To choose to configure an IPv4 address, type y and press Enter. 21.
To use a static address instead of a DHCP address, type n and press Enter. 22.
Type <<vcenter_ip_address>> and press Enter. 23.
Type <<vcenter_netmask>> and press Enter. 24.
Review the IP address and subnet mask. Type y and press Enter to complete the configuration. 25.
Type 2 and press Enter to configure the default gateway IP address. 26.
Type 0 and press Enter to choose eth0 as the interface to associate with the default gateway. 27.
Type <<vcenter_default_gateway>> and press Enter. 28.
Press Enter to set no value for the IPv6 default gateway and to complete the configuration. 29.
Type 3 and press Enter to configure the host name of the VMware vCenter Server. 30.
Type <<vcenter_FQDN>> and press Enter to accept the configuration change. 31.
Type 4 and press Enter to configure the DNS information for the VMware vCenter Server. 32.
Type <<DNS_1_IP>> and press Enter. 33.
Type <<DNS_2_IP>> and press Enter to accept the configuration changes. 34.
Type 1 and press Enter to exit the configuration dialog box. 35.
Type exit and press Enter to log out of the prompt. 36.
Follow the instructions on the welcome screen to open a browser window to the URL shown. 37.
65 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Type root for the user name and vmware for the password. Click Login. 38.
66 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Select the checkbox beside Accept license agreement, then click Next to continue. 39.
Select the button beside Set custom configuration. Then click Next to continue. 40.
67 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click Next to accept an embedded database. 41.
68 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Enter a password in the password fields. Then click Next to accept an embedded single-sign-on 42.(SSO) deployment.
69 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Select the checkbox beside Active Directory Enabled to configure user permissions based on Active 43.Directory
®.
Enter the domain name, administrator user name, administrator password, and click Next to continue. 44.
70 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Review the configuration settings and click Start to complete the configuration. The VMware vCenter 45.Server will create all of the necessary configurations and database structures specified previously. This step might require several minutes.
Click Close after the configuration is complete. 46.
12.2 Changing the Root Password for VMware vCenter Server Appliance Web Client
To set up VMware vCenter Server on the VMware vCenter Server virtual machine, complete the following
steps:
Using the VMware vCenter Server Appliance web client, log into the newly created VMware vCenter 1.
Server using root as the user and vmware as the password.
71 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click the Admin tab. 2.
72 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Type vmware in the Current administrator password field. 3.
Type <<vcenter_root_password>> in the New administrator password field. 4.
Type <<vcenter_root_password>> in the Retype the new password field. 5.
Type <<vcenter_warning_email>> in the Email for expiration warning field. 6.
73 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click Submit. After the changes have been saved, “Operation was successful” should appear at the 7.top of the window in green letters.
12.3 Configuring Single Sign-On Using VMware vSphere Web Client
To configure the VMware vCenter Single Sign-On service using Active Directory with integrated Microsoft
Windows authentication, complete the following steps:
74 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Launch the VMware vSphere Web Client by opening a web browser and navigating to 1.https://<<vcenter_server_ip>>.
Click Log in to vSphere Web Client. 2.
Log in with the user name [email protected] and password <<password>>. 3.
On the homepage, click Administration in the left pane. 4.
Under Single Sign-On, click Configuration. 5.
Click the Identity Sources tab. 6.
Click the plus sign to add a new source. 7.
Select Active Directory (Integrated Windows Authentication) as the identity source type. 8.
Provide the DNS domain name <<dns_domain_name>> and use the machine account. 9.
Click OK. 10.
Click Users and Groups in the Single Sign-On section. 11.
Click the Groups tab and select the Administrators group. 12.
75 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
In the lower pane of the Group Members section, click the plus sign to add a member. 13.
Select the domain <<dns_domain_name>>, select the user on the list whom you want to set as an 14.
administrator, and click Add.
Click Check Names. 15.
Click OK when prompted for confirmation. 16.
Click OK. The newly added administrator user will appear in the Group Members section. 17.
Repeat steps 12 through 17 by selecting the Users group in step 12. 18.
Note: This document discusses the use of both the VMware vSphere thick client and VMware vSphere Web Client wherever appropriate. To facilitate this, add the FlexPod admin user to the Administrators group in the VMware vSphere thick client as well.
76 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
12.4 Setting Up VMware vCenter Server
Using a web browser, navigate to https://<<vcenter_ip>>. 1.
Click the link labeled Log in to vSphere Web Client. 2.
77 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
If prompted, run the VMware remote console plug-in. 3.
Log in using the VMware vCenter administrator user name [email protected] and 4.
the password.
78 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click the vCenter link in the left pane. 5.
79 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click the Datacenters link in the left pane. 6.
To create a Data center, click the icon in the center pane that has a green plus symbol above it. 7.
Type FlexPod_Express_DC as the data center name. 8.
Click the VMware vCenter Server available in the list. Click OK to continue. 9.
Right-click the data center FlexPod_Express_DC in the list in the center pane. Then click New 10.
Cluster.
80 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Name the cluster FlexPod_Express_Management. 11.
Check the box beside DRS. Leave the default values. 12.
Check the box beside vSphere HA. Leave the default values. 13.
81 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click OK to create the new cluster. 14.
Click FlexPod_Express_DC in the left pane. 15.
Click the Related Objects tab. 16.
Right-click FlexPod_Express_Management in the center pane and click Add Host. 17.
82 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
In the Host field, enter either the IP address or the host name of one of the VMware ESXi hosts. Click 18.Next.
Type root as the user name and the root password. Click Next to continue. 19.
Click Yes to accept the certificate. 20.
Review the host details and click Next to continue. 21.
Assign a license and click Next to continue. 22.
Click Next to continue. 23.
Click Next to continue. 24.
83 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Review the configuration parameters. Then click Finish to add the host. 25.
Repeat steps 17 to 25 to add the remaining VMware ESXi hosts to the cluster. 26.
Note: Two VMware ESXi hosts will be added to the cluster for the FlexPod Express small configuration. Four VMware ESXi hosts will be added to the cluster for the FlexPod Express medium configuration.
12.5 Setting Up a Microsoft Windows Template
To create a Microsoft Windows template, complete the following steps:
Log in to the VMware vCenter Server by using the VMware vSphere Client. 1.
In the VMware vSphere Client, select the FlexPod_Express_Management cluster in the left pane. 2.
Right-click the cluster and select New Virtual Machine. 3.
Select Custom and click Next. 4.
Enter a name for the virtual machine. Click Next. 5.
Select infra_datastore_1. Click Next. 6.
Select Virtual Machine Version: 8. Click Next. 7.
Verify that the Microsoft Windows option and the Microsoft Windows Server® 2008 R2 (64-bit) 8.version are selected. Click Next.
Select two virtual sockets and one core per virtual socket. Click Next. 9.
Select 4GB of memory. Click Next. 10.
Select two NICs total. 11.
For NIC 1, select the MGMT-Network option and the VMXNET 3 adapter.
For NIC 2, select the NFS-Network option and the VMXNET 3 adapter.
Click Next. 12.
84 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Keep the LSI Logic SAS option for the SCSI controller selected. Click Next. 13.
Keep the Create a New Virtual Disk option selected. Click Next. 14.
Make the disk size at least 60GB. Click Next. 15.
Click Next. 16.
Select the checkbox for Edit the Virtual Machine Settings Before Completion. Click Continue. 17.
Click the Options tab. 18.
Select Boot Options. 19.
Select the Force BIOS Setup checkbox. 20.
Click Finish. 21.
In the left pane, expand the cluster field by clicking the plus sign (+). 22.
Right-click the newly created virtual and click Open Console. 23.
Click the third button (green right arrow) to power on the virtual machine. 24.
Click the ninth button (CD with a wrench) to map the Microsoft Windows Server 2008 R2 SP1 ISO. 25.Then select Connect to ISO Image on Local Disk.
Navigate to the Microsoft Windows Server 2008 R2 SP1 ISO, select it, and click Open. 26.
Click the BIOS Setup Utility window and use the right arrow key to navigate to the Boot menu. Use 27.the down arrow key to select CD-ROM Drive. Press the plus (+) key twice to move CD-ROM Drive to the top of the list. Press F10 and Enter to save the selection and exit the BIOS setup utility.
The Microsoft Windows installer boots. Select the appropriate language, time and currency format, 28.and keyboard. Click Next.
Click Install Now. 29.
Make sure that the Windows Server 2008 R2 Standard (Full Installation) option is selected. Click 30.Next.
Read and accept the license terms and click Next. 31.
Select Custom (advanced). Make sure that Disk 0 Unallocated Space is selected. Click Next to allow 32.the Microsoft Windows installation to complete.
After the Microsoft Windows installation is complete and the virtual machine has rebooted, click OK to 33.set the administrator password.
Enter and confirm the administrator password and click the blue arrow to log in. Click OK to confirm 34.the password change.
After logging in to the virtual machine desktop, from the virtual machine console window select the 35.virtual machine menu. Under Guest, select Install/Upgrade VMware Tools. Click OK.
If prompted to eject the Microsoft Windows installation media before running the setup for the 36.VMware tools, click OK. Then click OK again.
In the dialog box, select Run setup64.exe. 37.
In the VMware Tools installer window, click Next. 38.
Make sure that Typical is selected and click Next. 39.
Click Install. 40.
If prompted to trust software from VMware, select the checkbox to always trust and click Install. 41.
Click Finish. 42.
Click Yes to restart the virtual machine. 43.
After the reboot is complete, select the virtual machine menu. Under Guest, select Send Ctrl+Alt+Del 44.and then enter the password to log in to the virtual machine.
Set the time zone for the virtual machine and the IP address, gateway, and host name. 45.
85 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Note: A reboot is required.
If necessary, activate Microsoft Windows. 46.
Log back in to the virtual machine and download and install all required Microsoft Windows updates. 47.
Note: This process requires several reboots.
Right-click the virtual machine in VMware vCenter and click Clone to Template. 48.
Enter the name w2k8-template for the clone. 49.
Select the cluster FlexPod_Express_Management as the target cluster to host the template. Click 50.
Next.
Select infra_datastore_1. Click Next. 51.
Click Finish. 52.
13 NetApp Virtual Storage Console 5.0 Deployment Procedure
This section discusses the procedure for deploying NetApp Virtual Storage Console (VSC) 5.0.
13.1 NetApp VSC 5.0 Preinstallation Considerations
The following licenses are required to run NetApp VSC on storage systems that run clustered Data
ONTAP 8.2.2:
Protocol licenses (NFS)
NetApp SnapRestore (for backup and recovery)
NetApp SnapManager suite
13.2 Installing NetApp VSC 5.0
To install the NetApp VSC 5.0 software, complete the following steps:
Log into the VMware vCenter Server using the vSphere Web Client. 1.
In the right pane, click VMs and Templates. 2.
Select win2k8-template in the left pane and right-click it. 3.
Select Deploy VM from this Template. 4.
Provide a name for the NetApp VSC virtual machine and select FlexPod_Express_DC as the location 5.for the virtual machine. Click Next.
Select the FlexPod_Express_Management cluster. Click Next. 6.
Select infra_datastore_1. Click Next. 7.
Click Next. 8.
Review the virtual machine settings and click Finish. 9.
Select the newly created virtual machine and select Power on the virtual machine in the right pane. 10.
Right-click the virtual machine and select Open Console. 11.
Log in to the virtual machine as Administrator, assign an IP address, and join the machine to the 12.Active Directory domain. Install the current version of Adobe Flash Player on the virtual machine. Install all Microsoft Windows updates on the virtual machine.
Download the x64 version of NetApp Virtual Storage Console 5.0 from the NetApp Support site. 13.
Right-click the file downloaded and select Run As Administrator. 14.
On the Installation Wizard Welcome page, click Next. 15.
86 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Select the checkbox to accept the message. Click Next. 16.
Select the backup and recovery capability. Click Next. 17.
87 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Note: The backup and recovery capability requires an additional license.
Click Next to accept the default installation location. 18.
Click Install. 19.
88 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click Finish. 20.
Note: During the VSC installation, an error message is sometimes displayed indicating that the NetApp vSphere Plugin Framework has not started. This issue occurs if the installer fails to generate SSL certificates during the installation. If you receive this error message, you must run the below command from the VSC installation directory:
cd C:\Program Files\NetApp\Virtual Storage Console\bin
vsc ssl setup -generate-passwords
13.3 Registering NetApp VSC with VMware vCenter Server
To register NetApp VSC with the VMware vCenter Server, complete the following steps:
A browser window with the registration URL should open automatically when the installation phase is 1.complete. If a browser doesn’t open, open a browser on the NetApp VSC virtual machine and navigate to https://localhost:8143/Register.html.
Click Continue to this website (not recommended). 2.
In the Plug-in Service Information section, in the drop-down list, select the local IP address that the 3.VMware vCenter Server uses to access the NetApp VSC server.
In the VMware vCenter Server Information section, enter the host name or IP address, user name 4.(FlexPod admin user), and user password for the VMware vCenter Server. Click Register to complete the registration.
89 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Upon successful registration, the storage controllers are discovered automatically. 5.
Note: The storage discovery process will take some time.
13.4 Updating Credentials for Storage Resources
To discover storage resources for the monitoring and host configuration and the provisioning and cloning
capabilities, complete the following steps:
Using the VMware vSphere Web Client, log in to the VMware vCenter Server as the FlexPod admin 1.user. If the VMware vSphere web client was previously opened, close it and then reopen it.
On the homepage, click the Home tab and click Virtual Storage Console. 2.
90 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
In the navigation pane, select Storage Systems if it is not selected by default. 3.
91 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Right-click the unknown controller and select Modify. 4.
Enter the storage cluster management IP address in the Management IP address field. Enter admin 5.as the user name and the admin password as the password. Make sure that Use SSL is selected. Click OK.
92 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Click OK to accept the controller privileges. 6.
Refresh the VMware vSphere Web Client to view the updated information. 7.
13.5 Using Optimal Storage Settings for VMware ESXi Hosts
NetApp VSC allows automated configuration of storage-related settings for all VMware ESXi hosts that
are connected to NetApp storage controllers. To use these settings, complete the following steps:
On the Home screen, choose vCenter > Hosts and Clusters. For each VMware ESXi host, right-click 1.and choose NetApp VSC > Set Recommended Values for these hosts.
93 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Check the settings that are to be applied to the selected VMware vSphere hosts. Click OK to apply 2.the settings.
Note: This function sets values for host bus adapters (HBAs) and converged network adapters (CNAs), sets appropriate paths and path-selection plug-ins, and verifies appropriate settings for software-based I/O (NFS and Small Computer System Interface over IP [iSCSI]).
Click OK. 3.
94 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
13.6 Performing NetApp VSC 5.0 Backup and Recovery
Prerequisites for Using the Backup and Recovery Capability
Before you begin using the backup and recovery capability to schedule backups and restore your
datastores, virtual machines, or virtual disk files, you must make sure that the storage systems that
contain the datastores and virtual machines for which you are creating backups have valid storage
credentials.
If you are planning to use the NetApp SnapMirror® update option, add all the destination storage systems
with valid storage credentials.
Backup and Recovery Configuration
The following steps detail the procedure for configuring a backup job for a datastore.
On the Home screen, click Storage on the Home tab. 1.
Right-click the datastore that you need for the backup. Choose NetApp VSC > Backup > Schedule 2.Backup.
Note: If you prefer one time back, then choose Backup Now instead of Schedule Backup.
95 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Type a backup job name and description. 3.
Click Next. 4.
Click Next. 5.
Select one or more backup scripts if any are available and click Next. 6.
96 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Select the hourly, daily, weekly, or monthly schedule that you want for this backup job and click Next. 7.
Use the default VMware vCenter credentials or type the user name and password for the VMware 8.vCenter Server and click Next.
97 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
Specify backup retention details according to your requirements. Enter an e-mail address so that you 9.can receive e-mail alerts. You can add multiple e-mail addresses by using semicolons to separate e-mail addresses. Click Next.
Review the summary page and click Finish. If you want to run the job immediately, select the Run Job 10.Now option. Click Finish.
Click OK. 11.
98 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
On the storage cluster interface, you can disable automatic Snapshot copies of the volume by typing 12.this command:
volume modify –volume infra_datastore_1 –snapshot-policy none
To delete any existing automatic Snapshot copies that have been created on the volume, type the 13.following command:
volume snapshot show –volume infra_datastore_1
volume snapshot delete –volume infra_datastore_1 -snapshot *
Press Y to confirm the deletion. 14.
14 Bill of Materials
This section details the hardware and software components used in validating both the small and medium
FlexPod Express configurations included in this document.
14.1 Small Configuration
Table 10 lists the components of the small configuration.
Table 10) Small configuration components.
Part Number Product Description Quantity Required
Cisco Components
Network Switches
N3K-C3048-FA-L3 Cisco Nexus 3048 Std Airflow (port side exhaust) AC P/S LAN Ent 2
N2200-PAC-400W N2K/N3K AC Power Supply Std airflow (port side exhaust) 4
CAB-C13-C14-AC Power cord C13 to C14 (recessed receptacle) 10A 4
N3K-C3048-BAS1K9 Cisco Nexus 3048 Base License 2
N3K-C3048-LAN1K9 Cisco Nexus 3048 LAN Enterprise License 2
N3K-C3048-FAN Cisco Nexus 3048 Fan Module Port-side Exhaust 2
N3K-C3064-ACC-KIT Cisco Nexus 3064PQ Accessory Kit 2
N3KUK9-602U2.3 Cisco NX-OS Release 6.0(2)U2(3) 2
CON-SNT-48FAL3 Cisco SMARTNET®8X5XNBD Nexus 3048 Std Airflow AC P/S LAN Ent
2
Cisco UCS Compute
99 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
UCSC-C220-M3S
Cisco UCS C220 M3 SFF w/o CPU mem HDD PCIe PSU w/ rail kit 2
UCS-CPU-E52650B
2.60 GHz E5-2650 v2/95W 8C/20MB Cache/DDR3 1866MHz 4
UCS-MR-1X082RY-A 16GB DDR3-1600-MHz RDIMM/PC3-12800/dual rank/1.35v 16
CAB-C13-C14-AC Power cord, C13 to C14 (recessed receptacle), 10A 4
UCSC-PSU-450W 450W power supply for C-series rack servers 4
N2XX-ABPCI03-M3 Broadcom 5709 Quad Port 1Gb w/TOE iSCSI for M3 Servers 2
UCS-SD-32G-S 32GB SD Card for Cisco UCS servers 4
N20-BBLKD UCS 2.5 inch HDD blanking pane 16
UCSC-HS-C220M3 Heat Sink for Cisco UCS C220 M3 Rack Server 4
UCSC-PCIF-01F Full height PCIe filler for C-Series 2
UCSC-RAIL1 Rail Kit for C220, C22, C24 rack servers 2
CON-SNT-C220M3SF SMARTNET 8X5XNBD Cisco UCS C220 M3 SFF w/o 2
NetApp Components
FAS2520A-001-R6 FAS2520 High Availability System 2
X80102A-R6-C Bezel,FAS2520,R6,-C 1
FAS2520-111-R6-C FAS2520,12x900GB,10K,-C 1
X1558A-R6-C Power Cable,In-Cabinet,48-IN,C13-C14,-C 2
SVC-FLEXPOD-SYSTEMS
Systems Used in FlexPod Solution, Attach PN 1
X6560-R6-C Cable,Ethernet,0.5m RJ45 CAT6,-C 1
X6561-R6 Cable,Ethernet,2m RJ45 CAT6 2
X6557-EN-R6-C Cbl,SAS Cntlr-Shelf/Shelf-Shelf/HA,0.5m,EN,-C 2
DOC-2520-C Documents,2520,-C 1
X5518A-R6-C Kit,FAS2XXX,-C,R6 1
OS-ONTAP-CAP2-1P-C OS Enable,Per-0.1TB,ONTAP,Perf-Stor,1P,-C 108
SWITCHLESS 2-Node Switchless Cluster 1
SW-2-2520A-SMGR-C SW-2,SnapManager Suite,2520A,-C 2
SW-2-2520A-SRESTORE-C
SW-2,SnapRestore,2520A,-C 2
SW-2-2520A-FLEXCLN-C
SW-2,FlexClone,2520A,-C 2
SW-2-2520A-ISCSI-C SW-2,iSCSI,2520A,-C 2
SW-ONTAP8.2.2-CLM SW,Data ONTAP 8.2.2,Cluster-Mode 2
100 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
NetApp Components
SW-2-2520A-CIFS-C SW-2,CIFS,2520A,-C 2
SW-2-2520A-NFS-C SW-2,NFS,2520A,-C 2
SVC-A2-IN-NBR-E HW Support,Standard2 Replace,Inst,NBD,e 1
SW-SSP-A2-IN-NBR-E SW Subs,Standard2 Replace,Inst,NBD,e 1
SVC-INST-A2-IN1-NBR-E
Initial Install,Standard2 Replace,Inst,NBD,e 1
CS-OS-SUPPORT-ONTAP
OS Support Entitlement,ONTAP 1
SES-SYSTEM SupportEdge Standard, Premium or equivalent service from an authorized support services partner
1
1
1SupportEdge Premium required for cooperative support.
14.2 Medium Configuration
Table 11 lists the components for the medium configuration.
Table 11) Medium configuration components.
Part Number Product Description Quantity Required
Cisco Components
Network Switches
N3K-C3048-FA-L3 Cisco Nexus 3048 Std Airflow (port side exhaust) AC P/S LAN Ent 2
N2200-PAC-400W N2K/N3K AC Power Supply Std airflow (port side exhaust) 4
CAB-C13-C14-AC Power cord C13 to C14 (recessed receptacle) 10A 4
N3K-C3048-BAS1K9 Cisco Nexus 3048 Base License 2
N3K-C3048-LAN1K9 Cisco Nexus 3048 LAN Enterprise License 2
N3K-C3048-FAN Cisco Nexus 3048 Fan Module Port-side Exhaust 2
N3K-C3064-ACC-KIT Cisco Nexus 3064PQ Accessory Kit 2
N3KUK9-602U2.3 Cisco NX-OS Release 6.0(2)U2(3) 2
CON-SNT-48FAL3 Cisco SMARTNET 8X5XNBD Nexus 3048 Std Airflow AC P/S LAN Ent 2
Cisco UCS Compute
UCSC-C220-M3S Cisco UCS C220 M3 SFF w/o CPU mem HDD PCIe PSU w/ rail kit 4
UCS-CPU-E52650B 2.60 GHz E5-2650 v2/95W 8C/20MB Cache/DDR3 1866MHz 8
UCS-MR-1X082RY-A 16GB DDR3-1600-MHz RDIMM/PC3-12800/dual rank/1.35v 32
CAB-C13-C14-AC Power cord, C13 to C14 (recessed receptacle), 10A 8
UCSC-PSU-450W 450W power supply for C-series rack servers 8
101 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
N2XX-ABPCI03-M3 Broadcom 5709 Quad Port 1Gb w/TOE iSCSI for M3 Servers 4
UCS-SD-32G-S 32GB SD Card for Cisco UCS servers 8
N20-BBLKD Cisco UCS 2.5 inch HDD blanking pane 32
UCSC-HS-C220M3 Heat Sink for Cisco UCS C220 M3 Rack Server 8
UCSC-PCIF-01F Full height PCIe filler for C-Series 4
UCSC-RAIL1 Rail Kit for C220, C22, C24 rack servers 4
CON-SNT-C220M3SF SMARTNET 8X5XNBD Cisco UCS C220 M3 SFF w/o 4
NetApp Components
FAS2520A-001-R6 FAS2520 High Availability System 2
X80102A-R6-C Bezel,FAS2520,R6,-C 1
FAS2520-111-R6-C FAS2520,12x900GB,10K,-C 1
X1558A-R6-C Power Cable,In-Cabinet,48-IN,C13-C14,-C 2
SVC-FLEXPOD-SYSTEMS
Systems Used in FlexPod Solution, Attach PN 1
X6560-R6-C Cable,Ethernet,0.5m RJ45 CAT6,-C 1
X6561-R6 Cable,Ethernet,2m RJ45 CAT6 2
X6557-EN-R6-C Cbl,SAS Cntlr-Shelf/Shelf-Shelf/HA,0.5m,EN,-C 2
DOC-2520-C Documents,2520,-C 1
X5518A-R6-C Kit,FAS2XXX,-C,R6 1
OS-ONTAP-CAP2-1P-C OS Enable,Per-0.1TB,ONTAP,Perf-Stor,1P,-C 108
SWITCHLESS 2-Node Switchless Cluster 1
SW-2-2520A-SMGR-C SW-2,SnapManager Suite,2520A,-C 2
SW-2-2520A-SRESTORE-C
SW-2,SnapRestore,2520A,-C 2
SW-2-2520A-FLEXCLN-C
SW-2,FlexClone,2520A,-C 2
SW-2-2520A-ISCSI-C SW-2,iSCSI,2520A,-C 2
SW-ONTAP8.2.2-CLM SW, Data ONTAP 8.2.2,Cluster-Mode 2
SW-2-2520A-CIFS-C SW-2,CIFS,2520A,-C 2
SW-2-2520A-NFS-C SW-2,NFS,2520A,-C 2
SVC-A2-IN-NBR-E HW Support,Standard2 Replace,Inst,NBD,e 1
SW-SSP-A2-IN-NBR-E SW Subs,Standard2 Replace,Inst,NBD,e 1
SVC-INST-A2-IN1-NBR-E
Initial Install,Standard2 Replace,Inst,NBD,e 1
102 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
NetApp Components
CS-OS-SUPPORT-ONTAP
OS Support Entitlement,ONTAP 1
SES-SYSTEM SupportEdge Standard, Premium or equivalent service from an authorized support services partner
1
1
15 FlexPod Express with Cisco UCS Director
Cisco UCS Director is an orchestration and automation tool for the converged data center and is
designed to simplify management. This tool allows you to connect to all the different infrastructure
components to control all aspects of the data center. Cisco UCS Director manages computing, network,
and storage resources in physical, virtual, and bare-metal environments. It offers capabilities such as
single-pane monitoring, provisioning, and orchestration. Cisco UCS Director is used as the management
tool for the FlexPod Express cloud discussed in this document. For instructions about how to set up the
Cisco UCS Director software and to quickly deploy a virtual machine self-service portal, refer to
http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/unified-computing/whitepaper-
c11-731143.html.
16 Conclusion
FlexPod Express is the optimal shared infrastructure foundation to deploy a variety of IT workloads. This
platform is both flexible and scalable for multiple use cases and applications. VMware vSphere is one
common use case as the virtualization solution, which is described in this document. The flexibility and
scalability of FlexPod Express enable customers to start out with a right-sized infrastructure that can
ultimately grow with and adapt to their evolving business requirements.
References
NetApp FAS25XX Series Storage Controllers: http://www.netapp.com/in/products/storage-systems/fas2500/
Cisco UCS C-Series Servers: http://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-c-series-rack-servers/index.html
Cisco Nexus 3048 Switches: http://www.cisco.com/c/en/us/products/switches/nexus-3048-switch/index.html
VMware vSphere 5.5: http://pubs.vmware.com/vsphere-55/index.jsp
1 SupportEdge Premium required for cooperative support.
103 FlexPod Express with VMware vSphere 5.5 U1: Small and Medium Configurations Implementation Guide
© 2014 NetApp, Inc. All Rights Reserved.
NetApp provides no representations or warranties regarding the accuracy, reliability, or serviceability of any information or recommendations provided in this publication, or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS, and the use of this information or the implementation of any recommendations or techniques herein is a customer’s responsibility and depends on the customer’s ability to evaluate and integrate them into the customer’s operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.
© 2014 NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, AutoSupport, DataFabric, Data ONTAP, FlexClone, FlexPod, FlexVol, SnapManager, SnapMirror, SnapRestore, and Snapshot are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. Cisco, Cisco Nexus, Cisco UCS, and SMARTnet are registered trademarks of Cisco Systems. Microsoft, Active Directory, Windows, and Windows Server are registered trademarks of Microsoft Corporation. VMware, vMotion, and VMware vSphere are registered trademarks and ESXi and vCenter are trademarks of VMware, Inc. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such. TR-4328-0914
Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exact product and feature versions described in this document are supported for your specific environment. The NetApp IMT defines the product components and versions that can be used to construct configurations that are supported by NetApp. Specific results depend on each customer's installation in accordance with published specifications.