dell emc ready solution for vmware vcloud nfv 3.1 ... · 7 dell emc ready solution for vmware...
TRANSCRIPT
Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Deployment Automation Software Guide for VMware NFV 3.1 with VMware Integrated OpenStack 5.1 with Kubernetes
Dell Engineering May 2019
2 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Revisions
Date Description
May 2019 Initial release for Deployment Automation framework SW v3.1.1.1
The information in this publication is provided “as is.” Dell Inc. and its suppliers makes no representations or warranties of any kind with respect to the
information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software that is described in this publication requires an applicable software license.
Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Big
Cloud Fabric (BCF) is the trademark or registered trademark of Big Switch Networks, Inc. Other trademarks may be the property of their respective
owners. Published in the USA.
Dell believes that the information in this document is accurate as of its publication date. The information is subject to change without notice.
3 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Table of contents
Revisions............................................................................................................................................................................. 2
Introduction ......................................................................................................................................................................... 6
1 Solution prerequisites ................................................................................................................................................... 7
1.1 Deployment server ............................................................................................................................................. 7
1.1.1 Install ESXi on deployment server ...................................................................................................................... 8
1.1.2 Customize ESXi ................................................................................................................................................ 12
1.2 Create standard vSwitch on deployment server ............................................................................................... 16
1.3 Create port group on deployment server .......................................................................................................... 18
1.4 Physical switch configuration ............................................................................................................................ 19
1.5 Create datastore on Deployment server .......................................................................................................... 19
1.6 Deployment Automation VM ............................................................................................................................. 21
2 Supported configuration ............................................................................................................................................. 22
2.1 Dell EMC NFV hardware for vSAN Ready Node .............................................................................................. 22
3 Deployment environment prerequisites ...................................................................................................................... 23
4 Getting started ............................................................................................................................................................ 24
4.1 OVA deployment............................................................................................................................................... 24
4.1.1 Overview of OVA directory file system ............................................................................................................. 27
4.2 VMware license bundle key .............................................................................................................................. 29
4.3 User input ......................................................................................................................................................... 29
5 Big cloud fabric OVA deployment............................................................................................................................... 31
5.1 BCF Configuration ............................................................................................................................................ 31
5.2 Configure Big Cloud Fabric (BCF) controllers .................................................................................................. 32
5.2.1 Configure the first BCF Controller .................................................................................................................... 32
5.2.2 Configure second BCF Controller ..................................................................................................................... 35
5.2.3 Configure virtual IP address for cluster ............................................................................................................ 37
5.2.4 Access the BCF GUI ........................................................................................................................................ 38
5.3 Leaf switches configuration using BCF GUI ..................................................................................................... 39
5.3.1 Install and configure fabric switches with zero touch fabric ............................................................................. 39
5.3.2 Provision switches in the BCF Controller ......................................................................................................... 40
5.3.3 Switch installation ............................................................................................................................................. 40
5.3.4 Verify Switch Light operating system installation ............................................................................................. 41
5.4 Resolve common warnings and errors ............................................................................................................. 42
4 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5.4.1 Resolve suspended switches ........................................................................................................................... 42
5.4.2 Resolve switches with mismatched ONIE and CPLD firmware ....................................................................... 43
5.4.3 Resolve switches without management address ............................................................................................. 44
5.4.4 Leaf interfaces not in interface group ............................................................................................................... 45
5.4.5 BCF validation from CLI ................................................................................................................................... 46
5.5 Create Interface group ...................................................................................................................................... 47
5.6 Creating tenants ............................................................................................................................................... 50
5.7 Creating segments ........................................................................................................................................... 52
5.7.1 Link the segment to Interface group ................................................................................................................. 53
5.8 BGP configuration............................................................................................................................................. 56
5.8.1 Creating Neighbors ........................................................................................................................................... 58
5.8.2 Segment interface ............................................................................................................................................ 59
5.9 Configure remote log server for BCF................................................................................................................ 61
5.10 Update cutsheet values .................................................................................................................................... 62
6 Run deployment orchestrator ..................................................................................................................................... 63
7 Logging and reporting ................................................................................................................................................ 64
7.1 Sample main log file structure .......................................................................................................................... 64
7.2 Sample summary log file structure ................................................................................................................... 65
7.3 Sample inventory log file structure ................................................................................................................... 65
8 Supported features ..................................................................................................................................................... 66
8.1 Remediation ...................................................................................................................................................... 66
8.1.1 Non-supported workflows ................................................................................................................................. 66
8.1.2 Supported workflows ........................................................................................................................................ 67
8.2 Scalability .......................................................................................................................................................... 68
8.3 Cleanup ............................................................................................................................................................ 68
9 Additional configuration requirements ........................................................................................................................ 70
9.1 Time zone ......................................................................................................................................................... 70
9.2 Internet connectivity to stamp ........................................................................................................................... 70
10 Supported workflows for Remediation ....................................................................................................................... 71
10.1 ESXi installation and configuration ................................................................................................................... 71
10.1.1 Install ESXi on Dell EMC PowerEdge R640 and R740xd servers using iDRAC9 ..................................... 71
10.1.2 Customize ESXi ......................................................................................................................................... 76
10.2 VMware vCenter Server Appliance deployment ............................................................................................... 82
5 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10.2.1 Stage 1: Deploy OVA file for Management vCenter Server Appliance with embedded PSC .................... 82
10.2.2 Stage 2: Set up Management vCenter Server Appliance with embedded PSC ........................................ 88
10.2.3 Change vSAN default storage policy for management vCSA .................................................................... 90
10.2.4 Assign license to Management vCSA ........................................................................................................ 91
10.2.5 Add AD authentication for Management vCSA .......................................................................................... 93
10.2.6 Stage 1: Deploy OVA file for Resource vCenter Server Appliance ........................................................... 96
10.2.7 Stage 2: Set up resource vCenter Server Appliance with embedded PSC ............................................. 102
10.2.8 Change VMware vSAN default storage policy for resource vCSA .......................................................... 104
10.2.9 Assign license to Resource vCSA ........................................................................................................... 106
10.2.10 Add AD authentication for Resource vCSA ............................................................................................. 107
10.3 NSX-T deployment and setup ........................................................................................................................ 110
10.3.1 Install NSX Manager Virtual Appliance .................................................................................................... 111
10.3.2 Installation of controller and cluster from NSX Manager ......................................................................... 117
10.3.3 NSX Edge installation .............................................................................................................................. 123
10.4 Enable VMware enhanced vMotion compatibility ........................................................................................... 128
10.4.1 Enable VMware EVC for management cluster ........................................................................................ 129
10.4.2 Enable VMware EVC for resource cluster ............................................................................................... 130
10.5 Enable vSphere DRS ..................................................................................................................................... 130
10.6 Enabling vSphere availability .......................................................................................................................... 131
10.7 Set up anti-affinity rules .................................................................................................................................. 132
10.7.1 Create an anti-affinity rule ........................................................................................................................ 133
A Show running configs on leaf switches .................................................................................................................... 135
B References ............................................................................................................................................................... 147
6 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Introduction
The Dell EMC with VMware vCloud NFV platform automates the deployment of VMware vCloud NFVi using
VMware vCloud NFV OpenStack Edition 3.1 with NSX-T and Dell EMC PowerEdge servers. This guide
follows the Two-pod Design as described in VMware vCloud NFV OpenStack Edition Reference Architecture
3.1 guide.
This guide provides the information required to install the Dell EMC Ready Solution bundle. This guide
provides step by step information to create a deployment server, deploy the deployment VM, and run the
deployment automation script that consists of:
• Orchestrator
• Automation script
• OVA
• ISO
• Other required files
The automation process includes pre-deployment validation, installation, configuration, and integration of the
following components:
• Big cloud fabric (BCF)
• ESXi
• Active Directory (AD)
• Domain Network System (DNS)
• Network Time Protocol (NTP)
• vCenter Server Appliance (vCSA)
• vSAN
• NSX-T
• vRealize Log Insight (vRLI)
• vRealize Operation Manger (vROPs)
• VMware Integrated OpenStack (VIO)
• Kubernetes
7 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
1 Solution prerequisites
Note: Before starting with the NFV solution make sure all the hardware is configured and firmware are
updated as mentioned in Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Deployment Automation Hardware Guide.
1.1 Deployment server
Deployment sever provides the ability to an Administrator to successfully deploy the Dell EMC Ready Solution
bundle.
Figure 1 displays the network topology for the deployment server and its network connections:
ToR switch - Top of Rack (S4048T-ON)
Leaf switch - Z9100-ON or S6010-ON
• ToR switch using 10G interface for BCF (p-switch management network) – vmnic0
• ToR switch using 10G interface for the ESXI management network – vmnic1
• ToR switch using 10G interface for the OOB network – vmnic2
• ToR switch using 10G interface for the Management network– vmnic3 (stamp)
• Leaf In-band port 1 using 10G interface – vmnic4
• Leaf In-band port 2 using 10G interface – vmnic5
• Leaf In-band port 3 using 10G interface – vmnic6
• Leaf In-band port 4 using 10G interface – vmnic7
• Leaf1 switch using 10G/25G interface for physical switch network
• Leaf2 switch using 10G/25G interface for physical switch network
8 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Deployment server network connections
Note: ESXi 6.7U1 must be installed on a bare-metal deployment server.
1.1.1 Install ESXi on deployment server Prerequisites:
• iDRAC should be configured and accessible
• ESXi iso file on local machine
• ToR switch should be configured
Perform the following steps to install ESXi on deployment server:
1. Use the designated URL to launch iDRAC 9.
2. Enter the required user name and password, then click Log In:
9 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
iDRAC login page
3. From the Dashboard screen, click Launch Virtual Console from the Virtual Console section.
The iDRAC Virtual Console screen displays.
4. On the navigation bar, click Connect Virtual Media.
iDRAC 9 Virtual Console window
5. On the Virtual Media screen, in the Map CD/DVD section, click Choose File, and select the ESXi
image file from your local machine.
6. Click Map Device then click Close.
10 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Connect Virtual Media Screen
7. In the navigation bar, click Next Boot, then select Virtual CD/DVD/ISO and save it.
Next Boot select screen
8. In the navigation bar, click Power then select Power Cycle System (cold boot).
Power Cycle System (cold boot) selection
The ESXi installation process begins.
9. From the Welcome to the VMware ESXi 6.7.0 Installation screen, press Enter to continue.
VMware ESXi installation screen
11 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10. Review the contents of the End User License Agreement (EULA) and if you agree to the terms,
press F11.
11. From the Disk installation screen, use the arrow keys to highlight the Dell Internal Dual SD storage
device, then press Enter.
Disk selection screen
12. From the Confirm Disk Selection window, press Enter to confirm the selection.
13. Verify that US Default is the option highlighted in the Keyboard layout window, then press Enter to
continue.
14. In the Enter a root password window, enter the root password, enter it again to confirm, then press
Enter. The Scanning system screen displays.
Scanning system status screen
15. On the Confirm Install window, press F11 to confirm the installation.
Fig- confirm installation screen
16. Once the installation is complete, on the Installation Complete window, press the Enter key to
reboot the system.
Installation Complete confirmation window
12 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Once the system reboot completes, the Direct Console User Interface (DCUI) screen displays.
Direct Console User Interface (DCUI) screen
1.1.2 Customize ESXi The System Customization screen allows users to customize various ESXi system settings such as:
• Passwords
• Management configuration
• Restart options
• Keyboard settings
• Troubleshooting options
• System reset configurations
Follow the below steps to access the System Customization screen:
1. From the DCUI screen, press F2.
2. Enter the required user credentials, then press Enter.
DCUI screen authentication
3. On the System Customization screen, use the keyboard arrow keys and select the Customize
option, then press Enter.
System customization screen
13 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
1.1.2.1 Management network configuration To configure the OOB management network perform the following steps:
1. From the System Customization screen, use the keyboard arrow keys and select Configure
Management Network, then press Enter.
2. To update the network adapters, use the keyboard arrow keys and select the Network Adapter
option, then press Enter.
Connected Network Adapters
1.1.2.2 Change IPv4 configuration Follow the below steps to configure the static IPv4 address:
1. On the System Customization screen, select Configure Management Network, then select IPv4
Configuration, then press the Enter key.
2. Select Set static IPv4 and network configuration, then provide the required IPv4 address, subnet
mask, and default gateway.
3. Press Enter to save the changes made.
Note: The IPv4 addresses used in the example are for demonstration purposes only.
Setting static IPv4 address
14 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
1.1.2.3 Change DNS configuration Follow the below steps to change the DNS configuration:
1. On the Configure Management Network screen, use keyboard arrow keys to select DNS
Configuration then press Enter.
DNS Configuration option
2. From the DNS Configuration screen, use keyboard arrow keys to select Use the following DNS
server addresses and hostname option.
3. In the fields provided, enter the required Primary DNS Server, Alternate DNS Server IP, and
Hostname information in the fields provided, then press Enter.
Adding DNS server and hostname
4. Add the domain name in the Suffixes field then press Enter to save the settings.
Setting DNS suffix
15 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5. Press Enter to restart the management network.
Restart of Management Network option
6. Once the network restarts, select the Test Management Network and press Enter. This pings the
configured default gateway, primary and alternate DNS servers, and resolve the configured
hostname.
1.1.2.4 Troubleshooting options Follow the below steps to troubleshoot ESXi issues:
1. Use the keyboard to highlight the Troubleshooting Options then press Enter.
2. From the options provided, use the arrow to select the desired troubleshooting option, then press
Enter.
1.1.2.5 Assign license to ESXi 1. From the browser, open ESXi and click Manage, then select Licensing.
2. In the License key field, enter the required license key and click Check license.
Assign ESXi license
16 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
3. Click Assign license then click Close.
ESXi license assignment screen
1.2 Create standard vSwitch on deployment server
By default, vSwitch0 is available on the deployment server. The user is required to create following additional
virtual switches on the deployment server:
vSwitch details
vSwitch name Uplink MTU (bytes) Link discovery Security
vSwitch0 vmnic2 1500 Bytes Listen/CDP For Promiscuous mode and Forged transmits, enable the Reject radio button
vSwitch1 vmnic4 9000 Bytes Listen/CDP For Promiscuous mode and Forged transmits, enable the Accept radio button
vSwitch2 vmnic5 9000 Bytes Listen/CDP For Promiscuous mode and Forged transmits, enable the Accept radio button
vSwitch3 vmnic6 9000 Bytes Listen/CDP For Promiscuous mode and Forged transmits, enable the Accept radio button
vSwitch4 vmnic7 9000 Bytes Listen/CDP For Promiscuous mode and Forged transmits, enable the Accept radio button
BCF vmnic0 9000 Bytes Listen/CDP For Promiscuous mode and Forged transmits, enable the Reject radio button
Stamp176 vmnic3 9000 Bytes Listen/CDP For Promiscuous mode and Forged transmits, enable the Reject radio button
esxi_mgmt_200 vmnic1 9000 Bytes Listen/CDP For Promiscuous mode and Forged transmits, enable the Reject radio button
To create a standard vSwitch on the deployment server, perform the following steps:
1. From a web browser, navigate to the deployment server IP address, then log-in into it using the
necessary credentials.
17 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Deployment server log in page
2. From the navigation pane, click Networking.
3. Click the Virtual switches tab, then select Add standard virtual switch. The Add standard virtual
switch screen displays.
4. In the vSwitch Name field, enter the desired vSwitch name.
5. Use the vSwitch details in Table 1 to select the MTU, Uplink, Mode, and Protocol options.
6. Use the vSwitch details in Table 1 to select the Security details for the remaining fields, then click
Add.
Add standard virtual switch settings screen
7. Repeat the steps in this section to create additional vSwitches on the deployment server as specified
in Table 1.
18 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
1.3 Create port group on deployment server
On the deployment server by default, a VM network port group with VLAN ID 0 is created. While creating an
additional port group, you must assign the VLAN and vSwitch to the port group as specify in Table 2:
Port group details
Port group name
Description VLAN ID
Virtual switch
Security
VM network For iDRAC (OOB management network)
0 vSwitch0 For Promiscuous mode and Forged transmits, enable the Inherit from vSwitch radio button
Inbandport1 For BCF VMs (In-band network on leaf1 for BGP routing protocol)
4095 vSwitch1 For Promiscuous mode and Forged transmits, enable the Inherit from vSwitch radio button
Inbandport2 For BCF VMs (In-band network on leaf2 for BGP routing protocol)
4095 vSwitch2 For Promiscuous mode and Forged transmits, enable the Inherit from vSwitch radio button
Inbandport3 For BCF VMs (In-band network on leaf2 for BGP routing protocol)
4095 vSwitch3 For Promiscuous mode and Forged transmits, enable the Inherit from vSwitch radio button
Inbandport4 For BCF VMs (In-band network on leaf2 for BGP routing protocol)
4095 vSwitch4 For Promiscuous mode and Forged transmits, enable the Inherit from vSwitch radio button
BCF For BCF VMs (OOB BCF management network)
- BCF For Promiscuous mode and Forged transmits, enable the Inherit from vSwitch radio button
PG-176 For deployment and stamp VMs (Management network)
176 Stamp176 For Promiscuous mode and Forged transmits, enable the Inherit from vSwitch radio button
Esxi_mgmt_200
For rack ESXi servers (Esxi management network)
200 Esxi_mgmt_200
For Promiscuous mode and Forged transmits, enable the Inherit from vSwitch radio button
To create a port group on deployment server, perform the following steps:
1. From a web browser, navigate to the deployment server IP address and enter the necessary log in
credentials.
2. From the navigation panel, click Networking.
3. On the Port groups tab, then click Add port group. The Add port group screen displays.
Add port group option
4. In the Name field, enter the Port group name.
5. In the vSwitch Name field, select the desired vSwitch name from the drop down.
19 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
6. Using the information provided in Table 2, enter the required port group details, required VLAN ID
information, and select the required virtual switch.
7. Use the Table 2 port group details to assign the necessary security settings.
8. Click Add.
Add port group settings screen
9. Repeat the steps in this section to create additional port groups on the deployment server as
specified in Table 2.
1.4 Physical switch configuration
Note: For VLAN configuration on the ToR switch, see the Switch configuration for TOR section within the Dell
EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform Deployment Automation
Hardware Guide.
1.5 Create datastore on Deployment server
Follow the below steps to create the datastore on ESXi deployment server:
1. Login into ESXi host using vSphere web client.
2. From the Home screen, select Storage, then click New datastore.
3. On Select Creation Type screen, select Create new VMFS datastore then click Next.
20 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Select creation type Screen
4. In the Name field, enter the name of the datastore, select a non-SSD device, and then click Next.
Select device screen
5. From the Select partitioning options screen, select how you would like to partition the device, then
click Next.
Select partitioning options Screen
6. Review the options selected from the Ready to complete screen and if no changes are required,
click Finish.
21 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
1.6 Deployment Automation VM
In this document, a deployment VM is used to deploy the solution. The CentOS Linux -7.6 automation
deployment VM is used in this guide as a base operating system platform for the deployment of the NFV
infrastructure. This VM will be deployed on deployment server. The automation deployment VM contains the
licenses for VMware software OVA, ISO, and other licenses required for the deployment. The automation
deployment VM also contains Google Chrome, to access the GUI, and SSH utility.
Note: To deploy the VM, ensure the Dell servers and network is accessible.
The automation deployment VM contains code to perform all the steps involving installation, configuration,
and verification of the VMware software stack.
The OVA package contains all the required software packs on the Automation deployment VM to run the
infrastructure successfully. Navigate to /root/Test/vCloud-Nfv-3.1/software to access these software packs:
• Big Switch Network Big Cloud Fabric (BCF) - 4.7.0 – Controller
• AD-DNS-Server.ova
• NTP_VM.ova
• VMware-VMvisor-Installer-6.7.0.update01-10302608.x86_64-DellEMC_Customized
• VMware-VCSA-all-6.7U1
• VMware-vRealize-Log-Insight-4.7
• vRealize-Operations-Manager-Appliance-7.0
• NSX-T Manager 2.3
• nsx-controller-2.3
• nsx-edge-2.3
• VMware-OpenStack-5.1
• VMware-OpenStack-for-Kubernetes-5.1
• contentPacks
• managementPacks
22 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
2 Supported configuration
2.1 Dell EMC NFV hardware for vSAN Ready Node
Dell EMC PowerEdge R640/740 or R740 servers’ minimum required BIOS and firmware versions that are
tested for the Dell EMC Ready Bundle for NFV platform are as follows:
Dell EMC PowerEdge R640/R740 or R740xd firmware versions
Product Firmware version
BIOS 1.4.9
iDRAC with Lifecycle Controller 3.32.32.32
rNDC - Intel(R) 4P X550-t - 10 Gbps 18.5.18
PCIe - Intel(R) 25 GbE 2P XXV710 18.5.17
(Optional) PCIe - Intel(R) 10 GbE 2P X710 18.5.17
(Optional) PCIe - QLogic 25GB 2P QL41262 14.07.07
HBA330 ADP/Mini storage Controller (Dell EMC PowerEdge R740xd/R640)
15.17.09.06
Backplane/Storage Enclosures (Dell EMC PowerEdge R640/R740) 4.26
Backplane/Storage Enclosures (Dell EMC PowerEdge R740xd) 2.25
Note: If QLogic 25GB nics are used the link speed for each of the servers is to be changed from Auto-
negotiate to 25 GB links.
Note: To change the QLogic 25GB link speed from Auto-negotiate to 25 GB links See the section 2 Change
QLogic link speed from Auto-negotiate to 25GB link speed in Dell EMC Ready Solution for VMware vCloud
NFV 3.1 OpenStack Edition Platform Deployment Automation Operations Guide.
Note: All the servers should have clean setup. There should be no partitions on HDD’s and SSD’s.
Dell Networking tested BIOS and firmware versions
Product Instance (count) Version
S4048T-ON firmware out of band management 1 Dell OS 9.14
Z9100-ON or S6010-ON firmware leaf-switch 2 SWL-OS-BCF-4.7.0(0)
Note: For additional hardware configuration and setup information, see the Dell EMC Ready Solution for
VMware vCloud NFV 3.1 OpenStack Edition Platform Deployment Automation Hardware Guide.
23 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
3 Deployment environment prerequisites The Dell EMC and VMware vCloud NFV deployment automation solution is bundled in a Linux-based OVA
file. The Automation deployment VM contains all of the software dependencies required to run the
Orchestrator.
OVA operating system details:
• CentOS 7.6 or above
Prerequisites:
• An additional VMware vSphere Hypervisor (ESXi) server present inside/outside of ESXi server version
6.7 U1.
• Three VMXNET3 network adapter.
• Automation deployment VM with connectivity to stamp specific ToR that can reach stamp networks, ESXi
management, and Corp network.
• Deployment server configured with IPv4 and static IP’s.
• Deployment server has date and time synchronized with pre-existing NTP server.
• Internet proxy settings are disabled on Automation deployment VM.
• USER_INPUT.xlsm file is configured and ready for use, with the appropriate inputs and parameters.
• Verify that the MTU of the physical switch is set to 9216.
Note: For additional information, see the Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack
Edition Platform Deployment Automation User Input File Guide.
24 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
4 Getting started Prior to starting the OVA deployment, verify that:
• The ESXi server is configured as specified in the Deployment server section.
• The physical network topology configuration as per the Dell EMC Ready Solution for VMware vCloud NFV
3.1 OpenStack Edition Platform Deployment Automation Hardware Guide.
4.1 OVA deployment
Prerequisites:
• Deployment server with ESXi 6.7U1
• Port groups must be configured as specified in Create port group on deployment server.
• TOR switch must be configured as specified in Dell EMC Ready Solution for VMware vCloud NFV 3.1
OpenStack Edition Platform Deployment Automation Hardware Guide.
• Datastore should be configured
Follow the below steps to deploy OVA:
1. Open the Deployment server in the browser using vSphere web client, in the navigation pane, right-
click on Host, and select Create/Register VM.
Deployment server
2. On the New virtual machine window, select creation type as Deploy a virtual machine from an OVF
or OVA file, and click Next.
3. Enter virtual machine name and select the OVF and VMDK files or OVA for the VM required to
deploy in the box, then click Next.
4. Select the datastore, and click Next.
5. From Deployment options screen, select the networks VM Network, PG-176 and esxi_mgmt_200
for vnic1, vnic2 and vnic3 respectively and set Disk provisioning to Thin provisioning, then click
Next.
25 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Deployment options
6. Review the configuration, and click Finish.
7. Configure Static IP address using GUI:
a. Use one of the below method to open Network window:
> Click the network icon and then select Network Settings on top-left corner of the screen.
> Navigate to: Applications >System Tools > Settings >Network.
Network window
26 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
b. On the Network window, click the settings icon next to each ethernet adapter to set IPV4
network information.
Settings IPV4
8. Click Apply to update the IP information and exit from the menu.
9. Restart network services for implementing the changes or restart the VM.
Note: Once the VM is ready, from the terminal ping all the gateways of the network adapters attached to the
VM. For example, Ping 100.67.176.254 (for OOB mgmt.) ping 192.168.176.254 (for vm mgmt) and ping
192.168.200.254 (esxi mgmt.).
The OVA deployed on the Deployment server contains all the required software dependencies, an
Orchestrator, and a CLI-based end-user interface. The Orchestrator enables the user to trigger the
automation deployment.
To initialize orchestrator open the terminal window from automation deployment VM and change the directory
to /Test/vCloud-Nfv-3.1/src/orchestration or check the Orchestrator help using the following
command:
#python orchestrator.pyc –-help
27 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
• The Help command displays all of the workflows in sequence and optional arguments:
Orchestrator help verification commands
The Orchestrator supports the following optional preferences:
Optional Orchestrator arguments
• The skipWorkflowsTill <<Workflow Name>> command skips the workflows specified in the
command.
EX- python orchestrator.pyc --skipWorkflowsTill predeployment –-csvInput
“<.csv_File_path>”
4.1.1 Overview of OVA directory file system To access the OVA directory file system:
1. Log in to the CentOS automation VM using the required credentials.
2. Click ESXi host menu > Home > Other Location > Test >vCloud-Nfv-3.1.
In the vCloud-Nfv-3.1 directory, following folders are displayed:
• software
• src
28 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
vCloud-Nfv-3.1 folder
4.1.1.1 Software directory The software directory contains the OVA files of the software invoked by scripts.
Software directory
4.1.1.2 SRC directory The src directory contains the core and orchestration directories as well as the requirements.txt file.
SRC directory
Each of the folders contains the following:
• core: Directory contains common and component-based libraries used by the Orchestrator for
automation deployment.
• orchestration: Directory contains the pyc and .yaml files, as well as logs and workflow directories as
follows:
29 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
▪ orchestrator.pyc: Main script that runs a complete NFV cloud solution.
▪ release.yaml: Consists of build and date information.
▪ Logs: Directory to save the main, summary, and inventory log files.
▪ Config: Directory contains the configuration files for the Orchestrator.
▪ Workflows: Directory contains the code for all of the workflows called by the orchestrator.pyc for the
deployment of NFV components.
• requirements.txt – This file contains the third-party software information required to install the
automation VM and run the Orchestrator.
requirements.txt software information
4.2 VMware license bundle key
VMware licenses can be obtained for each component or a bundle key can be used. VMware provides the
bundle license key for all vCloud NFV components. Use of the VMware bundle key requires a separate
license obtained for each optional VMware component, with each license manually added into the
USER_INPUT.xlsm file by the user.
4.3 User input
The users are required to provide the appropriate input data into the USER_INPUT.xlsm file. The
USER_INPUT.xlsm file will be provided along with OVA package. The input data is logically categorized to
avoid redundancy. Values for some of the parameters are optional and set to a default standard value if, the
user does not configure the values.
The USER_INPUT.xlsm file has for all the components of the Management and Edge and Resource pods
configurations as well as Big Cloud Fabric (BCF) Controller configurations. The Management pod
30 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
components include ESXi, Auxiliary components, VSAN, Network, VRLI, VROPS, NSX-T, VIO, Kubernetes,
and VCSA components.
The resource pod components include ESXi, VCSA, VSAN, NSX-T, and Network components.
NOTE: Please follow the instruction provided in the Dell EMC Ready Solution for VMware NFV Platform
Deployment Automation User Input Template File 3.1 to fill all the required configuration values in the
USER_INPUT.xlsm file.
31 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5 Big cloud fabric OVA deployment BCF script will deploy 2 BCF controller’s VM. User need to configure both the controllers. One will be the active master controller and second will be the standby.
Follow the below steps to deploy the BCF controllers on Deployment server:
1. Login to the Automation deployment VM as instructed in section 4.1.
2. Navigate to the following path in Automation deployment VM :
<root@user>cd /Test/ vCloud-Nfv-3.1/src/orchestration/
Run the following command:
[root@localhost orchestration]# python deployBcfOva.pyc --help
usage: deployBcfOva.pyc [-h] --ipaddress IPADDRESS --username USERNAME --
password PASSWORD --datastore DATASTORE --management_network
MANAGEMENT_NETWORK --pswitch_control_network PSWITCH_CONTROL_NETWORK --
inband_network_ctrl_01 INBAND - 01 INBAND - 02 --inband_network_ctrl_02
INBAND – 03 INBAND - 04 [--ctrl_vm_name CTRL_VM_NAME]
Example:
[root@ localhostorchestration] #python deployBcfOva.pyc --ipAddress
100.67.176.18 --username root --password <PASSWORD> --datastore DATASTORE --
management_network PG-176 --pswitch_control_network BCF --
inband_network_ctrl_01 INBAND - 01 INBAND - 02 --inband_network_ctrl_02
INBAND – 03 INBAND - 04 [--ctrl_vm_name BCF]
3. Descriptions of optional arguments:
4. -h, --help show this help message and exit
--ipaddress Enter IP address of ESXi deployment server --username Enter username of ESXi deployment server --password Enter password of ESXi deployment server --datastore Enter datastore on ESXi deployment server --management_network Enter management network port group name --pswitch_control_network Enter the p-switch control network --inband_network_ctrl_01 Enter 2 inband network port group for controller 1 --inband _network_ctrl_02 Enter 2 inband network port group for controller 2 --ctrl_vm_Name Enter the VM name for BCF controller. Two BCF controllers will be
deployed with [vmName-01] and [vmName-02]
5.1 BCF Configuration
Once the BCF Controllers are deployed, configure the BCF controller as described in this section.
32 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5.2 Configure Big Cloud Fabric (BCF) controllers
5.2.1 Configure the first BCF Controller Follow the below steps to configure the first BCF Controller
1. Open the VM console of the first BCF Controller from Deployment Server.
VM Console
2. Log in as admin (no password). Do you accept the EULA for this product? (Yes/No/View)
displays.
3. Review the contents of the EULA if desired, and enter Yes to continue.
4. Enter and confirm the Emergency recovery user password for the controller.
EULA and Emergency recovery password prompts
5. From the Hostname prompt, enter the hostname for the first controller, for example bcfcntrl1.
6. Under Management network options: select [1] IPv4 only.
7. Enter the values at the corresponding prompts, for example:
• IPv4 address: 192.168.176.90
• IPv4 prefix length: 24
• IPv4 gateway (Optional): 192.168.176.254
• DNS server 1 (Optional): 192.168.176.250
33 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
• DNS server 2 (Optional): not used in this example
• DNS search domain (Optional): dellnfv.com
When complete, the console displays the following screen:
Hostname and management network settings
8. From the Controller cluster options: select [1] Start a new cluster.
9. Enter the cluster name, for example, bcf-cluster.
10. From the Cluster description, enter a description. It is required to add a description.
11. At the Cluster administrator password prompt, enter a password and retype to confirm.
Create a new cluster
12. Under NTP server options: select your preferred option. In this example, [2] Use custom NTP
servers is selected.
13. Press Enter.
34 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
NTP server selection
A summary of the configuration settings displays.
14. Review the settings and select [1] Apply settings.
Configuration summary – first BCF Controller
When complete, the First-time setup is complete! message displays.
Set up complete
15. Press Enter. The controller hostname and log in prompt displays.
Controller login prompt
35 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5.2.2 Configure second BCF Controller Configuration of the second BCF Controller similar as first controller configuration, except that the second
controller joins the existing cluster configured on the first controller.
1. Open the VM console of the second BCF Controller.
2. Log in as admin (no password).
3. Review the terms provided in the EULA and if you agree, then accept the terms.
4. Enter and confirm the Emergency recovery user password.
5. From the Hostname prompt, enter the hostname for the second controller, for example, bcfcntrl2.
6. Under Management network options: select [1] IPv4 only.
7. Enter the values at the corresponding prompts, for example:
• IPv4 address: 192.168.176.91
• IPv4 prefix length: 24
• IPv4 gateway (Optional): 192.168.176.254
• DNS server 1 (Optional): 192.168.176.250
• DNS server 2 (Optional): not used in this example
• DNS search domain (Optional): dellnfv.com
When complete, the console displays the following screen:
Management network configuration – second BCF Controller
Next, the Controller Clustering section is displayed.
8. From the Controller Clustering section, locate the Controller cluster options: select [2] Join an
existing cluster.
9. Enter 192.168.176.90 in the Existing controller address command prompt.
Note: The Existing controller address is the IP address of the first controller.
36 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10. Enter the required password in the Cluster administrator password field.
Note: This password was previously configured on the first controller.
11. Retype the password to confirm.
Joining an existing cluster
The Configuration settings summary for the second controller displays.
12. Review the settings and select [1] Apply settings.
Configuration summary on second BCF Controller
13. When complete, the console displays the following screen:
Note: The message Please verify that: Secure control plane is NOT configured displays. By default, the
secure control plane is not configured.
Applying settings on second controller
37 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
14. Select [1] Continue connecting (the above info is correct) to proceed. When complete, the First-
time setup is complete! message displays.
Settings applied on second controller
15. Press the Enter from the keyboard. The controller login screen for the second controller displays.
16. Log in as admin. The command prompt displays and indicates this controller is in the standby role.
Log in prompt and command prompt on second (standby) controller
Note: The standby controller is read only. Configuration commands made from the command line must be run
on the active controller.
5.2.3 Configure virtual IP address for cluster As a best practice, set a virtual IP (VIP) address for the cluster. This allows you to connect to the
management port of the active node using an IP address that does not change even if the active controller
fails over and the role of the standby controller changes to active.
Follow the below steps to configure the cluster VIP address:
1. Log in to the console of the active controller locally or remotely using secure shell (SSH).
2. Use the following commands to set the cluster VIP address:
>enable
#configure
#controller
#virtual-ip 192.168.176.92 (example)
Setting the cluster virtual IP address
3. Enter the show controller command from either the active or standby controller to verify the
cluster settings.
4. Verify that the Cluster Virtual IP address is correct and that the Redundancy status is redundant.
38 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Show controller command output
5.2.4 Access the BCF GUI The BCF GUI is accessible from a browser by navigating to the VIP address of the cluster.
Note: For more information, see the Deployment and Best Practices Guide for Big Switch Networks Big Cloud
Fabric with VMware NSX.
Follow the below steps to access BCF GUI:
1. In the Automation deployment VM, open the web browser and enter the cluster VIP address. You are
redirected to a secure login page.
Connecting to the BCF GUI
2. Log in as admin using the password created during controller setup. The BCF dashboard displays
like the following:
39 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
BCF dashboard
5.3 Leaf switches configuration using BCF GUI
This section covers the Manual Configuration of BCF based on the Dell EMC Ready Solution for VMware
vCloud NFV 3.1 OpenStack Edition Platform. This section provides step-by-step configuration of BCF
Software Defined Networking (SDN) solution integrated with VMware vSAN.
A BCF consist of logical tenants and logical segments:
• Logical tenants: A logical grouping of layer 2 and layer 3 networks. Tenants are similar in functions as
Virtual Routing and Forwarding (VRF) entity. Each tenant in BCF uses logical router and creates a layer 3
boundary to separates network traffic from other tenants.
• Logical Segments: logical segments are layer 2 networks and like VLANs. These segments consist of
logical ports and endpoints. In each tenant, you can create different segments and establish layer 2
boundaries for each tier.
Pre-requisites:
• Make sure that BCF Controllers are correctly deployed.
5.3.1 Install and configure fabric switches with zero touch fabric Fabric switches are shipped with an ONIE network-enabled boot image. On switch boot up, each switch gets
the switch light operating system software from the BCF Controller This section covers the installation,
configuration, and deployment of BCF switches with zero touch fabric.
Note: Before proceeding, verify that all leaf switches, spine switches, and BCF Controllers are physically
connected to the p-switch network.
40 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5.3.1.1 Zero touch fabric overview Zero touch fabric, or ZTF, uses the Open Network Install Environment (ONIE) boot loader for the automation
of switch installation and configuration. Fabric switches are shipped with an ONIE network-enabled boot
image. On switch boot up, each switch gets the switch light operating system software from the BCF
Controller.
5.3.1.2 Switch compatibility requirements The BCF Controller performs compatibility checks during fabric bring up. If verification fails, switches are
suspended, and the fabric error message is displayed. The following commands on the BCF Controller help
resolve issues and understand the compatibility issues.
show fabric error suspended-switches
show switch all inventory
5.3.2 Provision switches in the BCF Controller Table 5 provides the MAC addresses, switch names, roles, and leaf groups used for the provision switches in
this section.
Switch provisioning details
Model MAC address Switch name Fabric role Leaf group
Z9100-ON XX: XX: XX: XX: XX: XX Leaf1 Leaf Rack1
Z9100-ON XX: XX: XX: XX: XX: XX Leaf2 Leaf Rack1
Note: The BCF Controller CLI or GUI may be used to provision the switches. The GUI is used in this
example.
5.3.3 Switch installation Follow the below steps to bring up the fabric switches:
1. Notate the MAC address of the switch.
Note: The MAC address is usually printed on the top surface of the switch and may also be available on a
plastic pullout.
2. Using a browser, navigate to the VIP address of the BCF Controller cluster and log in.
3. In the BCF GUI, navigate to Fabric tab then from the drop down click on Switches.
4. To provision the leaf switches, click the (+) icon to open the Provision Switch dialog box.
5. Complete the fields by referring values in Table 5:
a. Name - enter the switch name to use for the leaf switch.
b. MAC Address - enter or paste the MAC address corresponding to the leaf switch.
c. Fabric Role - select the Leaf box.
d. Leaf Group - enter the appropriate Leaf Group for the switch.
41 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Note: Defaults are used for the remaining items.
After information for the first leaf switch is entered, the Provision Switch dialog box appears as shown.
Provision switch
6. Click Save and repeat steps from 4 to 6 for the remaining leaf switches.
5.3.4 Verify Switch Light operating system installation To verify successful installation of the Switch Light operating system:
1. Navigate to the BCF Controller GUI.
2. From the BCF Controller GUI, click Fabric, then Switches.
3. Review each of the switches, MAC addresses, names, connection status, fabric status, and fabric
roles.
Switch summary in BCF Controller GUI
Note: If switches remain in not connected state after switch installation goes to switch console and manually
reboot the switches.
42 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5.4 Resolve common warnings and errors
To view current warnings and errors:
1. Navigate to the BCF Controller GUI.
2. From the BCF Controller GUI, click Visibility, then click Fabric Summary, or you can click the red
errors and warnings message indicator in the upper-left corner of the GUI.
Errors and warnings message indicator
Note: From the Fabric Summary page, errors and warnings may be shown or hidden by selecting or
deselecting the category in the left panel.
5.4.1 Resolve suspended switches The Dell EMC Networking Z9100-ON leaf switches may appear under the Suspended Switches section with
the message that ASIC supported as leaf only in forwarding-mode high-bandwidth as shown in Figure
61.
Suspended switches listing
Follow the below steps to set the forwarding mode to high bandwidth for leaf switches:
1. Navigate to the BCF Controller GUI.
2. From the BCF Controller GUI, click Settings, then Fabric Settings.
3. From the Fabric Settings screen, select the Tools icon.
4. In the left-navigation pane of the Fabric Settings dialog box, select Forwarding Mode.
5. In the right-navigation pane, move the High Bandwidth slider to the right. All other sliders are moved
to the left.
Fabric Settings screen
43 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
6. Click Submit.
7. Return to the Visibility > Fabric Summary page to verify that suspended switches are resolved.
5.4.2 Resolve switches with mismatched ONIE and CPLD firmware Some switches may be listed with mismatched ONIE or complex programmable logic device, or CPLD,
firmware.
5.4.2.1 Resolve ONIE switch mismatch Follow the below steps to resolve the ONIE switch mismatch:
1. Scroll down to the Switches with Mismatched ONIE listing and click the switch name.
2. From the switch page, click the Actions tab.
3. On the left side of the page, select Manage Firmware. The Manage Switch Firmware dialog box
displays.
4. Move the CPLD slider to N, and the ONIE slider to Y.
Note: The CPLD and ONIE firmware must be upgraded separately. Dell EMC recommends you upgrade the
ONIE firmware first, followed by the CPLD firmware.
5. Click to select the Reboot switch right away box, then click Upgrade. The switch reboots and ONIE
firmware is updated.
Note: This can be observed in the switch console.
6. Repeat the steps in this section for the remaining switches listed within the Switches with
Mismatched ONIE.
7. Click the Refresh icon on the Visibility and Fabric Summary page to verify that the ONIE issues are
resolved.
After the ONIE mismatch issues are resolved, perform the following steps to resolve the CPLD
mismatches:
8. Scroll down to the Switches with Mismatched CPLD section.
9. Double-click the switch name. The Switch properties screen displays.
10. Click to select the Actions tab.
11. On the left side of the page, select Manage Firmware. The Manage Switch Firmware dialog box
displays.
12. Verify that the CPLD slider is set to Y, and that the other sliders are set to N.
13. Click to place a check in the Reboot switch right away box, then click Upgrade. The switch reboots
and CPLD firmware is updated.
Note: This process can be observed within the switch console and may take 10-20 minutes to complete.
14. After the switch reboots and the CPLD firmware installation is complete, power cycle the switch by
removing all the power cables.
Note: Wait until all LEDs are off for approximately 5-10 seconds before reconnecting the power cables.
15. Repeat the steps for the remaining switches listed in the Switches with Mismatched CPLD.
44 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
16. Click the Refresh icon on the Visibility and Fabric Summary page to verify that the CPLD issues
are resolved.
Updated firmware
5.4.3 Resolve switches without management address Once the leaf switches are configured, an error identifies the leaf switches that require an IP address configuration.
Switches without Management Address listing
Follow the below steps to resolve the error:
1. Navigate to the BCF Controller GUI.
2. From the BCF Controller GUI, click Fabric, then click Switches.
3. Next to the IP Address Allocation listing, click the Tools icon. The Configure Switch IP Allocation
window displays.
4. In the Configure Switch IP Allocation dialog box, move the slider to Enabled.
5. Click the Add (+) icon to open the Create IP Range dialog box.
6. Within the Start IP field, enter the starting IP address.
7. Enter the ending IP address within the End IP address field.
8. From the Subnet Mask Length field, use the arrows to select the desired length for the subnet mask.
Note: The example provided uses the 192.168.176.93-95 range with a subnet mask length of 24.
45 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
IP address range
9. After entering the required ranges, click Append, and then click Submit.
When complete, the IP Address Allocation section of the Fabric > Switches page displays the resolved
IP addresses, with each one used for each leaf switch in the topology.
IP address pool configured
5.4.4 Leaf interfaces not in interface group Interface groups provide active-active load balancing and failover among members of the group. Connected
leaf edge ports that are not configured in groups, display in the Leaf Interfaces Not in Interface Groups
section.
The VMware integration process will automatically configure the interface groups and resolves these
warnings.
46 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
List of leaf interfaces not in interface groups
5.4.5 BCF validation from CLI The commands within this section, help with the validation of the fabric configuration. These commands are
run from the active or standby controller.
5.4.5.1 Show fabric error command The show fabric error command displays fabric errors. The errors are also found within the BCF
Controller GUI on the Visibility > Fabric Summary page under Errors. This command should return None at
this point as shown below.
bcfcntrl1> show fabric error
None.
Note: To see items shown in the GUI on the Visibility > Fabric Summary page under Warnings, run the
show fabric warning command. At this stage of deployment, warnings display for interfaces not
configured in interface groups.
5.4.5.2 Show link command The show link command identifies all inter-switch links that are operational. This includes leaf-to-leaf (peer
links) and leaf-spine links. Links are discovered using Cisco Discovery Protocol (CDP).
Inter-switch links between leafs are used for the topology in this deployment.
bcfcntrl1> show link
47 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
show link command results
5.5 Create Interface group
In this section, logical router interfaces are configured to enable communication between segments and
tenants. Interface groups create a logical route between two components that can be mapped with different
tenants and segments. An interface group can be linked with multiple segments and tenants with different
VLAN IDs. Refer the USER_INPUT.xlms file.
Dell EMC recommends that, create following interface groups specified in Table 6:
Interface Group Port details
Interface group name Switch type
Leaf group
Members Mode Backup mode
For TOR: The ethernet30 interfaces from Leaf 1 and Leaf 2 are each physically connected to a single S4048T-ON which is acting as the core router.
To-Tor Leaf Group Rack 1 Leaf1- ethernet 30 and Leaf2-ethernet 30
LACP LACP
For Management pod: The Ethernet interfaces from Leaf 1 and Leaf 2 are physically connected to each ESXi host.
esxi10.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
Leaf Group Rack 1 Leaf1-ethernet 1:1 Leaf2-ethernet 1:1
LACP LACP
esxi11.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
Leaf Group Rack 1 Leaf1- ethernet 1:2 Leaf2-ethernet 1:2
LACP LACP
esxi12.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
Leaf Group Rack 1 Leaf1- ethernet 1:3 Leaf2-ethernet 1:3
LACP LACP
esxi13.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
Leaf Group Rack 1 Leaf1- ethernet 1:4 Leaf2-ethernet 1:4
LACP LACP
For Edge and Resource pod: The Ethernet interfaces from Leaf 1 and Leaf 2 are physically connected to each ESXi host.
esxi6.dellnfv.com-infra-mgmt-VDS-vmnic4
Leaf Group Rack 1 Leaf1- ethernet 3:1 Static Static
esxi7.dellnfv.com-infra-mgmt-VDS-vmnic4
Leaf Group Rack 1 Leaf1- ethernet 3:2 Static Static
48 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
esxi8.dellnfv.com-infra-mgmt-VDS-vmnic4
Leaf Group Rack 1 Leaf1- ethernet 3:3 Static Static
esxi9.dellnfv.com-infra-mgmt-VDS-vmnic4
Leaf Group Rack 1 Leaf1- ethernet 3:4 Static Static
esxi6.dellnfv.com-infra-mgmt-VDS-vmnic6
Leaf Group Rack 1 Leaf2- ethernet 3:1 Static Static
esxi7.dellnfv.com-infra-mgmt-VDS-vmnic6
Leaf Group Rack 1 Leaf2- ethernet 3:2 Static Static
esxi8.dellnfv.com-infra-mgmt-VDS-vmnic6
Leaf Group Rack 1 Leaf2- ethernet 3:3 Static Static
esxi9.dellnfv.com-infra-mgmt-VDS-vmnic6
Leaf Group Rack 1 Leaf2- ethernet 3:4 Static Static
esxi6.dellnfv.com-Overlay-Network-VDS-vmnic5
Leaf Group Rack 1 Leaf1- ethernet 7:1 Static Static
esxi7.dellnfv.com-Overlay-Network-VDS-vmnic5
Leaf Group Rack 1 Leaf1-ethernet 7:2 Static Static
esxi8.dellnfv.com-Overlay-Network-VDS-vmnic5
Leaf Group Rack 1 Leaf1-ethernet 7:3 Static Static
esxi9.dellnfv.com-Overlay-Network-VDS-vmnic5
Leaf Group Rack 1 Leaf1-ethernet 7:4 Static Static
esxi6.dellnfv.com-Overlay-Network-VDS-vmnic7
Leaf Group Rack 1 Leaf2- ethernet 7:1 Static Static
esxi7.dellnfv.com-Overlay-Network-VDS-vmnic7
Leaf Group Rack 1 Leaf2--ethernet 7:2 Static Static
esxi8.dellnfv.com-Overlay-Network-VDS-vmnic7
Leaf Group Rack 1 Leaf2-ethernet 7:3 Static Static
esxi9.dellnfv.com-Overlay-Network-VDS-vmnic7
Leaf Group Rack 1 Leaf2-ethernet 7:4 Static Static
esxi6.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
Leaf Group Rack 1 Leaf1- ethernet 11:1 Leaf2- ethernet 11:1
LACP LACP
esxi7. dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
Leaf Group Rack 1 Leaf1- ethernet 11:2 Leaf2- ethernet 11:2
LACP LACP
esxi8. dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
Leaf Group Rack 1 Leaf1- ethernet 11:3 Leaf2- ethernet 11:3
LACP LACP
esxi9. dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
Leaf Group Rack 1 Leaf1- ethernet 11:4 Leaf2- ethernet 11:4
LACP LACP
49 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
To create an interface group:
Note: See the Table 6 to create an interface group and enter the values in various required fields.
1. In the BCF GUI, navigate to Fabric > Interface Groups.
2. On the Interface Groups page, click the icon to open the Create Interface Group screen.
3. On the Create Interface Group screen, fill the fields as per the followings:
a. Name: Enter the name of interface group.
b. Description: Enter a brief description about interface group.
c. Admin Status: Move the Slider to Up.
d. Mode: From the drop-down list, select one of following mode:
> Static: To create static mode interface group.
> LACP: To create a Link Aggregation Control Protocol (LACP) interface group.
e. Backup Mode: From the drop-down list, select one of following mode:
> Static: To create backup mode as a static mode interface group.
> LACP: To create backup mode as a LACP interface group.
f. Click Next.
Create Interface Group Info tab
4. From the Leaf Group drop-down list select Rack 1 to display the leaf switches.
5. Click or drag the ports to add in Members section.
The ports added in the Members section will be connected logically.
Note: For example, Ethernet30 is selected under Leaf1 and Leaf2 and these ports are physically connected
with ToR. This will create a logical routing between Leaf1, Leaf2, and ToR.
50 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Leaf switches
6. Click Save to create interface group.
Note: Repeat the above steps to create interface groups use the information specified in Table 6.
5.6 Creating tenants
We recommended that you must create following tenants:
• Tenant_mgmt: For management pod
• Tenant_res: For Edge and Resource pod
To create tenants:
1. In the BCF GUI, navigate to Logical > Tenants.
2. On the Tenants page, click the icon to open the Create Tenants screen.
51 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Create Tenant screen
3. Enter the tenant name in the Name field.
4. Enter a brief description about tenant in the Description field.
5. Slide the Multicast toggle button to enable multicast for tenant.
6. Click Save to create tenant.
Tenant
Note: Repeat the above steps in this section to create more Tenants.
52 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5.7 Creating segments
Dell EMC recommends that, create following Segments specified in Table 7:
Segment details
Tenants Segment name QoS traffic class
Tenant_mgmt Segment-vMotion-400 Traffic Class 0
Tenant_mgmt Segment-vSAN-300 Traffic Class 0
Tenant_res Segment-To-Tor Traffic Class 0
Tenant_res Segment-To-Leaf Traffic Class 0
Tenant_res Segment-vMotion-res-400 Traffic Class 0
Tenant_res Segment-vSAN-res-300 Traffic Class 0
Tenant_res Segment-edge-overlay-700 Traffic Class 0
To create Segments:
Note: See the Table 7 to create Segments and enter the values for Tenants, Name, and Traffic Class.
1. In the BCF GUI, navigate to Logical > Segments.
2. On the Segments page, click the icon to open the Create Segments screen.
Create Segments screen
3. From the Tenant drop-down list, select a tenant to link with segments.
4. Enter the segment name in the Name field.
5. Enter a brief description about segment in the Description field.
6. From the QoS Traffic Class drop-down list, select Traffic Class 0.
53 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Traffic class
7. Click Next, then click Save to create segment.
Note: Repeat the above steps to create Segments specified in Table 7.
5.7.1 Link the segment to Interface group Link each Segment with Interface Groups as specified in Table 8:
Link segments to interface groups
Segment name Interface group name VLAN ID
Segment-vMotion-400 esxi10.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 400
esxi11.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
esxi12.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
esxi13.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
Segment-vSAN-300 esxi10.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 300
esxi11.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
esxi12.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
esxi13.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
Segment-To-Tor Tor 30
Segment-vMotion-res-400
esxi6.dellnfv.com-infra-mgmt-VDS-vmnic4 400
esxi7.dellnfv.com-infra-mgmt-VDS-vmnic4
esxi8.dellnfv.com-infra-mgmt-VDS-vmnic4
54 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
esxi9.dellnfv.com-infra-mgmt-VDS-vmnic4
esxi6.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi7.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi8.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi9.dellnfv.com-infra-mgmt-VDS-vmnic6
Segment-vSAN-res-300
esxi6.dellnfv.com-infra-mgmt-VDS-vmnic4 300
esxi7.dellnfv.com-infra-mgmt-VDS-vmnic4
esxi8.dellnfv.com-infra-mgmt-VDS-vmnic4
esxi9.dellnfv.com-infra-mgmt-VDS-vmnic4
esxi6.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi7.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi8.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi9.dellnfv.com-infra-mgmt-VDS-vmnic6
Segment-edge-overlay-700
esxi6.dellnfv.com-infra-mgmt-VDS-vmnic4 700
esxi7.dellnfv.com-infra-mgmt-VDS-vmnic4
esxi8.dellnfv.com-infra-mgmt-VDS-vmnic4
esxi9.dellnfv.com-infra-mgmt-VDS-vmnic4
esxi6.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi7.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi8.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi9.dellnfv.com-infra-mgmt-VDS-vmnic6
esxi6.dellnfv.com-Overlay-Network-VDS-vmnic5
esxi7.dellnfv.com-Overlay-Network-VDS-vmnic5
esxi8.dellnfv.com-Overlay-Network-VDS-vmnic5
esxi9.dellnfv.com-Overlay-Network-VDS-vmnic5
esxi6.dellnfv.com-Overlay-Network-VDS-vmnic7
esxi7.dellnfv.com-Overlay-Network-VDS-vmnic7
esxi8.dellnfv.com-Overlay-Network-VDS-vmnic7
esxi9.dellnfv.com-Overlay-Network-VDS-vmnic7
Segment-To-Leaf esxi6.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
30
55 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
esxi7.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
esxi8.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
esxi9.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
To link the interface group with segments, follow the below steps:
Note: See the Table 8 to link the segments with interface groups and enter the values for Interface Groups,
and VLAN.
1. In the BCF GUI, navigate to Logical > Segments.
2. On the Segments page, click the segment name. In this example we have used Segment-vSAN-res-
300.
Segments
3. In the Interface Group Membership section, click the icon.
Interface group membership
4. On the Create Interface Group Membership Rule window, enter the details as follows:
a. From the Interface Group drop-down list, select an interface group to link with segment.
b. In the VLAN field, enter the VLAN ID for interface group.
c. In the Rule Description field, add a description.
d. Click Save.
56 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Create Interface Group Membership Rule
The interface group is added to the segment.
Note: Repeat the above steps to add multiple interface group in all segments as provided in Table 8.
5.8 BGP configuration
In this section, BGP is configured on the BCF tenant logical routers for adjacency with their respective
routers.
Pre-requisites:
• BGP must be configure on the TOR
To Configure BGP:
1. In the BCF GUI, navigate to Logical > Tenants and click on Tenant-res to configure BGP.
2. In the left pane under Logical Router, ensure that Routes is selected.
3. Under the Border Gateway Protocol, click Configuration under BGP - Gateway Protocol.
Navigation pane
The BGP Configuration is added to the right pane.
57 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
4. On the BGP Configuration section, click the icon to open the BGP Configuration screen.
5. On the BGP Configuration screen:
a. In the Router IP Address field, set the router IP to 172.16.61.1
b. In the Local Autonomous System ID field, set the local AS no. to 64502
c. In the IPv4 Address field, set the IP to 172.16.61.1
d. Click Next.
BGP Configuration Info tab
6. On the Options tab:
a. Set the Max Parallel Routes Installed Per Route to 2. This will determine the number of parallel
routes can be install per route.
b. Move the slider to Yes and enable the following options:
> Push Connected Routes to Remote Router
> Redistribute Statically Configured Routes
> Log Updates from BGP Neighbor
c. Enable the Graceful Restart option by moving the slider to Enabled.
d. Click Submit.
BGP Configuration Options tab
58 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5.8.1 Creating Neighbors Dell EMC recommends that, create following Neighbors specified in Table 9:
Neighbors details
Name IP address Remote autonomous system ID
BGP connect time
Hold time after restart
Store inbound routing table updates
eBGP hop control
Hop count
Configure next hop self
Inbound/ Outbound route map
TOR 192.168.30.254 65001 60 180 Yes eBGP Multihop
2 No No Route
NSX-Tier0
172.16.60.10 65002 60 180 Yes eBGP Multihop
2 No No Route
VIO-K-Tier0
172.16.60.20 65003 60 180 Yes eBGP Multihop
2 No No Route
To create neighbors:
Note: See the Table 9 to add the Neighbors and enter the values in various required fields.
1. On the same tenant page, in the BGP Neighbors section, click the icon to open the Create BGP
Neighbor window.
a. Enter the neighbor name in the Name field.
b. Enter the neighbor description in the Description field.
c. Enter the neighbor IP address in the IP Address field.
d. Enter the unique remote autonomous ID in the Remote Autonomous System ID field.
e. Click Next.
Create BGP Neighbor Info tab
2. On the Options tab:
59 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
a. Set the BGP Connect Time and Hold Time After Restart.
b. Move the Store Inbound Routing Table Updates slider to Yes.
c. In the eBGP Hop Control section, select eBGP Multihop, then set the Hop Count to 2.
d. Click Next.
Create BGP Neighbor Options tab
3. Click Save to create BGP Neighbor.
Note: Repeat the steps in this section to create BGP Neighbors as provided in Table 9.
5.8.2 Segment interface Dell EMC recommends that you add following segments to the Segment Interface as specified in Table 10:
Segment interface details
Segment name IP CIDR Prefix
Segment-To-Tor 192.168.30.2 24
Segment-To-Leaf 172.16.60.2 24
Segment-vMotion-res-400 192.200.4.254 24
Segment-vSAN-res-300 192.200.3.254 24
Segment-edge-overlay-700 192.200.7.254 24
60 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
To configure segment interfaces:
Note: See the Table 10 to add the segment interface and enter the values in various required fields.
1. In the BCF GUI, navigate to Logical > Tenants and click the name of the required tenant to open the
tenant configuration page.
2. In the left pane scroll-down and click Segment Interfaces.
Segment Interfaces
The Segment Interfaces is added to the right pane.
3. On the right pane, in the Segment Interfaces section, click the icon. The Create Logical
Segment Interface window opens:
Create Logical Segment Interface screen
4. From the Logical Segment drop-down list, select the desired segment to configure, and click Next.
5. On the Subnet tab, click the icon. The Create Subnet window opens.
61 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Create Subnet screen
6. Enter the segment interface IP Address and Prefix. The value in subnet mask field will automatically
populate.
7. Click Append to add subnet, then click Next.
8. Click Save to add segment interface.
Note: Repeat the above steps to create multiple Segment Interfaces as specified in Table 10.
See Appendix A for more details on complete running configuration Leaf switches.
5.9 Configure remote log server for BCF
You can configure BCF to send log messages to a remote logging server.
Prerequisites:
• Configured vRLI server to receive the logs.
Procedures:
1. Log in to the BCF GUI.
2. Click Settings and then click Logging.
3. Click Remote logging and toggle the Switch remote to Yes.
4. Click + to add syslog server.
5. In the field provided, enter the following hostname: vrli123.dellnfv.com
6. In the Port no: field, enter 514.
7. From the Log level: field, select Warning.
62 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5.10 Update cutsheet values
NOTE: Once the configuration is done, you must enter the BCF controllers IP, VIP, BGP, and other
information using the information in the USER_INPUT.xlsm file as shown in Figure 86 and in Figure 87.
Environment Details under configuration tab
Routing info under Network Configuration tab
63 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
6 Run deployment orchestrator The Automation deployment VM contains executables solution that can be triggered using the orchestrator
provided with the Automation deployment VM.
Note: Before running orchestrator make sure TOR and Leaf switches are configured properly.
Parameters can be changed from the USER_INPUT.xlsm file provided with ova package.
To run the Orchestrator, perform the following steps:
1. Login to the Deployment VM as instructed in OVA deployment.
2. Navigate to the operating system CLI and change the directory to the Solution folder:
<root@user>cd /Test/ vCloud-Nfv-3.1/src/orchestration/
Following options are available to run orchestrator:
1. To execute the complete NFV deployment, run the following command:
[root@localhostorchestration] # python orchestrator.pyc –-csvInput
“<.csv_File_path>”
2. To skip the workflows till the workflow specify the workflow name in the command:
[root@localhostorchestration] # python orchestrator.pyc --
skipWorkflowsTill <workflowName> –-csvInput “<.csv_File_path>”
3. To list the workflows supported, run the following command: [root@localhostorchestration]
# python orchestrator.pyc –-help
64 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
7 Logging and reporting This section provides information to monitor the live deployment status and how to reference the log files. The
status of each workflow and its tasks are displayed on the terminal and are pushed to the logs.
This solution generates the following log files:
• Main log – The main log file contains all of the detailed logs with the log level set as DEBUG. These logs
provide detailed information when more logs at task level are required.
• Summary log – The summary log file contains high level logs with the log level set as INFO. Summary
logs display the tasks executed in workflow.
• Inventory log – The inventory log file contains summarized information of inventory files such as
iDRACs, ESXis, and other components in tabular format. These logs provide detailed picture of all
inventory components, access details and their distribution across Management and Edge/Resource
POD.
For every new execution or retrigger (in case of failure), new log files are created with the latest time stamp.
Logs are located in the src/orchestrator/logs folder.
In case of mail log, the log levels can be configured. The following log levels are supported:
• Critical
• Exception
• Error
• Warning
• Info
• Debug
7.1 Sample main log file structure
The following is a sample of the main log file structure:
Sample main log file structure
65 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
7.2 Sample summary log file structure
The following is a sample of the summary log file structure:
Sample summary log file structure
7.3 Sample inventory log file structure
The following is a sample of the inventory log file structure:
Sample inventory log file structure
66 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
8 Supported features
8.1 Remediation
Remediation allows the end user to manually remediate a workflow after a failure occurs during orchestration,
so that the deployment may continue.
For manual remediation steps, see the Table 12 for the supported remediation workflows and necessary
clean-up and mapped playbook section.
If an error occurs while executing a workflow on the console, it requires instruction inputs from the user to
either continue or terminate the deployment. The continue option requires validation to verify the success of
manual remediation and resumes the workflow.
8.1.1 Non-supported workflows
Note: Errors encountered during the execution of the workflows provided below, require the user to debug the
error from the logs by entering the appropriate command to terminate the deployment.
The following workflows are not supported for remediation:
Non-supported workflows
Workflow name Workflow description
preDeployment Pre deployment validation workflow.
preparevsan Prepare vSAN workflow.
auxservices Auxiliary Services (AD, DNS, NTP) workflow.
mgmtvnetwork Management vNetwork workflow.
mgmtvsan Management vSAN workflow.
resourcevnetwork Resource vNetwork workflow.
resourcevsan Resource vSAN workflow.
vcsaha VCSA HA workflow
nsxtconfiguration NSX-T Configuration Workflow
vrlivropsvio vRealize Log Insight (vRLI), vRealize Operations Manager (vROps) and VMware Integrated OpenStack (VIO) workflow.
kubernetes Kubernetes workflow
67 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
8.1.2 Supported workflows Table 12 contains the details for remediation supported workflows.
Note: If the user is not able to debug the issue as mentioned in Action Column, a manual cleanup is required
to delete the VM deployed in that workflow. After that need to follow the Remediation steps mentioned in
Section for remediation column.
Note: For timeout errors that occur during the execution of a workflow, verify the manual remediation steps
and enter the yes command to continue with the next workflow.
Supported workflows
Workflow Name Section for remediation Action
ESXI Workflow (esxi) See the ESXi installation and configuration section within the ESXi installation and configuration.
Debug the issue and remediate the steps accordingly.
Management VCSA workflow (managementvcsa)
See the following sections:
• Stage 1: Deploy OVA file for Management vCenter Server Appliance with embedded PSC
• Stage 2: Set up Management vCenter Server Appliance with embedded PSC
• Change vSAN default storage policy for management vCSA
• Assign license to Management vCSA
• Add AD authentication for Management vCSA
Debug the issue and remediate the steps accordingly or delete the VM’s and follow the remediation steps.
Resource VCSA workflow (resourcevcsa)
See the following sections
• Stage 1: Deploy OVA file for Resource vCenter Server Appliance
• Stage 2: Set up resource vCenter Server Appliance with embedded PSC
• Change VMware vSAN default storage policy for resource vCSA
• Assign license to Resource vCSA
• Add AD authentication for Resource vCSA
Debug the issue and remediate the steps accordingly or delete the VM’s and follow the remediation steps.
NSX-T Deployment Workflow (nsxtdeployment)
See the NSX-T deployment and setup Debug the issue and remediate the steps
68 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
accordingly or delete the VM’s and follow the remediation steps.
Resource Pod post Configurtation Workflow (respodpostconfig)
See the following sections
• Enable VMware enhanced vMotion compatibility
• Enable vSphere DRS
• Enabling vSphere availability
• Set up anti-affinity rules
• Create an anti-affinity rule
Debug the issue and remediate the steps accordingly.
Management Pod Post Configuration Workflow (mgmtpodpostconfig)
See the following sections
• Enable VMware enhanced vMotion compatibility
• Enable vSphere DRS
• Enabling vSphere availability
• Set up anti-affinity rules
• Create an anti-affinity rule
Debug the issue and remediate the steps accordingly.
8.2 Scalability
The scalability feature supports expansion of ESXi servers on both Management and Edge/Resource pods
from a minimum of four servers, to a maximum of 64 servers. The Deployment Automation supports four
servers in the Management Pod, and a range of 4-64 servers for the Edge/Resource pod.
Note: If additional Management Pod Servers are required, this may be done after completion of Deployment
Automation.
For scalability, users must provide the details using the IP range of all ESXi servers to be deployed in the 4-
Server Management Pod and 4-to-64-Server Edge/Resource Pod. With the ESXi and iDRAC server details to
indicate where ESXi servers are deployed. All ESXi servers are deployed as a scalability feature to contribute
to vSAN Datastore and ESXi host cluster on both the 4-Server Management and 4-to-64-Server
Edge/Resource pods.
8.3 Cleanup
If the setup is partially deployed, all ESXi hosts are UP, vSAN datastore is created, and configured to run from
scratch, then perform the cleanup process before triggering the deployment. A separate
orchestrator_cleanup.pyc script will perform the cleanup by deleting all VMs from all ESXi hosts and vSAN
datastore.
69 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Follow the below steps to execute the script cleanup:
1. Log in to the CentOS automation VM.
2. Open the terminal and navigate to the following file path: <root@user>cd /Test/vCloud-Nfv-
3.1/src/orchestration
3. Run the following command: python orchestrator_cleanup.pyc –-csvInput
“.csv_file_path”.
Note: SSH must be enabled on all ESXi hosts for the cleanup script to run successfully.
In case cleanup script fails to cleanup vSanDatatore, ssh to each esxi using ssh utility and run the below
commands:
#esxcli vsan storage list
#esxcli vsan storage remove -s [SSD-DEVICE-ID] {Only for IS SSD: true}
#esxcli vsan cluster leave
70 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
9 Additional configuration requirements
9.1 Time zone
By design, the VMware ESXi only supports the UTC time zone. Time zones other than UTC are not
supported.
9.2 Internet connectivity to stamp
Internet connectivity is required for Microsoft Windows activation. The following components use Microsoft
Windows OVA:
• Auxiliary component (AD/DNS/NTP)
71 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10 Supported workflows for Remediation This section describes to users how to manually remediate a workflow after a failure occurs during
orchestration, so that the deployment may continue.
If an error occurs, while executing a workflow on the console, then it requires instruction inputs from the user
to either continue or terminate the deployment. The continue option requires validation to verify the success of
manual remediation and resumes the workflow.
This guide includes Manual steps for Supported workflows for remediation in automation.
Note: Use the information within the USER_INPUT.xlsm file to provide correct values.
10.1 ESXi installation and configuration
To create an NFV infrastructure, install ESXi on the Dell EMC PowerEdge R640 and R740xd servers based
on vSAN Ready Node.
10.1.1 Install ESXi on Dell EMC PowerEdge R640 and R740xd servers using
iDRAC9 Prerequisites:
• Verify the minimum required hardware firmware versions on the servers as follows:
Product Firmware version
BIOS 1.4.9
iDRAC with Lifecycle Controller 3.32.32.32
rNDC - Intel® 4P X550-t - 10 Gbps 18.5.18
PCIe - Intel® 25 GbE 2P XXV710 18.5.17
(Optionl) PCIe - Intel® 10 GbE 2P X710 18.5.17
(Optional) PCIe - QLogic 25GE 2P QL41262 14.07.07
HBA330 ADP/Mini storage Controller (Dell EMC PowerEdge R640 and R740xd)
15.17.09.06
Backplane (Dell EMC PowerEdge R740xd) 4.26
• If QLogic 25GE nics are used the link speed for each of the servers is to be changed from Auto-
negotiate to 25 GB links.
72 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Note: To change the Qlogic 25GB link speed from Auto-negotiate to 25 GB links See the section 2 Change
Qlogic link speed from Auto-negotiate to 25GB link speed in Dell EMC Ready Solution for VMware vCloud
NFV 3.1 OpenStack Edition Platform Deployment Automation Operations Guide.
• ESXi Installer 6.7U1 or later ISO file
• iDRAC with at least 16 GB SD Card enabled
Note: Verify that the storage controller or host bus adapter is in pass-through mode.
Follow the below steps to install ESXi:
1. Use the designated URL to launch iDRAC 9.
2. Enter the required user name and password in the fields provided, then click Log In:
iDRAC login page
3. On the Dashboard screen, in the Virtual Console section click Launch Virtual Console.
73 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
iDRAC 9 dashboard screen
The iDRAC Virtual Console screen displays.
4. On the navigation bar, click Connect Virtual Media.
iDRAC 9 Virtual Console window
5. On the Virtual Media screen, in the Map CD/DVD section, click Choose File and select the ESXi
image file from your local machine, and click Map Device then click Close.
Connect Virtual Media Screen
74 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
6. From the navigation bar, select Next Boot, then select Virtual CD/DVD/ISO and save it.
Next Boot select screen
7. From the navigation bar, click Power then select Power Cycle System (cold boot).
Power Cycle System (cold boot) selection.
The ESXi installation process begins.
8. From the Welcome to the VMware ESXi 6.7 Installation screen, press Enter to continue.
VMware ESXi installation screen
9. Review the contents of the End User License Agreement (EULA) and if you agree to the terms,
press F11.
10. From the disk installation screen, use the arrow keys to highlight the Dell Internal Dual SD storage
device, then press Enter.
75 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Disk selection screen
11. From the Confirm Disk Selection window, press Enter to confirm the selection.
12. Verify that US Default is the option select in the Keyboard layout window, then press Enter.
13. In the Enter a root password window, enter the desired root password, enter it again to confirm,
then press Enter. The Scanning system… window displays.
Scanning system status screen.
14. From the Confirm Install window, press F11 to confirm the installation.
Fig- confirm installation screen
15. When the installation is complete, press Enter in the Installation Complete window to reboot the
system.
Installation Complete confirmation window
Once the system reboot completes, the Direct Console User Interface (DCUI) screen displays.
76 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Direct Console User Interface (DCUI) screen
10.1.2 Customize ESXi The System Customization screen allows users to customize various ESXi system settings such as:
• Passwords
• Management configuration
• Restart options
• Keyboard settings
• Troubleshooting options
• System reset configurations
Follow the below steps to access the System Customization screen:
1. From the DCUI screen, press F2.
2. Provide the required user credentials in the fields provided then press Enter.
DCUI screen authentication
3. On the System Customization screen, use the arrows to highlight the option to customize, then
press Enter.
77 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
System customization screen
10.1.2.1 Management network configuration 1. From the System Customization screen, use the arrow keys to highlight Configure Management
Network, then press Enter.
2. To update the network adapters, use the arrow keys to highlight the Network Adapter option, then
press Enter.
3. Use the arrow keys to select the appropriate network adapter for the host default management
network connection and use two or more adapters to fault tolerance and load balancing.
Connected Network Adapters
10.1.2.2 Set VLAN ID for ESXi management network (optional) As an option, the VLAN ID for the ESXi management network can be set. Follow the below steps to set the
VLAN ID for the ESXi management network:
1. On the System Customization screen, select Configure Management Network then select VLAN,
and press Enter. (Optional).
2. Enter the configured VLAN ID, and press Enter to save the change.
VLAN ID setting
78 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10.1.2.3 Change IPv4 configuration Follow the below steps to configure the static IPv4 address:
1. On the System Customization screen, select Configure Management Network using the
keyboard, highlight the IPv4 Configuration listing, then press the Enter key using the keyboard,
highlights the IPv4 Configuration listing, then press the (Enter) key.
IPv4 configuration
2. From the options provided, select Set static IPv4 and network configuration and enter the required
IPv4 address, subnet mask, and default gateway.
3. Press Enter to save the changes made.
Setting static IPv4 address
10.1.2.4 Change DNS configuration Follow the below steps to change the DNS configuration:
1. On the Configure Management Network screen, use keyboard arrow keys to select DNS
Configuration then press Enter.
79 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
DNS Configuration option
2. From the DNS Configuration screen, use the arrow keys to select Use the following DNS server
addresses and hostname: option.
3. In the fields provided, enter the required Primary DNS Server, Alternate DNS Server IP and
Hostname information in the fields provided, then press Enter.
Adding DNS server and hostname
4. Add the domain name in the Suffixes field then press Enter to save the settings.
Setting DNS suffix
5. Press Enter to restart the management network.
80 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Restart of Management Network option
6. Once the network restarts, select the Test Management Network and press Enter. This pings the
configured default gateway, primary and alternate DNS servers, and resolve the configured
hostname.
10.1.2.5 Troubleshooting options Follow the below steps to troubleshoot ESXi issues:
1. Use the keyboard to highlight the Troubleshooting Options then press Enter.
2. From the options provided, use the arrow to highlight the desired troubleshooting option, then press
Enter.
10.1.2.6 Assign license to ESXi 1. From a browser, open ESXi and click Manage, then select Licensing.
2. In the License key field, enter the required license key and click Check license.
Assign ESXi license
3. Click Assign license then click Close.
81 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
ESXi license assignment screen
10.1.2.7 Set SSH policy To set SSH policy:
1. Navigate to your ESXi server’s embedded host client using the IP address or domain name.
2. From the left navigation bar, select Manage to access the settings for your host.
3. Select the Services tab then highlight the TSM-SSH service (SSH).
4. Right-click on a service or click the Actions menu item to set the policy to Start and stop with host.
Service tab
82 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10.1.2.8 Set firewall rules Perform the following steps to set the firewall rules:
1. SSH to the ESXi Host with valid credentials.
2. Run the following command to disable firewall rule:
esxcli network firewall set --enabled false
3. Run the following command to get the status:
esxcli network firewall get
Firewall rules
Note: Repeat the steps provided in the ESXi installation and configuration section to install and configure the
remaining ESXi servers.
10.2 VMware vCenter Server Appliance deployment
In the vCloud NFV 3.1 architecture, the deployment of the vCenter Server Appliance is done in two stages.
The first deployment is for the VCSA instance that manages the management cluster, and the second VCSA
deployment manages the resource cluster.
Prerequisites:
• ESXi 6.7U1 is configured and in the running state
• AD, DNS, and NTP are running
• Manual creation of forward and reverse lookup entries for all VCSA instances on the DNS server before
deployment
• See the Inventory layout table in section Solution archive within the Dell EMC Ready Solution for VMware
vCloud NFV 3.1 OpenStack Edition Platform Deployment Automation Hardware Guide to deploy VCSA
on ESXi server
10.2.1 Stage 1: Deploy OVA file for Management vCenter Server Appliance with
embedded PSC Stage 1 of the deployment process involves the deployment of the OVA file included in the vCenter Server
Appliance installer, as a vCenter Server Appliance with an embedded PSC. To deploy the OVA file:
1. Mount the VMware-VCSA-all-6.7.x ISO on Windows VM Deployment server.
2. In the vCenter Server Appliance installer, navigate to the vcsa-ui-installer directory, then to the
subdirectory for your operating system, and run the installer executable file.
a. For Microsoft Windows, go to the win32 subdirectory, and run the installer.exe file.
83 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
b. For Linux, go to the lin64 subdirectory, and run the installer file.
3. From the vCenter Server Appliance 6.7U1 Installation screen, click Install.
Stage 1 Deploy appliance screen
4. From the Introduction screen, review the installation overview then click Next.
Introduction screen
5. Review the information provided within the End User License Agreement (EULA) and if you agree
to the terms, clicks Next.
6. On the Select deployment type screen, select Install vCenter Server with an Embedded Platform
Services Controller from the Embedded Platform Services Controller section, then click Next.
84 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Select deployment type screen
7. On the Appliance deployment target screen, enter the ESXi host, HTTPS port, user name, and
password in the fields provided, then click Next.
Appliance deployment target screen
8. Review the contents of the Certificate Warning, then click Yes to accept the certificate.
Certificate Warning screen
9. From the Set up appliance VM screen:
85 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
a. In the VM name field, enter the VM name specified for the vCenter Server Appliance.
b. In the Set root password, create a root password.
c. In the Confirm root password, re-enter the password, then click Next.
Set up appliance VM screen
Note: The appliance name can contain upper and lower case letters, however special characters such as %,
\, or / are not permitted. The appliance name must not exceed 80 characters in length.
10. From the Select deployment size screen, use the drop-down from the Deployment size and
Storage size sections to select the sizes necessary for the vCenter Server Appliance for your
vSphere inventory, then click Next.
Select deployment size screen
86 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
11. From the Select datastore screen, select the Install on an existing datastore accessible from the
target host radio button, then:
a. Check the Enable Thin Disk Mode box.
b. Enter the Datacenter and Cluster Name then click Next.
Select datastore screen
12. On the Configure Network Settings screen, enter the appropriate network settings and click Next.
Configure network settings screen
87 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
13. From the Ready to complete stage 1 screen, review the deployment settings for the vCenter Server
Appliance and click Finish.
Ready to complete stage 1 screen
The OVA deployment process starts.
Stage 1 status window
14. Once the OVA deployment is complete, click Continue to proceed with the stage 2 deployment
process.
Stage 1 completion and continuation screen
88 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10.2.2 Stage 2: Set up Management vCenter Server Appliance with embedded PSC Once the OVA deployment is complete, you are redirected to stage 2 of the deployment process to set up and
start the services of the newly deployed vCenter Server Appliance with an embedded Platform Services
Controller.
1. Review the information within the Introduction to stage 2 page, then click Next.
Stage 2 Introductory screen
2. On the Appliance configuration screen, click the Time synchronization mode drop-down and
select Synchronize time with NTP servers., then enter the NTP server IP address in the NTP
Servers field:
Appliance configuration screen
Note: From this screen, the option to enable remote SSH access to the appliance is provided.
89 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
3. On the SSO configuration screen, enter the vCenter Single Sign-On domain name and
administrator password, then click Next.
SSO configuration screen
4. On the Configure CEIP screen, if interested in VMware Customer Experience Improvement
Program, check the box and click Next.
5. On the Ready to complete page, review the configuration settings for the vCenter Server Appliance,
click Finish, and then click OK. Stage 2 of the deployment process and set up of the appliance is
complete.
Ready to complete screen
The Appliance setup is initiated.
90 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
6. Once the initial setup is complete, you can click the client to go to the vSphere Web Client and log in
to the vCenter Server instance in the vCenter Server Appliance, or you can click the
https://<vcenter_server_appliance_fqdn>:443 link to access the vCenter Server Appliance Getting
Started page. Otherwise, click Close to exit the wizard.
Stage 2 completion screen
10.2.3 Change vSAN default storage policy for management vCSA Prior to deploying additional VMs to the system, you must first update the VMware vSAN storage policy.
1. Using the required credentials, log in to the VMware vCenter Server Virtual Appliance (vCSA) web
client.
2. Click the Home icon and select Policies and Profiles.
3. From the navigation pane, select vSAN Default Storage Policy, and click the Manage tab.
4. Select the Rule-set 1:VSAN option.
vSAN Default Storage Policy screen
5. Click the Edit button. The Rule-set 1 screen displays.
6. Locate the Primary level of failures to tolerate option and enter 0 in the field provided.
7. From the Force provisioning drop-down, click to select Yes.
91 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Rule-set 1 settings screen
8. Click OK to save the changes.
Rule-set 1 change confirmation
10.2.4 Assign license to Management vCSA Follow the below steps to assign licenses:
1. Log in to the Management vCenter through vSphere Web Client.
2. Click the Home icon and select Administration.
3. In the Navigation page, click Licenses.
Licenses tab
4. On the Licenses tab, click the Add (+) icon.
92 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5. On the New licenses screen, in the License keys field, enter the license key, then Next.
License key field
6. Add a name for the license in the field provided and click Next.
License Name field
7. From the Ready to complete page, review the license information and click Finish.
The license displays in the Licenses tab and is added to vCenter.
Listing of licenses within Licenses tab
93 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
8. Click the Assets tab, then the Solutions tab.
9. Select the vCenter Server from the Solutions list.
10. From the All Actions drop-down menu, select Assign license.
11. Select the license you want to apply and then click OK to assign the license.
10.2.5 Add AD authentication for Management vCSA Prerequisites:
• Deployment of the Management vCSA
Follow the below steps to add AD authentication:
1. Using the required credentials, log in to the VMware vCenter Server Virtual Appliance (vCSA) web
client.
Note: You can access the vCSA web client by navigating to the following URL:
<https://VCSA_FQDN_OR_IP>
2. Log-in as single sign-on administrator.
Note: Use the password that was set during the installation process.
3. Click the Home icon and select Administration.
4. On the navigation pane, select System Configuration, and click Nodes.
5. In the Nodes, select the FQDN and go to the Mange tab.
Manage screen
6. From the Manage screen, locate the Active Directory section and click the Join button.
94 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Manage screen
7. In the fields provide, enter Domain name, user name and password.
Note: The Organizational unit information is optional.
8. Click OK.
Active Directory credentials screen
9. Reboot the node to apply the changes.
Node reboot screen
95 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10. After the node reboots, log in to the VMware vCenter Server.
11. Click the Home icon and select Administration.
12. On the navigation pane, select System Configuration, and click Nodes.
13. In the Nodes, select the FQDN and go to the Mange tab.
14. On the Manage screen, locate the Active Directory section and confirm that the domain is listed in
the Domain field.
Active Directory screen
15. Click the Home button and select Administration > Single Sign-On, and then Configuration.
16. From the Configuration screen, click the Identity Sources tab, and then click the Add (+) icon to
add an identity source. The Add Identity source window displays.
Identity Sources screen
17. From the Add identity source window, select the Active Directory (Integrated Windows
Authentication) source type from the options listed.
Active Directory (Integrated Windows Authentication) screen
Note: The underlying system must be a member of the Active Directory domain.
96 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
18. Enter the domain name in the field provided then click Next.
Domain name field
19. Review the domain name and then click Finish.
Domain name confirmation screen
The identity source displays in the Identity Sources tab.
Identity Sources tab with new identity listed
10.2.6 Stage 1: Deploy OVA file for Resource vCenter Server Appliance Stage 1 of the deployment process involves the deployment of the OVA file included in the vCenter Server
Appliance installer, as a vCenter Server Appliance with an embedded PSC on the management vCenter.
1. Mount the VMware-VCSA-all-6.7.x ISO on a compatible workstation.
2. In the vCenter Server Appliance installer, navigate to the vcsa-ui-installer directory, then to the
subdirectory for your operating system, and run the installer executable file.
a. For Microsoft Windows, go to the win32 subdirectory, and run the installer.exe file.
97 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
b. For Linux, go to the lin64 subdirectory, and run the installer file.
3. From the vCenter Server Appliance 6.7U1 Installation screen, click Install.
Stage 1 Deploy appliance screen
4. On the Introduction screen, review the installation overview, then click Next.
5. Reviews the information provided within the End User License Agreement (EULA) and if you agree to
the terms, click Next.
6. On the Select Deployment type screen, select Install vCenter Server with an Embedded Platform
Services Controller from the Embedded Platform Services Controller section then click Next.
Select deployment type screen
7. On the Appliance deployment target screen, enter vCenter Server name, HTTPS port, User
name, and Password in the fields provided, then click Next.
98 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Appliance deployment target screen
8. Review the contents of the Certificate Warning, then click Yes to accept the certificate.
Certificate Warning window
9. On the Select folder screen, select the ManagementDatacenter option and then click Next.
99 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Select folder screen
10. On the Select compute resource screen, navigate to the ESXi to deploy, then click Next.
Select compute resource screen
11. On the Set up appliance VM screen:
a. In the VM name field, enter the VM name specified for the vCenter Server Appliance.
b. In the Set root password, create a root password.
c. In the Confirm root password, re-enter the password, then click Next.
100 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Set up appliance VM screen
Note: The appliance name can contain upper and lower case letters, however special characters such as %,
\, or / are not permitted. The appliance name must not exceed 80 characters in length.
12. On the Select deployment size screen, use the drop-down from the Deployment size and Storage
size sections to select the sizes necessary for the vCenter Server Appliance for your vSphere
inventory, then click Next.
Select deployment size screen
101 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
13. On the Select datastore screen, select vsanDatastore and click Enable Thin Disk Mode, then click
Next.
Select datastore screen
14. On the Configure Network Settings screen, enter the appropriate network settings and click Next.
15. On the Ready to complete stage 1 screen, review the deployment settings for the vCenter Server
Appliance and click Finish.
Ready to complete screen
The OVA deployment process starts.
102 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Stage 1 status window
16. Once the OVA deployment is complete, click Continue to proceed with the stage 2 deployment
process.
10.2.7 Stage 2: Set up resource vCenter Server Appliance with embedded PSC Once the OVA deployment is complete, you are redirected to stage 2 of the deployment process to set up and
start the services of the newly deployed vCenter Server Appliance with an embedded Platform Services
Controller.
1. Review the information within the Introduction to stage 2 page, then click Next
Stage 2 Introduction screen
2. On the Appliance configuration screen, from the Time synchronization mode drop-down list
select Synchronize time with NTP servers, then enter the NTP server IP address in NTP servers
field:
103 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Appliance configuration screen
Note: From the Appliance configuration screen, the option to enable the SSH access to the appliance can
be selected.
3. On the SSO configuration screen, enter the vCenter Single Sign-On domain name, User name,
and Administrator password, then click Next.
SSO configuration screen
4. On the Configure CEP screen, if interested in participating in the VMware Customer Experience
Improvement Program, click to place a check the box and click Next.
5. From the Ready to complete page, review the configuration settings for the vCenter Server
Appliance, click Finish, and then click OK. Stage 2 of the deployment process and set up of the
appliance is complete.
104 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Ready to complete screen
6. After the initial setup finishes, you have the option to click Close to complete the setup process, or
you can click the https://<vcenter_server_appliance_fqdn>:443 link to open the vCenter Server
Appliance Getting Started page.
Stage 2 Complete status screen
10.2.8 Change VMware vSAN default storage policy for resource vCSA Prior to deploying additional VMs to the system, you must first update the VMware vSAN storage policy.
1. Using the required credentials, log in to the VMware vCenter Server Virtual Appliance (vCSA) web
client.
2. Click the Home icon then select Policies and Profiles.
105 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
3. In the navigation pane, select vSAN Default Storage Policy, then click the Manage tab.
4. Select the Rule-set 1: VSAN option.
vSAN Default Storage Policy screen
5. Click the Edit button. The Rule-set 1 screen displays.
6. Locate the Primary level of failures to tolerate option and enter 0 in the field provided.
7. From the Force provisioning drop-down, click to select Yes.
Rule-set 1 settings screen
8. Click OK to save the changes.
Rule-set 1 change confirmation
106 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10.2.9 Assign license to Resource vCSA Follow the below steps to assign licenses:
1. Log in to the Resource vCenter through vSphere Web Client.
2. Click the Home icon and select Administration from the drop down.
3. In the navigation pane, select Licenses.
Licenses tab
4. Click the Add (+) icon and in the field provided, enter the license key, then Next.
License key field
5. Add a name for the license in the field provided and click Next.
License Name field
The license displays in the Licenses tab and is added to vCenter.
107 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Listing of licenses within Licenses tab
6. Click the Assets tab, then the Solutions tab.
7. Select the vCenter Server from the Solutions list.
8. From the All Actions drop-down menu, select Assign license.
9. Select the license you want to apply and then click OK to assign the license.
10.2.10 Add AD authentication for Resource vCSA Prerequisites:
• Deployment of the Resource vCSA
Follow the below steps to add AD authentication:
1. Using the required credentials, log in to the VMware vCenter Server Virtual Appliance (vCSA) web
client.
Note: You can access the vCSA web client by navigating to the following URL:
<https://VCSA_FQDN_OR_IP>
2. Log-in as single sign-on administrator.
Note: Use the password that was set during the installation process.
3. Click the Home icon, and select Administration.
4. On the navigation pane, select System Configuration, and click Nodes.
5. In the Nodes, select the FQDN and go to the Mange tab.
6. From the Manage screen, locate the Active Directory section and click the Join button.
108 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Manage screen
7. In the fields provide, enter Domain name, user name and password.
Note: The Organizational unit information is optional.
8. Click OK.
Active Directory credentials screen
9. Reboot the node to apply the changes.
Node reboot screen
10. After the node reboots, log in to the VMware vCenter Server.
11. Click the Home icon and select Administration.
109 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
12. On the navigation pane, select System Configuration, and click Nodes.
13. In the Nodes, select the FQDN and go to the Mange tab.
14. On the Manage screen, locate the Active Directory section and confirm that the domain is listed in
the Domain field.
Active Directory screen
15. Click the Home button and select Administration > Single Sign-On, and then Configuration.
16. From the Configuration screen, click the Identity Sources tab, and then click the Add (+) icon to
add an identity source. The Add Identity source window displays.
Identity Sources screen
17. From the Add identity source window, select the Active Directory (Integrated Windows
Authentication) source type from the options listed.
Active Directory (Integrated Windows Authentication) screen
Note: The underlying system must be a member of the Active Directory domain.
18. Enter the domain name in the field provided then click Next.
110 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Domain name field
19. Review the domain name and then click Finish.
Domain name confirmation screen
The identity source displays in the Identity Sources tab.
Identity Sources tab with new identity listed
10.3 NSX-T deployment and setup
Prerequisites:
• Review the necessary hardware requirements for NSX-T outlined in the System Requirements for NSX-T
section of the NSX-T Installation Guide.
• VMware ESXi 6.7U1
• VMware vCenter Server 6.7U1
111 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
VMware NSX-T Manager 2.3 OVA
▪ An NSX vSwitch is based on a vSphere distributed switch or VDS, which provides uplinks for host
connectivity to the top-of-rack or ToR physical switches.
▪ As a best practice, VMware recommends the planning and preparation of the vSphere distributed
switch before installing NSX for vSphere.
▪ NSX services are not supported on vSphere standard switch.
▪ VM workloads must be connected to a vSphere distributed switch (VDS) to use NSX services and
features.
▪ For each host cluster that uses NSX, all hosts within the cluster must be attached to a common VDS.
▪ To simplify a deployment, each cluster of hosts is associated with only one VDS, even though some
of the VDSs span multiple clusters.
10.3.1 Install NSX Manager Virtual Appliance The NSX Manager provides a graphical user interface and REST API for the creation, configuration, and
monitoring of NSX-T components such as logical switches, logical routers, and firewalls. The NSX Manager
provides an aggregated system view and is the centralized network management component of NSX. NSX
Manager is installed as a virtual appliance on any ESX host in the vCenter environment.
The NSX Manager virtual machine is packaged as an OVA file, which allows for the use of the vSphere Web
Client to import the NSX Manager into the datastore and virtual machine inventory.
Only one instance of NSX Manager can be installed in an environment. When NSX Manager is deployed on
an ESXi host, the vSphere high availability (HA) feature can be used to ensure the availability of NSX
Manager.
Note: The NSX Manager virtual machine installation includes VMware Tools. Do not attempt to upgrade or
remove VMware Tools on the NSX Manager.
• See the Inventory layout table in section Solution archive within the Dell EMC Ready Solution for VMware
vCloud NFV 3.1 OpenStack Edition Platform Deployment Automation Hardware Guide to deploy NSX VM
on ESXi host.
Follow the below steps to install the NSX Manager Virtual Appliance:
1. Locate the NSX-T Unified Appliance OVA or OVF file.
2. In web browser, log in to the vSphere Web Client.
3. Select VMs and Templates, then right-click on vCenter Server, and select Deploy OVF Template.
4. On the Deploy OVF Template window, click the Browse button and locate the NSX-T Unified
Appliance.ova file on your computer, then click Next.
Note: Deploy the NSX manager OVA file within the Management Cluster vCenter server.
112 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
OVF name and location screen
5. In the Name field, enter the NSX manager name, then select the datacenter to deploy NSX controller
and click Next.
6. On the Select a resource screen, select the NSX-T manager resource, then click Next. In the
example provided, ManagementCluster is selected.
Select a resource
7. Review the details of the OVA template and click Next.
113 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
OVF deployment details screen
8. On the Select configuration screen, select Medium from the Configuration drop-down list and click
Next.
Select configuration screen
9. In the Select storage screen:
a. From the Select virtual disk format drop-down list, select Thin provision.
b. From the VM storage policy drop-down list, select vSAN Default Storage Policy.
c. Select the datasotore and click Next.
114 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Select storage screen
10. On the Select networks screen, select a destination network from the drop-down list, and click Next.
In the example provided, VM-Management-Network is selected.
Select networks
11. On the Customize template screen, specify the root, admin, and audit user passwords.
12. Scroll down, in the Network properties section, fill the following fields:
a. Hostname: Enter the host name
b. Default IPv4 Gateway: Enter the IP address of default gateway for NSX-T Manager
c. Management Network IPv4m Address: The IPv4 address for the first interface
d. Management Network Netmask: The netmask for the first interface
115 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Network properties section
13. In the DNS section:
a. In the DNS Server list field, enter the IP address for DNS Server for NSX-T Manager.
b. In the Domain Search List field, enter the domain name for NSX-T Manager.
14. In the Services Configuration section:
a. In the NTP Server List field, enter the IP address for NTP server.
b. If required, check the Enable SSH and Allow root SSH logins check boxes to enable SSH
service.
DNS and Services Configuration
116 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
15. Click Next.
16. Verify the all details and click Finish to start deployment.
Ready to complete screen
17. After completion of the deployment:
a. Power on NSX Manager VM from vSphere Web Client:
> To Power ON the NSX-T Manager, in the navigation pane, right click NSX-T Manager, and
then click Power > Power on.
b. After the NSX Manager VM boots up completely, connect to NSX Manager GUI using the URL:
https://<IP/FQDN of NSX Manager>
NSX Manager Login screen
c. Review the EULA, and if you agree to the terms, click to select the check box to accept the EULA
and click CONTINUE.
117 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
EULA screen
d. Click SAVE to finish.
10.3.2 Installation of controller and cluster from NSX Manager The NSX controller is an advanced distributed state management system that provides control plane
functions for NSX-T logical switching and routing functions. It serves as the central control point for all logical
switches within a network and maintains information about all hosts, logical switches, and logical routers. NSX
Controllers control the devices that perform packet forwarding. These forwarding devices are known as virtual
switches.
Virtual switches, such as NSX managed virtual distributed switch, or N-VDS, (previously known as
Hostswitch) and Open vSwitch (OVS), reside on ESXi and other hypervisors such as KVM.
Prerequisites:
• Deployment of the NSX Manager
• Deployment of vCenter Server and vSphere ESXi hosts
• Register vSphere ESXi host to the vCenter Server
• vSphere ESXi host has the necessary CPU, memory, and hard disk resources to support 12vCPUs, 48
GB RAM, and 360 GB storage
• Refer the USER_INPUT.xlsm file to create three NSX-T Controllers
• See the Inventory layout table in section Solution archive within the Dell EMC Ready Solution for vCloud
NFV 3.1-OpenStack Deployment Automation Hardware Guide to deploy controller VM on the ESXi host
Follow the below steps to deploy the NSX controller cluster:
1. Locate the NSX-T Controller OVA or OVF file.
2. In web browser, log-in to the vSphere Web Client.
3. Select VMs and Templates, then right-click to Management Cluster, and select Deploy OVF
Template.
118 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
4. From the Deploy OVF Template window, select Local file radio button, then click the Browse button
and locate the NSX-T Controller.ova file on your computer, then click Next.
Note: Deploy the NSX Controller OVA file within the Management Cluster vCenter server.
Select Template screen
5. In the Name field, enter the NSX-T controller name, then select the datacenter to deploy NSX-T
controller and click Next.
Select name and location screen
6. On the Select a resource screen, select the NSX-T Controller resource, then click Next.
119 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Select a resource
7. Review the details of the OVA template then click Next.
Review details window
8. On the Select configuration screen, select Medium from the Configuration drop-down list and click
Next.
120 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Select configuration screen
9. In the Select storage screen:
a. From the Select virtual disk format drop-down list, select Thin provision.
b. From the VM storage policy drop-down list, select vSAN Default Storage Policy.
c. Select the datasotore, and click Next.
Select storage screen
10. On the Select networks screen, select a destination network from the drop-down list, and click Next.
In the example provided, VM-Management-Network is selected.
121 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Select networks
11. On the Customize template screen, specify the root, admin, and audit user passwords.
12. Scroll down, in the Network properties section, fill the following fields:
a. Hostname: Enter the host name.
b. Default IPv4 Gateway: Enter the IP address of default gateway for NSX-T Controller.
c. Management Network IPv4m Address: The IPv4 address for the first interface.
d. Management Network Netmask: The netmask for the first interface.
Network properties section
13. In the DNS section:
a. In the DNS Server list field, enter the IP address for DNS Server for NSX-T controller.
b. In the Domain Search List field, enter the domain name for NSX-T controller.
122 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
14. In the Services Configuration section:
a. In the NTP Server List field, enter the IP address for NTP server.
b. If required, check the Enable SSH and Allow root SSH logins check boxes to enable SSH
service.
DNS and Services Configuration
15. Click Next.
16. Verify the details and click Finish to start deployment.
Ready to complete screen
123 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
After completion of deployment, power on the NSX-T Controller VM from the vSphere Web Client.
17. To Power ON the NSX-T Controller, in the navigation pane, right-click on the NSX-T Controller, then
click Power > Power on.
Note: Repeat the steps in this section to deploy two more NSX-T Controllers.
10.3.3 NSX Edge installation Prerequisites:
• If a vCenter Server is registered as a compute Manager in NSX-T, use the NSX Manager UI to configure
a host as an NSX Edge node and automatically deploy it on the vCenter Server.
• Verify that the vCenter Server vSAN datastore on which the NSX Edge is being installed, has a minimum
of 120 GB memory available.
• Verify that the vCenter Server cluster or host has access to the specified networks and vSAN datastore in
the configuration.
• Refer the UserInput.xlsm file to create two NSx-T Edge VMs.
• See the Inventory layout table in section Solution archive within the Dell EMC Ready Solution for vCloud
NFV 3.1 - OpenStack Deployment Automation Hardware Guide to deploy Edge VM’s on ESXi host.
Follow the below steps to deploy the NSX Edge cluster:
1. Locate the NSX-T Edge OVA or OVF file.
2. In web browser, log-in to the vSphere Web Client.
3. Select VMs and Templates, then right-click on Management Cluster.
4. Click to select Deploy OVF Template.
5. On the Deploy OVF Template window, select Local file radio button, then click the Browse button
and locate the NSX-T Edge.ova file on your computer, then click Next.
Note: Deploy the NSX Edge OVA file within the Management Cluster vCenter server.
Select Template screen
124 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
6. In the Name field, enter the NSX-T Edge name, then select the datacenter to deploy NSX-T Edge
and click Next.
Select name and location screen
7. On the Select a resource screen, select the NSX-T edge resource, then click Next.
Select a resource
8. Review the details of the OVA template then click Next.
125 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Review details window
9. On the Select configuration screen, select Medium from the Configuration drop-down list and click
Next.
Select configuration screen
10. In the Select storage screen:
a. From the Select virtual disk format drop-down list, select Thin provision.
b. From the VM storage policy drop-down list, select vSAN Default Storage Policy.
c. Select the datasotore and click Next.
126 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Select storage screen
11. On the Select networks screen, select destination networks from the drop-down list, and click Next.
Select networks
12. On the Customize template screen, specify the root, admin, and audit user passwords.
13. Scroll down, in the Network properties section, fill the following fields:
a. Hostname: Enter the host name
b. Default IPv4 Gateway: Enter the IP address of default gateway for NSX-T Edge
c. Management Network IPv4m Address: The IPv4 address for the first interface
d. Management Network Netmask: The netmask for the first interface
127 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Network properties section
14. In the DNS section:
a. In the DNS Server list field, enter the IP address for DNS Server for NSX-T edge.
b. In the Domain Search List field, enter the domain name for NSX-T edge.
15. In the Services Configuration section:
a. In the NTP Server List field, enter the IP address for NTP server.
b. If required, check the Enable SSH and Allow root SSH logins check boxes to enable SSH
service.
DNS and Services Configuration
128 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
16. Click Next.
17. Verify the all details and click Finish to start deployment.
Ready to complete screen
After completion of deployment, Power ON NSX-T edge VM from vSph ere Web Client.
18. To Power ON the NSX-T edge, in the navigation pane, right-click on the NSX-T edge, then Power >
Power on.
Note: Repeat the above steps to deploy another NSX-T edge VM.
10.4 Enable VMware enhanced vMotion compatibility
To improve CPU compatibility between hosts that have varying CPU feature sets, you can hide some host
CPU features from the virtual machine by placing the host in an Enhanced vMotion Compatibility (EVC)
cluster. Hosts in an EVC cluster and hosts that you add to an existing EVC cluster must meet EVC
requirements.
Prerequisites:
• Power off all virtual machines in the cluster that are running on hosts with a feature set greater than the
EVC mode that you intend to enable.
• All hosts in the cluster must meeting the following requirements.
Requirements Description
ESX/ESXi version ESX/ESXi 6.7U1
129 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
vCenter Server The host must be connected to a vCenter Server system.
CPUs A single vendor, either AMD or Intel.
Advanced CPU features enabled Enable these CPU features in the BIOS if they are available:
• Hardware virtualization support (AMD-V or Intel VT)
• AMD No eXecute(NX)
• Intel eXecute Disable (XD)
10.4.1 Enable VMware EVC for management cluster Follow the below steps to enable EVC on management cluster:
1. Select the Management Cluster in the vSphere Web Client.
2. Click the Configure tab.
3. From the Configuration tab, navigate to VMware EVC and click Edit.
4. Select the CPU vendor and VMware EVC Mode appropriate for the hosts to add to the cluster, then
click OK.
Change EVC Mode screen
130 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
10.4.2 Enable VMware EVC for resource cluster Follow the below steps to enable EVC on resource cluster:
1. Navigate to the Resource Cluster in the VMware vSphere Web Client.
2. Click the Configure tab.
3. From the Services tab, navigate to VMware EVC and click Edit.
4. Select the CPU vendor and VMware EVC Mode appropriate for the hosts to add to the cluster, then
click OK.
10.5 Enable vSphere DRS
Follow the below to enable vSphere DRS on both the management and resource clusters:
1. From the VMware vSphere Web Client, navigate to the management cluster.
2. Click the Configure tab, Services, then click vSphere DRS.
vSphere DRS screen
3. Click Edit.
4. From the ManagementCluster-Edit Cluster settings window, click to place a check in the Turn ON
vSphere DRS check box.
5. Set the DRS Automation to Partially Automated state.
6. Set the Power Management to Off, and then set the Advanced Options to None.
7. Click OK.
Edit Cluster Settings screen
131 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
8. Verify that the vSphere DRS is Turned ON status displays.
vSphere DRS status screen
9. Repeat the steps provided within this section, on the resource cluster.
10.6 Enabling vSphere availability
Perform the following steps on both the management and resource clusters:
1. Within the VMware vSphere Web Client, navigate to the management cluster.
2. Click the Configure tab.
3. Navigate to Services and click vSphere Availability.
4. Click Edit.
vSphere availability screen
5. Click to place a check in the Turn ON vSphere HA and Turn on Proactive HA check boxes.
6. Click the Failures and Responses section and select Enable Host Monitoring.
7. Keep rest of the options to their defaults, then click OK.
132 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
vSphere Availability screen
8. Repeat the steps in this section, on the resource cluster.
10.7 Set up anti-affinity rules
The affinity rule setting establishes a relationship between two or more VMware VMs and hosts. Anti-
affinity rules tell the vSphere hypervisor platform to keep virtual entities separated.
Anti-affinity rules are created to keep two or more VMs separated on different hosts.
Note: The anti-affinity rules can be created for management and resource cluster VMs. The creation of a
separate anti-affinity rule for the VMs listed is recommended for the respective cluster. Dell EMC
recommends the execution of this section only after every component has been deployed and configured.
The following is a list of management cluster VMs that must be kept on different hosts using an anti-affinity
rule:
Management VM cluster list:
Rule name Create rule For VM’s
mgmtVCSARule mgmt VCSA active passive vc witness vc
resVCSARule resource VCSA active passive vc witness vc
nsxControllerRule NSX controller1 NSX controller2 NSX controller13
The following is a list of resource cluster VMs that must be kept on different hosts using an anti-affinity rule:
Resource cluster list:
Rule name Create rule for VMs
Edge_Rule Edge01 Edge02
133 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
Prerequisites:
• All components of the management cluster are deployed
• All components of the resource cluster are deployed
10.7.1 Create an anti-affinity rule Follow the below steps to create an anti-affinity rule:
1. From the VMware vCenter GUI, click the desired cluster, such as ManagementCluster, then click
the Configure tab.
2. From the Configuration screen, locate the VM/Host Rules and click the Add button.
VM Host Rules screen
3. In the Create VM/Host Rule dialog box, enter a Name for the rule, for example vRLI_Rule and click
to place a check in the Enable rule check box.
4. From the Type drop-down menu, select Separate Virtual Machines, then click Add.
Create VM/Host Rule configuration screen
134 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
5. From the Add Rule Member screen, select the virtual machines to keep on different hosts then click
OK.
Add Rule Member screen
6. Click OK to create the rule.
VM/Host Rule creation screen
NOTE: Repeat the steps in this section to set the Anti-Affinity rule for the management and resource cluster
VMs.
135 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
A Show running configs on leaf switches
Below mentioned running config is a reference to check the configuration done in above sections on the BCF
through GUI.
Login into one of the controllers using ssh utility and run below command:
bcfcntrl1>
bcf01> enable
bcf01# show running-config
!
! Big Cloud Fabric 4.7.0 (bcf-4.7.0 #186)
! Current Time: 2019-01-31 08:12:18 UTC
!
version 1.0
! ntp
ntp server 192.168.176.249
! logging
logging remote vrli123.dellnfv.com
logging remote vrli126.dellnfv.com
! aaa
aaa accounting exec default start-stop local
! local
local node
hostname bcf01
interface management
!
ipv4
ip 192.168.176.90/24 gateway 192.168.176.254
method manual
dns server 192.168.176.250
dns search dellnfv.com
!
ipv6
method manual
! user
user admin
full-name 'Default admin'
hashed-password method=PBKDF2WithHmacSHA512,salt=w5heJjvUBnf-
Rb10jG5ysg,rounds=25000,ph=true,mF7aAQF-so7srCuqgEy-QvpNy-OXyvx394sPbO-
J3NiW_SnAzJuY950J1Adz1jVjF7b6G9v-2tecXS0oCLuljg
136 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
! group
group admin
associate user admin
! controller
controller
cluster-name bcfcluster
virtual-ip 192.168.176.92
access-control
!
access-list api
1 permit from ::/0
2 permit from 0.0.0.0/0
!
access-list gui
1 permit from ::/0
2 permit from 0.0.0.0/0
!
access-list ns-api
1 permit from ::/0
2 permit from 0.0.0.0/0
!
access-list ssh
1 permit from ::/0
2 permit from 0.0.0.0/0
!
access-list vce-api
1 permit from ::/0
2 permit from 0.0.0.0/0
! switch
switch Leaf1
fabric-role leaf
leaf-group Rack1
mac 34:17:eb:46:a8:80
interface ethernet11:1
interface ethernet11:2
interface ethernet11:3
interface ethernet11:4
interface ethernet13:1
interface ethernet13:2
interface ethernet1:1
interface ethernet1:2
interface ethernet1:3
interface ethernet1:4
interface ethernet30
interface ethernet3:1
137 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
interface ethernet3:2
interface ethernet3:3
interface ethernet3:4
interface ethernet7:1
interface ethernet7:2
interface ethernet7:3
interface ethernet7:4
switch Leaf2
fabric-role leaf
leaf-group Rack1
mac 34:17:eb:46:cc:80
shutdown
interface ethernet1
interface ethernet11:1
interface ethernet11:2
interface ethernet11:3
interface ethernet11:4
interface ethernet13:1
interface ethernet13:2
interface ethernet1:1
interface ethernet1:2
interface ethernet1:3
interface ethernet1:4
interface ethernet3:1
interface ethernet3:2
interface ethernet3:3
interface ethernet3:4
interface ethernet7:1
interface ethernet7:2
interface ethernet7:3
interface ethernet7:4
! fabric
fabric
forwarding-mode high-bandwidth
!
ipam switch
dns-server 192.168.176.250
gateway 192.168.176.254
ip-range 192.168.176.93 192.168.176.95 subnet-mask-length 24
allocate
! interface-group
interface-group To-Tor
mode lacp
member switch Leaf1 interface ethernet30
138 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
member switch Leaf2 interface ethernet30
backup-mode lacp
interface-group esxi10.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
mode lacp
member switch Leaf1 interface ethernet1:1
member switch Leaf2 interface ethernet1:1
backup-mode lacp
interface-group esxi11.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
mode lacp
member switch Leaf1 interface ethernet1:2
member switch Leaf2 interface ethernet1:2
backup-mode lacp
interface-group esxi12.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
mode lacp
member switch Leaf1 interface ethernet1:3
member switch Leaf2 interface ethernet1:3
backup-mode lacp
interface-group esxi13.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6
mode lacp
member switch Leaf1 interface ethernet1:4
member switch Leaf2 interface ethernet1:4
backup-mode lacp
interface-group esxi6.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
mode lacp
member switch Leaf1 interface ethernet11:1
member switch Leaf2 interface ethernet11:1
backup-mode lacp
interface-group esxi6.dellnfv.com-Overlay-Network-VDS-vmnic5
member switch Leaf1 interface ethernet7:1
interface-group esxi6.dellnfv.com-Overlay-Network-VDS-vmnic7
member switch Leaf2 interface ethernet7:1
interface-group esxi6.dellnfv.com-infra-mgmt-VDS-vmnic4
member switch Leaf1 interface ethernet3:1
interface-group esxi6.dellnfv.com-infra-mgmt-VDS-vmnic6
member switch Leaf2 interface ethernet3:1
interface-group esxi7.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
mode lacp
139 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
member switch Leaf1 interface ethernet11:2
member switch Leaf2 interface ethernet11:2
backup-mode lacp
interface-group esxi7.dellnfv.com-Overlay-Network-VDS-vmnic5
member switch Leaf1 interface ethernet7:2
interface-group esxi7.dellnfv.com-Overlay-Network-VDS-vmnic7
member switch Leaf2 interface ethernet7:2
interface-group esxi7.dellnfv.com-infra-mgmt-VDS-vmnic4
member switch Leaf1 interface ethernet3:2
interface-group esxi7.dellnfv.com-infra-mgmt-VDS-vmnic6
member switch Leaf2 interface ethernet3:2
interface-group esxi8.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
mode lacp
member switch Leaf1 interface ethernet11:3
member switch Leaf2 interface ethernet11:3
backup-mode lacp
interface-group esxi8.dellnfv.com-Overlay-Network-VDS-vmnic5
member switch Leaf1 interface ethernet7:3
interface-group esxi8.dellnfv.com-Overlay-Network-VDS-vmnic7
member switch Leaf2 interface ethernet7:3
interface-group esxi8.dellnfv.com-infra-mgmt-VDS-vmnic4
member switch Leaf1 interface ethernet3:3
interface-group esxi8.dellnfv.com-infra-mgmt-VDS-vmnic6
member switch Leaf2 interface ethernet3:3
interface-group esxi9.dellnfv.com-NSXT-External-Network-VDS-vmnic8-vmnic9
mode lacp
member switch Leaf1 interface ethernet11:4
backup-mode lacp
backup-member switch Leaf2 interface ethernet11:4
interface-group esxi9.dellnfv.com-Overlay-Network-VDS-vmnic5
member switch Leaf1 interface ethernet7:4
interface-group esxi9.dellnfv.com-Overlay-Network-VDS-vmnic7
member switch Leaf2 interface ethernet7:4
interface-group esxi9.dellnfv.com-infra-mgmt-VDS-vmnic4
140 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
member switch Leaf1 interface ethernet3:4
interface-group esxi9.dellnfv.com-infra-mgmt-VDS-vmnic6
member switch Leaf2 interface ethernet3:4
! tenant
tenant Tenant_mgmt
description auto-created
multicast-enable
origination vsphere-mgmt-vc
segment Segment-vMotion-400
member interface-group esxi10.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 vlan
400
member interface-group esxi11.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 vlan
400
member interface-group esxi12.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 vlan
400
member interface-group esxi13.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 vlan
400
segment Segment-vSAN-300
member interface-group esxi10.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 vlan
300
member interface-group esxi11.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 vlan
300
member interface-group esxi12.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 vlan
300
member interface-group esxi13.dellnfv.com-infra-mgmt-VDS-vmnic4-vmnic6 vlan
300
!
segment Tenant_mgmt-10
description ' 1 vSphere portgroups: API-Access-Network'
id Tenant_mgmt-10
origination vsphere-mgmt-vc
!
segment Tenant_mgmt-176
description ' 1 vSphere portgroups: VM-management-Network'
id Tenant_mgmt-176
origination vsphere-mgmt-vc
!
endpoint vm-33-00-0c-29-26-61-e0
description 'AD-DNS-Server[vm-33] (Network adapter 1) on ESXi Host
esxi10.dellnfv.com [192.168.176.250]'
mac 00:0c:29:26:61:e0
origination vsphere-mgmt-vc
!
endpoint vm-34-00-0c-29-10-66-d0
141 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
description 'NTP-Server[vm-34] (Network adapter 1) on ESXi Host
esxi10.dellnfv.com [192.168.176.249]'
mac 00:0c:29:10:66:d0
origination vsphere-mgmt-vc
!
endpoint vm-35-00-0c-29-a4-83-63
description 'mgmtVcsaActive[vm-35] (Network adapter 1) on ESXi Host
esxi10.dellnfv.com [192.168.176.101]'
mac 00:0c:29:a4:83:63
origination vsphere-mgmt-vc
!
member interface-group esxi10.dellnfv.com-VM-mgmt-VDS-MgmtLag1 vlan 176
description esxi10.dellnfv.com-VM-mgmt-VDS-MgmtLag1-176
origination vsphere-mgmt-vc
!
segment Tenant_mgmt-20
description ' 1 vSphere portgroups: VCSA-HA-Network'
id Tenant_mgmt-20
origination vsphere-mgmt-vc
!
segment Tenant_mgmt-200
description ' 1 vSphere portgroups: ESXi-Management-Network'
id Tenant_mgmt-200
origination vsphere-mgmt-vc
!
endpoint esxi10.dellnfv.com-vmk0
description 'VMKernel [192.168.200.10] with Fault Tolerance, Management
traffic service'
ip 192.168.200.10
mac 24:6e:96:a4:1f:c8
origination vsphere-mgmt-vc
!
endpoint esxi11.dellnfv.com-vmk0
description 'VMKernel [192.168.200.11] with Fault Tolerance, Management
traffic service'
ip 192.168.200.11
mac 24:6e:96:a4:23:90
origination vsphere-mgmt-vc
!
endpoint esxi12.dellnfv.com-vmk0
description 'VMKernel [192.168.200.12] with Fault Tolerance, Management
traffic service'
ip 192.168.200.12
mac 24:6e:96:a4:23:2c
origination vsphere-mgmt-vc
!
endpoint esxi13.dellnfv.com-vmk0
142 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
description 'VMKernel [192.168.200.13] with Fault Tolerance, Management
traffic service'
ip 192.168.200.13
mac 24:6e:96:a4:1b:34
origination vsphere-mgmt-vc
!
member interface-group esxi10.dellnfv.com-ESXi-Management-VDS-MgmtLag1 vlan
200
description esxi10.dellnfv.com-ESXi-Management-VDS-MgmtLag1-200
origination vsphere-mgmt-vc
!
member interface-group esxi11.dellnfv.com-ESXi-Management-VDS-MgmtLag1 vlan
200
description esxi11.dellnfv.com-ESXi-Management-VDS-MgmtLag1-200
origination vsphere-mgmt-vc
!
member interface-group esxi12.dellnfv.com-ESXi-Management-VDS-MgmtLag1 vlan
200
description esxi12.dellnfv.com-ESXi-Management-VDS-MgmtLag1-200
origination vsphere-mgmt-vc
!
member interface-group esxi13.dellnfv.com-ESXi-Management-VDS-MgmtLag1 vlan
200
description esxi13.dellnfv.com-ESXi-Management-VDS-MgmtLag1-200
origination vsphere-mgmt-vc
!
segment Tenant_mgmt-300
description ' 1 vSphere portgroups: VSAN-Network'
id Tenant_mgmt-300
origination vsphere-mgmt-vc
!
endpoint esxi10.dellnfv.com-vmk1
description 'VMKernel [192.200.3.10] with vSAN traffic service'
ip 192.200.3.10
mac 00:50:56:67:ec:c8
origination vsphere-mgmt-vc
!
endpoint esxi11.dellnfv.com-vmk1
description 'VMKernel [192.200.3.11] with vSAN traffic service'
ip 192.200.3.11
mac 00:50:56:61:bd:60
origination vsphere-mgmt-vc
!
endpoint esxi12.dellnfv.com-vmk1
description 'VMKernel [192.200.3.12] with vSAN traffic service'
ip 192.200.3.12
mac 00:50:56:69:97:d3
143 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
origination vsphere-mgmt-vc
!
endpoint esxi13.dellnfv.com-vmk1
description 'VMKernel [192.200.3.13] with vSAN traffic service'
ip 192.200.3.13
mac 00:50:56:65:d9:10
origination vsphere-mgmt-vc
!
member interface-group esxi10.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 300
description esxi10.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-300
origination vsphere-mgmt-vc
!
member interface-group esxi11.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 300
description esxi11.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-300
origination vsphere-mgmt-vc
!
member interface-group esxi12.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 300
description esxi12.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-300
origination vsphere-mgmt-vc
!
member interface-group esxi13.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 300
description esxi13.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-300
origination vsphere-mgmt-vc
!
segment Tenant_mgmt-400
description ' 1 vSphere portgroups: vMotion-Network'
id Tenant_mgmt-400
origination vsphere-mgmt-vc
!
endpoint esxi10.dellnfv.com-vmk2
description 'VMKernel [192.200.4.10] with vMotion traffic service'
ip 192.200.4.10
mac 00:50:56:6a:42:9c
origination vsphere-mgmt-vc
!
endpoint esxi11.dellnfv.com-vmk2
description 'VMKernel [192.200.4.11] with vMotion traffic service'
ip 192.200.4.11
mac 00:50:56:63:60:59
origination vsphere-mgmt-vc
!
endpoint esxi12.dellnfv.com-vmk2
description 'VMKernel [192.200.4.12] with vMotion traffic service'
ip 192.200.4.12
mac 00:50:56:6d:fe:dc
origination vsphere-mgmt-vc
!
144 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
endpoint esxi13.dellnfv.com-vmk2
description 'VMKernel [192.200.4.13] with vMotion traffic service'
ip 192.200.4.13
mac 00:50:56:65:7e:6e
origination vsphere-mgmt-vc
!
member interface-group esxi10.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 400
description esxi10.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-400
origination vsphere-mgmt-vc
!
member interface-group esxi11.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 400
description esxi11.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-400
origination vsphere-mgmt-vc
!
member interface-group esxi12.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 400
description esxi12.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-400
origination vsphere-mgmt-vc
!
member interface-group esxi13.dellnfv.com-Infra-mgmt-VDS-MgmtLag1 vlan 400
description esxi13.dellnfv.com-Infra-mgmt-VDS-MgmtLag1-400
origination vsphere-mgmt-vc
tenant Tenant_res
description auto-created
multicast-enable
origination vsphere-tenant-resvc
logical-router
!
bgp
graceful-restart
local-as 64502
log-neighbor-changes
maximum-paths 2
no logging-flag events
no logging-flag keepalives
no logging-flag updates
no redistribute-ospf
no shutdown
protocol-ip 172.16.61.1
redistribute-connected
redistribute-static
router-id 172.16.61.1
!
neighbor NSX-Tier0
connect-time 60
ebgp-ttl multihop 2
hold-time 180
145 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
neighbor-ip 172.16.60.10 remote-as 65002
soft-reconfiguration inbound
!
neighbor Tor
connect-time 60
ebgp-ttl multihop 2
hold-time 180
neighbor-ip 192.168.30.254 remote-as 65001
soft-reconfiguration inbound
!
neighbor VIO-K-Tier0
connect-time 60
ebgp-ttl multihop 2
hold-time 180
neighbor-ip 172.16.60.20 remote-as 65003
soft-reconfiguration inbound
interface segment Segment-To-Leaf
ip address 172.16.60.2/24
interface segment Segment-To-Tor
ip address 192.168.30.2/24
interface segment Segment-edge-overlay-700
ip address 192.200.7.254/24
interface segment Segment-vMotion-res-400
ip address 192.200.4.254/24
interface segment Segment-vSAN-res-300
ip address 192.200.3.254/24
segment Segment-To-Leaf
member interface-group esxi6.dellnfv.com-NSXT-External-Network-VDS-vmnic8-
vmnic9 vlan 30
member interface-group esxi7.dellnfv.com-NSXT-External-Network-VDS-vmnic8-
vmnic9 vlan 30
member interface-group esxi8.dellnfv.com-NSXT-External-Network-VDS-vmnic8-
vmnic9 vlan 30
member interface-group esxi9.dellnfv.com-NSXT-External-Network-VDS-vmnic8-
vmnic9 vlan 30
segment Segment-To-Tor
member interface-group To-Tor vlan 30
segment Segment-edge-overlay-700
member interface-group esxi6.dellnfv.com-Overlay-Network-VDS-vmnic5 vlan 700
member interface-group esxi6.dellnfv.com-Overlay-Network-VDS-vmnic7 vlan 700
member interface-group esxi6.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 700
member interface-group esxi6.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 700
member interface-group esxi7.dellnfv.com-Overlay-Network-VDS-vmnic5 vlan 700
member interface-group esxi7.dellnfv.com-Overlay-Network-VDS-vmnic7 vlan 700
member interface-group esxi7.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 700
member interface-group esxi7.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 700
member interface-group esxi8.dellnfv.com-Overlay-Network-VDS-vmnic5 vlan 700
146 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
member interface-group esxi8.dellnfv.com-Overlay-Network-VDS-vmnic7 vlan 700
member interface-group esxi8.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 700
member interface-group esxi8.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 700
member interface-group esxi9.dellnfv.com-Overlay-Network-VDS-vmnic5 vlan 700
member interface-group esxi9.dellnfv.com-Overlay-Network-VDS-vmnic7 vlan 700
member interface-group esxi9.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 700
member interface-group esxi9.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 700
segment Segment-vMotion-res-400
member interface-group esxi6.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 400
member interface-group esxi6.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 400
member interface-group esxi7.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 400
member interface-group esxi7.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 400
member interface-group esxi8.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 400
member interface-group esxi8.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 400
member interface-group esxi9.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 400
member interface-group esxi9.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 400
segment Segment-vSAN-res-300
member interface-group esxi6.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 300
member interface-group esxi6.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 300
member interface-group esxi7.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 300
member interface-group esxi7.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 300
member interface-group esxi8.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 300
member interface-group esxi8.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 300
member interface-group esxi9.dellnfv.com-infra-mgmt-VDS-vmnic4 vlan 300
member interface-group esxi9.dellnfv.com-infra-mgmt-VDS-vmnic6 vlan 300
tenant test-mgmt
multicast-enable
147 Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform
B References
For additional information, reference the following documentation:
Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform Deployment Automation
Hardware Guide
Dell EMC Ready Solution for VMware vCloud NFV 3.1 OpenStack Edition Platform Deployment Automation
User Input File
VMware vCloud NFV OpenStack Edition Reference Architecture 3.1