validated reference design netscaler sdx platform - citrix · validated reference design netscaler...

1Citrix.com

Validated Reference Design

Validated Reference Design NetScaler SDX Platform

This guide focuses on providing guidelines to customers on NetScaler SDX Security Isolation and Feature Updates based on their use cases.

Validated Reference Design

2Citrix.com | Validated Reference Guide for NetScaler SDX Update

Validated Reference DesignNetScaler SDX Platform Update

Contents

Section 1: 3

Overview 3

SDX Security 4Virtual Machine (VM) Isolation: 4NIC Isolation: 4CPU Isolation: 4SSL Isolation: 5Memory Isolation: 5Disk Storage Isolation: 6

SDX Update 6Default SSL Certificate 6

New SDX Platform Features: 7Cluster Link Aggregation (CLAG) 7

CLAG NOTES: 7

SDX Platform Improvements: 7VPX Scaling: 7SNMP Standardization: 7SNMP Alarm Thresholds and Timeout Options: 7Multiple DNS Servers: 8Optional Admin User While VPX Provisioning: 8CPU Visualizer: 8Clean Install for All Platforms: 8

Section 2: 8NetScaler SDX 8

Default SSL Certificate Use Case 10

New Features 11Cluster Link Aggregation (CLAG) Use Case 11

NetScaler Default SSL Certificate Configuration 13Cluster Link Aggregation (CLAG) Configuration 13Configure Virtual Server (2 steps required) 14SNMP Alarm Thresholds and Timeout Options 14Multiple DNS Servers Configuration 15Optional Admin User While VPX Provisioning 15CPU Visualizer Configuration 15

Section 4: 15Configuration Example: NetScaler SDX CLAG 15



Section 1:

NetScaler SDX Overview

Citrix NetScaler is an all-in-one application delivery controller that makes applications run up to five times better, reduces application ownership costs, optimizes the user experience and ensures that applications are always available by using:

• Advanced L4-7 load balancing and traffic management

• Proven application acceleration such as HTTP compression and caching

• An integrated application firewall for application security

• Server offloading to significantly reduce costs and consolidate servers

As a 10 year Gartner leader of service and application delivery, Citrix NetScaler is deployed in thou-sands of networks around the world to optimize, secure and control the delivery of most common enterprise and cloud services. Deployed directly in front of web and database servers, NetScaler combines high-speed load balancing and content switching, http compression, content caching, SSL acceleration, application flow visibility and a powerful application firewall into an integrated, easy-to-use platform. Meeting SLAs is greatly simplified with end-to-end monitoring that transforms network data into actionable business intelligence. NetScaler allows policies to be defined and managed using

a simple declarative policy engine with no programming expertise required.

NetScaler SDX Overview

Citrix NetScaler SDX is a service delivery networking platform for enterprise and cloud datacenters. An advanced virtualized architecture supports multiple NetScaler instances on a single hardware ap-pliance, while an advanced control plane unifies provisioning, monitoring and management to meet the most demanding multi-tenant requirements - all with the industry-leading performance of the Citrix NetScaler MPX system architecture. Full resource isolation supports guaranteed performance SLAs as well as high availability, software version control, data separation and independent policy management. NetScaler SDX provides a foundation for the consolidation of services in the enterprise, and it is also optimized to deliver cloud-based services.

NetScaler SDX provides a platform to run multiple independent instances of key services to meet the unique requirements of individual business units, critical applications and service provider clients. En-terprise and service provider clients gain dedicated control over their delivery infrastructure, including services such as load balancing, security and application acceleration. Complete isolation of per-client traffic helps satisfy security and compliance mandates and eases operational administration through version control and life-cycle management.

The NetScaler SDX platform combines the power of NetScaler MPX hardware with advanced vir-tualization and I/O acceleration to support aggregate performance. Each NetScaler instance can reach high speed to meet the application needs of business tenants. The virtual architecture protects system resources to optimize application delivery functionality and ensure that individual NetScaler instances do not impact the performance SLAs of any other instance.

NetScaler SDX offers flexible licensing to meet both enterprise and service provider requirements. The solution includes licenses to run five independent NetScaler instances and provides a growth path to increase the number of concurrent instances. The popular Citrix pay-as-you-grow licensing program lets customers scale solution performance to meet future business needs while protecting



Unified provisioning, monitoring and management of multiple concurrent NetScaler instances through a single control plane streamlines multi-tenant operations. Not only is each NetScaler instance managed in-dependently, but each NetScaler can run a different software version and support independent IP address-ing schemes to preserve end-to-end isolation of application traffic between different clients.

SDX Security

Virtual Machine (VM) Isolation:SDX security starts at the hypervisor layer with virtual machine (VM) isolation. This VM isolation begins at the Virtual Machine Monitor (VMM) or Domain 0, which is known as the virtualization layer upon which virtualization architectures are built. The VMM or Domain 0 becomes the primary interface between a VM and the physical hardware. NetScaler SDX leverages Single Route Input/output Virtualiza-tion (SR-IOV) technology as defined by the PCI-Special Interest Group (SIG) to secure the virtual machine and provide VM isolation. SR-IOV technology significantly reduces virtualized network processing over-heads, and it provides secure and predictable mechanisms for sharing Input/output (I/O) devices among multiple virtual machines.

Specifically, SDX utilizes the Intel implementation of SR-IOV to secure the virtual machine and provide VM isolation. Intel has worked with PCI-SIG to define the SR-IOV specification. SR-IOV provides VM iso-lation by providing dedicated I/O to virtual machines bypassing the software virtual switch in the Virtual Machine Manager completely. Intel Ethernet Controllers improve data isolation among virtual machines.

Virtual Functions are a significant feature of SR-IOV. These lightweight PCIe functions allow a single physical port to look like multiple ports. Virtual Functions allow multiple virtual machines to have direct assignment on the same port. This feature increases scalability of the number of virtual machines through more efficient I/O device sharing while maintaining VM isolation.

NIC Isolation:NIC isolation on the NetScaler SDX is achieved by utilizing SR-IOV. SR-IOV allows virtualization of a NIC into multiple virtual instances and provides them to NetScaler VPX instances. These virtualized NIC in-stances can bypass the hypervisor and thereby improve performance. By utilizing SR-IOV, each instance achieves full network isolation for Layer 3 and above and Layer 2 isolation by using VLAN tagging of each instance on the NetScaler SDX.

The Intel Fortville NIC implements the Virtual Functions discussed previously under virtual machine isolation. Each NIC (SDX platform dependent) has 20 Virtual Functions assigned to it. However, Virtual Functions assigned are platform dependent. The number of Virtual Functions is set at 20 to allow for large VPX support (as much as 180 Gbps throughput per VM on the SDX 25000 series). Setting the num-ber of Virtual Functions at 20 on the Fortville NIC has two key roles. The first is to achieve NIC isolation while allowing users to provision VPX using maximum resources from the SDX. The second is to enable hardware Receive Side Scaling (RSS) for the interfaces.

CPU Isolation:The NetScaler SDX implements Intel Virtualization Technology for Directed I/O (VT-d) to achieve CPU isolation. Using Intel VT-d to create virtual machines, a virtual machine monitor (VMM) or hypervisor acts as a host and takes full control of the platform hardware. The VMM the presents the guest software (operating system and application software) with an abstraction of the physical machine. The VMM also retains selective control of processor resources, physical memory, interrupt management, agement, and data I/O. This action improves reliability and security through device isolation using hardware assist-ed remapping, and it improves I/O performance and availability by direct assignment of devices.

5Citrix.com | Validated Reference Guide for NetScaler SDX Platform Update


and data I/O. This action improves reliability and security through device isolation using hardware assisted remapping, and it improves I/O performance and availability by direct assignment of devices.

Additionally, VT-d restricts direct memory access (DMA) to pre-assigned domains or physical memory regions. This is achieved through direct memory access-remapping (DMA-remapping). VT-d DMA-remapping logic in the chipset resides between the DMA capable peripheral I/O devices and the computer’s physical memory, and it is programmed by the computer system software. In a virtualization environment, that system software is the VMM. In a native environment where there is no virtualization software, the system software is the native operating system. DMA-remapping translates the address of incoming DMA requests to the correct physical memory address and performs checks for permissions to access the physical address.

VT-d also enables the system software to create multiple protection domains. Each protection domain is an isolated envi-ronment containing a subset of the host physical memory. Depending on the software usage model, the DMA protection domain may represent memory allocated to the virtual machine (VM), or the DMA memory allocated by a guest oper-ating system driver running in a VM, or as part of the VMM itself. The VT-d architecture enables the system software to assign one or more I/O devices to a protection domain. As a result, DMA isolation is achieved by restricting access to the protection domain’s physical memory from I/O devices not assigned to it (using address-translation tables). This provides the necessary isolation to assure separation between each virtual machine’s computer resources.

SSL Isolation:The NetScaler SDX also leverages SR-IOV support to achieve SSL isolation. This document will address SSL isolation in terms of the type of chipset. Since the Cavium Nitrox N2 chipset does not have SR-IOV pass-thru support, it is beyond the scope of this document and will not be addressed.

The Cavium Nitrox N3 SSL chipset uses SR-IOV pass-thru support to achieve isolation. Each Nitrox N3 chip has 56 SSL cores. The number of chips per SDX is platform dependent. There are eight (8) virtual functions (VF) per chip, and each virtual function is assigned to seven (7) SSL cores. By controlling the assignment of the virtual functions in this manner, SSL isolation is achieved.

The Intel Coleto Creek SSL chipset uses SR-IOV pass-thru support to achieve isolation as well. However, the Intel Coleto Creek chipset does not have cores, rather it has crypto engines. As a result, allocation of SSL is done using Crypto Capacity Management. With the Coleto Creek chipset, there are 32 virtual functions (VF) assigned per chip.

As stated previously, allocation of SSL is accomplished using Crypto Capacity Management. Crypto Capacity Management uses Asymmetric Crypto Units (ACU) and Symmetric Crypto Units (SCU) to allocate SSL capacity. An ACU equals one (1) op-eration per second (ops) of a specified algorithm (RSA) 2K (2048-bit key size) decryption. SCU equals 1Mbps for a specified operation type (cipher + authentication) algorithm (AES-128-CBC + SHA256-HMAC) with 1024 bytes buffer size.

Crypto Capacity Management uses Crypto Virtual Interfaces to represent basic access to the SSL hardware. When these Crypto Virtual Interfaces are exhausted, the SSL hardware cannot be further assigned to NetScaler VPX instances. These Crypto Virtual Interfaces are also known as virtual functions, and there are 32 available per chip. They are a read-only entity, and they are automatically allocated by the NetScaler SDX appliance. In this manner, SSL isolation is achieved on the Coleto Creek SSL chipset.

Memory Isolation:Memory isolation equates to virtualizing the memory and keeping track of it. Memory isolation starts in real hardware with actual hardware memory. This memory is also known as the physical memory, and it is divided into 4k-blocks called physical frames. Those physical frames are addressed by their physical frame number (pfn). Physical frame numbers usu-ally start at 0 and are contiguous (x86 computers). On x86 computers, a description of which physical frame numbers are available for use by memory is in the E820 map, and it is provided by the BIOS to operating systems at boot.



When virtualizing, operators must provide the guest with virtual “physical memory address space.” Virtual “physical address space” is described in the E820 map provided to the guest. These spaces are called guest physical frame numbers (gpfns). Real hardware backing this guest virtual physical memory address space is known as machine frames (mfns). Every guest physical frame number has a machine frame number behind it backing it up. As a result, actual hardware memory is virtualized.

Next, guest-to-machine memory translation occurs. Guest physical frame numbers have to start at 0 and be contiguous, but machine frames which back them can come from anywhere in memory. Therefore, a record must be kept to match the guest physical frame numbers to the machine frames. For this reason, every virtual machine has a physical-to-machine translation table (p2m table) to map guest physical frame number space to machine frame number space. Each guest physical frame number will have an entry in the table and every usable bit of RAM will have a machine frame number behind it to back it up. This process is done by the domain builder in domain 0.

After guest-to-machine memory translation occurs, memory is allocated. The guest driver will ask the host to allocate a free page of memory. After allocating a page, the guest driver will put the page on its list of pages and find a guest physical frame number for that page. The guest driver will choose one memory page on its list that it has allocated and ask the host to put some memory behind the guest physical frame number. If the host determines that the guest is allowed to increase its memory, the host will allocate a machine frame number and put it in the physical-to-machine translation table behind that guest physical frame number.

After memory has been allocated, memory must be released when it is no longer being used. However, there must be a process for memory release. The guest driver will ask the guest operating system (OS) for a free page of memory that it can return to the host. After allocating a page, the guest driver will put it on its list of pages and find the guest physical frame number for that page. The guest driver then tells the host it can take the memory behind the guest physical frame number back. The host will replace the machine frame number in that guest physical frame number space with “invalid entry” and put the machine frame number on its own free list. Now, that free memory is potentially available for use by another virtual ma-chine. If the guest were to attempt to read or write this memory now, it would crash; however, it will not because the guest OS thinks the page is in use by the memory driver. The memory driver will not touch the memory and the OS will not use it for anything else. Therefore, memory isolation is achieved.

Disk Storage Isolation:Disk storage volumes or partitions are created for each virtual machine (VM) or VPX. In the NetScaler SDX, each VPX/VM gets approximately 20GB of storage, but this amount can vary depending on the amount of RAM. The SDX and the XenServer Ecosystem use an underlying Logical Volume Manager Technology (LVM) to manage the storage volumes or partitions that are created for each VPX/VM. The LVM is used to configure mirroring and striping of the logical volumes to provide data redundancy and increase I/O per-formance. File systems are created on logical volumes and logical volume devices are mounted the same way that these operations are performed on a physical volume. Additionally, the LVM is non-disruptive and transparent to users. However, the LVM will not allow one VPX/VM to write into the storage file system of another VPX/VM. This is how the NetScaler SDX achieves disk storage isolation.

SDX UpdateDefault SSL CertificatePrevious to this change, the default SSL certificate on the Service Virtual Machine (SVM) of the NetScaler SDX was set as a 1024-bit key certificate. The purpose of this change is to increase the size of the default SSL certificate on the SVM to a 2048-bit key certificate. The user must delete the existing default SSL certif-icate and restart the SVM in order to have this change take effect. The user must delete the existing default certificate and key from /var/mps/ssl_certs and /var/mps/ssl_keys respectively.



New SDX Platform Features:Cluster Link Aggregation (CLAG)SDX can support Cluster Link Aggregation Groups on a per interface basis, so long as there is only one network per SDX interface. This method is rarely used with SDX clustering due to the limitation to only one VLAN per interface. CLAG is an L2 Channel that can be either Static or Dynamic and the upstream switch sees a single cluster MAC address in the ARP table. CLAG is used to distribute traffic across clusters using a fat pipe.

CLAG NOTES:1. A separate physical medium is required for Client connection steering and node-to-node commu-nications. 2. Cluster Heartbeats cannot be exchanged over the CLAG interfaces. 3. Standalone VPX appliances are not supported with CLAG, some ESX and KVM versions can support CLAG.

SDX Platform Improvements:

VPX Scaling:The purpose of VPX scaling is to enable hardware (HW) Receive Side Scaling (RSS) for the Intel Fortville interfaces on the NetScaler SDX and to enable users to provision VPX using maximum resources from the SDX. As a result, SVM allows VPX with 16 cores on the 25xxx 40G appliances and 10 cores on the 14xxx 40G appliances. Additionally, SVM enables VPX to use cores from both of the sockets (each appliance has two sockets). In order to do this, the maximum number of virtual functions (VF) per interface is set at 20 VF. Previously, the maximum number of VF per interface was 32 VF for a 40G interface. Enabling HW RSS and thereby enabling VPX scaling allows maximum throughput per VM of up to 180Gbps on all 16 cores on the 25xxx series SDX. Throughput per VM on non-RSS capable SDX appliances is 35Gbps.

SNMP Standardization:SNMP standardization allows a standard SNMP (Simple Network Management Protocol) MIB-2 (Manage-ment Information Base-II) table walk for interfaces and channel details to be conducted on the NetScaler SDX. Previously, SDX interface details were not exposed through the standard MIB-2 OID (Object Identifier). Channel details were not exposed over SNMP at all. As a result, an SDX admin had to use vendor-specific OID’s to poll SDX interfaces. Now, interface and channel details on the SDX are exposed through the stand-ard SNMP MIB-2 table through OID .1.3.6.1.2.1.2.

SNMP Alarm Thresholds and Timeout Options:SNMP alarm thresholds and timeout options were added to the NetScaler SDX configuration to allow SNMP alarm thresholds and timeout frequency to be configured. Previously, alarm thresholds were not configurable. An alarm was raised only once until it was cleared, and the timeout frequency was not config-urable at all. Now, SDX allows setting a threshold for SNMP alarms. A threshold is configured as a percent-age. If a threshold is configured and the current usage for that particular monitor goes above the threshold, then a high event is raised. Once the usage goes below the threshold, a clear event is raised. SDX also allows setting the frequency for SNMP alarms. A timeout is configured in minutes. Once an event is raised, the SVM will wait for the configured timeout minutes before repeating that event (if it does not get cleared within that time period).



Multiple DNS Servers:Multiple DNS (Domain Name System) Servers allow support for configuring additional DNS servers on the NetScaler SDX. Previously, the SDX supported only a single DNS server on an SVM. If that DNS server became unreachable, then the SVM would not be able to reach any of the configured hosts. This could also lead to a total block on external authentica-tion. SVM now supports configuring as many as two additional DNS servers. This adds redundancy to the DNS configura-tion on the SDX. The primary (First) server is strictly IPv4 since it is configured on Xen (DOM0), but additional DNS servers can be a combination of IPv4 and IPv6.

Optional Admin User While VPX Provisioning:The SVM on the NetScaler SDX previously created a mandatory admin user account in a VPX while provisioning that VPX. This feature may have been objectionable to enterprises that have tight security policies. Now, the SVM has made creation of the admin account optional in a VPX while provisioning. This saves admins the extra step they would have to take to delete this account after provisioning.

CPU Visualizer:Previously, the CPU layout on the SVM of the NetScaler SDX had a fluid form but its representation was static and tabular. There was no visible distinction between committed, shared, reserved, or available CPU cores. The administrator could not determine the number of VM’s that could be provisioned in dedicated or shared mode, and load distribution across CPU sockets was not visible. The CPU Visualizer allows users to determine the number of CPU cores committed, shared, reserved, or available. The CPU Visualizer allows users to determine the number of VM’s that can be provisioned in dedicated or shared mode, and it allows users to view load distribution across CPU sockets.

Clean Install for All Platforms:A clean install for all NetScaler SDX platforms allows a clean install of images for all existing deployed appliances. Previously, a clean install was supported only for newly manufactured SDX platforms (containing a 10GB factory partition). This made the feature una-vailable for existing appliances in the field. Now, SVM checks for space on the factory partition in order for clean install preparation. If enough space is found, then a clean install can proceed. Otherwise, the user is informed that insufficient space exists to perform a clean install.

Section 2:

NetScaler SDXThe Citrix NetScaler SDX appliance is a multitenant platform on which a user can provision and manage multiple virtual NetScaler machines (instances). The SDX appliance addresses cloud computing and multitenancy requirements by allowing a single ad-ministrator to configure and manage the appliance and delegate the administration of each hosted instance to tenants. The SDX appliance enables the appliance administrator to provide each tenant the following benefits:

• One complete instance, with each instance having the following privileges: • Dedicated CPU and memory resource • A separate space for entities • The independence to run the release and build of their choice • Lifecycle independence • A completely isolated network. Traffic meant for a particular instance is sent only to that instance.

The Citrix NetScaler SDX appliance provides a Management Service that is pre-provisioned on the appliance. The Management Ser-vice provides a user interface (HTTP and HTTPS modes) and an API to configure, manage, and monitor the appliance, the Manage-ment Service, and the instances. A Citrix self-signed certificate is prepackaged for HTTPS support. Citrix recommends that you use the HTTPS mode to access the Management Service user interface.



A virtualized multi-tenant ADC should offer datacenter managers the following capabilities:

• High consolidation density – Enabling a large number of ADC instances to run on a single platform, each with its own policy, configuration and dedicated system resources.

• Complete isolation of ADC resources – With 100% isolation of compute, memory and ADC processing resources (including SSL acceleration and data compression) ensuring that the performance of one ADC instance never impacts another.

• Full ADC feature support – Consolidation requires that all existing ADC footprints can be consolidated without a loss of functionality.

• Pay-As-You-Grow Scalability – Datacenter managers must have the ability to scale overall ADC capacity on-de-mand without adding additional hardware.

NetScaler SDX specifically offers these capabilities because it enables multiple, independent, full-featured NetScaler instances to run on a single physical appliance. NetScaler SDX is an optimized combination of two proven solutions in their own right, NetScaler VPX and Citrix XenServer. It enables today’s organizations to reduce their ADC footprint and total cost of ownership (TCO) by pursuing opportunities for both horizontal and vertical consolidation of discrete, standalone ADC devices. NetScaler SDX squarely meets the four fundamental requirements for a natively virtualized ADC consolidation solution:

• Density – Up to 115 NetScaler ADC instances can run independently on a single NetScaler SDX platform, depend-ing on the SDX platform. This impressive level of density supports the most ambitious consolidation projects.

• Isolation – All critical system resources, including memory, CPU and SSL processing capacity are assigned to individual NetScaler instances. This is essential to ensuring that resource demands made by one tenant do not negatively impact other tenants running on the same physical system. It also provides greater security for each ADC instance by providing full separation of traffic flows.

• Full ADC Functionality – NetScaler SDX supports 100 percent of the ADC functionality available with both hardware-based NetScaler MPX appliances and software-based NetScaler VPX virtual appliances. This enables NetScaler SDX to consolidate all existing ADC deployments with virtually no policy constraints.

• Pay-As-You-Grow – The Pay-As-You-Grow option delivers on-demand elasticity enabling organizations to easily scale ADC capacity to keep pace with application traffic growth. Since it leverages a software-based architecture, NetScaler SDX can scale performance and capacity with a simple software key, eliminating expensive hardware purchases and upgrades.

NetScaler SDX Use CasesFor networking components (such as firewalls and Application Delivery Controllers), support for multi-tenancy has historically involved the ability to carve a single device into multiple logical partitions. This approach allows different sets of policies to be implemented for each tenant without the need for numerous, separate devices. Traditionally, it is severely limited in terms of the degree of isolation that is achieved.

The NetScaler SDX appliance is not subject to the same limitations. In the SDX architecture, each instance runs as a separate virtual machine (VM) with its own dedicated NetScaler kernel, CPU resources, memory resources, address space, and bandwidth allocation. Network I/O (input/output) on the SDX appliance not only maintains aggregate sys-tem performance but also enables complete segregation of each tenant's data-plane and management-plane traffic. The management plane includes the 0/x interfaces. The data plane includes the 1/x and 10/x interfaces. A data plane can also be used as a management plane.

The most common use cases for an SDX appliance relate to consolidation and reducing the number of networks re-quired while maintaining management isolation. Following are the basic consolidation scenarios or use cases:

• Consolidation when the Management Service and the NetScaler instances are in the same network • Consolidation when the Management Service and the NetScaler instances are in different networks but all the

instances are in the same network



• Consolidation across security zones• Consolidation with dedicated interfaces for each instance• Consolidation with sharing of a physical port by more than one instance

NetScaler SDX:

Section 3:

Default SSL Certificate Use CasePreviously, the default SSL certificate on the Service Virtual Machine (SVM) of the NetScaler SDX was set as a 1024-bit key certificate. The purpose of this change is to increase the size of the default SSL certificate on the SVM to a 2048-bit key certificate. The user must delete the existing default SSL certificate and restart the SVM in order to have this change take effect. The user must delete the existing default certificate and key from /var/mps/ssl_certs and /var/mps/ssl_keys respectively.

Increases the size of the default SSL certificate on the SVM of the NetScaler SDX to a 2048-bit key certifi-cate.



New Features

Cluster Link Aggregation (CLAG) Use CaseSDX can support Cluster Link Aggregation Groups on a per interface basis, so long as there is only one network per SDX interface. (This method is rarely used with SDX clustering due to the limitation to only one VLAN per interface.) CLAG is an L2 Channel that can be either Static or Dynamic and the upstream switch sees a single cluster MAC address in the ARP table. CLAG is used to distribute traffic across clusters using a fat pipe. Previously, CLAG support was miss-ing from the SDX. Now, CLAG is supported on the SDX.

CLAG NOTES:1. A separate physical medium is required for Client connection steering and node-to-node communications. 2. Cluster Heartbeats cannot be exchanged over the CLAG interfaces. 3. VPX appliances are not supported with CLAG, some ESX and KVM versions can support CLAG.

Provides CLAG support for SDX.

Improvements

VPX Scaling Use CaseThe purpose of VPX scaling is to enable hardware (HW) Receive Side Scaling (RSS) for the Intel Fortville interfaces on the NetScaler SDX and to enable users to provision VPX using maximum resources from the SDX. As a result, SVM allows VPX with 16 cores on the 25xxx 40G appliances and 10 cores on the 14xxx 40G appliances. Additionally, SVM enables VPX to use cores from both of the sockets (each appliance has two sockets). In order to do this, the maximum number of virtual functions (VF) per interface is reduced to 20 VF. Previously, the maximum number of VF per interface was 32 VF for a 40G interface. Enabling HW RSS and thereby enabling VPX scaling allows maximum throughput per VM of up to 180Gbps on all 16 cores on the 25xxx series SDX. Throughput per VM on non-RSS capable SDX appliances is 35Gbps.

Enable HW RSS for the Intel Fortville interfaces on the NetScaler SDX and enable users to provision VPX using maxi-mum resources from the SDX.

SNMP Standardization Use CaseSNMP standardization allows a standard SNMP (Simple Network Management Protocol) MIB-2 (Management Informa-tion Base-II) table walk for interfaces and channel details to be conducted on the NetScaler SDX. Previously, SDX inter-face details were not exposed through the standard MIB-2 OID (Object Identifier). Channel details were not exposed over SNMP at all. As a result, an SDX admin had to use vendor-specific OID’s to poll SDX interfaces. Now, interface and channel details on the SDX are exposed through the standard SNMP MIB-2 table through OID .1.3.6.1.2.1.2.

Allow a standard SNMP MIB-2 table walk for interfaces and channel details to be conducted on the NetScaler SDX.



SNMP Alarm Thresholds and Timeout Options Use CaseSNMP alarm thresholds and timeout options were added to the NetScaler SDX configuration to allow SNMP alarm thresholds and timeout frequency to be configured. Previously, alarm thresholds were not configurable. An alarm was raised only once until it was cleared, and the timeout frequency was not configurable at all. Now, SDX allows setting a threshold for SNMP alarms. A threshold is configured as a percentage. If a threshold is configured and the current usage for that particular monitor goes above the threshold, then a high event is raised. Once the usage goes below the threshold, a clear event is raised. SDX also allows setting the frequency for SNMP alarms. A timeout is configured in minutes. Once an event is raised, the SVM will wait for the configured timeout minutes before repeating that event (if it does not get cleared within that time period).

Allow SNMP alarm thresholds and timeout frequency to be configured.

Multiple DNS Servers Use CaseMultiple DNS (Domain Name System) Servers allow support for configuring additional DNS servers on the NetScaler SDX. Previously, the SDX supported only a single DNS server on an SVM. If that DNS server became unreachable, then the SVM would not be able to reach any of the configured hosts. This could also lead to a total block on external au-thentication. SVM now supports configuring as many as two additional DNS servers. This adds redundancy to the DNS configuration on the SDX. The primary (First) server is strictly IPv4 since it is configured on Xen (DOM0), but additional DNS servers can be a combination of IPv4 and IPv6.

Allow support for configuring additional DNS servers on the NetScaler SDX.

Optional Admin User While VPX Provisioning Use CaseThe SVM on the NetScaler SDX previously created a mandatory admin user account in a VPX while provisioning that VPX. This feature may have been objectionable to enterprises that have tight security policies. Now, the SVM has made creation of the admin account optional in a VPX while provisioning. This saves admins the extra step they would have to take to delete this account after provisioning.

Allow the SVM to create an optional admin account in a VPX while provisioning.

CPU Visualizer Use CasePreviously, the CPU layout on the SVM of the NetScaler SDX had a fluid form but its representation was static and tabular. There was no visible distinction between committed, shared, reserved, or available CPU cores. The user could not determine the number of VM’s that could be provisioned in dedicated or shared mode, and load distribution across CPU sockets was not visible. The CPU Visualizer allows users to determine the number of CPU cores committed, shared, reserved, or available. The CPU Visualizer allows users to determine the number of VM’s that can be provi-sioned in dedicated or shared mode, and it allows users to view load distribution across CPU sockets.

The CPU Visualizer:

• Allows users to determine the number of CPU cores committed, shared, reserved, or available.• Allows users to determine the number of VM’s that can be provisioned in dedicated or shared mode.• Allows users to view load distribution across CPU sockets.



Clean Install Procedure for All Platforms Use CaseA clean install for all NetScaler SDX platforms allows a clean install of images for all existing appliances in the field. Previously, a clean install was supported only for newly manufactured SDX platforms (containing a 10GB factory parti-tion). This made the feature unavailable for existing appliances in the field. Now, SVM checks for space on the factory partition in order for clean install preparation. If enough space is found, then a clean install can proceed. Otherwise, the user is informed that insufficient space exists to perform a clean install.

Allow a clean install of images for all existing appliances in the field.

Configuration

NetScaler Default SSL Certificate Configuration

• Delete the existing default SSL certificate and restart the SVM.• Delete the default SSL certificate and key from /var/mps/ssl_certs and /var/mps/ssl_keys respectively.

Cluster Link Aggregation (CLAG) ConfigurationSDX Build

Configure NetScaler SVMs

Install the SVM bundle.

Configure NetScaler Instances

Provision the NetScaler instances from the SVM with NSIP and default gateway on the cluster backplane VLAN.

Cluster BuildConfigure Cluster Instance

Create a Cluster instance from the SVM on appliance (1)Assign the unique Cluster IP (CLIP) Address

Configure Additional Cluster Nodes

Join Cluster node members from each SVM on which they are hosted into the Cluster Group (Default _Group).

Configure the Cluster Node Priorities

The default SVM assigned priority for a cluster node is 31. This is set on all nodes and can affect the cluster coordina-tor (CCO) election process when a node member leaves or joins the cluster. Change the priority for each node to give first, second and third priority for CCO elections. Setting the Cluster Priority ID allows for management of the Default CCO process.



Route ManagementPolicy Based Routes

Use Policy Based Routes to manage CLIP and SNIP communication to the SVM instance. The may require more than one PBR rule depending on the next hop definition and how many nodes are in the cluster.

Adjust Default Route Change the Default Route to forward all VServer traffic to the production network, from the default route that was assigned by the SVM at time of provisioning.

Create Spotted SNIP(s) From the Cluster IP, configure a spotted SNIP for each member node of the cluster with an IP address from the Traffic VLAN. These IP addresses all need to be on the same network segment. Enable Dynamic Routing. A Spotted SNIP is required for each cluster node that will participate in ECMP.

Configure Virtual Server (2 steps required)

• Create LBVserver for HTTP service Enable RHI State on the LBVserver

• Configure the Striped VIP to the corresponding Vserver Set the LSA value = 1 Set the OSPF AREA value (xx) Set Host Route Gateway (Striped SNIP)

Configure Dynamic RoutingThe dynamic routing configuration (OSPF in this example) for the SDX cluster should be completed on the Cluster Coordinator (CCO) via the Cluster IP address (CLIP).

OSPF ConfigurationDefining the “owner-Node” command for each member node of the cluster will allow the CCO to distribute the OSPF routes to all members of the cluster.

SNMP Alarm Thresholds and Timeout OptionsThe following configuration steps are configured in the user interface (UI):

• Configure alarm name.• Check the box to enable the alarm.• Set the alarm severity.• Set the alarm threshold.• Set the alarm timeout time in minutes.• Click OK



Multiple DNS Servers ConfigurationThe following configuration steps are configured in the user interface (UI):

• Select the interface to be configured. • Configure the appliance supportability IP address. • Configure the gateway IP address.• Configure the primary DNS IP address (must be an IPv4 IP address)• Configure the appliance management IP address.• Configure the netmask.• Configure additional DNS IP address

• May configure up to two additional DNS servers.• May be a combination of IPv4 and IPv6 IP addresses.

Optional Admin User While VPX ProvisioningThe following configuration steps are configured in the user interface (UI):

• In the Resource Allocation box under “Instance Administration” check the “Add Instance Administration” box to enable the optional admin user account while provisioning a VPX instance.

• Go to the “Instance Administration” box• Configure the optional admin user name.• Configure the optional admin user password.• Confirm the optional admin user password.

• If no admin user account is desired during VPX provisioning, simply leave the “Add Instance Administration” box unchecked in the Resource Allocation box.

CPU Visualizer ConfigurationThe following views are available for the CPU Visualizer:

• Summary view.• Hyper thread view.

Section 4:

Configuration Example: NetScaler SDX CLAG The following NetScaler SDX configuration delineates the CLAG configuration:

Configuration Example Created Instances from SVM set ns config -IPAddress 192.168.7.11 -netmask 255.255.255.0 add vlan 301 -sdxVlan YES set interface 10/1 -lacpMode ACTIVE -lacpKey 2 -ifnum LA/2 set interface 10/2 -lacpMode ACTIVE -lacpKey 2 -ifnum LA/2 set interface 10/3 -lacpMode ACTIVE -lacpKey 3 -ifnum LA/3 set interface 10/4 -lacpMode ACTIVE -lacpKey 3 -ifnum LA/3



set channel LA/2 -tagall ON set channel 0/LA/3 add route 0.0.0.0 0.0.0.0 192.168.7.1 -routeType STATIC

set ns config -IPAddress 192.168.7.12 -netmask 255.255.255.0 add vlan 301 -sdxVlan YES set interface 10/1 -lacpMode ACTIVE -lacpKey 2 -ifnum LA/2 set interface 10/2 -lacpMode ACTIVE -lacpKey 2 -ifnum LA/2 set interface 10/3 -lacpMode ACTIVE -lacpKey 3 -ifnum LA/3 set interface 10/4 -lacpMode ACTIVE -lacpKey 3 -ifnum LA/3 set channel LA/2 -tagall ON set channel 0/LA/3 add route 0.0.0.0 0.0.0.0 192.168.7.1 -routeType STATIC

set ns config -IPAddress 192.168.7.13 -netmask 255.255.255.0 add vlan 301 -sdxVlan YES set interface 10/1 -lacpMode ACTIVE -lacpKey 2 -ifnum LA/2 set interface 10/2 -lacpMode ACTIVE -lacpKey 2 -ifnum LA/2 set interface 10/3 -lacpMode ACTIVE -lacpKey 3 -ifnum LA/3 set interface 10/4 -lacpMode ACTIVE -lacpKey 3 -ifnum LA/3 set channel LA/2 -tagall ON set channel 0/LA/3 add route 0.0.0.0 0.0.0.0 192.168.7.1 -routeType STATIC

Initial Created Cluster Config add cluster instance 1 add cluster node 0 192.168.7.11 -state ACTIVE -backplane 0/LA/3 add cluster node 1 192.168.7.12 -state ACTIVE -backplane 1/LA/3 add cluster node 2 192.168.7.13 -state ACTIVE -backplane 2/LA/3 bind cluster nodegroup DEFAULT_NG -node 0 bind cluster nodegroup DEFAULT_NG -node 1 bind cluster nodegroup DEFAULT_NG -node 2 enable cluster instance 1 add ns ip 192.168.7.14 255.255.255.255 -type CLIP -vServer DISABLED -mgmtAccess ENABLED add route 0.0.0.0 0.0.0.0 192.168.7.1 -routeType STATIC

Add traffic VLAN to cluster add vlan 401

Add spotted SNIPs: add ns ip 192.168.6.11 255.255.255.0 -vServer DISABLED -gui DISABLED -dynamicRouting ENABLED -owner Node 0 add ns ip 192.168.6.12 255.255.255.0 -vServer DISABLED -gui DISABLED -dynamicRouting ENABLED -owner Node 1 add ns ip 192.168.6.13 255.255.255.0 -vServer DISABLED -gui DISABLED -dynamicRouting ENABLED -owner Node 2



Add striped SNIP: add ns ip 192.168.6.14 255.255.255.0 -vServer DISABLED -gui DISABLED

BIND traffic VLAN to cluster interfaces and striped SNIP address bind vlan 401 -ifnum 0/LA/2 -tagged bind vlan 401 -ifnum 1/LA/2 -tagged bind vlan 401 -ifnum 2/LA/2 -tagged bind vlan 401 -IPAddress 192.168.6.14 255.255.255.0

Add PBR for cluster backplane traffic add ns pbr Traffic_PBR ALLOW -srcIP "!=" 192.168.7.11-192.168.7.14 -nextHop 192.168.7.1 -priority 10 apply ns pbrs

Change default route to use traffic VLAN add route 0.0.0.0 0.0.0.0 192.168.6.1 -routeType STATIC rm route 0.0.0.0 0.0.0.0 192.168.7.1

Enable Dynamic Routing enable ns feature OSPF

Add Server: add server HTTP-Test 127.0.0.1

Create Service or Service Group: add service HTTP-Test-Svc HTTP-Test HTTP 80 -gslb NONE -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -sp ON -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

Create LB vServer add lb vserver HTTP-Test-LBVS HTTP 192.168.5.1 80 -persistenceType NONE -cltTimeout 180 -RHIstate ACTIVE

Bind Service to LB vServer bind lb vserver HTTP-Test-LBVS HTTP-Test-Svc

Enable Dynamic Routing on VIP for LB vServer add ns ip 192.168.5.1 255.255.255.255 -type VIP -snmp DISABLED -hostRoute ENABLED -hostRtGw 192.168.6.14 -ospfLSAType TYPE1 -ospfArea 51

Configure OSPF in VTYSH config t ! log syslog ! log record-priority ! interface lo0 ! interface vlan0 !



interface vlan107 ip ospf priority 0 ! router ospf 1 owner-node 0 ospf router-id 192.168.6.11 exit-owner-node owner-node 1 ospf router-id 192.168.6.12 exit-owner-node owner-node 2 ospf router-id 192.168.6.13 exit-owner-node auto-cost reference-bandwidth 100000 redistribute kernel passive-interface vlan0 area 51 stub network 192.168.6.0/24 area 51 ! exit ! write mem !

NetScaler SDX: Improvements

SNMP Standardization



SNMP Alarm Thresholds and Timeout Options



Multiple DNS Servers



Optional Admin User While VPX Provisioning

Add VPX



Edit VPX



CPU Visualizer

Summary View



Hyper thread View



Appendix A: References

The following links provide further reading on NetScaler SDX security isolation such as virtual machine isolation, NIC isolation, CPU isolation, SSL isolation, and memory isolation:

“Security Recommendations When Deploying Citrix XenServer,”https://www.citrix.com/content/dam/citrix/en_us/documents/white-paper/security-recommenda-tions-when-deploying-citrix-xenserver.pdf

“Use Cases for NetScaler SDX Appliances,”

http://docs.citrix.com/ja-jp/sdx/11/use-cases-for-netscaler-sdx.html

“Consolidation without compromise,”https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/consolidation-with-out-compromise.pdf

“NetScaler multi-tenancy shootout: SDX vs. Traffic Domains,” https://www.citrix.com/blogs/2013/05/28/netscaler-multi-tenancy-shootout-sdx-vs-traffic-domains/

“Maximizing Multi-tenancy with Citrix NetScaler,”https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/maximizing-multi-tena-cy-with-citrix-netscaler-en.pdf

“CITRIX SYNERGY: FIVE (VERY COOL) THINGS YOU DIDN’T KNOW WERE POSSIBLE WITH NETSCALER,” https://wagthereal.com/2011/05/31/citrix-synergy-five-very-cool-things-you-didnt-know-were-possible-with-net-scaler/

“NetScaler SDX Appliance with SR-IOV and Intel-VTd,”https://richardegenas.com/tag/sr-iov/

“INTEL VIRTUALIZATION TECHNOLOGY (INTEL VT),”https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-tech-nology.html

“Intel Virtualization Technology for Directed I/O (VT-d): Enhancing Intel platforms for efficient virtualization of I/O devices,”

https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhancing-in-tel-platforms-for-efficient-virtualization-of-io-devices

https://www.citrix.com/content/dam/citrix/en_us/documents/white-paper/security-recommendations-when-

https://www.citrix.com/content/dam/citrix/en_us/documents/white-paper/security-recommendations-when-

http://docs.citrix.com/ja-jp/sdx/11/use-cases-for-netscaler-sdx.html

https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/consolidation-without-c

https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/consolidation-without-c

https://www.citrix.com/blogs/2013/05/28/netscaler-multi-tenancy-shootout-sdx-vs-traffic-domains/

https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/maximizing-multi-tenacy

https://www.citrix.com/content/dam/citrix/en_us/documents/products-solutions/maximizing-multi-tenacy

https://wagthereal.com/2011/05/31/citrix-synergy-five-very-cool-things-you-didnt-know-were-possible-

https://wagthereal.com/2011/05/31/citrix-synergy-five-very-cool-things-you-didnt-know-were-possible-

https://richardegenas.com/tag/sr-iov/

https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualizatio

https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualizatio

https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhan

https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhan

28Citrix.com | Solution Guide | Validated Reference Guide for NetScaler SDX Update

Solution GuideValidated Reference Design Guide for NetScaler SDX Update

About Citrix

Citrix (NASDAQ: CTXS) is a leader in mobile workspaces, providing virtualization, mobility management, net-

working and cloud services to enable new ways to work better. Citrix solutions power business mobility through

secure, personal workspaces that provide people with instant access to apps, desktops, data and communica-

tions on any device, over any network and cloud. This year Citrix is celebrating 25 years of innovation, making

IT simpler and people more productive. With annual revenue in 2013 of $2.9 billion, Citrix solutions are in use at

more than 330,000 organizations and by over 100 million users globally. Learn more at www.citrix.com.

Copyright © 2014 Citrix Systems, Inc. All rights reserved. Citrix, NetScaler MPX, NetScaler SDX, NetScaler, Cloud-Bridge and AppFlow are trademarks of Citrix Systems, Inc. and/or one of its subsidiaries, and may be registered in the U.S. and other countries. Other product and company names mentioned herein may be trademarks of their respective companies.

validated reference design netscaler sdx platform - citrix · validated reference design netscaler...

Documents