hosted desktop and evolution of hardware server technologies - 2015 edition

White Paper Hosted Desktop - The Evolution of Hardware-Assisted Server Technologies

1 citrix.com

2015 Edition Hosted Desktop The Evolution of Hardware-Assisted Server Technologies

Ahmed Sallam, VP and CTO Hardware, Security, IP and Emerging Solutions.


2 citrix.com

Executive Summary New in the 2015 Edition The first version of this report was published in 2014. Since its initial publication, this report has gained attention from a variety of Citrix Hardware Server Partners, CIOs and IT personnel. It has become a good reference while evaluating possible technology offerings that help improve their experience deploying Hosted Virtual Desktops / Virtual Desktop Infrastructure (VDI). The aim of this updated report is to provide a quick summary of key evolving server technologies along with the main benefits, which include: improved security, flexibility, agility and resilience along with reduced cost, loss of service and downtime. Many newly emerging server technologies are making VDI operate faster at reduced cost with improved speed and density. For example:

• In-memory computing speeds up database performance many fold compared to off-chip processing.

• Using flash memory to replace mechanical hard drives increases IOPs by many folds. The 2015 Edition provides coverage for the following evolving server technologies:

• Fabric-Based Infrastructure / Unified Computing • Fabric-Based Computing • Persistent Memory and In-Memory Computing (IMC)-Optimized Servers • Server Virtual IO

Finally, we include a new reference to the ARM SBSA initiative, which includes improved content quality and technical accuracy. Overview Four key server hardware technologies are shaping the future of Desktop Virtualization:

1. Hardware-Assisted System Virtualization 2. Hardware-Assisted System Security 3. Hardware Servers Physicalization 4. Integrated, Converged and Fabric based Infrastructure

Hardware-assisted virtualization is happening everywhere for CPUs, Memory, I/O and GPUs. Virtualization allows XenDesktop to scale up to take the best advantage of existing compute power in system hardware. Microservers are driving innovation further letting desktop physicalization scale to take advantage of low-cost, commodity hardware yielding better performance per watt, higher density and lower cost. Hardware-assisted security is changing the face of computing making IT infrastructure safer at the bottom of the system architecture stack outside the reach of software. Citrix is actively engaged with the hardware ecosystem vendors to ensure better design and enablement of various types of hardware-assisted features to enable delivering unique and unprecedented enterprise mobility experience.


3 citrix.com

This report provides the reader with enough technical insights covering the four emerging server technology areas. The content targets Citrix customers and field engineers who have basic understanding of data centers infrastructure architecture as well as system virtualization. The paper is not intended for those readers looking for deep technical description of each technology or for those readers looking for a high-level, not so-technical description. Converged Infrastructure aims at consolidating, combining and unified the essential hardware and software elements of the data center including: compute, storage, networking, power, cooling, management, connectivity, redundancy, and security into a modular, flexible, scalable, converged infrastructure with built-in intelligence and support for future technology expansion. Various technology alternatives are available in the market today, including: "converged system", "unified computing", "fabric-based computing", and "dynamic infrastructure". A unified integrated and converged data center infrastructure provides various long-term advantages including:

• Lowering cost of capital expenses resulting from higher utilization, less cabling, and fewer network connections. Also, lowering operating cost as a result of reduced labor via increased intelligent automation of data center management along with consolidation of servers, storage and network management infrastructure teams.

• Increasing IT agility by virtualizing IP and Fiber Channel storage networking allowing for single console management.

• Predictable scaling costs as solutions are purchased as a single unit with compute and storage for a fixed number of seats, with variable available levels of performance. This allows IT to pay-as-they-grow via scale-out expansion.

Citrix has collaborated with x86 Hardware server vendors such as Intel, AMD, HP, IBM, Dell, and Cisco along with Microsoft® at an engineering level to ensure that IT providers can benefit from software, hardware, and service solutions that are jointly tested, certified, and tuned to deliver optimal server performance. Citrix has also collaborated with ARM Holding and its ecosystem vendors to deliver baseline system architecture for evolving ARM servers Architecture. As part of that effort, Citrix held the status of a founding member of Linaro Enterprise Group in 2013 and 2014. Furthermore, Citrix has ported Linux Xen Project hypervisor into the ARM architecture, which has been and is being validated with ecosystem vendors such as: HP, AMD, Cavium and AppliedMicro.


4 citrix.com

Table of Contents Background

Evolution of Server Physicalization and Software Defined Servers

Hardware-Assisted System Virtualization

GPU Virtualization: The art of sharing GPUs across virtual machines

Intel® Hardware-Assisted Security Technologies

Evolving Converged Server Fabric Technologies

Fabric-Based Infrastructure / Unified Computing

Fabric-Based Computing

Persistent Memory and In-Memory Computing (IMC)-Optimized Servers

Server Virtual IO

Closing Notes

References


5 citrix.com

Background Introduction For over two decades Desktop Virtualization has revolutionized the IT industry through reduced cost, simplified centralized management, better security, flexibility, visibility, scalability and higher availability. Citrix XenDesktop has been the industry leading solution for both desktops and applications virtualization in the datacenter and as a service in the cloud. Hardware server technologies have played a key role in enabling desktop virtualization. This paper talks about specific current and emerging server hardware technologies that make desktop virtualization faster, simpler, safer, less expensive and highly scalable.

Intel, NVIDIA, AMD and HP In this paper we cover many of Intel’s server hardware technologies. The focus on Intel’s technologies is natural given Intel’s market leadership as the provider of very large-scale hardware compute servers. NVIDIA has recently developed their technology for server GPU virtualization that is also covered. AMD and HP have collaborated closely to deliver x86 Microservers, which address the growing need for system physicalization and this topic is covered in the paper as well.

Hosted Windows Desktops on ARM microservers Citrix XenDesktop runs today on top of x86-based hardware servers. XenDesktop manages Windows in the enterprise and as a cloud-based desktop service. ARM-based Microservers are growing in popularity entering the market with specific focus on web, cloud and big data workloads. Citrix has been active in the ARM microserver space as outlined below:

• Collaborating and engaging closely with ARM Corporation on server architecture and specification.

• Engaging with ARM hardware microserver providers like AppliedMicro, Cavium, AMD and Marvell.

• Being an active member of Linaro’s Enterprise Group. • Porting the Citrix Xen Project Hypervisor to the ARM architecture.

Citrix has collaborated closely with ARM Holdings and its key partners in unifying the hardware server architecture along with the software stack powering servers in the enterprise on-premise data centers and with the cloud infrastructure. Some of the key developments are:

• Supporting the ARM Server Base System Architecture (SBSA) specifications. • Porting the Xen Project Hypervisor to the ARMv8A 64 bit architecture.

For interested readers, the SBSA provides the following capabilities to ARM ecosystem:

• Single OS image for all ARMv8-A based servers. • Application developers can target a single OS image for All ARMv8-A servers. • OS / Firmware vendors have well defined platform target.

From an operating system support standpoint, the focus of ARM microserver ecosystem vendors has been mainly on Linux given that Windows server OS has not been made available as yet on the ARM architecture. Microsoft has not publicly disclosed plans for doing so in the near future. Given these reasons, ARM architecture will not be covered in this paper.


6 citrix.com

Evolution of Server Physicalization and Software Defined Servers In the rapidly growing Internet of Things environment, many things we do every day, such as checking email accounts, posting onto social media sites, browsing web pages, and searching web indexes or portals are not necessarily compute-intensive. They do however; have high I/O throughput and memory footprint requirements. IT architects working at this scale typically use cluster techniques to run massively parallel workloads that distribute data across many nodes, often in cloud environments. Using typical server x86 CPUs designed for compute-intensive enterprise applications in these environments means underutilizing compute capacity and wasting energy. Distributed workloads in cloud environments often run at low processor utilization levels of 20% or less, yet administrators pay for the cost of a premium CPU. Virtualization has historically addressed the issue of low CPU and GPU utilization by allowing IT architects to consolidate multiple workloads that are somewhat balanced, such as enterprise applications or infrastructure-as-a-service. Virtualization provides great resiliency to the data centers allowing virtual machines to shift to alternative redundant hardware resources when a primary system resource fails or runs out of capacity. Physicalization, on the other hand addresses the need to scale-out applications and web serving, where the I/O component is much larger and the amount of processing required per unit of data is much smaller. In these environments, consolidating through virtualization effectively reduces the network, memory, and I/O bandwidth per unit of data, which makes the large I/O problem worse. Physicalization takes the approach of using energy-efficient CPUs that balance performance and cost to match the needs of data-intensive applications.

Figure 1: XenDesktop managing hosted desktops in scale-out physical data centers.

The datacenter environment is diversifying both in terms of the infrastructure and the market segments including storage, communications, cloud, HPC, and traditional enterprise. Each area


7 citrix.com

has a unique requirement, which provides an opportunity for targeted solutions to best cover these needs. Microservers are characterized by large numbers of server nodes configured to share infrastructure, such as power and cooling fans in a common chassis. A microserver is typically comprised of many small one-socket servers sharing a chassis; fans, power supplies and a common interconnect to achieve improved flexibility, higher efficiency and density. The microserver represents a new server architecture characterized by many lightweight server nodes bundled together in a shared chassis infrastructure. This topology is designed specifically for density, lower power per node, reduced costs, and increased operational efficiency. By sharing common fans, switching, power supplies, and the metal chassis, microservers streamline rack infrastructure and improve density over standard rack options.

Figure 2: Innovation in Server Form factors.

Figure 3: Intel ATOM C2000 four SoCs Card

The Intel® Atom® processor C2000 product family is Intel’s second-generation 64-bit server System on Chip (SoC) manufactured in a low power SoC 22nm process. Their focus is on enabling high density with high performance providing 2, 4, and 8 core product models at 6-20 Watts of power consumption. That extends Intel’s existing portfolio of products that service the cloud service providers. Optimized for parallel software that benefit most from more individual servers with sufficient I/O between nodes including static web servers, simple content delivery node, distributed memory caching (memcached), entry dedicated hosting, cold storage, and any of the afore-mentioned uses that have an additional need for acceleration of cryptographic communications such as entry level security appliances and switches.


8 citrix.com

Up to four Intel® Atom® SoC nodes can be added on to a Server System Infrastructure (SSI) module. Multiple SSI modules can be added to a single microserver chassis to expand the number of accessible nodes. This allows for optimization of rack density as compared to other single unit servers. HP® Moonshot Hyperscale Microservers HP Moonshot System is a new server design that addresses the speed, scale and specialization required for the new style of IT that is emerging around the converging trends of mobility, cloud, social media, and big data. With billions of people connected with each other and with businesses over the Internet, many of them from mobile devices, there is a rapidly escalating demand for digital content and experiences. The connection of almost any device to the Internet has become known as the Internet of Things (IoT). These devices can gather and process data, provide a service, and seamlessly interact with other devices. The IoT presents businesses with new ways to drive market differentiation, deepen customer relationships, and deliver profitability. These specialized IoT solutions require a new style of computing, one that can achieve optimal performance and efficient scaling. A key issue that overwhelms IT managers in hyperscale environments is the sheer number of devices they must manage, power, and cool. With today’s rack-mount x86 platforms, you can have between 20 and 40 servers in a 42U rack. Scale-out optimized platforms like HP ProLiant SL can increase the density to 80 servers in each rack. Each server comes with its own management controller, network controllers, storage controllers, OS instance, device drivers, and so on. So every time you add a server, you must also procure multiple I/O devices and manage, secure, power, and cool them. While HP Blade System c-Class enclosures also provide a shared infrastructure, the HP Moonshot System takes the sharing to a new level by integrating the processor and chipset onto a single piece of silicon and sharing other resources across the system. Dedicated hosting companies use large numbers of traditionally architected servers, hitting the wall for power, cooling and space. The HP Moonshot System uses an innovative new architecture that results from one simple design tenet: to align purpose-built modules with the right workload to provide optimal results for dedicated hosting environments. HP Moonshot System is a software-defined server platform achieving efficiency and scale by aligning just the right amount of compute, memory and storage to get the work done, enabling IT to capitalize on the major growth trend of IoT. Traditional servers rely on dedicated components, including management, networking, storage, and power cords and cooling fans in a single chassis. In contrast, the Moonshot system shares these chassis components and is capable of supporting 45 servers per 4.3U chassis. This provides the ability to generate greater revenue from a smaller footprint while driving down operational costs.


9 citrix.com

Figure 4: HP Moonshot 1500 Chassis rear vi

Figure 5: HP Moonshot 1500 Chassis front view Each software defined sever contains its own dedicated memory, storage, storage controller, and two NICs (1Gb). For monitoring and management, each server contains management logic in the form of a Satellite Controller with a dedicated internal network connection (100 Mb). HP Moonshot System provides application-specific processing for targeted workloads. Creating a fabric infrastructure capable of accommodating a wide range of application-specific workloads requires highly flexible fabric connectivity. This flexibility allows the Moonshot System fabric architecture to adapt to changing requirements of hyperscale workload interconnectivity. Moonshot management is achieved via support of the Command-Line (CLI) and Intelligent Platform Management (IMPI) Interfaces. These provide the primary gateway for node management, aggregation, inventory, power capping, firmware management and aggregation along with asset management and deployment. Citrix XenDesktop powering HP® - AMD® Microservers


10 citrix.com

At HP Discover 2013 in Barcelona, Spain, HP unveiled a new member of the Moonshot platform called the Converged System 100 for Hosted Desktops designed exclusively with AMD for Citrix XenDesktop. The system is supported for Citrix customers using XenDesktop 7.1 and Provisioning Services 7.1. Independent compute and graphics processing unit (GPU) per user when combined with the high-density of the HP Converged System 100 for Hosted Desktops delivers a full-powered PC desktop experience to all types of enterprise users. Workers now enjoy consistent performance and quality of service, no matter what individual workloads they are running and including business graphics and multimedia applications. The HP Converged System 100 for Hosted Desktops consists of a 4.3U HP Moonshot 1500 Chassis that holds up to 45 AMD-based cartridges. Each cartridge has four independent servers (PC-on-a-chip), with each server supporting one desktop. The dedicated GPU per-user enables PC-quality multimedia capabilities. Combined with HP Moonshot and data center hosting efficiencies, this non-persistent delivery model provides a compelling cost per user. A complete solution including compute, storage, and networking, the HP Converged System 100 for Hosted Desktops hosts up to 180 desktops per chassis. With no SAN or virtualization layer to install and manage, IT administrators will experience less complexity. And with pre-determined sizing and fewer workload images, desktop provisioning time is greatly reduced. The main feature that only XenDesktop 7.1 provides is the capability for the Standard VDA to leverage the native GPU for Direct X enabled applications, for example, without the need of the HDX 3D Pro VDA that was always the case before for leveraging GPUs.


11 citrix.com

Figure 6: Moonshot AMD m700 Cartridge

The HDX 3D Pro VDA is required for higher end CAD applications, which also require a higher end GPU than what is inside the M700 cartridge. Consider the NVIDIA K2 and XenServer GPU pass through with HP BL380 Gen 8 blades here for HDX 3D Pro for those higher end users, which is a separate architecture than Moonshot. Throughout the development of the Moonshot platform Citrix, HP, and AMD worked very closely to ensure HDX compatibility. During that time, Citrix developers were able to enhance the XenDesktop 7.1 VDA WDDM driver to be able to provide optimizations that are now capable of leveraging the AMD graphics cards, which are a standard on the Moonshot HDI platform. This new WDDM driver enhancement now allows for a superior HDX experience that can directly leverage the GPU for each node! Citrix XenApp powering HP® - Intel® Microservers HP Moonshot and Citrix family of products has grown to include support for Citrix XenApp 7.5 on ProLiant m710 cartridge. m710 is powered by Intel E3 chipset with integrated Iris Pro Graphics and is another example of the PC-on-a-Chip server architecture cartridges. Enabling an out-of-the-box XenApp HDX experience through Intel Iris Pro Graphics for each XenApp user. This


12 citrix.com

bare-metal architecture does not require a hypervisor and simplifies the setup time and deployment for XenApp without waiting on the underlying management platform to be enabled or a GPU to be virtualized. Each ProLiant m710 cartridge is enabled with 4 SODIMM Slots with 8GB of DDR3L-1600 low voltage memory per slot for a total maximum configuration of 32GB RAM per cartridge. While 32GB of RAM is a smaller amount of memory than a full size blade, it enables you to create micro XenApp instances similar to what you would use in virtualized VM instance of XenApp, but without the hypervisor. Integrating CPU and GPU on server boards is a great advancement for Hosted Desktops. Delivering on demand graphics for rich applications combined with Intel’s Turbo Boost Technology with speeds up to 3.2 Ghz. Delivering graphics in the past for rich applications or content that call for OpenGL, OpenCL, and Web GL has always been a challenge when graphics cards were not present. Now with a solution like the HP Moonshot ProLiant m710 for XenApp, a user has on demand enablement of graphics for those scenarios. Historically, IT needed to schedule time to take down servers and insert a GPU to enable graphic intensive workload. With microserver architecture like HP Moonshot and ProLiant m710 cartridge those challenges fade away. ProLiant m710 cartridge comes with integrated 2x 10gb Mellanox ConnectX®-3 network adapters supporting RDMA over Converged Ethernet or RoCE. These adapters seamlessly integrate into the Intel QM87 chipset on the ProLiant m710 cartridge. With two 10gb adapters, the transfer speed for users accessing content such as videos, high-resolution images, and large files are instantaneous.

Figure 7

The performance tests of ProLiant m710 were very predictable with linear scalability proofing that the cartridge can handle a multitude of XenApp applications even up to intense graphical applications that leverage the GPU on demand. The reader may refer to the Citrix Blog for more insights into benchmarking data and diagrams.


13 citrix.com

Hardware-Assisted System Virtualization Core benefits Virtualization solutions allow multiple operating systems and applications to run in independent partitions all on a single computer. Using virtualization capabilities, one physical computer system can function as multiple "virtual" systems. Virtual partitioning needs to be achieved from the hardware level at the very bottom and enabled all the way up through upper software layers. System hardware is composed of CPUs, memory, GPUs and I/O devices like networks and storage in particular. Every one of those hardware components has to be pre-designed or capable of running multiple isolated virtual environments on top. Server hardware and software hypervisors have evolved in the past few years to provide virtualization assistance across CPUs, GPUs, memory, network and storage. For over two decades Citrix has been the industry leader in applications virtualization. Our flagship product XenApp has been behind the streamlined operations in hospitals, enterprises, schools, factories, airports, governments, etc. As server virtualization became possible Citrix delivered a full desktop virtualization experience not only allowing apps to be virtualized with isolated access but also desktops. Virtualization provides the ability to isolate software components running them in isolated containers with inbound and outbound access control. With such level of isolation and access control virtualization allows companies like Citrix to revolutionize the way desktops and apps are delivered and secured driving us into new era of safer and full enterprise mobility.

Figure 8: XenDesktop managing hosted desktops in virtual data centers

Intel’s family of Xeon server processors provides support for hardware-based technologies enabling Desktop and Applications virtualization and security. The following section of the paper


14 citrix.com

will cover specifically the following technologies: Intel VT, VT-x, VT-d, TXT, OS Guard, VMCS Shadowing (nesting of hypervisors) and AES-NI. Responsive and secure desktop virtualization requires tight integration between the virtualization machine monitor / hypervisor software that is used to deploy and manage virtual machines and the underlying hardware platform. XenServer is the Citrix open source hypervisor product for server and cloud virtualization. XenServer takes advantage of many server hardware provided technologies. XenDesktop, which runs on top of many commercial hypervisors, gets the benefits of many of those direct interfaces between XenServer, the hypervisor and Intel server hardware. Some of those benefits will be covered in coming sections.

Challenges with software based system virtualization The design of the Intel’s protected mode architecture provides four protection rings, ring 0 to ring 3, out of which ring 0 is most privileged used for running operating system kernel along with device drivers and ring 3 is used to run user mode applications. Software modules running in ring 0 have enough privilege to directly access certain processor; memory and I/O control structures, addresses and registers. One approach to software-based virtualization is called ring de-privileging which involves running guest OS at a higher ring than ring 0. Various techniques have been generally used for software-based virtualization: (1) binary translation, inducing a trap and emulate model, (2) shadowing of memory and I/O pages and (3) devices and chipset emulation. Those techniques increase software complexity affecting its performance and reliability greatly, increase the size of what is needed to establish a Trusted Computing base (TCB) and suffer from the absence of sufficient protection across boundaries. Another popular technique is para-virtualization, which involves modifying and porting the operating system to run within the target virtual machine environment. The obvious price of para-virtualization is not being able to run operating systems code unmodified in virtual environments. Intel® Virtualization Technology (Intel® VT) Intel® Hardware-based Virtualization Technology (Intel® VT) improves the fundamental flexibility and robustness of traditional software-based virtualization solutions by accelerating key functions of the virtualized platform. This efficiency offers benefits to the IT as it speeds up the transfer of platform control between the guest operating systems (OSs) and the virtual machine manager (VMM)/hypervisor. Enabling the VMM to uniquely assign CPUs and Memory pages to guest OSs, Intel VT performs various virtualization tasks in hardware, like memory address translation, which reduces the overhead and footprint of virtualization software and improves its performance. Intel® Virtualization Technology for Directed I/O (VT-D) Intel VT-d is the other part of the Intel Virtualization Technology hardware architecture. VT-d addresses the loss of native performance or of native capability of a virtualized I/O device by providing hardware isolation and translation mechanisms that enable to VMM to directly assign the device to a VM. In this model, the VMM restricts itself to a controlling function for enabling direct assignment of devices to its partitions. Rather than invoking the VMM for all (or most) I/O requests from a partition, the VMM is invoked only when guest software accesses protected resources (such as I/O configuration accesses, interrupt management, etc.) that impact system functionality and isolation.


15 citrix.com

Intel VT-d enables protection by restricting direct memory access (DMA) of the devices to pre-assigned domains or physical memory regions. This is achieved by a hardware capability known as DMA-remapping. The VT-d DMA-remapping hardware logic in the chipset sits between the DMA capable peripheral I/O devices and the computer’s physical memory. In a virtualization environment the system software is the VMM. In a native environment where there is no virtualization software, the system software is the native OS. DMA-remapping translates the address of the incoming DMA request to the correct physical memory address and perform checks for permissions to access that physical address, based on the information provided by the system software.

GPU Virtualization: The Art of Sharing GPUs Across Virtual Machines

Figure 9: NVIDIA vGPU GRID

As Intel made great advancements to hardware CPU and I/O virtualization, parallel progress was made around GPU hardware virtualization. NVIDIA® GRID™ vGPU™ brings the full benefit of NVIDIA hardware-accelerated graphics to virtualized solutions. This provides exceptional graphics performance for virtual desktops by sharing a single GPU among multiple users. GRID vGPU provides hardware acceleration across multiple virtual desktops while delivering a high performance graphics experience, with economic benefits over a dedicated GPU per each user. Operating systems still use NVIDIA native graphic drivers allowing seamless support without impacting applications features or compatibility. Furthermore, the graphics commands of each virtual machine are passed directly to the GPU, without requiring additional translation by


16 citrix.com

the hypervisor. This transparent support allows GPU hardware to be virtually divided delivering ultimate shared virtualized graphics performance. As said earlier, Citrix HDX 3D Pro uses the native NVIDIA GPU driver installed directly in the guest OS. With NVIDIA GRID cards, this ensures full application-level compatibility. As a result of that, any application certified to work with NVIDIA cards would be fully supported on NVIDIA vGPU GRID. Citrix HDX 3D Pro supports OpenGL 4.3 and DirectX 11 applications on both desktop and server platforms. Application vendors are actively working with NVIDIA and Citrix to certify their applications for compliance. It is worth noting here that such kind of compliance does not happen transparently with software-based GPU virtualization.

Figure 10: XenDesktop supporting NVIDIA vGPU GRID

To provide the reader with further explanation of how this works, as shown in the diagram above, each virtual machine directly accesses a part of the physical card, called the “vGPU”. The vGPU assignment provides direct frame buffer access to video memory residing on the GPU. This direct access minimizes lag time and provides a highly responsive user experience, even when rendering large and complex 3D models. XenDesktop and XenServer take advantage of such advanced server-side GPU rendering to provide knowledge workers, power users, and designers the ability to perform at their best with no interruption. NVIDIA GRID™-accelerated XenDesktop is an ideal solution for 3D graphics-intensive applications like remote workstations as users get full experience of the local PC while running on a virtual desktop served residing in the data center. XenDesktop existing software GPU pass-through and hardware sharing technologies have delivered great value for graphically intensive applications such as Adobe Photoshop, Dassault SolidWorks, Ansys Workbench and Autodesk Applications. Combining the benefits of that with the vGPU technology will deliver unprecedented value at much lower cost.


17 citrix.com

A wide range of graphics, video and CAD intensive applications including medical and industrial imagery products are now fully interactive with NVIDIA GRID. By leveraging GRID technology with full 3D and compute API support through the latest NVIDIA Quadro® drivers, users will be able to take advantage of thousands of applications that run OpenGL 4.3, Microsoft DirectX9, 10, 11, or NVIDIA CUDA® 5.0. It is worth noting that Citrix is actively working with NVIDIA along with major server vendors such as HP, Dell, Cisco and IBM to ensure software integration is done and available for use with XenDesktop sessions on XenServer hypervisors.

Intel Hardware-Assisted Security Technologies Challenges with Traditional Software-based Security Traditional design of computer hardware architecture did not distinguish between running legitimate and illegitimate software modules. As a result of that, any piece of software code could boot the system hardware taking full control before the firmware boots the user operating system installed on the system. This boot-time control has been behind many key Advanced Persistent Threats (APTs) that have taken place in the past few years steeling corporates key valuable digital assets; challenging stability and viability of world’s economy. Usage of cryptographic algorithms has been used as a key element of ensuring confidentiality of data exchanged across the Internet and stored on persistent storage. But cryptographic algorithms are very computationally extensive. Thus their usage has been limited to situations in which their overhead over system response time is acceptable. In coming sections, the paper will talk about some key security technologies to address the need to protect the boot-elements of the hardware, to establish a Trusted Compute Base (TCB) and to accelerate adoption of cryptographic algorithms.

Intel Platform Protection Technologies To address malware infections taking place underneath the operating system, malware protection has to start from the BIOS. Intel BIOS Guard Technology (IBGT) ensures that updates made to system BIOS flash are secure. Any update made to system BIOS is cryptographically verified by a guard module using a protected agent running in protected system memory. Another related technology is Intel’s Platform Trust Technology (IPTT), which provides platform functionality for credential storage and key management used by Windows 8. Both technologies bring great value to XenDesktop hosted desktops as they ensure that the physical hardware is protected and secure from boot-record malware infections preventing an entry point used by Advanced Persistent Threats (APTs). Intel OS Guard (IOSG) is another key security feature preventing instruction execution from user mode memory pages while the CPU is in supervisor mode. IOSG helps to prevent common attacks that seek to use privilege escalation to gain control of a platform or execute malware. IOSG can be enabled via a Windows 8 boot loader option. With XenDesktop centralized management and policy enforcement, IT admins can force the OS Guard feature policy to be always turned on for Windows 8.


18 citrix.com

Intel Trusted eXecution Technology (TXT) Intel TXT® is a feature available in the Intel® Xeon® processor. It establishes a root of trust through measurements when the hardware and pre-launch software components are in a known good state. Intel TXT brings the security advantages of microkernel model to actual platform with enhancements. For a cloud environment, Intel® TXT is able to Measure Launch (ML) the BIOS, hypervisor and attest the integrity of each VM individually.

Figure 11: TXT benefits to virtualized data centers and clouds


19 citrix.com

Utilizing the result, XenDesktop along with a VMM like XenServer, administrators can set policies for sensitive data and workload placement onto groups of servers known as trusted compute pools. Those trusted compute pools with Intel®TXT support IT compliance by protecting virtualized XenDesktop data centers against attacks toward hypervisor and BIOS, firmware, and other pre-launch software components. With Intel TXT, IT can run XenDesktop virtual desktops on a trusted server, protecting enterprises workload and data avoiding compromising security and enhancing IT compliance.

Figure 12: XenDesktop and XenServer support for TXT-based measurement and attestation.

Intel® AES-NI and Secure Key Technology Intel® AES-NI is a new encryption instruction set that improves on the Advanced Encryption Standard (AES) algorithm and accelerates the encryption of data in the Intel® Xeon® processor family. AES NI is a set of new instructions to the Intel architecture implementing some intensive sub-steps of the AES algorithm into the hardware accelerating execution of the AES application. AES NI minimizes application performance concerns inherent in traditional cryptographic processing providing enhanced security by addressing side channel attacks on AES associated with traditional software methods of table look-ups. Intel® Secure Key is a new instruction added to the Intel® 64 and IA-32 Architectures called RDRAND with an underlying Digital Random Number Generator (DRNG) hardware implementation. The DRNG using the RDRAND instruction is useful for generating high-quality keys for cryptographic protocols. Encryption is a basic tool to ensure confidentiality of data at rest and through the wires protecting against man in the middle attacks. With AES NI offloading of encryption, cryptography can become a common tool used whenever data confidentiality is needed without having to worry about processing speed and slowness of overall system operations.


20 citrix.com

XenDesktop manages virtual machines as they run on top of server hypervisors like XenServer and Hyper-V. Various types of security compliance and regulations require the content of VMs with sensitive private data to be encrypted. AES-NI makes this possible. Today XenDesktop gets the value of AES-NI via the lower level hypervisor as those hypervisors code rely on AES-NI for acceleration and key security. Windows OS and some of its applications can take advantage of AES-NI. XenDesktop IT admins can get the value of Windows in-bound usage of AES-NI directly by providing the right set of configuration to the Windows VM. Intel® VMCS Shadowing Technology Citrix realized long ago that newer usage models are emerging that would require two or more Virtual Machine Monitors (VMMs) to be hosted on the same client system. Citrix has been heavily engaged with Intel® to take advantage of new hardware capabilities designed to accelerate nesting of hypervisors (VMMs). Intel® VMCS Shadowing greatly reduces the frequency with which the guest VMM must access the root VMM in a nested environment. With Intel VMCS Shadowing, the root VMM is able to define a shadow VMCS in hardware. A guest VMM can access this shadow VMCS directly, without interrupting the root VMM. Since the shadow VMCS is implemented in hardware, required accesses can be completed nearly as fast as in a non-nested environment.

Figure 13: Intel VMCS Shadow Tables

As explained above XenDesktop relies on hypervisors’ interfaces for providing an abstracted hardware-independent view of the data center and cloud hardware. XenDesktop uses hypervisor interfaces available from XenServer, VMware Virtual Center and Microsoft System Center Virtual Machine Manager to achieve that purpose. Such capabilities will allow XenDesktop to deploy custom-driven in-guest VMs that yield better security, availability and robustness of desktops. A good example is McAfee’s Deep Defender, which provides advanced protection using a form of system virtualization furnished by a lightweight hypervisor, or Virtual Machine Monitor (VMM), known as DeepSAFE. Unlike server hypervisors like XenServer, DeepSAFE does not provide full system and I/O virtualization. Instead, it uses hardware-assisted virtualization to monitor and control memory and processor operations, which provides the foundational layer for Deep Defender security functions. Together, XenDesktop and Deep Defender provide a breadth and depth of security that neither can provide alone. VMCS shadowing is a revolutionary technology as it opens the doors widely for custom VM-level virtualization-derived feature. As more companies deliver guest-VM based micro-visors, XenDesktop IT administrators would be able to deploy separate custom-built guest-VM


21 citrix.com

hypervisors (micro-visors) separately per-VM bases. For instances, XenDesktop IT admins can deploy a micro-visor that improves system security and recoverability in one VM while deploying another micro-visor that improves system availability, fault-tolerance and measurability to another VM with both VMs running within the same XenDesktop virtual infrastructure. Those key benefits would be more realized in XenDesktop managed appliance-type VMs that run a single particular mission critical application like a web or a DB server for instance.

Figure 14: XenDesktop and XenServer support for nested upper guest hypervisors (micro-visors)

Evolving Converged Server Fabric Technologies What Is Driving Data Center Change? Virtualization essentially changed everything. With virtualization, the basic unit became not the server, but the virtual machine. With a small number of virtual machines, an organization could simply buy a bigger server to handle the new technology. As the number of virtual machines increases, however, the network needs a different approach. Instead of networking servers, it needs to network virtual machines. To meet this new requirement, a strong connection needs to exist between the server and the network. This is when a network and a server start to become a fabric. A fabric should provide transparency so that virtual machines are visible on both the server and the network, with capabilities to help ensure that security policies follow the virtual machine. IT infrastructure, however, will not change overnight. Instead, it will evolve. In fact, most IT departments are not even close to virtualizing all their servers or completely transforming IT into a private cloud service. The reason is simple: they do not have to do so. In fact, IT will likely always have a mix of traditional data center (with physical mapping of applications to server and storage resources) and virtualized data center (with applications that map to virtual machines) technologies, while at the same time being ready for emerging trends such as big data and high performance computing.


22 citrix.com

Figure 16: Benefits of Unified Computing/ Fabric-Based Infrastructure

Figure 15: Application Evolution from Data Center to the Cloud IT now needs to consider how to evolve its infrastructure to best support virtualization and cloud computing, while building on the high availability, security, and application awareness that are fundamental features of the current data center, and particularly of the network. To bring all these pieces together, IT is turning to the concept of a data center infrastructure fabric. Unified Computing/ Fabric-Based Infrastructure The term ”fabric” is used by different vendors, analysts and IT groups to describe different things. Gartner offers a definition of “fabric” that can be applied across the industry: “A set of compute, storage, memory and I/O components joined through a fabric interconnect and the software to configure and manage them.” A fabric thus provides the capability to reconfigure all system components - server, network, storage, and specialty engines – at the same time, the flexibility to provide resources within the fabric to workloads as needed, and the capability to manage systems holistically. Cisco delivers a fabric-based infrastructure with the Cisco Unified Data Center platform. The Cisco Unified Data Center changes the economics of the data center by unifying computing, storage, networking, and management resources into a single, fabric-based platform designed to increase operating efficiency, simplify the data center, and provide business agility. Unlike other solutions, which add layers of management software to achieve integration, the Cisco Unified Data Center is specifically designed for virtualization and automation and enables on-demand provisioning from shared pools of infrastructure across physical and virtual environments. This approach allows IT to move from being a cost center to providing IT services that create competitive advantage.


23 citrix.com

Citrix XenDesktop VDI Customers who elect to build their own privately hosted DaaS services will benefit greatly from the agility, security, high availability, scalability, openness, resilience and flexibility associated with Fabric Based Infrastructure. In addition, some of the benefits available from the convergence of storage, networks and compute into the form of a fabric include:

• Standardized interfaces: Since a unified port can be configured as either an Ethernet port or a Fiber Channel port.

• Role-based access control (RBAC): Helping to ensure that different groups can have access to different configuration parameters of the switch, as required and helping ensure that multiple groups can use a common switching platform.

• Consolidate fabrics: The Fiber Channel over Ethernet (FCoE) protocol allows Ethernet and Fiber Channel to run over a common 10-GB connection. Bringing Ethernet and Fiber Channel together through the FCoE protocol.

• Inclusion of storage in the converged fabric through native interfaces helps to ensure that, when the traffic needs to be broken out natively, storage managers can accomplish this at the appropriate places in the network.

Citrix sees clear advantages of Cisco Unified Data Center Platforms especially for delivering both Desktops as a Service and Workspace as a Service. Citrix is building on our Citrix Service Provider (CSP) Reference Architecture to take advantage of the new performance, features and integration capabilities of the new Cisco UCS M-Series and C3160 Rack Servers to simplify administration and management across multiple data centers. We’ll pool our resources to streamline the process of provisioning and on-boarding new users while making sure the system delivers the rich HDX mobile user experience people expect from Citrix.

Figure 17: Cisco Unified Data Center Reference Architecture


24 citrix.com

Fabric-based computing (FBC) FBC is a modular form of computing in which a system can be aggregated from separate (or disaggregated) building-block modules connected over a fabric or switched backplane. Fabric-based infrastructure (FBI) differs from FBC by enabling existing technology elements to be grouped and packaged in a fabric-enabled environment, while the technology elements of an FBC solution will be designed solely around the fabric implementation model. Fabric Computing refers to a consolidated high-‐performance computing system consisting of loosely coupled storage, networking and parallel processing functions linked by high bandwidth interconnects (such as 10 Gigabit Ethernet and InfiniBand) The main advantages of fabrics are that a massive concurrent processing combined with a huge, tightly-coupled address space makes it possible to solve huge computing problems (such as those presented by delivery of cloud computing services) and that they are both scalable and able to be dynamically reconfigured. Persistent Memory, Solid-State Dual-in-Line Memory Modules (DIMMs) With the virtualization trends in data centers, servers demand higher I/O throughput to match the capabilities of multi-core processors and increased amounts of memory, allowing the higher number of virtual machines (VMs) to be hosted on a single physical system. Higher I/O throughput, including storage I/O, can help achieve better server utilization and a higher number of VMs per server. Higher storage I/O throughput can help improve user and business productivity by lowering overall response time. Data warehouses and business analytics are other examples of the workload that requires higher storage I/O throughput to allow faster data processing, making strategic business decisions in a timely manner.

A Non-Volatile DIMM (NVDIMM), is a module that can be integrated into the main memory of an industry standard compute platform (i.e. server), perform workloads at DRAM speeds (i.e. DDR3), yet be persistent and provide data retention in the event of a power failure or system crash. By understanding that flash based SSDs have the potential to radically accelerate application performance, but at the same time, being cognizant of some major shortcomings (endurance, performance and high availability) that present themselves when integrating SSDs into data centers. Non-Volatile DIMM, uses a trusted paradigm, DRAM access in main memory, to provide the fastest possible I/O performance, practically infinite write endurance (the Achilles heel of flash) and all this delivered in a solution that provides increased levels of data security and high availability.

Figure 18: Cost per gigabyte and latency for RAM, solid-state storage, and HDD


25 citrix.com

Figure 19: IBM Lenovo NVDIMM (eXFlash DDR3 Storage DIMMs)

Figure 20: Changes required to operating system file systems to support NVDIMMs.

This marriage of DRAM and NAND technology delivers a high speed and low latency “non-volatile /persistent” memory module. Designed from the ground up to support unlimited write activity, it performs at DDR3 speeds (12GBytes/second) and can sustain itself from host power failure or a system crash. Since it sits directly on the faster memory channel, rather than the storage channel, the flash memory in a solid-state DIMM does not face storage channel bottlenecks as input/output operations increase, as it would in a traditional storage system. Because of the direct and scalable interface to the CPU, these NAND flash-based SSDs can achieve drastically lower latencies (at least 50% lower) than any existing solid-state storage solution, and can function as more affordable and nonvolatile alternatives to DRAM memory if their slower access speeds are acceptable for the required application. Use of any solid-state DIMMs generally requires a mix, or all, of the following: support by the host chipset, optimization for the OS, and optimization for the server hardware. This may limit adoption in the short term.


26 citrix.com

Figure 21: Difference in data access between NVDIMMs (Showing IBM Lenovo eXFlash DIMMs) and SSDs.

NVDIMMs can be mounted to act as a drive attaching to applications on demand. In such case, they operate as a Persistent RAMDisk (PRAMDisk). Some examples of such use case are:

• As an alternative to SSD/HDD storage for highly accessible VDI users files. • Improving VDI images Boot time via an instant on storage space. • To hold MemcacheDB files which can store VDI critical metadata key value pairs.

NVDIMM helps in systems that are IO sensitive requiring higher input/output requests per second (IOPS) and throughput (measured in MBps or GBps). Therefore, NVDIMM can speed image-editing systems like medical imaging, surveillance scanning, movie rendering and editing CAD. Combining this capability with server side GPUs will change the landscape of 3D image editing and processing in VDI environments. XenDesktop takes greater advance of Scale out servers with GPUs and NVDIMM given its bare-metal provisioning capabilities that does not require a hypervisor change to support NVDIMM. Server Virtual I/O Virtual I/O works by providing a layer of abstraction between the server and the I/O devices so that, if an OS and application are moved — either manually or dynamically — from one physical server to another, then the Media Access Control (MAC) and/or World Wide Name (WWN) address can stay with the OS and/or application. In other words, without this type of virtual I/O, a new MAC or WWN address would have to be assigned each time an OS and/or application is moved to a new physical location. This type of server virtual I/O can be implemented in a fabric switch at the firmware level or on a chip. Virtual I/O demand is not exclusive to blades, but blade and modular server topology is particularly well suited to the technology, and the rack-dense nature of blades more easily justifies the investment in virtual I/O. This I/O flexibility benefits scale-out application workloads and, increasingly, database management system (DBMS) for transactional and analytic use, as well as for high availability (HA) and disaster recovery (DR) assistance. The demand for virtual I/O for servers is growing steadily, as blade-based platforms increase their footprint and the blade user community routinely adopts virtualization technology for new implementations. This capability is available through multiple vendor strategies. Solutions such as HP Virtual Connect (deployed on the HP BladeSystem c-Class) and Cisco's Unified Computing System (UCS) offer a switch-based structure. Oracle offers hardware-based solutions leveraged from the acquisition of Xsigo Systems in 2012. Niche third parties also offer hardware-based


27 citrix.com

solutions that can be deployed with blades and rack-optimized servers from multiple hardware vendors. The alternative option is to deploy at the firmware level; this is feasible with blade chassis from several vendors, including HP, IBM, Dell and Oracle. Firmware-based virtual I/O permits MAC address virtualization within the standard design.

Closing Notes Citrix XenDesktop Hosted Desktops VDI allows IT to realize important benefits that traditional PC environments can’t match:

• Improved security and compliance with centralizing desktops, data, and applications • Enhanced worker productivity anywhere, anytime, any device and secure mobility • Streamlined desktop support managing all desktops with no interruptions • Improved business agility scaling and adapting to changes quickly

This paper has shown to the reader how those benefits can be enabled and realized in fundamentally two different architectural scenarios:

1. A virtualized environment powered by hardware-assisted virtualization of CPU, memory, GPU and I/O.

2. A physicalized environment powered by integrated large number of PCs and servers on a single chip as in the case of microservers.

From an IT admin perspective, whether the infrastructure is virtualized or physicalized XenDesktop will work uniformly the same and users will get the benefit of Hosted Desktops whether they’re deployed in the datacenter or in the cloud.

XenDesktop VMs on scale up virtualized servers. XenDesktop bare-metal provisioning on scale out physical servers.

Figure 22: Citrix XenDesktop unified centralized management and security for system virtualization and physicalization.


28 citrix.com

References • Citrix Collaboration with ARM unifying hardware server architecture. • Citrix Collaboration with Intel bringing of nested hardware virtualization to market. • Intel® Hardware-Assisted Virtualization Technology • Intel® Trusted Execution Technology (TXT®) • Mitigating threats in the cloud using Intel® TXT • Intel® Virtualization Technology for Directed I/O • An Introduction to SR-IOV Technology • Intel® AES NI Technology • Intel® Atom C2000 Processor Technical Overview • HP® Moonshot System software defined servers • Citrix XenDesktop, HP Moonshot, and AMD! • Citrix XenApp is now available on HP Moonshot with Intel graphics! • NVIDIA® Virtual GPU • Benchmarking NVIDIA® vGPU for XenServer and XenDesktop • Blog entry on Citrix and HP Moonshot • Blog entry on GPU sharing technologies • Blog entry on Citrix, AMD® and HP® Moonshot • Intel and Citrix collaboration around nesting of VMMs • HP ConvergedSystem 100 and XenDesktop brief • Citrix Collaboration with ARM to unify server architecture and bringing Xen to ARM. • Xen ARM with Virtualization Extension. • Xen on ARM white paper. • Intel microserver white paper. • HP Converged System 100 for Hosted Desktops with Citrix XenDesktop. • IBM SmartCloud Desktop Infrastructure. • Cisco Desktop Virtualization Solutions with Citrix XenDesktop. • Gartner Hype Cycle for Storage Technologies, 2014. • Gartner Hype Cycle for Server Technologies, 2014. • SNIA NVDIMM Working Group. • Cisco Fabric-‐Based Infrastructure for Virtualization and Cloud Computing • IBM Lenovo eXFlash Memory-‐Channel Storage in Enterprise Solutions • Linux Foundation XenProject and Citrix XenServer.

About the author Ahmed Sallam is Citrix cross-functional VP and CTO leading technology development, emerging solutions and products strategy in new emerging era of smart devices, IoT, IIoT, IoE, system virtualization, server physicalization and security. His focus is on new emerging end-to-end solutions ranging from devices to networks to clouds across Citrix lines of products. He also drives Intellectual Property growth opportunities and monetization strategy. Works closely with software and hardware ecosystem partners integrating into Citrix open platforms. Prior to Citrix, Ahmed was CTO of Advanced Technology and Chief Architect at McAfee/ Intel. developing global threat intelligence along with (PPP) Proactive Predicative Preventive anti-malware security solutions. Ahmed is the co-inventor and architect of Intel/ McAfee’s DeepSAFE technology and co-designer of VMware’s VMM CPU security technology known as VMsafe. Prior to McAfee, Ahmed was a Senior Architect with Nokia’s security division and a Principal Engineer


29 citrix.com

at Symantec. Prior to that, Mr. Sallam was a founding engineer / director / chief architect at three start-ups. Ahmed is a renowned expert across the industry well known for pioneering new models in computer system virtualization, security and management delivering flexible, safer, well-managed and secure computing experience. He holds 31 issued patents along with over 40 pending patent applications. He is a frequent keynote and session speaker at many conferences including: Citrix Synergy, McAfee Focus, RSA Conference, VMworld and ARM TechCon. Contacting the author: Ahmed Sallam can be contacted through his LinkedIn page, Twitter and Citrix email.