mercury interactive's load runner for citrix

MMeerrccuurryy IInntteerraaccttiivvee’’ss LLooaaddRRuunnnneerr ffoorr CCiittrriixx

By Citrix Consulting Services

Citrix Systems, Inc.

Notice

The information in this publication is subject to change without notice.

THIS PUBLICATION IS PROVIDED “AS IS” WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT. CITRIX SYSTEMS, INC. (“CITRIX”), SHALL NOT BE LIABLE FOR TECHNICAL OR EDITORIAL ERRORS OR OMISSIONS CONTAINED HEREIN, NOR FOR DIRECT, INCIDENTAL, CONSEQUENTIAL OR ANY OTHER DAMAGES RESULTING FROM THE FURNISHING, PERFORMANCE, OR USE OF THIS PUBLICATION, EVEN IF CITRIX HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES IN ADVANCE.

This publication contains information protected by copyright. Except for internal distribution, no part of this publication may be photocopied or reproduced in any form without prior written consent from Citrix.

The exclusive warranty for Citrix products, if any, is stated in the product documentation accompanying such products. Citrix does not warrant products other than its own.

Product names mentioned herein may be trademarks and/or registered trademarks of their respective companies.

Copyright © 2002 Citrix Systems, Inc., 851 West Cypress Creek Road, Ft. Lauderdale, Florida 33309-2009 U.S.A. All rights reserved.

Version History

Ed Rodriguez Original September 3, 2002

Table of Contents TABLE OF CONTENTS ................................................................................................................................................................III

INTRODUCTION.............................................................................................................................................................................1

CITRIX LOAD TESTING CHALLENGES .......................................................................................................................................2

THE CITRIX ICA CLIENT INTEGRATION .....................................................................................................................................4 LOADRUNNER FOR CITRIX FUNCTIONALITY .......................................................................................................................................4

MERCURY LOADRUNNER AND CITRIX ......................................................................................................................................6 LOADRUNNER FOR CITRIX SOLUTION...............................................................................................................................................6

SETUP AND EXECUTION OF A LOADRUNNER FOR CITRIX SOLUTION.................................................................................8 INSTALLATION OF THE LOADRUNNER FOR CITRIX SOLUTION ...............................................................................................................8 INSTALLATION OF LOADRUNNER 7.51 ..............................................................................................................................................8 INSTALLATION OF LOADRUNNER FOR CITRIX SUPPORT FILES ...........................................................................................................10 DEVELOPING CITRIX VIRTUAL USER (VUSER) SCRIPTS....................................................................................................................10 TIPS FOR WORKING WITH CITRIX VUSER SCRIPTS ...........................................................................................................................10 EXECUTING A CITRIX LOADRUNNER TEST.......................................................................................................................................11

SUMMARY ...................................................................................................................................................................................14

APPENDIX A: LOADRUNNER FOR CITRIX PROOF OF CONCEPT .........................................................................................15 PLANNING THE TEST: REQUIREMENTS AND ENVIRONMENT SPECIFICATION........................................................................................15

Configuration Details.............................................................................................................................................................15 Software and Platform Specifications....................................................................................................................................15 Server Specifications ............................................................................................................................................................16 Network Environment............................................................................................................................................................17 Client Configuration...............................................................................................................................................................18 Server Connection Configuration ..........................................................................................................................................18 Metrics ..................................................................................................................................................................................19 User Experience....................................................................................................................................................................19 Network Utilization ................................................................................................................................................................20

CREATING VUSER SCRIPTS: USER PROFILING ...............................................................................................................................20 vUser Initialize.......................................................................................................................................................................20

Mercury Interactive’s LoadRunner for Citrix iii

Actions ..................................................................................................................................................................................20 vUser End .............................................................................................................................................................................21

CREATING THE SCENARIO.............................................................................................................................................................21 RUNNING THE SCENARIO ..............................................................................................................................................................22 MONITORING THE SCENARIO .........................................................................................................................................................22 ANALYZING THE RESULTS .............................................................................................................................................................22 VUSER SCRIPT SOURCE CODE......................................................................................................................................................22

Mercury Interactive’s LoadRunner for Citrix iv

Introduction When deploying a Citrix Server-Based Computing (SBC) environment, primary concerns often include performance, scalability and availability. This requires significant capacity planning and decisions regarding the expected user loads and hardware specifications. Capacity planning involves making the best decision to cater for the system hardware by finding the best price-to-performance ratio. Should we have a handful of high-end servers, or a larger number of smaller machines to handle the load? What’s the maximum number of users per server? At what point will the system performance become unusable?

These questions are typically asked during the planning/proof of concept phase of a Citrix MetaFrame deployment and can be answered by performing scalability tests on anticipated production hardware. Typical scalability tests are executed within sessions on the Citrix MetaFrame servers rather than on the client machines instantiating the sessions. This results in additional overhead and potentially inaccurate or skewed results.

This document provides an overview of what LoadRunner for Citrix provides to IT administrators when used in scalability and functional testing. LoadRunner for Citrix allows realistic actions to be invoked on ICA clients, and performance as well as resource bottlenecks can be measured. The document will focus on:

• Citrix load testing challenges

• The Citrix ICA Client Integration

• The LoadRunner for Citrix performance testing and monitoring solution

• How to setup and execute a Mercury Interactive’s LoadRunner solution in a Citrix MetaFrame environment

At the conclusion of this paper, an Appendix is provided providing detailing a limited implementation of Mercury LoadRunner for Citrix.

Mercury Interactive’s LoadRunner for Citrix 1

Citrix Load Testing Challenges Today, load testing of a Citrix MetaFrame server is typically done by executing a server based script using a Winbatch engine or Visual Basic Scripting engine on a MetaFrame server. The Citrix Server Test Kit (CSTK) is primarily used to employ this type of testing solution. The benefit of this strategy is cost, as the CSTK is a free product available on Citrix’s website: www.citrix.com. However, this scenario is less than desirable for the following reasons:

• A server-driven solution is intrusive, due to the additional CPU and memory footprint for each started session. Intrusive tests can never yield truly representative results.

• A server-driven solution is unrealistic as it provides only the server-to-client side of the exchange. No key press or mouse-click information is ever sent to the server.

• Currently there is no Unix GUI playback engines, thus this solution cannot be applied to MetaFrame for UNIX. A client-driven solution would have no platform dependencies on the server side.

• Server-side script playback is rendered unrealistic in a thin-client environment due to all client-server events being activated not by events on the client, but by events on the server.

Additionally, to instantiate the number of sessions needed to generate the maximum load a server can handle requires numerous client devices with the traditional Citrix Program Neighborhood ICA client. Another challenge in Citrix load testing is to be able to monitor the performance of the system across all the tiers to help isolate bottlenecks wherever they may reside.

Most existing solutions today rely on deploying server-side agents to generate the load which adversely affects the accuracy of the solution and do not measure true client-side performance. The alternative is running a subset of client-side users; however this approach does not provide scaling to production levels.


http://www.citrix.com/

ICA Screen Updates

No Keypresses

or mouseclicks

MetaFrame Farm

Client Machines

• Requires Server side Agents

• Script is executed on Server, not client

Figure 1: Standard Testing Scenario

Figure 1 illustrates a standard testing scenario where many client machines are necessary to launch sessions on the MetaFrame server(s) under test. Once a connection is made, the script that executes is run server-side, thus eliminating all ICA client-to-server events and communication.


The Citrix ICA Client Integration The Citrix ICA Client Integration with Mercury Interactive is an extension of the ICA Client Object that first shipped with the MetaFrame XP 1.0 Client CD and was included with the ICA Win32 Client, version 6.01. The initial release of the ICA Client Object provided an application programming interface (API) to the Citrix Win32 ICA Client. Using this API, the ICA Client Object can be embedded into applications and scripts to launch ICA sessions while shielding the mechanics of accessing a Citrix Server Farm. This integration extends this functionality by exposing functions that allow programmatic control of actions within an ICA session such as mouse movements and keystrokes. Additionally, modifications have been made to ICA client code to allow the launching of a session with minimal resource requirements; thus allowing numerous connections to be made and controlled from a single client machine.

Client Based Scalability Testing advantages (Pre-deployment):

• Simulate hundreds of users with just a few workstations.

• Load testing tools can be used which can effectively emulates real environments.

• Define, test, and measure performance of definable business transactions.

• Pinpoint potential bottlenecks and resolve before going production.

• Reduce risk by load testing just before going live.

Application Performance Management (Post-deployment)

• End to End monitoring of infrastructure, including MetaFrame systems (synthetic transactions).

• Segmented metrics provide information for increasing performance and capacity planning of infrastructure.

LoadRunner for Citrix Functionality The LoadRunner for Citrix integration provides the ability to launch multiple ICA sessions from a single client machine as well as programmatically control the actions of a session from the client machine. The following list briefly outlines the core functionality exposed through the Citrix Client integration. These items are explained in detail in the “readme” and API function list included with the integration.

• Scalability

• Exposes 4 modes of operations: Normal, Headless, Windowless, and Renderless.

• Windows Information

• Exposes some window events such as re-size / move / create / destroy, etc.

• Allow querying of window information such as caption / style / position, etc.

• Input Events

• Expose events such as Mouse Move / Mouse Click / Key Down, etc.

• Input Simulation

• Allow simulation of mouse and keyboard events.


• Other APIs

• ScreenShot API to get screen hash or actual bitmap.

• Ping API to get network latency.


Mercury LoadRunner and Citrix Mercury Interactive (Mercury) is the leading provider of enterprise testing and performance management. Citrix and Mercury have teamed to jointly develop the first native performance testing and monitoring solution for Citrix-deployed applications.

Mercury’s industry standard load testing product LoadRunner, and application performance management product Topaz, have both been customized to support Citrix environments. This means that customers can now accurately test and monitor their Citrix-based systems before and after deployment to ensure high performance, scalability and availability.

LoadRunner for Citrix Solution LoadRunner works by emulating hundreds or thousands of ICA clients with just a handful of machines to create a scalable Citrix load testing solution. Up to 100 connections can be made from a single workstation (dependent on machine hardware) using the LoadRunner for Citrix integration. Sessions are initiated and workflow scripts are executed on client machines; simulating real life scenarios and generating realistic network and resource loads. Since scripts controlling user sessions are executed on the client machines, no additional overhead is placed on the MetaFrame servers eliminating the potential inaccuracies that are inherent when testing with the CTSK.

The virtual users are created using a recording technology that captures the ICA traffic between the client and server into a high level, easy to read, maintainable test script. These scripts can be easily modified to represent real users with their own sets of data and replay speeds. By licensing key Citrix technology, LoadRunner’s virtual users generate exactly the same traffic as the ICA client and “fool” the system into believing it is under the load of real production users.

By measuring the end-to-end response time of user transactions, LoadRunner helps identify performance problems from the end users perspective. Using a suite of real-time performance monitors that measure the system performance across all the tiers, LoadRunner can also help isolate these problems wherever they lie in the system. Finally, all the test scripts created in pre-deployment load testing with LoadRunner can be reused as-is in Topaz for performance monitoring once the application goes live.

As with all testing strategies, tests are only as accurate at the workflows utilized to emulate real life scenarios. For example, scripts that emulate users running a word processing program for typing and saving of files may not produce accurate results if printing is usually performed but not incorporated into the script’s workflow. Thus, when developing virtual user scripts, it is important that the scripts replicate the actions of real users to obtain accurate results.

Application Backend

LoadRunner For Citrix

ICA

LoadRunner Controller

Citrix Server + or Server Farm

Citrix (ICA) Virtual Users


Figure 2: LoadRunner and Citrix Testing Scenario

Figure 2 illustrates a testing scenario using Mercury LoadRunner and the Citrix Client integration. In this solution, client machines running LoadRunner virtual users can emulate real users accessing the Citrix MetaFrame farm and backend resources. All scripts are run from the client machines, eliminating any server side overhead producing realistic measurements. A LoadRunner Controller is used to administer the test and aggregate performance measurements.


Setup and Execution of a LoadRunner for Citrix Solution This section details the steps to setup a load test utilizing LoadRunner for Citrix.

Installation of the LoadRunner for Citrix Solution Download the latest Citrix ICA Client for LoadRunner from the Citrix Developer Network website: http://apps.citrix.com/CDN. This file has the latest Headless ICA Client as well as an ATL based sample Application (ATLICO) and a VB based sample (VBWindows) that exercise the ICA Client integration. Installation of the Headless ICA client is identical to the standard ICA client. Register the ICA Client Object (ICO) by using the following command line: "C:\Program Files\Citrix\ICA Client\wfica32.exe" /setup

Installation of LoadRunner 7.51 Windows System Requirements

In order to successfully run LoadRunner you need the following minimum system configuration. Refer to http://www-svca.mercuryinteractive.com/products/loadrunner/technical/ for the most updated installation requirements:

Figure 3: LoadRunner System Requirements

Two types of LoadRunner installations are available:

• Standalone Installation installs LoadRunner on the hard disk of a single-user computer. • Network Installation installs LoadRunner on a network drive, making LoadRunner available to all users who

have access to that network drive.


http://apps.citrix.com/CDN

http://www-svca.mercuryinteractive.com/products/loadrunner/technical/

http://www-svca.mercuryinteractive.com/products/loadrunner/technical/

You can install the Controller, virtual user components, or both. In addition, you can choose to install the full LoadRunner configuration, a load generator configuration, or a custom configuration. Choosing custom configuration lets you specify which LoadRunner components to install.

1. Insert CD and run D:\setup where D: is the CDRom drive letter. 2. In the registration information dialog, insert the Maintenance Number found in the maintenance pack shipped with

LoadRunner (or packing slip). The number is on a sticker in the upper right corner. 3. In the installation type dialog box, select Standalone Installation. 4. In the LoadRunner 7.5 License dialog box, enter the LoadRunner license code from you license agreement. If you

are evaluating LoadRunner or waiting for a permanent license key, enter the license key TEMPORARY. This will let you run up to 25 virtual users for a period of 10 days. For this install, we will be using LoadRunner in evaluation mode, thus entering TEMPORARY.

5. For setup type, select Typical Installation. This installs all commonly-used LoadRunner components, including the LoadRunner Controller, GUI and DB Virtual User components, online documentation, and script samples. Use this option for the machine that controls the vUsers. The Load Generator option installs only the components needed for running virtual users to generate load. Use this option for machines that are used only to generate load and not to control vUsers.

6. On the Choose Destination Location dialog, select the location where you want to install LoadRunner. To select a different location, click Browse, choose a directory, and click OK.

7. The Select Program Folder dialog box appears. 8. In the Select Program Folder dialog box, specify a program folder, or accept the default folder, LoadRunner. 9. Click Next to proceed. The installation settings that you selected are displayed in a read-only window. To select

different installation settings, click Back. To begin installation, click Next. The installation process begins. 10. If you installed LoadRunner’s samples, a message tells you to run the samples setup from the LoadRunner

Samples program group to complete the samples installation. 11. If you installed LoadRunner’s online documentation, a message tells you to run the Acrobat Reader setup after the

LoadRunner setup is complete. You use the Acrobat Reader to view the LoadRunner online documentation. If not already installed, install Acrobat Reader from Start > Programs > LoadRunner > Documentation > Acrobat Reader Setup

12. Click OK to continue with the setup procedure. The User Login Settings dialog box appears. 13. Select one of the following user choices:

• Allow virtual users to run on this machine without user login: LoadRunner will automatically log in to the network from the Load Generator machine, so the virtual users can run without any manual intervention. Enter the network domain where the user machine resides, a user name, and password. Note that you must have administrator privileges. Note: You must boot and log in to the system at least once after LoadRunner installation before the automatic login can work.

• Manual log in to the Load Generator machine: The user must manually log into the network from the Load Generator machine for each session of running vUsers.

14. Select Manual log in to the Load Generator machine 15. The Registration Information dialog box appears. Click Next to complete the installation. 16. The setup program prompts you to restart the computer. Note: The following message appears the first time that a user logs on after a full standalone LoadRunner installation: "HCL Inetd CPLInquire cpl 8af40c". This message relates to the HCL (Hummingbird Communications Ltd.) Inetd service, when Inetd is installed as a Windows NT service. Please close the message box and continue to start your computer.

17. If the samples are desired, run the Samples setup after LoadRunner setup is complete.

Instructions for obtaining a permanent license key

Go to the following URL, where you will be required to submit relevant licensing information, to obtain a permanent license key. http://support.merc-int.com/licenses/LR Note: Once this is completed, it takes approximately 2 business days to receive your license number.


http://support.merc-int.com/licenses/LR

Installation of LoadRunner for Citrix Support Files 1. Unzip LR_For_Citrix_ICA.zip to the c:\mercury interactive\LoadRunner (assuming this is where you installed

LoadRunner) directory. Overwrite any files if prompted. 2. Execute c:\mercury interactive\LoadRunner\bin\citrix.bat. This will register all the necessary DLLs with the system. 3. One of the Citrix support files located in the LoadRunner\bin directory is called CitrixClientImpl.dll. If this file has a

time stamp earlier than 09/03/2002, it must be replaced with a later version to support wildcard names for dialog boxes.

Developing Citrix Virtual User (vUser) Scripts Citrix vUser scripts emulate the Citrix ICA protocol communication between a Citrix client and MetaFrame server. LoadRunner comes with a tool called VuGen that enables the generation of scripts by recording user actions in a Citrix session from a client machine. When a user performs actions on the MetaFrame server, VuGen generates functions that describe these actions. Each function begins with a ctrx prefix and emulates the analog movements of the mouse and keyboard. In addition, the ctrx functions provide synchronization mechanisms to prevent the replay of scripts from loosing timing. Examples of synchronization include waiting for specific windows to open or for a denoted bitmap to appear prior to the script resuming playback. These script synchronization mechanisms are achieved by enhancing vUser scripts to include transactions, rendezvous points, and control-flow structures. For details, see the Creating vUser Scripts guide included with the LoadRunner suite.

When developing scripts it is also important to define script parameters and run-time settings. Parameters are defined for fixed-values recorded into the vUser script. By substituting fixed-values with parameters, the same business process can be repeated many times using different values. An example of this is logging into a Citrix session. When recording the logon activities of a Citrix session, the user’s credentials entered are captured and hard coded into the script. By using parameters, each vUser executing the script can log into the server with a different user account’s credential set. Run-time settings control the vUser behavior during the execution of a vUser script. These settings include the pacing, logging, think time, and connection information.

Workflows to emulate a Citrix user are separated into three different scripts. vUser_init is used to execute the connection and logon to the MetaFrame server. The Actions script is used to perform the actual application workflows. The Actions are repeated a configurable number of times. vUser_end is used to perform any clean up and disconnect from the MetaFrame server.

A .pdf file exists detailing the development of Citrix vUser scripts and configuring the Citrix Monitor on the LoadRunner controller. This document (Developing Citrix vUser Scripts.pdf) is available on Mercury’s support website: http://support.merc-int.com. You must register with Mercury to access this site.

Tips for Working with Citrix vUser Scripts The following tips are listed in the Developing Citrix vUser Scripts document.

Recording Tips

• When recording a session, make sure to perform the complete business process, starting with the connection and ending with the cleanup. End your session at a point from where you could start the entire process from the beginning.

• Configure the Citrix server to completely close a session. Open the Citrix Connection Configuration dialog box. Choose Start > Programs > Citrix > MetaFrame XP > Citrix Connection Configuration. Double-click on the ica-tcp connection name. The Edit Connection dialog box appears. Click on the Advanced button. In the bottom section of the dialog box, clear the inherit user config check box adjacent to the On a broken or timed-out connection list box. Change the entry for this list box to reset.


http://support.merc-int.com/

• Record the connection process into the vuser_init section, and the closing process in the vuser_end

section. This will prevent you from performing iterations on the connection process.

• Display settings of 1024 x 768 are recommended on the recording machine. This will allow the Citrix window, which is 800 x 600 to be displayed properly.

• When opening expanded menu options, click explicitly on each option—do not depend on the expanding menu. For example, Start > Programs > Microsoft Word, be sure to click on the word Programs.

• If having trouble with window or dialog box names not being consistent, edit the script to use wildcards (*). For example, ctrx_set_window("Spelling and Grammar:*”);

Replay Tips

• To prevent overloading by multiple vUsers while connecting, set an initialization quota or use a ramp up schedule from the Controller’s scheduler.

• For best results, do not disable think time in the run-time settings. Think time is especially relevant after ctrx_set_window and before ctrx_type functions, which require time to stabilize.

Debugging Tips

• You can view additional replay information in the Extended log. You enable Extended logging from the Run-Time settings (F4 Shortcut key) Log tab. You can view this information in the Execution Log tab, or the output.txt file in the script’s directory. When running a LoadRunner scenario each vUser generates its own execution log located in C:\mercury interactive\LoadRunner\scripts\ScriptName\res\log\scriptname_vUserID.log.

• When an error occurs, VuGen saves a snapshot of the screen to the script’s output directory. You can view the bitmap to try to determine why the error occurred.

• To check the version of the server, make sure the MetaFrame XP server is installed. Select Citrix Connection Configuration on the server’s console toolbar and choose Help > About.

Executing a Citrix LoadRunner Test As with all testing strategies, a clearly defined testing process helps to ensure accurate and repeatable results. The following section is an overview of LoadRunner’s six-step process for testing applications under load. The details of this process can be found in the LoadRunner Controller User’s Guide.


Figure 4: LoadRunner Testing Methodology

Step1: Planning the test. Successful testing requires development of a thorough test plan. A clearly defined test plan ensures that the LoadRunner scenarios that are developed will accomplish the load testing objectives. Load test planning involves:

• Analyzing the application to determine hardware and software components, the system configuration, and typical usage patterns.

• Defining testing objectives (i.e. Maximum user load, application upgrade compatibilities, identifying bottlenecks).

• Planning the LoadRunner implementation. This involves defining the scope of performance measurements, defining vUser workflows, Selecting vUsers, and choosing test hardware.

Step2: Creating vUser Scripts. vUsers emulate human users interacting with the system under test. vUser scripts contain the actions that each virtual user will perform during a scenario execution. These scripts should emulate what real users typically do with the system in a production environment.

Step3: Creating the Scenario. A scenario describes the events that occur in a testing session. A scenario includes defining the client machines that vUsers will run on, scripts that vUsers will execute, and a specified number of vUsers or vUser groups that run the scenario. Scenarios are created using the LoadRunner Controller.

Step4: Running the Scenario. User load is emulated by instructing multiple vUsers to perform tasks simultaneously. Before executing a scenario, configuration and scheduling is defined. This determines how all the load generators and vUsers behave when the scenario is run. While a scenario is run, LoadRunner measures and records the transactions defined in the vUser scripts. The performance of the system under test (hardware resources) is also monitored and captured for later analysis.

Step5: Monitoring the Scenario. Using LoadRunner’s online run-time performance monitors, all components of the system under test can be monitored. Additional details on Monitoring MetaFrame servers resides in the LoadRunner


Developing Citrix vUser Scripts and Configuring the Citrix Monitor Document Supplement, Version 7.51: Developing Citrix vUser Scripts.pdf and is available on Mercury’s support website: http://support.merc-int.com.

Step6: Analyzing Test Results. Throughout execution of a scenario, LoadRunner records the performance of the system under test at different load levels. LoadRunner’s graphs and reports can be utilized to analyze the systems performance. For more information about LoadRunner’s reports and graphs, see the LoadRunner Analysis User‘s Guide.


http://support.merc-int.com/

Summary In the past, testing a Citrix MetaFrame environment was performed by using server side agents to run scripts from within a Citrix session or utilizing a subset of real users to drive load or a combination of both. Scripts and agents that run server side skew the accuracy of a scalability test since only server to client communication is emulated and agents impose an overhead on the server’s resources. While leveraging real users is the most realistic approach, it is near impossible to obtain enough users to generate production level loads.

With the advent of the Citrix ICA Client integration with the Mercury LoadRunner and Topaz products, system scalability testing and monitoring can be performed accurately with production level user loads. The solution allows for client actions to be scripted and executed on client machines with minimal resource usage. This results in the ability to execute numerous Citrix sessions from a single client machine. With the integration into Mercury’s suite of products, tests can be administered, monitored, and analyzed with ease in a controlled environment. For this document, Topaz was not evaluated, but a later evaluation is planned.


Appendix A: LoadRunner for Citrix Proof of Concept The following sections details requirements, architecture specifications, execution, and conclusions from a Proof of Concept using Mercury’s LoadRunner for Citrix solution.

The tests were executed on a two-processor server with 2 GB of memory. This server configuration was used because it is typical of most MetaFrame deployments. A Windows 2000 Server, MetaFrame XPe/Feature Release 2 platform was used and all sessions and scripts were initiated using the Mercury LoadRunner version 7.51. The scripts executed client based commands on the applications under test (AUT), Word 2000 and Internet Explorer 6.0.

It is critical to note these tests were performed using a unique environment. Because of this and significant variances in application usage patterns, the results are not representative of all production environments and may vary significantly if there are differences in the network infrastructure, server configuration, Farm layout, and application usage. This proof of concept was performed to demonstrate the usage of Mercury LoadRunner testing in a Citrix MetaFrame environment, as well as serve as a template for future testing and validation. It should not replace scalability tests or pilots planned for your implementation.

At the when this proof of concept was performed, the Citrix LoadRunner functionality was still under development. While all stages of the proof of concept were executed, unfortunately do to errors in the functionality of several LoadRunner for Citrix functions, the script did not execute properly and results could not be obtained or analyzed.

Planning the Test: Requirements and Environment Specification This section describes the test environment, setup, and execution of the server sizing and scalability project.

Configuration Details

The following list details the architecture and configurations that were used to conduct the scalability analysis:

• Users were assigned a mandatory profile

• Each user saved their files on a the local MetaFrame server

• No logon scripts were executed for the sessions

• Mercury LoadRunner, version 7.51, was used to simulate concurrent user sessions

• The Mercury LoadRunner Load Generator was installed on the client device to run Virtual Users that initiate the ICA connections and run vUser scripts to emulate user workflows.

• A custom script was created for Microsoft Word 2000 and Internet Explorer 6.0 using LoadRunner’s VuGen recording utility.

• LoadRunner Controller’s Citrix MetaFrame XP Resource Monitor was used to gather metrics.

Software and Platform Specifications

The following tables outlines the software requirements for each server and/or client device used during the project.


MetaFrame XP/ Microsoft® Office

MetaFrame XP Zone Data Collector (ZDC)

Client Devices

Windows 2000 Server with SP 2

MetaFrame XPe with FR 2

Office 2000 Premium

Internet Explorer 5.5

Performance Monitor

Windows 2000 Server with SP 2

MetaFrame XPe with FR 2

Office 2000 Premium

Internet Explorer 5.5

Windows 2000 Server with SP2

Citrix ICA Client Headless Version (6.21.025)

LoadRunner 7.51 Load Generator

Server Specifications • Compaq ProLiant DL360

• Dual Intel Pentium® III 1.4 MHZ Processors

• 2GB SDRAM Memory

• One 18 GB disk drive, Ultra3 SCSI with 10K disk RPM

• Two Compaq NC7780 Gigabit NIC (embedded, only one active, running at 100mb) 10/100 WOL (Wake On LAN)


Network Environment

Client Devices

1 2 3 4 5 6

7 8 9101112

AB

12x

6x

8x

2x

9x

3x

10x

4x

11x

5x

7x

1x

Ethe

rnet

A

12x

6x

8x

2x

9x

3x

10x

4x

11x

5x

7x

1x

C

Laptop

Workstation Computer

MetaFrame OfficeProductivity Servers

MetaFrame ZoneData Collector

LoadRunner 7.51Controller

Figure 5: Proof of Concept Network Layout

The servers and client devices were connected to an Ethernet switch with a 100mb backbone. The users’ profiles were stored on locally on the MetaFrame server. For this proof of concept, the LoadRunner controller was also used as the client device executing vUser sessions, but any number of client devices can be used.


Client Configuration

The following MetaFrame XP client settings/features were enabled/configured.

• Data Compression = Enabled

• Windows Colors = Use Server Default

• Encryption level = Use Server Default

• SpeedScreen Latency Reduction = Use Server Default

Server Connection Configuration

The following chart outlines the connection configuration settings for the MetaFrame XP server. All connections used the ICA protocol over TCP/IP.

• Timeout Setting for Connection = inherit user config

• Timeout Setting for Disconnection = 10

• Timeout Setting for Idle = 30

• Required Encryption = Basic

• AutoLogon = inherit client config

o Prompt for Password = No

• Initial Program = inherit client/user config

o Only Run Published Applications = NOT SELECTED

• User Profile Overrides = none

• On a broken or timed out connection = reset

• Reconnect sessions disconnected = inherit user config

• Shadowing = inherit user config

• Audio Quality = Medium

• Client Settings:

o Connect Client Drives at Logon = inherit user config

o Connect Client Printers at Logon = inherit user config

o Default to Main Client Printer = inherit user config

o By default, connect only the o client’s main printer = NOT SELECTED

• Client Mapping Overrides:

o Client Drive Mapping = ENABLED


o Windows Client Printer Mapping = ENABLED

o Client LPT Port Mapping = ENABLED

o Client COM Port Mapping = ENABLED

o Client Clipboard Mapping = ENABLED

o Client Audio Mapping = ENABLED

Metrics

The following table outlines the metrics monitored during the project. The values were gathered at the server level using the Citrix MetaFrame XP Resource Monitor (RM) integrated into the LoadRunner Controller’s Application Deployment Solutions monitor.

Resource Evaluated

Counter Object Source

Processor % Total Processor Time System RM

Processor Interrupts/sec System RM

Memory/Hard Disk

Pages Reads/Sec Memory RM

Memory Available Bytes Memory RM

Memory Committed Bytes Memory RM

Memory Free System PTE’s Memory RM

Memory % Usage Paging File RM

Hard Disk % Disk Time Physical Disk RM

Hard Disks File Data Operations/sec System RM

Hard Disk Current Disk Queue Length Physical Disk RM

Network Input Session Bandwidth ICA Session RM

Network Latency – Session Average ICA Session RM

User Experience

Transactions were placed into the vUser scripts to capture the amount of time a particular action took to complete. By analyzing these results, latency can be measured and user experience can be better quantified. However, there is no replacement for actual users measuring the “user experience”. By running real users in conjunction with the scripted test, a subjected analysis can be performed. This analysis can be divided into the following three categories. Integrating real users was not in the scope of this proof of concept and therefore was not performed.


Category Description

Good Screen updates are fluid and there is minimal latency.

Fair Screen updates are noticeable and latency is increased. However, the user is still able to function.

Poor The session becomes frozen or disconnected. Therefore, the user cannot continue his/her tasks.

Network Utilization

Network utilization testing was beyond the scope of this proof of concept. Network utilization plays an extremely important role in determining system scalability and user experience. A congested network will not only limit the number of sessions that can be executed concurrently, but will also affect the amount of latency each user feels within their Citrix session. Things to be considered include:

• Network link sizes and bandwidth utilization

• # of hops from clients to MetaFrame servers and backend resources

• Client actions (file transfers, printing and print job sizes, etc.)

• # of concurrent users

Creating vUser Scripts: User Profiling Microsoft Word 2000 and Internet Explorer 6.0 were used to perform this proof of concept. Microsoft Word was published as an application and Internet Explorer was invoked by clicking on an embedded link from within a Word document.

The following workflows were created to emulate basic usage of these applications. The script contains many areas where synchronization or transactions were added manually. The synchronizations kept the script running during periods of high latency while transactions were used to determine the amount of time a particular action took to complete.

Note: The scenarios used did not include saving to local client drives/remote file servers or printing. If these actions take place in the environment under test, they should be accounted for in vUser scripts to obtain accurate test results.

vUser Initialize 1. Logon a vUser.

Actions 1. Close Existing Document

2. Open New Document

3. Save document with date/time stamp in the document name

4. Modify the documents properties (File -> Properties). Add a document title

5. Type a paragraph of text


6. Run the spell/grammar checker (Tools -> Spelling and Grammar)

// Start Transaction

7. Click on the embedded link www.citrix.com

8. Verify Internet Explorer was spawned. If not click it again.

// End transaction

9. Do a bitmap comparison to ensure the page is open.

10. Click on the Support link

11. Do a bitmap comparison to ensure the page is open.

12. Click on the Knowledge Base link

13. Do a bitmap comparison to ensure the page is open

14. Click on the search field

15. Type in “cmc”

// Start Transaction

16. Click the search button.


// End Transaction

18. Click on the link of the first result returned from the search


20. Close the browser (File -> Close)

21. Click on the Word Document to gain focus

22. Save the Document (File -> Save)

vUser End 1. Close Microsoft Word (File -> Exit)

Creating the Scenario A scenario describes the events that occur in a testing session. A scenario includes defining the client machines that vUsers will run on, scripts that vUsers will execute, and a specified number of vUsers or vUser groups that run the scenario. Scenarios are created using the LoadRunner Controller. For this proof of concept the LoadRunner controller was also used as the client machine to invoke vUser sessions. The vUsers ran a script generated from the workflow provided in the section above. The script can be found in the section vUser Script Source Code.


http://www.citrix.com/

Running the Scenario User load is emulated by instructing multiple vUsers to perform tasks simultaneously. Before executing a scenario, configuration and scheduling is defined. This determines how all the load generators and vUsers behave when the scenario is run. While a scenario is run, LoadRunner measures and records the transactions defined in the vUser scripts. The performance of the system under test (hardware resources) is also monitored and captured for later analysis. For this proof of concept, vUsers began execution 10 at a time every 5 minutes. They continue to repeat iterations of the Actions portion of the script until the test is stopped. The system under test was a single MetaFrame server, CCSlabs32, with Microsoft Word published. Unfortunately, do to errors in the functionality of several LoadRunner for Citrix functions, the script did not execute properly and results could not be obtained or analyzed.

Monitoring the Scenario Using LoadRunner’s online run-time performance monitors, all components of the system under test can be monitored. The Citrix MetaFrame server can be monitored to isolate performance problems or resource limitations. The Citrix MetaFrame XP Resource Monitor is an Application Deployment Solution monitor that provides performance information for the Citrix MetaFrame XP server. In order to obtain this data, you need to activate the Citrix MetaFrame XP server monitor before executing the scenario, and specify which measurements and resources you want to monitor.

Analyzing the Results Throughout execution of a scenario, LoadRunner records the performance of the system under test at different load levels. LoadRunner’s graphs and reports can be utilized to analyze the systems performance. Unfortunately, do to errors in the functionality of several LoadRunner for Citrix functions, the script did not execute properly and results could not be obtained or analyzed.

vUser Script Source Code The following code was generated by using the LoadRunner Generator Utility and enhanced by adding synchronization points, transactions, think times, and comments.

vuser_init ()

{ ctrx_set_connect_opt(APPLICATION, "#Word");

ctrx_set_connect_opt(NETWORK_PROTOCOL, "TCP/IP");

/* Parameters added for userid of the vuser so each login is with a different account */

ctrx_connect_server("ccslabs32", "lbuser<vuserid>", "lbuser<vuserid>", "ccslabs32");

lr_think_time(1);

ctrx_wait_for_event("LOGON");

ctrx_set_window("Document1 - Microsoft Word"); }

Actions ()

{

lr_think_time(2);

/* close existing document */

ctrx_key("f", MODIF_ALT);


ctrx_type("c");

ctrx_set_window("Microsoft Word");

lr_think_time(3);

/* Open new document */


lr_think_time(2);

ctrx_type("n");

ctrx_set_window("New");

lr_think_time(2);

ctrx_key("ENTER_KEY", 0);

lr_think_time(3);

/* Save document with time date stamp parameter so we can continue to use

the same paramater to reference the window through out the script */


ctrx_type("s");

ctrx_set_window("Save As");

/* The <savefile> parameter is a time/date stamp parameter that is persistent

for the whole iteration */

ctrx_type("LR test <savefile>");

lr_think_time(5);

ctrx_mouse_click(523, 340, LEFT_BUTTON, 0, "Save As");

ctrx_set_window("LR test <savefile> - Microsoft Word");

ctrx_type("this is a test.");

lr_think_time(2);

/* Modify the documents properties to give it a title */


lr_think_time(1);

ctrx_type("i");

ctrx_set_window("LR test <savefile> Properties");

ctrx_type("LoadRunner Test");

lr_think_time(4);

ctrx_mouse_click(227, 437, LEFT_BUTTON, 0, "LR test <savefile> Properties");

ctrx_unset_window("LR test <savefile> Properties");

lr_think_time(3);




/* Type a paragraph of text */

ctrx_type("This paragraph is being written to emulate usage of Microsoft Word through a Citrix MetaFrame S");

ctrx_key("BACKSPACE_KEY", 0);

ctrx_type("session. This session is being controlled by a LoadRunner Virtual User that is typing this paragraph.î);

ctrx_type("Since the ");



ctrx_type("is Virtual User\'s script was created by EdRod, this paragraph should talk about what a great person he is. Or, maybe not. So I\'m going to type a fw ");



ctrx_type("ew words that are miispelled so that I can runn the spell checker. Being a Virtual User is thd coolest thing in the world. I live sttres");





ctrx_type("ress free with only on ");


ctrx_type("e tal");


ctrx_type("sk. ");


ctrx_type(". namely run this script. Check ou th");




ctrx_type("t this web site: H");

lr_think_time(1);


ctrx_type("http://www.ct");



ctrx_type("itrix.com");

lr_think_time(3);


lr_think_time(1);


ctrx_type("Sincerely,");


lr_think_time(2);


lr_think_time(1);

ctrx_type("Virtual User");

lr_think_time(10);

/* Click the tools menu */

ctrx_mouse_click(207, 33, LEFT_BUTTON, 0, "LR test <savefile> - Microsoft Word");

lr_think_time(2);

/* Click the spelling and grammer menu item */


lr_think_time(2);

/* Begin spell check. Note the use of the wildcard for the window title*/

ctrx_set_window("Spelling and Grammar:*");

lr_think_time(3);

ctrx_mouse_click(404, 53, LEFT_BUTTON, 0, "Spelling and Grammar:*");

lr_think_time(3);


lr_think_time(4);


lr_think_time(3);


lr_think_time(3);


lr_think_time(3);


lr_think_time(3);



lr_think_time(3);


lr_think_time(3);


lr_think_time(3);


lr_think_time(3);


lr_think_time(5);


lr_think_time(3);


lr_think_time(3);


lr_think_time(2);

/* Finish spell and grammer check. Click OK button to the spell check complete dialog */

ctrx_set_window("Microsoft Word");

lr_think_time(5);

ctrx_mouse_click(411, 360, LEFT_BUTTON, 0, NULL);

ctrx_unset_window("Microsoft Word");

lr_think_time(4);

ctrx_set_window("LR test <savefile> - Microsoft Word");

/* Start transaction to open a web browser from an embedded URL. Use the

transaction to measure response time */

lr_start_transaction("open browser");

lr_think_time(3);


/* If the browser didn't appear within 10 seconds, click it again */

if (ctrx_win_exist("Citrix Systems, Inc. - Microsoft Internet Explorer", 10) == 1)


lr_think_time(2);

/* End transaction */

lr_end_transaction("open browser",LR_AUTO);

lr_think_time(5);


ctrx_set_window("Citrix Systems, Inc. - Microsoft Internet Explorer");

/* Click on the support link */


/* Sync and continue script once bitmap has changed */

ctrx_sync_on_bitmap_change(190, 185, 177, 38, 0, 100, CTRX_LAST);

lr_think_time(3);

/* Click on the Knowledge base link */




lr_think_time(5);

ctrx_set_window("Citrix ");

/* Click to gain focus of the search field */


/* Enter a query for CMC */

ctrx_type("cmc");

lr_think_time(5);

/* Start a transaction to measure the time it takes to complete the search */

lr_start_transaction("KB search");




/* End transaction */

lr_end_transaction("KB search",LR_AUTO);

lr_think_time(7);

/* Select the first result returned */

ctrx_mouse_click(341, 251, LEFT_BUTTON, 0, "Citrix ");



ctrx_set_window("C");

lr_think_time(8);

/* Close the browser */


lr_think_time(4);


ctrx_type("c");

lr_think_time(3);

/* Save the word document again */


lr_think_time(3);


lr_think_time(3);

ctrx_type("s");

vuser_end ()

{

lr_think_time(5);

/* Close Microsoft Word, which ends the session */


lr_think_time(3);

ctrx_type("x");

lr_think_time(1);

/* Disconnect from server */

ctrx_disconnect_server("ccslabs32");

}


851 West Cypress Creek Road

Fort Lauderdale, FL 33309 954-267-3000 http://www.citrix.com

Copyright © 2000 Citrix Systems, Inc. All rights reserved. Citrix, WinFrame and ICA are registered trademarks, and MultiWin and MetaFrame are trademarks of Citrix Systems, Inc. All other products and services are trademarks or service marks of their respective companies. Technical specifications and availability are subject to change without prior notice.

mercury interactive's load runner for citrix

Documents