qualisystems testshell - cloud automation and digital … · 2020-05-22 · test automation ......

QualiSystems TestShellTM

TestShell 5.0

Release notes

Release date: July 2013

Document version (Rev A.01)

TestShell 5.0 ii

Table of contents

Introduction ....................................................................................................... 6

TestShell 5.0 ...................................................................................................... 7

New features ................................................................................................................... 7

Lab management ...................................................................................................... 7

Test automation ........................................................................................................ 8

Updated features and functionality changes ................................................................. 10

Installation ............................................................................................................... 10

Licensing ................................................................................................................. 10

Test automation ...................................................................................................... 11

Known issues ................................................................................................................ 11

TestShell Portal ....................................................................................................... 11

Lab management .................................................................................................... 18

Test automation ...................................................................................................... 23

TestShell 4.8 SP3 ............................................................................................ 25

New features ................................................................................................................. 25

Lab management .................................................................................................... 25

Test automation ...................................................................................................... 29


Installation ............................................................................................................... 29

Lab management .................................................................................................... 30

Test automation ...................................................................................................... 35

Resolved issues ............................................................................................................ 35

Lab management .................................................................................................... 35

TestShell 4.8 SP2 Patch 2 .............................................................................. 37

New features ................................................................................................................. 37

Lab management .................................................................................................... 37

Resolved issues ............................................................................................................ 38

Lab management .................................................................................................... 38

Test automation ...................................................................................................... 39


Resolved issues ............................................................................................................ 41

Installation ............................................................................................................... 41

Test automation ...................................................................................................... 41

TestShell 4.8 SP2 ............................................................................................ 42

New features ................................................................................................................. 42

Installation ............................................................................................................... 42

TestShell 5.0 iii

Lab management .................................................................................................... 42

Test automation ...................................................................................................... 42


Lab management .................................................................................................... 44

Test automation ...................................................................................................... 46

Resolved issues ............................................................................................................ 48

Lab management .................................................................................................... 48

Test automation ...................................................................................................... 49


Resolved issues ............................................................................................................ 51

Test automation ...................................................................................................... 51


Resolved issues ............................................................................................................ 52

Lab management .................................................................................................... 52

Test automation ...................................................................................................... 53


New features ................................................................................................................. 54

Lab management .................................................................................................... 54


Lab management .................................................................................................... 54

Resolved issues ............................................................................................................ 55

Lab management .................................................................................................... 55

TestShell 4.8 SP1 ............................................................................................ 57

New features ................................................................................................................. 57

Lab management .................................................................................................... 57

Test automation ...................................................................................................... 59


Installation ............................................................................................................... 60

Lab management .................................................................................................... 61

Test automation ...................................................................................................... 64

Resolved issues ............................................................................................................ 66

Lab management .................................................................................................... 66

Test automation ...................................................................................................... 67

TestShell 4.8 Patch 1 ...................................................................................... 69

New features ................................................................................................................. 69

Lab management .................................................................................................... 69


Lab management .................................................................................................... 69

Test automation ...................................................................................................... 69

TestShell 5.0 iv

Resolved issues ............................................................................................................ 70

Lab management .................................................................................................... 70

TestShell 4.8 .................................................................................................... 72

New features ................................................................................................................. 72

Business intelligence ............................................................................................... 72

Lab management .................................................................................................... 73

Licensing ................................................................................................................. 73

Test automation ...................................................................................................... 74

Resolved issues ............................................................................................................ 75

Business intelligence ............................................................................................... 75

Test automation ...................................................................................................... 75

TestShell 5.0 v

Revision history

Rev Description Author Approved by Date

A.01 Added new and updated

features for version 5.0.

Added known issues for

the current release.

Mika Yehezkeli Nir Ben Eliezer 8/7/2013

Introduction New features

TestShell 5.0 6

Introduction

This document contains a summary of the relevant changes and issues for each version

of TestShell since version 4.8.

The changes in TestShell 5.0 are cumulative, and include all the updates from all

previous version.

New features – These sections describe the new features and functionality added

to this new version.

Functionality changes and updated features – These sections detail changed

features and functionality in this version, with a short explanation describing the

changes.

Resolved issues – These sections cover resolved problems since the previous

version.

Known issues – These sections cover some of the known problems with the

current version.

TestShell 5.0 New features

TestShell 5.0 7

TestShell 5.0

New features

Lab management

Reservation dashboard

Feature Description

New web-based lab

management portal

The Portal’s Reservation dashboard provides web

access to reservations that have been scheduled in the

Resource Manager client.

Reservation dashboard From the Portal’s Reservations dashboard, you can:

Filter reservations according to status and time

range. Admin users can also filter reservations

according to reservation owner.

View current, completed, and pending

reservations for any domain where you have

permissions

Login to another domain where you have

permissions

View reservation details, including the

reservation ID, start time, end time, status, and

owner

View resource details, including resource

attributes and availability

Activate and export topologies and view route

details.

Extend, end, and delete reservations


TestShell 5.0 8

API methods

Module Method Description

Resource &

topology

management \

Resources

GetResourceAvailability Gets availability details for

the specified resources.

Resource &

topology

management \

Resources

GetResourceAvailabilityInTimeRange Gets availability details for

the specified resources for a

specific time range.

Test automation

GUI automation utilities

Feature Description

GUI Automator tool The GUI Automator tool is now also available in TestShell

Driver Builder.

GUI repository assets GUI repository asset types have been added to TestShell

Driver Builder.

Job scheduling

Feature Description

New job scheduling

dashboard in the TestShell

Portal.

In the Portal’s Job Scheduling dashboard, you can:

Create jobs templates for executing shared

Studio tests

Manage job templates and run jobs

Manage execution servers and execution

capacity

Job templates A job template is a package of settings that defines a job. A

job is a package of settings for executing tests via the Job

Scheduling portal.

For every job template, you can:


TestShell 5.0 9

Feature Description

Start a new run.

Run a custom job.

Edit the job template.

Delete the job template.

View the job template history.

TestShell 5.0 adds

distributed runtime

architecture for execution of

multiple tests.

The Job Scheduler manages execution of enqueued,

planned, and pending jobs.

Total test execution capacity depends upon the number of

execution servers that are available to the portal. Each

installed server adds to the available execution capacity.

Control the Job Scheduling

portal’s test execution

capacity

For each execution server, you can:

Toggle the included status

Delete the server

View the server address and description

View the number of jobs currently running on

that server

Usages

Feature Description

Usages feature You can use the Usages feature to find where selected

variables, tests, and functions are in use.

You can also check for unused variables in the current

document.

New Usages pane The new Usages pane displays results for Usages

operations.

You can toggle display of the Usages pane via the View

tab’s new Usages icon.

Variable usages The three new variable usages operations include:

Finding where a selected variable is in use in the

current test or function

Finding all unused variables in the current test or

function

TestShell 5.0 Updated features and functionality changes

TestShell 5.0 10

Feature Description

Deleting unused variables

Updated features and functionality changes

Installation

Application Description

Execution Server installation The Execution server has been added to the TestShell

installer.

You can’t install more than one Execution server on a

machine. Each installed Execution server adds to the web

portal’s total test execution capacity.

You can update Execution Server configuration settings via

the TestShell Configuration wizard.

Easier component selection In the Components selection installation step, you can

select all products by putting the focus inside the product

tree area and clicking Ctrl + A.

Licensing

License Description

QS_TeamServer has been

replaced by

QS_TestShellServer

QS_ResourceManager is no

longer in use.

QS_TestShellServer now

covers both TestShell

Server and the Resource

Manager client.

The QS_TestShellServer license includes resource

manager attributes, max concurrent reservations, and max

ports.

TestShell 5.0 Known issues

TestShell 5.0 11

Test automation

Runtime architecture

Feature Description

Multithreading runtime

architecture

TestShell Runtime 5.0 allows execution of multiple tests.

Known issues

TestShell Portal

Supported platforms

Issue Solution

Portal Server supports Windows 7, Windows

Server 2008 and Windows 2012.

The web portal server is installed

automatically with the TestShell server, and

will automatically remove any previous

versions of IIS Express.

If a previous version of IIS Express is

already installed on the machine, the

TestShell installer will replace it with

IISexpress 8.0.

Browser compatibility

Issue Solution

The Portal client currently supports Chrome,

Firefox, and IE versions 8, 9, and 10.

Supported resolution is 1024*768 and

above. Working with 800*600 is possible,

but not recommended.

Permissions

Issue Solution


TestShell 5.0 12

Issue Solution

Windows authentication is currently not

supported.

Imported TestShell users will not be able to

login to the Web Portal with their Windows

credentials.

Users should login to the Portal with their

TestShell user names and passwords.

Job scheduling

Execution Server

Issue Description

Execution Server crashes following a

TestShell server crash.

Only one Execution server can be installed

on a Windows terminal server.

Browser quirks

Issue Description

Breadcrumbs aren’t displayed properly in

IE8.

In IE8, links in the Executive Summary

section of the report are not functional.

In some configurations, it may not be

possible to login to the web portal using

IE10.

This problem occurs due to a bug in

Microsoft ASP.NET, for which Microsoft

issued a hotfix.

If this happens, we recommend

downloading and installing the hotfix for this

issue.

Ranorex doesn’t support the most current

Google Chrome and Firefox versions.

http://support.microsoft.com/kb/2600088



TestShell 5.0 13

Connectivity

Issue Description

TestShell Server will crash if it loses

connectivity with the database for any

reason for attempts to perform a database

operation while the database is down.

It’s recommended to stop the TestShell

server service before taking the database

offline.

GUI

Issue Description

GUI settings in the Execution Servers list

are reset after refreshing the page.

If you refresh the Execution Servers page

after setting the list to Expand All, the list

will revert to its initial collapsed state.

It is not possible to sort the job queue.

The Job Scheduling’s job queue doesn’t

preserve the user’s filter selection when the

user leaves the page.

The Job Scheduling’s job queue loses the

user’s filter selection after performing

operations on a job template.

If the job template description exceeds the

available display area, you will not be able

to run, edit, or delete the job.

The job template has problems displaying

large variables that exceed the allotted

display area.

The Job Scheduling dashboard doesn’t

update the job queue automatically.

The count on running jobs/ jobs may be

outdated.

Handling of outdated items

Issue Description

Stopping a job that is running on an

Execution server that was deleted from the

Execution Servers list causes unhandled


TestShell 5.0 14

Issue Description

error.

Deleting an execution server with running

jobs doesn’t abort the running jobs.

The job scheduler allows you to delete an

execution server from the list, even if is

currently in use.

If you delete an execution server with

pending jobs waiting for an available

license, the jobs will attempt to run upon

obtaining a license.

If you delete an Execution server, tests

pending license availability will run once

they obtain a license.

Jobs attempting to run on phantom

execution servers remain enqueued

Deleting an execution server that is

required by a job template does not

invalidate the job template.

The job template will be enqueued, and the

job will attempt to run.

Deleting an execution server required by a

job template does not invalidate the job

template

Deleting an execution server sets the

execution server to None selected.

Recreating the execution server will not

reset the job template. The job template will

be enqueued, and the job will attempt to

run.

Reusing a deleted execution server name

causes an error.

If an execution server name was previously

in use, using it again for a new execution

server will throw an error.

Deleting a job template deletes its job

history.

After deleting a job template, its job history

will be inaccessible.

The job template allows users to select

outdated tests.

The list of available tests in the job

template wizard isn’t updated automatically.

An error occurs if you try to edit a job for test

that no longer exists.

Owner information is missing in job

templates for deleted users.

Job template owner information is not

saved for deleted users.


TestShell 5.0 15

Issue Description

Inconsistent handling of deactivated users.

Deactivated users are not blocked from the

portal if they were already logged in before

they were deactivated.

Job templates

Issue Description

If a job is sent to the Execution server while

all licenses are taken, the job owner will

receive a Job Started notification, even

though the job will only actually start running

when a license becomes available.

Foreign characters not supported in the job

template (Oracle specific issue)

If the admin password is changed while jobs

are running, the change won’t take effect

until the admin user logs out and back in.

If an admin user edits a job template that is

owned by another user, the admin will

become the job owner.


TestShell 5.0 16

Performance

Issue Description

If the TestShell Server reaches high CPU

capacity, GUI performance in the portal will

be sluggish.

See the QualiSystems support portal for

information about optimizing TestShell

performance.

Test results with large variable output

require high memory usage in the TestShell

server.

Permissions

Issue Description

Users with no domain defined cannot login

to the job scheduling portal.

Job templates don’t recognize Studio

permissions, allowing users to select tests

that are unavailable to them in Studio.

The job template will be valid, but the job

will fail, as the test will not be executed.

Working with local tests

Issue Description

Local tests on TestShell Server and

Execution Servers are not migrated

automatically when upgrading TestShell to a

new version.

Users must migrate local tests manually,

and copy the migrated versions to the

TestShell and Execution servers.

Check the support portal for more

information about working with local tests.

It is impossible to override default script

engines when working with local tests.

Test variables

Issue Description

Tests with password, session, and

waveform output variables are unsupported


TestShell 5.0 17

Issue Description

Password variables cannot be used as test

input parameters.

Time zone

Issue Description

Changing the time zone while logged into

the portal results saves results to the

TestShell server with the wrong client time.

Shutdown and restart the browser to

realign time zone settings with the

TestShell server.

Time changes implemented while logged

into the portal are not updated in the

TestShell server, and will cause errors when

running scheduled or recurring jobs.

Shutdown and restart the browser to

realign time zone settings with the

TestShell server.

Weeks are changed to days when editing a

job template.

Recurring jobs scheduled to run both before

and after changing from or to daylight saving

time, will not run at the adjusted hour.

Tool specific

Issue Description

Some tools do not support multiple test

execution. You cannot run simultaneous

tests when using GUI automation, Ranorex

steps, Capture Image, and IxNetwork tools.

The IxNetwork process closes abruptly when

the job that had opened it ends.

Users should explicitly end the session

before ending the test.

When running multiple jobs with the LabView

tool simultaneously, some return as

Completed With Error.


TestShell 5.0 18

Reservations dashboard

Issue Description

The Reservation dashboard doesn’t provide

domain information for the reservation.

Double clicking the dashboard filter causes

reservations to be displayed twice.

The Reservation dashboard doesn’t

preserve the user’s filter selection after

refreshing the page. (IE9 & IE10)

Lab management

Embedded resource inventory

Issue Solution

Move is not yet supported in the auto load

framework.

Moved resources are resources which

inventory driver determines are now

associated with another parent resource -

for example a card which has been moved

to another device.

Moving resources is not yet supported, but

development is ongoing.”

It isn't possible to set values of inherited

attributes using Auto Load.

Use the Resource Manager or Studio GUI to

update attribute values.

Domains

Issue Description

It’s possible to modify archived domains. It’s possible to change the topologies folder

and to rename archived domains.

Lab Monitor displays deleted domain

resources

Resources that are removed from a domain

will still be displayed in the Lab Monitor.


TestShell 5.0 19

Inventory operations

Issue Description

Updating values for inherited attributes Auto Load cannot update values for

inherited attributes.

Work-around: Update values for inherited

attributes in the Resource Settings and

Resource Configuration documents.

Performance

Issue Description

Route resolution in large databases In large databases, it can take 2-3 minutes

to indicate that route resolution has failed.

Copying resources with more than 12,000

sub-resources.

It can take more than 40 minutes to copy a

child resource with more than 12k

descendants.

Displaying sub resources with more than

60K child resources

Expanding a sub resource with more than

60K child resources in the Internal

Resources Explorer can take up to 3

minutes. No progress bar is displayed

during the rendering.

No progress indicator when opening a huge

resource (64K) in a large databases.

In large databases, it can take a long time to

open resources with more than 64,000 sub

resources.

Managing resources in large domains In large domains, it can take a long time to

open the domain document’s Resources

page.

Removing the Shared status of large

resources with more than 20K sub

resources in a reservation

It can take up to 9 minutes to change the

Shared status of an active resource with

more than 20,000 sub resources.

Adding domains in large databases It can take more than a minute to add a new

domain to databases with more than 6

million resources.

Running Auto Load can affect performance

of other operations.

For example, when Auto Load is running, it

can take more than 3 minutes to resolve a


TestShell 5.0 20

Issue Description

route, even if the route is requested on a

different station.

The Create Route window may take a long

time to open when creating a route between

two resources with thousands of

descendants each.

It can take up to five minutes to lock a

resource with thousands of child resources.

You may experience a decrease in

performance when creating a reservation

with more than 10K resources.

It may take a long time to add a large

number of resources to a reservation.

Permissions

Issue Description

Domains icon not displayed for admin users

in specific domains.

The Domains icon is only displayed when

logging into the Global domain.

Reservations

Issue Description

Displaying current reservations according to

their start date

In the Current Reservations panel, sorting

reservations according to their start date

sorts the reservations by month.

Cannot reset the default minimum duration

for a reservation

In some configurations, the

MinDurationForReservationInMinutes config

key doesn’t work.

Resources

Issue Description

Failure to clear mapping may result in Out of


TestShell 5.0 21

Issue Description

Sync status.

A route between two abstract ports will only

be resolved using L1 resources (direct

connections for abstract ports not

supported)

It’s not possible to update resource

configuration if the resource name has an

extra space at the end.

Possible problem when adding a resource to

a topology or to a domain's resource page.

This problem manifests itself in 2 different

ways:

1. In topologies 'Properties' window:

When adding a resource, while the

focus is on a folder or a topology in the

resource explorer, an exception will

appear.

2. In domains' 'Resource' page: When the

focus is on a folder or a topology in the

resource explorer, the 'Find Resources'

button will appear to be disabled.

Routing and physical connections

Issue Description

Updating tapped route information using

the ONPATH L1 driver library version

3.0.17

When using the ONPATH L1 driver library

version 3.0.17, connecting MCast based taps

results in inconsistency between Resource

Manager and the switch.

Removing routes from a reservation It’s not possible to remove a route on top of a

physical connection in a reservation. As a

workaround, remove one of the endpoint

resources.

Connecting a physical cabling request

will override physical existing

connections without warning.

Note that existing connections will be overwritten

when resolving a physical cabling request. Note

that only a domain or a system admin is allowed

to override existing physical connections.

When using TestShell API, it isn't Avoid creating routes over existing physical


TestShell 5.0 22

Issue Description

possible to connect a route which

overrides a physical connection.

connections when using TestShell API or use

the UpdatePhysicalConnection method instead.

For large databases, resolving routes

with no resolution may take up to 2-3

minutes.

It is possible to create a tap or a

unidirectional route overriding a physical

connection. This scenario will cause an

error message and the route in the

reservation may be out of sync.

Avoid creating tap or unidirectional routes

overriding physical connections.

TestShell API

Method Issue

CreateResource /

CreateResources

It’s faster to use CreateResource in a loop,

rather than CreateResources for creating

root resources.

FindResources/

FindResourcesInTimeRange

Output from the FindResources and

FindResourcesInTimeRange methods

doesn't include view-only resources when

the Show resources from all domains option

is set to Yes.

ReleaseTopologyResources The ReleaseTopologyResources method

doesn't release root resources.

GetGroupDomains When working with an Oracle database, the

GetGroupDomains method returns an empty

matrix for groups with view-only

permissions.

FindResources/


The FindResources and

FindResourcesInTimeRange methods

cannot search for resources by unique

identifier.

SetAttributeValue /

SetAttributesValues

The SetAttributeValue and

SetAttributesValues methods cannot update

values for inherited attributes.

Work-around: Update values for inherited


TestShell 5.0 23

Method Issue

attributes in the Resource Settings and

Resource Configuration documents.

GetReservationDetails The RouteType value of a connected cable

in reservation is listed as bi instead of cable.

Topologies

Issue Description

Resolving abstract topologies with many

models of the same type

It takes a long time to resolve an abstract

topology when there are a great number of

available candidates for each abstract

resource.

Reserving two topologies with shared

resources

Reserving multiple topologies with

conflicting physical cabling requests

causes an invalid state.

Connecting conflicting physical connections

in the reservation will cause an unhandled

exception to be thrown. In addition, the

conflicting cables will appear connected.

Reserving topologies with shared routes Some shared routes conflict types will not

be indicated in the Reservation Details

window. These conflicts will only be known

after initiating the reservation, which

resolves the reservation’s requested

routes.

Any found conflicts will be displayed at the

top of the Current Reservation diagram.

Test automation

Shared tests

Issue Description

Shared tests can include local tests. Running a shared test that includes a local

test will fail when executing the test outside

of the local hosting environment.


TestShell 5.0 24

Assets

Issue Description

Deleting asset folders in TestShell Studio Attempting to delete non-empty asset

folders causes an error in Studio.

You’ll be able to delete the folder after

deleting the folder contents.

Tools and libraries

Issue Description

If an Instruction tool step sets

an invalid default value for a

combo box field, the combo

box will initially display a valid

value from the data set.

However if the user clicks

Restore Defaults, the combo

box will display the invalid

value defined in the test.

Selecting Restore Defaults in an instruction tool combo

box returns an invalid default value.

Usages

Issue Description

Test refactoring Copying a test with references to another test that is

located under a folder with the same name as the test

creates an invalid state.

TestShell 4.8 SP3 New features

TestShell 5.0 25

TestShell 4.8 SP3

New features

Lab management

Admin operations

Module Feature Description

Resource

allocation /

Domain

management

Restructured domain

document

The Domains document has been

restructured to accommodate the

updated resource allocation features.

You can view the domain’s reservation

history, archive and restore domains,

and specify permissions for groups that

are associated with the domain.

You can allocate resources to a domain

via search results, by dragging and

dropping resources from the Resource

Explorer, or by adding topologies to the

domain.

Resource

allocation /

Domain

topologies

New Domains diagram In the new domain Diagram, you can

allocate resources to the domain, add

and remove topologies, and manage

topology routes.

You can also toggle display of the

topology’s physical layer.

The Domain diagram is optimized for

small to medium domains and can

display up to 200 resources.

Resource

allocation /

Domain

management

New Domains Explorer

pane

The Domains Explorer has been

upgraded to a stand-alone pane.

Use the Domains Explorer to add and

remove domains, select and open

domains, and toggle display of archived

domains.


TestShell 5.0 26


Resource

allocation /

Domain

management

Domain archiving You can now archive domains that are

no longer in use. You can toggle display

of archived domains in the Domains

Explorer pane.

To restore an archived domain, click the

toolbar’s Show archived domains icon

to enable display of the archived

domains. Double click to open the

domain, and click the Unarchive button

in the top-right corner of the domains

document.

Resource

allocation /

Domain groups

Added view only

permissions level for groups

associated with a domain

Admin users can grant groups view-only

access to a domain.

Groups granted view only permissions to

a domain, will be able to view its

resources or add them to topology. As a

user with view only access, however, it

is not possible to reserve resources or

affect them in any way.

Resource

allocation /

Domain groups

A Groups tab has been

added to the Resource

Configuration document.

In the new Groups tab, you can view

which groups have access to the

resource, and the permissions level for

each group.

Resource

management /

Domain

resources

A Domains tab has been

added to the Resource

Configuration document.

In the new Domains tab, you can review

or update the list of domains where the

resource has been allocated.

Resource

allocation /

Domain

resources

A Domains tab has been

added to the Groups

document.

You can now associate a group with

selected domains directly from the

Groups document.

Resource

allocation /

Domain

topologies

Admin level support for

displaying L1 switches and

sub resources in the

diagram documents.

Detailed display of L1 switches,

including the switch sub resources.

You can toggle display of L1 switches

and patch panels by checking or clearing

the Show physical layer option in the

diagram toolbar.


TestShell 5.0 27


Resource

allocation /

Domain

topologies

Smart route creation

automatically adds

connecting devices to the

domain diagram.

Resolving a request for a route segment

between two resources that are

connected via an L1 switch or patch

panel automatically adds the connector

resource to the domain and displays it

on diagram.

API methods

Module Method Description

User

management

/ Domains

SetGroupDomainPermission Use to set a group’s

permissions for the specified

domain.

User

management

/ Domains

ArchiveDomain Archives a specified domain

and deletes any scheduled or

recurring reservations.

User

management

/ Domains

UnarchiveDomain Restores an archived

domain, including its

associated resources,

topologies, and groups.

Reservation

operations /

Routing

UpdateRouteAliasesInReservation Updates attributes and

associated values for

specified routes via their

aliases.

Resource &

topology

management

/ Topologies

DeleteTopology Use to delete the specified

topology.

Resource &

topology

management

/

Connections

UpdatePhysicalConnections Define physical connections

(cable links) between

resources.

Configuration ExportFamiliesAndModels Exports the resource

families, models, attributes

and structure configuration.


TestShell 5.0 28

Resources


Resource

availability

status

Resource status icons and

tooltip

The icon now updates to indicate the

resource’s current availability status.

Detailed resource availability information

has been added to the resource tooltip.

Resource

accessibility

Show All Domains filter

toggles display of resources

outside the current domain.

The Home tab’s new Show All

Domains icon toggles display of

resources from other domains assigned

to the user.

When the Show All Domains toggle is

active, users will be able to view

resources from all assigned domains,

but will not be able to reserve or affect

them.

When the toggle is inactive, the system

will show only resources belonging to

the current domain.

Topologies


Routes New Cable route type The new Cable route type allows users

to create a requested cable route

between two resources that are not

currently connected or managed via an

L1 switch.

Routes Connecting / Disconnecting

cable routes updates the

resource connections

Users can connect the resources directly

by right clicking the route and choosing

Connect from the context menu.

Disconnecting a cable is allowed for

administrators only.

TestShell 4.8 SP3 Updated features and functionality changes

TestShell 5.0 29

L1 drivers

Driver Version Supported firmware

MRV_MCC_4870 3.0.17 MRV v4.8 mcc 07

Test automation

Traffic libraries

Library Version Description

IxN2X 1.0.0.23 IxN2X traffic library for working with

Ixia IxN2X traffic generator.


Installation


TestShell Controller &

TestShell Foundation

TestShell Controller and TestShell Foundation have been

removed from the standard installation package.

Upgrading to TestShell 4.8 SP3 will not make changes to

these applications if they are already installed. However it’s

still recommended to include both programs in the pre-

installation backup.

Resource Manager

configuration

Automatic updating of resource families, models, and

structures is now one of the Admin tasks in the TestShell

Server configuration utility.

You can import updated resource definitions directly in

Resource Manager via the Resource Families explorer,

after running the QsMigrationUtility.exe utility.

Upgrading from older

TestShell versions

You can now upgrade directly to TestShell 4.8 SP3 from

TestShell 4.3 or higher.

Note: while pre-installation backup and validation are

recommended for any upgrade, these are mandatory

processes when upgrading from any version prior to


TestShell 5.0 30


TestShell 4.6.

Note that you must also run ConvertDashboard

Binaries.exe to upgrade any existing dashboards.

Contact QualiSystems support before upgrading if your

TestShell implementation includes more than 100

associated test attributes.

TestShell server

configuration

Admin settings to the TestShell server have been

centralized into a single configuration step.

Lab management

Admin operations


Resource

management

Perform Auto Load and

Sync From operations on all

types of resources.

You can use the QsAutoInventory utility

to automate inventory operations.


TestShell 5.0 31

API methods

Module /

Update

Method Description

Resource &

Topology

Management /

Resources

FindResources


The added Show all domains

parameter allows the user to

retrieve content from the logged in

domain, or from all domains where

the user has permissions -

including view-only level

permissions.

The added Usage Summary

output parameter helps identify

available resources.

More outputs for each resource

includes the shared state,

reservation status, permissions

and exclude status.

Resource &

Topology

Management /

Resources

CreateResource

GetLockedResources

GetResourceList

The current permission is also

available in the result xml.

Resource &

Topology

Management

GetFolderContent

GetResourceDetails

The added Show resources from

all domains parameter allows the

user to retrieve content from the

logged in domain, or from all

domains where the user has

permissions - including view-only

level permissions.

The current permission is also

available in the result xml.

In 4.8 SP2 Patch 2, the

GetResourceDetails method was

updated to improve performance

for top level resources.

User

management /

Domains

AddGroupsToDomain The added View only parameter

allows admin users to set a

group’s permission levels for the

specified domain.

To change an existing group’s


TestShell 5.0 32

Module /

Update

Method Description

permission, use the new

SetGroupDomainPermissions

method.

The added View only parameter

allows admin users to set or

modify a group’s permission levels

for the specified domain.

User

management /

Domains

GetDomainDetails The added Include Archived

Domains parameter allows you to

retrieve domain details from

archived domains.

Performance


Domain,

Reservation,

and Topology

diagrams

GUI performance

Domain, Reservation,

and Topology

diagrams

Support for operations on large resources and

large databases.

TestShell API Improved

performance for large

scale delete

operations

Supports deletion of large root resources with

many sub resources.

Attributes Improved

performance of

adding and removing

attributes

Support for operations on large databases.


TestShell 5.0 33

Permissions


Login Users may login to a

view-only domain.

A user’s view only domains have been added to

the Login window’s list of available domains.

User permission levels are indicated for each

domain.

Resources


Resource

search

Added search specificity

levels and new resource

availability filters

Use the Name field to search for a

specific resource and its child resources.

The Available only option is now part of

a set of three optional search filters

located next to the Search button.

You can filter out unavailable resources,

connected resources, and resources

allocated to domains.

Resource

search

Find resources You can now search within a resource

directly from the resource context menu.

When searching within a resource, the

search results will be populated

automatically with the sub resources of

the selected resource.

Resource

search

Added shortcut to open

resource search.

Pressing Ctrl + F opens the resource

search pane.

Resource

search results

Added resource availability

to the search results grid

An Internal Usage column has been

added to the search results grid,

denoting the availability status for the

resource’s ports or lowest levels

resources.

The resource tooltip displays a summary

of the availability status for each search

result.

The usage information is currently not

available for sub resources.


TestShell 5.0 34


Resource

search results

Drill down links added to the

resource search results grid.

You can search inside any resource in

the search results grid by clicking the

link in the Name column.

You can search inside a resource

connection by clicking the link in the

Connected To column.

Reservations


Scheduler Toggle calendar

orientation

Toggle between horizontal and vertical

orientation for the selected time range.

Scheduler Enhanced navigation

for the selected time

range

Easier access to current and historical

reservation views.

Topologies Shared resource

attributes

You can now set the attributes of a shared

topology resource, even when it’s an active

resource.

New

reservations

Initiating new

reservations

It’s now possible to initiate a new reservation

from the Application menu, and from the

ribbon’s Home tab.

Current

Reservations

Ending reservations It’s no longer possible to end all reservation

from the Current Reservations panel. If needed,

you can multi-select all of the reservations and

use the End button to end all of the selected

reservations.

L1 drivers


MRV_MCC_4640 3.0.17 MRV v4.6 mcc 04

MRV_MCC_4840 3.0.17 MRV v4.8 mcc 04

ONPATH_Horizon_0244 3.0.17 Horizon 2.4.4

TestShell 4.8 SP3 Resolved issues

TestShell 5.0 35


CW_GLX4000 3.0.14 LXfw v1.10.0.0

APCON_CLI4 3.0.17 CLI3-4

FiberZone AFM 3.0.17 1.6.1

Test automation

Traffic libraries

Library Version Description

IxExplorer 1.0.0.19 Performance improvements and bug

fixes.

IxLoad 1.0.0.17 Performance improvements and bug

fixes.

IxNetwork 1.0.0.20 Performance improvements and bug

fixes.

Resolved issues

Lab management

TestShell API

Module / Feature Description

Loopback mapping Fixed an issue where, under specific circumstances,

loopback mapping in a topology would fail.


TestShell 5.0 36

Topologies

Module / Feature Description

Topologies Resolved an issue where, under certain circumstances,

activating a topology would not apply the requested

attribute values.

TestShell 4.8 SP2 Patch 2 New features

TestShell 5.0 37

TestShell 4.8 SP2 Patch 2

New features

Lab management

L1 drivers

Feature Description

New ONPATH driver 4.8 SP2 Patch 2 includes an updated ONPATH driver with

radically improved performance.

The new driver requires Horizon firmware version 2.4.4 or

higher.

QualiSystems strongly recommends updating the firmware

for compatible ONPATH L1 switches, to allow use of the

updated driver.

New MRV driver The new MRV driver provides improved handling of

unexpected disruptions in connectivity between the driver

and the L1 switch.

With previous versions, losing connectivity to the server

would not clear the port connection or log out the driver

from the switch, preventing the driver from logging back in

when connectivity was resumed.

Resources

Feature Description

This patch adds a set of

tools for performing batch

resource operations.

New API methods allow you to create delete, include,

exclude, and set attribute values for multiple resources.

TestShell 4.8 SP2 Patch 2 Resolved issues

TestShell 5.0 38

TestShell API

Method Description

GetResourceDetails The GetResourceDetails method has been updated to

provide significantly faster performance for top level

resources.

SetAttributesValues Sets new attribute values for the specified matrix of

resources and resource attributes.

CreateResources Creates multiple new resources.

DeleteResources Deletes a set of specified resources.

IncludeResources Includes a set of specified resources.

ExcludeResources Excludes a set of specified resources.

Resolved issues

Lab management

L1 drivers

Driver Description

ONPATH Resolved an issue where when ending a reservation with

both regular and tapped routes, Resource Manager would

consider all routes to be cleared, while in the L1 switch, the

routes were not cleared.

TestShell API

Method Issue

SetAttributeValue /

SetAttributesValues

Resolved a problem with setting attribute lookup values

using these methods.

GetScheduledReservations Resolved a problem with the GetScheduledReservations

method, where it would retrieve all reservations for the

specified time frame, including reservations outside of the


TestShell 5.0 39

Method Issue

user’s domains.

GetResourceDetails Resolved an issue where the GetResourceDetails method

would fail if any of the child resources had the same name

as one of the root resources.

Test automation

Libraries

Feature Issue

Library updates Fixed an issue where after updating a library, under specific

scenarios, the icons would not refresh to indicate that the

DLL had been successfully updated.

Assets

Feature Issue

Script activation Fixed an issue where activating a script asset would clear

the value of the Script Name field.

Tools and libraries

SNMP

Feature Issue

SetValues method Fixed an issue where the SetValues method was unable to

set specific types of values.


TestShell 5.0 40

SVN integration

Feature Issue

Integration with TestShell Resolved a number of errors that would occur when saving

changes to tests managed by SVN.

Resolved an issue where reverting a test to a previous

version would cause TestShell to crash.

Resolved an issue where changes to tests by multiple

users would cause SVN to display a loop of messages.

Clean Up Fixed an error that enabled the Clean Up function for

individual files.


TestShell 5.0 41


Resolved issues

Installation

Module Resolved

64-bit support Resolved a problem with Microsoft Windows on 64-bit machines

that would cause TestShell Studio and TestShell Server to crash

under several scenarios.

Test automation

Runner

Module Resolved

QsTestRunner Resolved an issue where QsTestRunner would ignore steps with

library methods, if running prior to opening Studio/Runner for the

first time.


TestShell 5.0 42

TestShell 4.8 SP2

New features

Installation

Feature Description

Support for 64-bit

systems

TestShell can now run in 64-bit mode on 64-bit systems.

TestShell 4.8 SP2 is installed as both a 32-bit application and

as a 64-bit application.

Support for Windows

Server 2012

TestShell 4.8 SP2 adds support for Windows Server 2012.

This applies to both TestShell clients and TestShell servers.

Lab management

TestShell API


User management New CopyDomainResources

method

Added a method for copying

resources from multiple source

domains to a target domain.

Resource

management

New AutoLoad method Added a method for performing

auto load of L1 switch structure

and attributes.

Test automation

Assets


Script assets Generic script engine and

custom script asset types

The new generic script engine

adds support for custom script

asset types.


TestShell 5.0 43

Data types


Variables New password data type The new password data type

allows you to create specific

password variables.

Password variables do not retain

their encryption outside of

TestShell.

For example, when writing a

password to file or transmitting it

to a command shell or Telnet,

TestShell will decrypt the

password before sending it.

Runner


QsTestRunner New command line test

runner

QsTestRunner is a command line

runner for executing tests without the

TestShell Runner GUI.

SVN integration SVN support for

TestShell Runner

TestShell Runner now supports the

SVN integration for checking out tests

from the SVN repository


TestShell 5.0 44

Tools and libraries


MatShell tool New MatShell password

functions

SP2 provides new MatShell password

functions for working with the new

password data type.

str2password

Converts an input string to an encrypted

password.

encryptedstr2password

Converts an encrypted input string to a

password variable.

password2encryptedstr

Converts a password variable to an

encrypted string.


Lab management

TestShell API

Module Update Description

Reservation

management

Changed reservation status

options in these methods:

CreateReservation,

CreateImmediateReservation,

GetCurrentReservations,

GetReservationDetails, and

GetScheduledReservations

The status parameter in the

relevant reservation methods

has been updated to align with

the status options (Pending,

Started, In use, Completed, and

Overtime) in the Filter by

reservation status feature.

User management GetDomainDetails The resources matrix now

contains the full name of a

resource in the Name column.

Topology

management

GetTopologyDetails output The GetTopologyDetails method

now displays the full resource

path for resources listed for each

topology route.


TestShell 5.0 45

Reservations


Scheduler Filter by reservation

status

The filter by reservation status option allows

you to display only reservations that are

Pending, Started, In use, Completed,

Overtime, or a combination of specified

status states.

Filter criteria is cumulative. If you add

additional filters, the Scheduler will display

only reservations with the specified states

for the specified conditions.

Note that unlike the other filters which are

static, reservation status is a dynamic state.

When filtering by reservation status, you

can use the Scheduler tab’s Refresh icon to

update the displayed reservations with real-

time results.

Scheduler GUI improvements A new Scheduler tab has been added to the

Resource Manager ribbon.

The view and time range buttons have been

removed from the Scheduler toolbar, and

added instead to the new Scheduler ribbon

tab.

Scheduler Navigation widget

added to Table view

The Scheduler’s navigation widget has

been moved to the Scheduler toolbar, and

is now also available in Table view.

Topologies


Topology diagram GUI improvements The topology diagram has been enhanced

to accommodate large reservations.


TestShell 5.0 46

Test automation

Assets


Software assets External .NET DLL

hosting

.NET DLL software assets are now

executed by default as external

processes.

This feature is designed to improve

performance of tests that include these

types of software assets, as external

hosting frees up internal Studio

memory.

Note that External .NET DLL hosting

runs as a 32-bit process, even on 64-bit

systems.

Admin operations

Module Update

TestShell Spy The TestShell Spy executable now requires elevated

permissions (must be run as admin).


TestShell 5.0 47

Tools and libraries


Database access

library

There can be only one active

transaction for each DLL

session.

In previous versions, it was

possible to open a new

transaction, perform some

commands against the database

and end the transaction on one

branch of execution while doing

the same on a parallel branch.

Each branch would use its own

transaction, even one there was

only one session containing the

library.

To ensure compatibility with the

updated .NET DLL hosting

feature, it is no longer possible

for multiple transactions to share

the same session.

It is still possible to run multiple

transactions in parallel, however,

each transaction must now run in

its own session.

Command Shell tool PowerShell 3.0 support TestShell 4.8 SP2 adds support

for PowerShell 3.0 Command

Shell sessions.

PowerShell 2.0 is still supported.

Security

Module Resolved

SSH encryption Added support for the AES256-CTR SSH encryption algorithm.

The other supported encryption algorithms include:

AES256-CBC, 3DES-CBC, AES192-CBC, and AES128-CBC.

EXE drivers Executable files compiled in Driver Builder must be run as admin.

Permissions in Driver Builder executables have been elevated to

facilitate running of command line programs.


TestShell 5.0 48

Resolved issues

Lab management

Performance

Module Resolved

Reservations Improved performance relating to refreshing and loading large

reservations.

Routes Improved performance for connecting multiple shared routes.

Drivers Fixed a possible out of memory issue that could happen if the

system contains very large drivers.

Reservations

Module Resolved

Reservation

management

Improved the Save reservation as topology feature. If a user

overwrites an existing topology by saving a reservation as one of

its included topologies, the topology in the reservation will now be

updated with these changes.

Resources

Module Resolved

Resource operations Resolved an issue where under some conditions, a resource

could not be deleted.

Resource

management

Fixed an issue where, under some circumstances, resource

attributes could not be deleted.


TestShell 5.0 49

TestShell API

Date/time alignment

Module Resolved

GetServerDateAndTime

method

Fixed an issue with the GetServerDateAndTime method’s

handling of the API’s date/time alignment. The method now

uses the correct time format specified in the configuration file.

Topologies

Module Resolved

Topology operations Fixed an issue where removing a route could cause an invalid state

if the route’s end point ports relied on a resource that was not

shared.

Topology operations Fixed an error when trying to connect shared and non-shared

routes in the same action.

Topology operations Fixed a problem where routes could not be deleted if several users

in the system were attempting to remove routes concurrently.

Test automation

Tools and libraries

Module Resolved

GUI Keyboard tool Resolved an issue with {key.End}, which can now be added to

key press sequences.

Source control

Module Resolved

Libraries Resolved a problem with checking in libraries to TestShell when

using an Oracle database.


TestShell 5.0 50

Variables

Module Resolved

Publishing Resolved an issue where, under certain circumstances, the

output values from two published variables would be swapped

when the test is called from another test.


TestShell 5.0 51


Resolved issues

Test automation

Tools and libraries

Module/Title Resolved

PowerShell support This patch resolves various issues relating to PowerShell.

Note that support for PowerShell 3.0 is available in TestShell

SP2.

Ranorex integration Fixed a problem which caused Ranorex license server to lease a

license when opening TestShell Studio, even though Ranorex

was not used.


TestShell 5.0 52


Resolved issues

Lab management

L1 drivers


Updated MRV driver for

MCC 4.8.04 L1

switches

The patch includes an updated MRV driver (version 3.0.13) to

fix a timeout issue with the MCC 4.8.04 L1 switch during route

connection operations.

Error message

displayed by mistake

when performing

mapping changes.

Resolved an issue where performing mapping changes after a

Sync From operation would produce an error message, even

though the actions were both successful.

Resources


Locking and unlocking Resolved a performance issue that occurred while locking many

resources concurrently (for example during the activation of a

large topology).

Topologies


Copying topologies

with shared routes

Copying a topology with shared routes within the Resource

Explorer tree copies the topology without any of the shared

routes.


TestShell 5.0 53

Test automation

Tools and libraries


Ranorex runtime

sessions

Resolved an issue where initiating a new runtime Ranorex

session in IE Explorer would cause Studio to hang.

TestShell 4.8 SP1 Patch 1 New features

TestShell 5.0 54


New features

Lab management

L1 drivers

Module Feature / Update Description

ONPATH 2.4

driver version

3.0.13:

A new, high

performance

ONPATH driver has

been added to the

patch release.

Designed to utilize only duplex connections

(except in the case of tapping routes), the new

driver is able to provide significantly improved

performance - especially when connecting and

disconnecting numerous mappings.

Note that with this driver, unidirectional

connections can only be added to ports that are

already connected via a duplex connection.

Any added unidirectional connections must be

cleared before removing a duplex connection

originating from the same port.


Lab management

Reservations


Reservations Ending

reservations

End Reservation clear mapping operations are now sent

as a bulk for each L1 switch, improving performance.


TestShell 5.0 55

Resources


Resources Adding and

copying

resources

Optimized database queries for adding and copying

resources, reducing the time it takes to complete these

operations –especially in large databases.

Routes and physical connections


Performance Connecting

and

disconnecting

Improved time and efficiency for connecting and

disconnecting routes.

Routes Shared route

behavior

Improved the default behavior for selecting whether a

new route should be shared.

Topologies


Reservations Saving a

reservation

into an

existing

topology.

Allows users to save a reservation as a topology, even if

the topology already exists.

Note that feature will overwrite the existing topology.

Resolved issues

Lab management

Reservations


End reservation Fixed an issue where the End Reservation button wasn’t always

enabled for selected reservations in the Current Reservations

list.


TestShell 5.0 56

Resources


Mapping Resolved an issue where the resource mapping table would be

displayed as empty for resources with more than 2500 ports.

Topologies


Copy topology Resolved an issue where the shared route status is not

preserved when copying a topology.

TestShell API


Resource naming Resolved an issue caused by using special characters in the

resource full name.

Resource naming Resolved an issue where you could not add a new subfolder if a

one of its parent folders had the same name.


TestShell 5.0 57

TestShell 4.8 SP1

New features

Lab management

TestShell API


User

management

New methods AddGroupsToDomain

AddNewDomain

AddResourcesToDomain

AddTopologiesToDomain

DeleteDomain

GetDomainDetails

RemoveGroupFromDomain

RemoveResourcesFromDomain

RemoveTopologiesFromDomain

UpdateDomainTopologiesFolder

Resource &

Topology

management

New API methods GetTopologyDetails

UpdateTopologyOwner

Reservation

management

New API methods UpdateReservationDescription

Domains

Feature Description

Added domain

functionality

Add a new domain, or delete an existing domain.

Add groups, resources, and topologies to a domain.

Remove groups, resources, and topologies from a domain.

Retrieve domain details.


TestShell 5.0 58

Topologies

Feature Description

Added topology

functionality

Update the topology owner and retrieve topology details.

Reservations

Feature Description

Updated reservation

functionality

Update the reservation description.

Reservations


Reservations Admin control over

asking whether

mapping should be

cleared at the end of

a reservation

Admin users can configure Resource Manager to

automatically clear mapping when ending a

reservation, or removing resources with mapping

from an active reservation.

To configure the message settings:

Open the

QualiSystems.ResourceManagement.Service.Plu

gin.config file from the TestShell Server

installation folder.

Look for the Reservation.AutoClearMapping key.

If the key does not exist, copy one of these

options to the file manually:

<add key="Reservation.AutoClearMapping"

value="true"/>

<add key="Reservation.AutoClearMapping"

value="false"/>

Set the Reservation.AutoClearMapping key to

either True or False.

Setting AutoClearMapping to True will

automatically clear mapping upon reservation

end, without displaying a message to the user.

Setting AutoClearMapping to False requires user

confirmation to clear mapping. The Clear


TestShell 5.0 59


Mapping message will be displayed, allowing

users to decide how to handle mapping issues on

a case by case basis.

Reservation

notifications

Separate admin

notification settings

Admin users will now receive notifications for

reservation activities according to the email

settings specified in the

QualiSystems.ResourceManagement.Service.Plu

gin.config file. These settings have preference

over reservation notification settings.

Reservation

notifications

Conflict warning

notification

A notification bar has been added to the active

reservation document to indicate that a conflict

exists.



Route

attributes

ONPATH L1

port flapping

control

This attribute allows users to configure turning ports on

and off at predetermined intervals.

Test automation

Traffic libraries


Traffic stencil New traffic library The IxExplorer library, version 1.0.0.13, was

added to the Traffic stencil.


TestShell 5.0 60

Tools and libraries

MatShell


MatShell tool New MatShell

date/time functions

SP1 provides new MatShell date/time functions

that take advantage of the flexible date/time

format options.

addtotime

Adds time to a given date

currenttime2str

Returns the current date and time as a string.

time2str

Returns a specified date and time as a string.

time2vect

Returns a specified date and time as a vector.

timediff

Gets the number of seconds between two date

and time instances.

You don't have to update tests that use the old

functions. Tests using the old date/time

functions will still work. However, you should use

the updated functions for all new tests. The old

functions do not appear in the updated MatShell

function list.


Installation


TestShell files Configurable

TestShell file location.

You can now modify the default location for

TestShell files.


TestShell 5.0 61

Lab management

L1 drivers


Drivers Updated L1

drivers

Apcon L1 Switch drivers are not part of the

TestShell 4.8 SP1 installation. If required, please

contact QualiSystems support team.

The ONPATH UCMS driver has been updated, and

can now be used without Horizon.

Drivers Renamed

ONPATH and

MRV L1 drivers

The names for ONPATH and MRV L1 drivers have

been changed to:

ONPATH 3900 HorizON 2.3.0

ONPATH 3900 HorizON 2.4.0

ONPATH 2900

ONPATH UCMS 1.2.0

MRV 4.6.04

MRV 4.8.04

Drivers MRV driver

support

TestShell supports MRV driver versions 4.8.04 and

4.6.04. Older driver versions will not be supported

from TestShell versions 4.8 SP1 and above

Drivers ONPATH blade

interoperability

Added ONPATH blade interoperability constraints.


TestShell 5.0 62

Reservations


Reservations Shared routes

in topologies

This capability allows multiple users to share routes, or

for an individual user to share routes over multiple

topologies.

If you save a topology as a reservation, any routes set to

shared state will be saved together with the topology.

Note that any user with access to a shared route can

make changes to the route.

Routes with abstract resources cannot be set to shared

state.

Reservations Reservation

unique ID

The Reservation ID has been added to the GUI in three

different places:

1. The current reservations table.

2. The active reservation details pane.

3. The reservation properties window.

Reservations Reservation

visibility

Non-admin users can now view current and scheduled

reservations for all users in their domain.

Reservations User

reservation

credentials in

TestShell

drivers

When creating a TestShell driver command, it is

sometimes required to perform operations via the

TestShell API. Using the TestShell API requires logging

into the system, so the driver needs to have the

username/password information to do so. It is now

possible to receive, in the context of the driver, the

reservation owner credentials, in order to log-in and use

the TestShell API.

To use this feature:

Add a new matrix type variable as an input parameter to

the driver called "reservation". The matrix should be

structured as follows:

Username

Password

Domain

ReservationId


TestShell 5.0 63


The value for each property will be automatically

assigned to the third column of the matrix when the

function is called by the TestShell Server.

Please note that the reservation matrix parameter is

optional. For backwards compatibility purposes, it is also

possible to get the "ReservationId" in the "resource"

matrix parameter.

Resources


Power

management

Apply power

operations on all ports

It’s now possible to perform power operations on

all ports for the specified resource.

Power

management

Power management

control for admin

users

Admin users can now perform power operations

on excluded resources. The resources can

remain in their users’ reservations during the

operation.

Attributes Attribute description

parameter and

filtering

The attribute description parameter will be

displayed as a tooltip when hovering over the

attribute, and as a column in the Attribute bank.

You can also filter the Attributes bank via the

Description column.

Commands Command input

column order

The column order was changed to have the

Name and Value first.

TestShell API

Date/Time alignment

Feature Description

Date/time format

options

TestShell is now able to output and accept date/time values in a

variety of formats.

For the TestShell API library and asset .dll file, the default

date/time format is taken from the operating system settings. It is

possible to override this default and specify the format explicitly

for each client. To do so, add the following line to the


TestShell 5.0 64

Feature Description

customer.config file located in the TestShell Studio installation

folder:

<add key="DateTimeFormat" value="dd/MM/yyyy HH:mm:ss"/>

This example configuration defines a specific date/time format.

You can specify a different one by changing the format.

For the TCL client the date/time format settings are stored in the

config.ini file which exists in the TCL client folder.

Power operations


Power

management

Apply power

operations on all ports

The ports parameter in the Power management

API is now optional.

Leave blank to perform power operations on all

ports for the specified resource.

Power

management

Power management

control for admin

users

Admin users can now perform power operations

on excluded resources.

The resources can remain in their users’

reservations during the operation.

Test automation

Tools and libraries


TestShell

libraries

Improved library

access

The Library DLLs have been also added to the

C:\TestShell folder for easier access when the

libraries are added as assets to a Driver Builder

project.

Traffic stencil Updated traffic

libraries

The current traffic libraries have been updated to

versions:

IxLoad 1.0.0.9

IxNetwork 1.0.0.11

TestCenter 1.0.0.17


TestShell 5.0 65

File management


Administration File recoverability SP1 adds file recoverability to TestShell files.

Deleted TestShell files are now saved to the

Recycle bin.

To restore deleted files, open the Recycle bin,

select the files to recover and click Restore.


TestShell 5.0 66

Runtime

Runner API


Runner Added argument to

the command line

QSTestLauncher.

When launching a test through the

QSTestLauncher, an additional report template

argument is available which allows selection of

the report template to be used automatically with

the end of the execution.

Resolved issues

Lab management

Reservations


Reservations Resolved an issue where a driver command could not be added

to a reservation start event if the resource was added to the

reservation via a topology.

Reservations Resolved an issue where a resource driver added to a

reservation via a topology could not be selected to launch on

reservation start.

Reservations Resolved an issue where creating a route in a reservation after

the reservation ended would not display an error.

Reservations Notifications for reservation end are now sent on actual end time

even if a reservation is terminated manually.



Routes Improved error reporting for failed routes.

Routes Resolved an issue where route configuration constraints were

not applied unless the user explicitly opened the route


TestShell 5.0 67


configuration dialog.

Routes Fixed a bug where deleting a reservation with routes would not

remove all the routes from the database, and added a migration

to remove leftover data.

Reservations, Routes Fixed an issue that made it possible to deactivate another

reservation’s shared route if only one L1 switch was used.

Routes Fixed an issue with route configuration for ONPATH, where the

configured high end speed would not set correctly on activation.

Routes Fixed a bug where the speeds of route ports would not intersect

correctly, causing more options to appear in the route

configuration document than were possible.

Resources


Resource search Fixed an Oracle bug in a specific scenario where a query would

have too many parameters.

Resource configuration Fixed the Sync to device operation order for ONPATH L1

switches.

Topologies


Topologies Duplicate clear mappings commands were sent to the driver in

some topology activation scenarios in cases where a conflicting

topology had to be cleared, causing errors in some L1 switches.

Test automation

Performance



TestShell 5.0 68


Studio Under certain circumstances rendering of Studio GUI would

become sluggish; this has now been fixed.

Runtime


TestShell Runner API Launching tests through Runner API with exe\bat works correctly

also when Runner is not opened.

Tools and libraries

GUI automation


Ranorex Ranorex installation issues were resolved in Ranorex 3.3.1

MatShell


Expressions “concat” expressions are operating as expected (concatvectors,

concatvertical, concathorizontal)

TestShell 4.8 Patch 1 New features

TestShell 5.0 69

TestShell 4.8 Patch 1

New features

Lab management

L1 drivers


Drivers ONPATH blade

interoperability

Added ONPATH blade interoperability

constraints.


Lab management

L1 drivers


Drivers MRV and ONPATH

drivers

Added updated drivers for MRV and ONPATH.

Test automation

Tools and libraries


GUI

automation

Ranorex support This patch adds support for Ranorex version

3.3.1

TestShell 4.8 Patch 1 Resolved issues

TestShell 5.0 70

Resolved issues

Lab management

Reservations


Reservations Resolved an issue where a driver command could not be added

to a reservation start event if the resource was added to the

reservation via a topology.

Reservations Resolved an issue where creating a route in a reservation after

the reservation ended would not display an error.

Topologies


Topologies Duplicate clear mappings commands were sent to the driver in

some topology activation scenarios in cases where a conflicting

topology had to be cleared, causing errors in some L1 switches.

Resources


Resource search Fixed an Oracle bug in a specific scenario where a query would

have too many parameters.

Resource configuration Fixed the Sync to device operation order for ONPATH L1

switches.



Routes Fixed a bug where deleting a reservation with routes would not

remove all the routes from the database, and added a migration

to remove leftover data.

TestShell 4.8 Patch 1 Resolved issues

TestShell 5.0 71


Routes Fixed an issue that made it possible to deactivate another

reservation’s shared route if only one L1 switch was used.

Routes Route configuration for ONPATH high end speed would not

correctly set the speed on activation.

Routes Fixed a bug where the speeds of route ports would not intersect

correctly, causing more options to appear in the route

configuration document than were possible.

Routes Resolved an issue where route configuration constraints were

not applied unless the user explicitly opened the route

configuration dialog.


TestShell 5.0 72

TestShell 4.8

New features

Business intelligence

Insight tools

Module/Title Description Solution

InSight Tools

– Installation

Mixing different database

type (SQL/Oracle) for the

TestShell and Queries

databases is not supported.

Use the same database type

(SQL/Oracle) for TestShell and InSight

Queries Databases.

InSight Tools

– reports

Saving a report without a

data source can damage the

database and prevent

generation of future reports.

Do not save a report without a data

source.

InSight Tools

– Query

based on

Wizard

When executing a query that

was created using the Query

Wizard, only the top 200

results are displayed in the

preview while in InSight

Portal there is no such

limitation. If the total number

of results exceeds 50,000

rows an error will be

displayed.

InSight Portal When opening the InSight

Portal. a blank page or a

”HTTP 500 – Internal Server

Error” may appear.

Please follow the Microsoft solution:



TestShell 5.0 73

Lab management

TestShell API


TestShell API TestShell API library The former Resource Manager library has been

revamped into a robust management API.

Renamed to reflect its broader cross application

capabilities, the new TestShell API library

includes dozens of new methods that includes

user & group management, additional

activation/scheduling capabilities, attribute

management and new reporting options.

Licensing

Module Feature /

Update

Description

License Procedure

Editor

The TestShell Studio and the TestShell Driver Builder

licenses now use the same license pool.


TestShell 5.0 74

Test automation

Tools and libraries


Interface Page Publishing variables A new interface was added to allow better

control of the published variables. It also allows

ordering the variables.

The former Interface dialog is now a full page

designed to allow better control of published

variables.

Incorporating some of the functionality that used

to be available from the Variable Properties

window, the new Interface page supports drag &

drop publishing of variables, automatic

publishing of new variables and enhanced

display options such as grouping and display

order.

SNMP SNMP Manager

library

The SNMP module has been replaced with a

new SNMP Manager library.

SNMP Trap Grab SNMP Traps The old SNMP Trap module has been replaced

with two separate tools:

The SNMP trap listener includes a filter wizard,

simplifying the process of determining which

traps to accept.

The Grab SNMP Traps tool provides enhanced

display and grouping options.

TestShell 4.8 Resolved issues

TestShell 5.0 75

Resolved issues

Business intelligence

Insight tools

Module Feature /

Update

Description

Insight Tools Reservations

table

A new table was added to allow building queries from

reservation information.

The query tool also includes pre-set queries.

Test automation

Tools and libraries


Explorers Fixed an error that occurred when copying and pasting a folder

of tests that took up more than 500 MB of storage space.

I/O Panel An issue that prevented the Description field from being

displayed under specific circumstances has been resolved.

PowerShell Under rare and specific circumstances, a stack overflow could

develop when working on Command Shell tool using PowerShell.

This is now fixed.

Studio Under certain circumstances, Studio would not open if libraries

had not been migrated to the current TestShell version. This has

now been fixed.

Transformations Xml – Select Node transformation no longer displays a warning

for XML sources when the doctype references an external DTD.

qualisystems testshell - cloud automation and digital … · 2020-05-22 · test automation ......

Documents