spirent cloudsure automation guide

v1.10.0

Spirent Communications

Spirent CloudSure Automation Guide


2

Table of Contents Getting Started ............................................................................................................ 3

1. AION Authentication ........................................................................................... 3

2. Find a solution ...................................................................................................... 5

3. Create a project .................................................................................................... 7

4. Add test cases to the project ................................................................................. 9

5. Configure test cases............................................................................................ 10

6. Run the project ................................................................................................... 15

7. Analyze results ................................................................................................... 21

8. Clean up results .................................................................................................. 22

Import a test package ................................................................................................ 23

API reference ............................................................................................................ 24

Example scripts ......................................................................................................... 24

Spirent Support ......................................................................................................... 25


3

Getting Started

Spirent CloudSure is a test solution framework for benchmarking and validating NFV

ecosystems, ultimately assuring your cloud investment. This manual will guide you through the

steps of finding the right solution and executing tests using the CloudSure REST API.

1. AION Authentication

The AION API supports obtaining an authorization token via OAuth 2.0. The endpoint to obtain

the bearer token requires a scope parameter which is the user's organization ID.

The organization ID can be obtained as follows:

Request:

curl -k -X GET \

https://my-aion-cluster/api/iam/organizations/default

Python example:

import requests

my_aion_cluster_url = "https://my-aion-cluster"

r = requests.get(f"{my_aion_cluster_url}/api/iam/organizations/default")

organization_id = r.json()["id"]

Response (example):

[

{

"id": "7fab9b0208aa456b92b7a88db774649d",

"serial": 2,

"name": "Spirent",

"domains": [

"spirent.com"

],

"subdomain": "spirent",

"locked": false,

"email_enabled": true,

"self_onboard": true,

"default_workspace": {

"id": "7fab9b0208aa456b92b7a88db774649d",

"name": "Spirent",

"description": "Organization's initial project"

}

}

]


4

The “id” property is used for the scope parameter in the following API call to obtain a bearer

token.

Request:

curl -k -X POST \

-H "Accept:application/json"

--data-urlencode "grant_type=password" \

--data-urlencode "username=my_email_addr" \

--data-urlencode "password=my_password" \

--data-urlencode "scope=my_org_id" \

https://my-aion-cluster/api/iam/oauth2/token

Python example:

import requests

my_aion_cluster_url = "https://my-aion-cluster"

username = "my_email_addr"

password = "my_password"

organization_id = "my_org_id"

r = requests.post(

url=f"{my_aion_cluster_url}/api/iam/oauth2/token",

json={

"grant_type": "password",

"username": username,

"password": password,

"scope": organization_id,

},

)

access_token = r.json()["access_token"]

Response:

{

"access_token":

"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsIng1dCI6IllUWmpPR0prWTJGak16TmpZVE01WlRoa

VlXWmlOR1V5Wm1ZME1qWXpPRFpsT0RkbU1XWTVOV1ZpTjJGbFlXWTBPVGd3TlRkaE56TTFOREUwTUR

Gak5nPT0ifQ.eyJjb25zIjoidXJuOmM0OTYzOGE2MjUzNzRiOTVhMjU0ZDQyZGRhMDZhZDBiOndvcm

tzcGFjZTpjNDk2MzhhNjI1Mzc0Yjk1YTI1NGQ0MmRkYTA2YWQwYiIsImV4cCI6MTQ5Njk2OTQ5Mywi

aXNzIjoidXJuOnJvb3Q6c2VydmljZTpvcmlvbi1pYW0iLCJuYmYiOjE0OTY4ODI3OTMsInN1YiI6In

VybjpjNDk2MzhhNjI1Mzc0Yjk1YTI1NGQ0MmRkYTA2YWQwYjp1c2VyOjg5ZmI2ZDZiMDM3ZTQ2NzJh

NWZjNmJiYzUxN2MwMTY1IiwidHlwIjoiYWNjZXNzIn0.qtrwaExQAauw5cWTfY40dU6o5TlhGQnY18

2v0Ji4Ll4SW6AL5WTMR93te82iEYWNuh6SR2E5ADYj7kfa4n7mhlFNuMyv1_Jf3SF5raFhGrCzcxdX

9V9E17-ZWEaNGeb1yU2r-b-

stqFqvdOQI4rgzohIzjVvCnvU5TT5fLxaNrNzSXN2qILmcI2hBc41NS1kDEHDGtTslIUcBmmnHh7v0

FpIz5THc8d94XH2omFs784kls2JIqOhoqe2UUi_WVVbVKZGa01W9oNpkfFZJZT-

qVt2MrKExMxOW1X23GhW705ZpoYbXnXbBfya4U145rjk4wjUaWMj9zhAygXbg6MEtw",

"token_type": "urn:ietf:params:oauth:token-type:access_token",

"expires_in": 86400,

"refresh_token":




tzcGFjZTpjNDk2MzhhNjI1Mzc0Yjk1YTI1NGQ0MmRkYTA2YWQwYiIsImV4cCI6MTQ5NzQ4Nzg5Mywi




5

NWZjNmJiYzUxN2MwMTY1IiwidHlwIjoicmVmcmVzaCJ9.MnFQxEO3d0sP0UL5yjX-3Gn-

NBs97mUQ5ZNE-sQ-TrmXnEQS59Ipm6nlKW709IHzy1K2-

d9LaWSvOixfcTbHsnCUQ2NeF0flJqnPCich5gRM4H8Gse9x4smaYRR6BI-

P7C21OlxjnrJEIh8ZlRXa3c7xUvAZEbh9ivzoaK8y9umupO6e9hkZzflUt_sukRQnvjRav4WSTAuws

qw8LeJL7A8b0OEWOFfneg5vjhbtE-

zJdYHPiDoMnDG0U_l_lzLL9EUjJgdohz4rMPBQZkH163vIihET2iksSsiYbmKvYz_Nhe80q-

f9W1151CHB8f_TUtgfWgU7ZIuq9U9-3M2upg",

"scope": "7fab9b0208aa456b92b7a88db774649d"

}

The access token will be used in an Authorization header in subsequent calls to the API.

The following is an example of the Authorization header value passed to the “-H” cURL option

using the access_token value from the response above. Note the inclusion of “Bearer” before the

token value in the header content. “Authorization: Bearer

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsIng1dCI6IllUWmpPR0prWTJGak16TmpZVE01WlRoaV

lXWmlOR1V5Wm1ZME1qWXpPRFpsT0RkbU1XWTVOV1ZpTjJGbFlXWTBPVGd3TlRkaE56TTFOREUwTURG

ak5nPT0ifQ.eyJjb25zIjoidXJuOmM0OTYzOGE2MjUzNzRiOTVhMjU0ZDQyZGRhMDZhZDBiOndvcmt

zcGFjZTpjNDk2MzhhNjI1Mzc0Yjk1YTI1NGQ0MmRkYTA2YWQwYiIsImV4cCI6MTQ5Njk2OTQ5Mywia

XNzIjoidXJuOnJvb3Q6c2VydmljZTpvcmlvbi1pYW0iLCJuYmYiOjE0OTY4ODI3OTMsInN1YiI6InV

ybjpjNDk2MzhhNjI1Mzc0Yjk1YTI1NGQ0MmRkYTA2YWQwYjp1c2VyOjg5ZmI2ZDZiMDM3ZTQ2NzJhN

WZjNmJiYzUxN2MwMTY1IiwidHlwIjoiYWNjZXNzIn0.qtrwaExQAauw5cWTfY40dU6o5TlhGQnY182

v0Ji4Ll4SW6AL5WTMR93te82iEYWNuh6SR2E5ADYj7kfa4n7mhlFNuMyv1_Jf3SF5raFhGrCzcxdX9

V9E17-ZWEaNGeb1yU2r-b-



qVt2MrKExMxOW1X23GhW705ZpoYbXnXbBfya4U145rjk4wjUaWMj9zhAygXbg6MEtw”

For python examples it should be assigned to variable without any prefix: access_token =




tzcGFjZTpjNDk2MzhhNjI1Mzc0Yjk1YTI1NGQ0MmRkYTA2YWQwYiIsImV4cCI6MTQ5Njk2OTQ5Mywi



NWZjNmJiYzUxN2MwMTY1IiwidHlwIjoiYWNjZXNzIn0.qtrwaExQAauw5cWTfY40dU6o5TlhGQnY18

2v0Ji4Ll4SW6AL5WTMR93te82iEYWNuh6SR2E5ADYj7kfa4n7mhlFNuMyv1_Jf3SF5raFhGrCzcxdX

9V9E17-ZWEaNGeb1yU2r-b-



qVt2MrKExMxOW1X23GhW705ZpoYbXnXbBfya4U145rjk4wjUaWMj9zhAygXbg6MEtw"

2. Find a solution

CloudSure provides a variety of solutions that are aimed at benchmarking and validating various

components of the NFV ecosystem. Solutions are delivered in the form of test packages where

each test package focuses on an NFV component (e.g., NFVI or MANO). Each test package

consists of one or more test case templates that are meant to benchmark and validate specific

aspects of that NFV component (e.g., Cross NUMA network bandwidth and latency).


6

Before you can start using the REST API, you will need to obtain an access token. CloudSure

authenticates API requests using OAuth 2.0. Refer to AION Authentication for information about

how to obtain an authorization token.

To list test packages

Request: curl -X GET \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/test-packages \

-H 'authorization: Bearer <YOUR_ACCESS_TOKEN>' \

-H 'content-type: application/json' \

-H 'x-spirent-metadata-only: 1'

Python example using CloudSure REST API client:

from cloudsure import TestPackagesApi

from cloudsure import configuration

cloudsure_app_vm_ip = "YOUR_CLOUDSURE_APP_VM_IP"

configuration.host = f"http://{cloudsure_app_vm_ip}/api"

configuration.access_token = "YOUR_ACCESS_TOKEN"

projects_api = TestPackagesApi()

test_packages = projects_api.get_test_package_list()

Python example using requests library:

import requests


access_token = "YOUR_ACCESS_TOKEN"

headers = {

"Authorization": f"Bearer {access_token}",

"content-type": "application/json",

"x-spirent-metadata-only": "1",

}

r = requests.get(f"http://{cloudsure_app_vm_ip}/api/test-packages",

headers=headers)

test_packages = r.json()

Response:

[

{

"id": "spirent_tst001ch6",

"name": "TST 001 Chapter 6 NFVi Benchmark",

"description": "A set of test cases derived from TST 001 Chapter 6 to

benchmark the NFV infrastructure.",

"author": "Spirent Communications",

"version": "1.0.0",

"license_ids": [

"TST001 CH6 NFVI"

],

"testcase_templates": [

{

"id": "spirent_tst001ch6.NFVI_BENCH_NTW_CROSS_NUMA",


7

"name": "Cross NUMA Bandwidth and Latency Characterization",

"description": "Characterize network data plane frame loss and latency

for a single compute host for network traffic that travels between network

ports, connected to same vSwitch, on same host, but on different NUMA nodes.

All traffic streams within the test travel VM -> vSwitch -> QPI -> vSwitch >

VM. No external network interfaces, internal routers, or firewalls are used

(unless integrated directly into the vSwitch).",

"dependencies": [

{

"type": "cloudsure",

"name": "Spirent CloudSure Framework",

"min_required_version": "1.0.0"

},

{

"type": "metrics_agent",

"name": "Spirent Metrics Agent",

"min_required_version": "197"

},

{

"type": "stc",

"name": "Spirent TestCenter",

"required_version": "4.97.7356"

}

],

"test_package_id": "spirent_tst001ch6",

"image_filename": "cross_numa_test_case.png",

"help_file_path": "doc/description.html"

},

{...}

]

}

]

If you are interested in Cross NUMA bandwidth and latency characterization, take note of the test

package ID (e.g., “spirent_tst001ch6”) and the test case template ID (e.g.,

“spirent_tst001ch6.NFVI_BENCH_NTW_CROSS_NUMA”). You will need this information

later.

3. Create a project

When that you have identified a test case template that interests you, projects are a great way to

organize sets of tests together and execute them consecutively.

To create a project

Request:

curl -X POST \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/projects \


-H 'accept: application/json' \


-d '{

"name": "My first project",

"description": "Collection of NFVI benchmark tests"

}'


8

Python example using CloudSure REST API client::

from cloudsure import ProjectsApi





projects_api = ProjectsApi()

project = projects_api.create_project(

{

"name": "cloudsurelib project",

"description": "Collection of NFVI benchmark tests",

}

)


import requests



headers = {



"accept": "application/json",

}

data = {


"description": "Collection of NFVI benchmark tests",

}

r = requests.post(url=f"http://{cloudsure_app_vm_ip}/api/projects", json=data,

headers=headers)

json_response = r.json()

Response:

{

"id": "d5d34842f2194fb09c854f188fe72f9d",


"created_at": "2019-03-18T18:18:53.902543156Z",

"updated_at": "2019-03-18T18:18:53.902543156Z",

"created_by": "c791989bd6ef472eb53851b584b37fa0",

"updated_by": "c791989bd6ef472eb53851b584b37fa0",

"description": "Collection of NFVI benchmark tests"

}

Take note of the project ID. You will need it to add test cases to your project.


9

4. Add test cases to the project

The newly created project is currently empty. Now you can add a test case to the project. Recall

the project ID you just recorded and the test case template ID you noted in step 2, because you

will need to provide those IDs when creating the test cases under your project.

To create a test case under your project

Request:

curl -X POST \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/testcases \




-d '{

"name": "My Cross NUMA test",

"description": "Characterize network bandwidth and latency when traffic

crosses between VMs on different NUMA nodes",

"testcase_template_id": "spirent_tst001ch6.NFVI_BENCH_NTW_CROSS_NUMA",

"project_id": "d5d34842f2194fb09c854f188fe72f9d"

}'


from cloudsure import TestcasesApi





testcases_api = TestcasesApi()

testcase = testcases_api.create_testcase(

{


"description": "Characterize network bandwidth and latency when

traffic crosses between VMs on different NUMA nodes",


"project_id": "d5d34842f2194fb09c854f188fe72f9d",

}

)


import requests



headers = {




}

data = {



10





}

r = requests.post(url=f"http://{cloudsure_app_vm_ip}/api/testcases",

json=data, headers=headers)


Response:

{

"id": "b53f4c169f244392bd826aa6af66d3e3",


"created_at": "2019-03-18T18:28:28.160026522Z",

"updated_at": "2019-03-18T18:28:28.160026522Z",



"run_count": 0,






"project_name": "My first project",

"version": "1.0.0",

"enabled": true,

"input_valid": false

}

Take note of the test case ID. You will need it to configure your test case. You may have noticed

that this test case has invalid input (“input_valid”: false). That is because no input values were

specified when you created the test case. That is the next step.

5. Configure test cases

Test cases are based on a test case template. A test case template contains the test code that will

be executed, and it exposes a set of inputs that you, the user, will provide. Each test case template

provides a JSON schema that specifies the format and structure of the test case input data that it

expects. You will need the test case template ID to retrieve the test case input schema.

To get the test case input schema for a test case template

Request:

curl -X GET \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/testcase-

templates/spirent_tst001ch6.NFVI_BENCH_NTW_CROSS_NUMA \


-H 'content-type: application/json'


from cloudsure import TestcaseTemplatesApi




11



testcase_template_api = TestcaseTemplatesApi()

testcase_template =

testcase_template_api.get_testcase_template("spirent_tst001ch6.NFVI_BENCH_NTW_

CROSS_NUMA")


import requests



headers = {



}

r = requests.get(

url=f"http://{cloudsure_app_vm_ip}/api/testcase-

templates/spirent_tst001ch6.NFVI_BENCH_NTW_CROSS_NUMA",

headers=headers,

)


Response (abbreviated):

{

"id": "spirent_tst001ch6.NFVI_BENCH_NTW_CROSS_NUMA",

"name": "Cross NUMA Bandwidth and Latency Characterization",

"description": "Characterize network data plane frame loss and latency for a

single compute host for network traffic that travels between network ports,

connected to same vSwitch, on same host, but on different NUMA nodes. All

traffic streams within the test travel VM -> vSwitch -> QPI -> vSwitch > VM.

No external network interfaces, internal routers, or firewalls are used

(unless integrated directly into the vSwitch).",

"dependencies": […],


"image_filename": "cross_numa_test_case.png",

"help_file_path": "doc/description.html",

"input_schema": {

"additionalProperties": false,

"properties": {

"addressing_method": {…},

"debug": {…},

"ignore_existing_vms_my_project": {…},

"ignore_existing_vms_non_vim": {…},

"ignore_existing_vms_other_project": {…},

"instantiation_type": {…},

"ip_version": {…},

"l4_protocol": {…},

"stcv_port_config": {…},

"test_network_count": {…},

"traffic_test": {…},

"vim": {…},

"vms_per_host": {…}


12

},

"required": [

"vim",

"vms_per_host",

"test_network_count",

"stcv_port_config",

"traffic_test"

],

"title": "Schema for NFVI_BENCH_NTW_CROSS_NUMA test input",

"type": "object"

}

}

The test case input schema can be quite large, so it may help to use an online JSON schema

validation tool (e.g., https://www.jsonschemavalidator.net). With this type of online tool, you can

copy and paste the input schema and then start creating the JSON test input data that conforms to

the schema.

Pro tip: Copy and paste the default JSON test case input data from the CloudSure UI code view

to use as a starting point.

When you have your test case input JSON data ready, you can update the test case with it. You

will need the test case ID you noted at the end of step 4.

To configure a test case with test case input

Request:

curl -X PUT \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/testcases/b53f4c169f244392bd826aa6af66d3

e3 \



-d '{

"id": "b53f4c169f244392bd826aa6af66d3e3",







"enabled": true,

"input": {

"addressing_method": "DHCP",

"instantiation_type": "VIM",

"ip_version": "IPV4",

"l4_protocol": "UDP",

"stcv_port_config": {

"speed": "10G"

},

"test_network_count": 2,

"traffic_test": {...},

"vim": {...},

"vms_per_host": 2

}

https://www.jsonschemavalidator.net/


13

}'


from cloudsure import TestcasesApi





testcase_template_api = TestcasesApi()

testcase_template = testcase_template_api.update_testcase(

testcase_id="b53f4c169f244392bd826aa6af66d3e3",

testcase_body={

"id": "b53f4c169f244392bd826aa6af66d3e3",


"description": "Characterize network bandwidth and latency when

traffic crosses between VMs on different NUMA nodes",




"enabled": True,

"input": {





"stcv_port_config": {"speed": "10G"},


"traffic_test": {},

"vim": {},

"vms_per_host": 2,

},

},

)


import requests



headers = {



}

data = {

"id": "b53f4c169f244392bd826aa6af66d3e3",







"enabled": True,


14

"input": {





"stcv_port_config": {"speed": "10G"},



"vim": {...},

"vms_per_host": 2,

},

}

r = requests.put(

url=f"http://{cloudsure_app_vm_ip}/api/testcases/b53f4c169f244392bd826aa6af66d

3e3",

json=data,

headers=headers,

)


Response (abbreviated):

{

"id": "b53f4c169f244392bd826aa6af66d3e3",


"created_at": "2019-03-18T18:28:28.160026522Z",

"updated_at": "2019-03-18T19:37:42.879959037Z",



"run_count": 0,






"project_name": "My first project",

"version": "1.0.0",

"enabled": true,

"input_valid": true,

"input": {





"stcv_port_config": {

"speed": "10G"

},



"vim": {...},

"vms_per_host": 2

}

}

Now you can see that the input is valid (“input_valid”: true).


15

6. Run the project

To recap, at this point you have a project with one test case, and it has valid input data. Recall the

project ID from step 3 and run your project!

To execute the project

Request:

curl -X POST -i \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/projects/d5d34842f2194fb09c854f188fe72f9

d/start \





from cloudsure import ProjectsApi





projects_api = ProjectsApi()

project_execution =

projects_api.start_project("d5d34842f2194fb09c854f188fe72f9d")


import requests



headers = {




}

r = requests.post(

url=f"http://{cloudsure_app_vm_ip}/api/projects/d5d34842f2194fb09c854f188fe72f

9d/start",

headers=headers,

)


Response:

HTTP/1.1 202 Accepted

Server: nginx/1.10.3

Date: Mon, 18 Mar 2019 20:03:42 GMT


16

Content-Type: application/json

Content-Length: 0

Connection: keep-alive

Location: {cloudsure-

_app_vm_ip}/api/executions/4e8fec3b969a4e2db83f36103d6761d0

Set-Cookie: csu-

session=MTU1MjkzOTQyMnxEdi1CQkFFQ180SUFBUkFCRUFBQVBQLUNBQUVHYzNSeWFXNW5EQVFBQW

1sa0JuTjBjbWx1Wnd3aUFDQXdNVEU1TlRVd1kySXpZVEUwTlROaU9EWTJaRFk1TURWa01HRmhaak0z

TVE9PXxJksQKYYMmrmiims52tfHO6FfJaMdkOcHaJmLspRHZtw==; Path=/; Expires=Wed, 17

Apr 2019 20:03:42 GMT; Max-Age=2592000

X-Request-Id: 6435118234451399413

X-Spirent-Api-Version: 1

{

"id": "4e8fec3b969a4e2db83f36103d6761d0",

"name": "Execution of project My first project",

"created_at": "2019-03-18T22:47:33.369289287Z",

"updated_at": "2019-03-18T22:52:18.793261179Z",



"app_inst_id": "d708d797871c3f6a4d873c3dd4f5c60d",


"status": "RUNNING",

"step_mode": false,

"executions": [

{

"id": "499c4f79244349a8bfb021b748ebf55a",

"name": "Execution of testcase My Cross NUMA test",

"created_at": "2019-03-18T22:47:33.666677321Z",

"updated_at": "2019-03-18T22:52:17.958762022Z",



"execution_id": "4e8fec3b969a4e2db83f36103d6761d0",

"testcase_id": "b53f4c169f244392bd826aa6af66d3e3",




"start_time": "2019-03-18T22:47:34.082431449Z",

"current_step": {

"name": "Run the Traffic Test",


"description": "Run the benchmark traffic test.",

"index": 10,

"total": 13

},

"live_results": {

"report_id": "http://{cloudsure_app_vm_ip}/orion-

res/results/vicrncrbivetvcfk?profileId=3c21428de59c4298a748a330d6f582c4"

}

}

]

}

Since executions are typically long-running operations, the REST API treats it as an

asynchronous operation. Notice that the “-i” option was specified in the cURL command to

include the response headers in the output. This is important because the Location header contains


17

the URL to track the project execution status. All you have to do now is poll the execution and

wait until its status is “DONE”.

To track execution status

Request:

curl -X GET \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/executions/4e8fec3b969a4e2db83f36103d676

1d0\




from cloudsure import ExecutionsApi





executions_api = ExecutionsApi()

execution = executions_api.get_execution("4e8fec3b969a4e2db83f36103d6761d0")


import requests



headers = {



}

r = requests.get(

url=f"http://{cloudsure_app_vm_ip}/api/executions/4e8fec3b969a4e2db83f36103d67

61d0",

headers=headers,

)


Response:

{

"id": "4e8fec3b969a4e2db83f36103d6761d0",


"created_at": "2019-03-18T22:47:33.369289287Z",

"updated_at": "2019-03-18T22:52:18.793261179Z",






"step_mode": false,


18

"executions": [

{

"id": "499c4f79244349a8bfb021b748ebf55a",


"created_at": "2019-03-18T22:47:33.666677321Z",

"updated_at": "2019-03-18T22:52:17.958762022Z",








"start_time": "2019-03-18T22:47:34.082431449Z",

"current_step": {




"index": 10,

"total": 13

},

"live_results": {



}

}

]

}

Each enabled test case under the project will be executed sequentially, and the execution

information for each test case is contained in the “executions” list of the execution resource.

Note that specific test cases can also be executed. Refer to API reference for more information.

This is what a completed execution looks like:

{

"id": "4e8fec3b969a4e2db83f36103d6761d0",


"created_at": "2019-03-18T22:47:33.369289287Z",

"updated_at": "2019-03-18T22:56:22.550779052Z",





"status": "DONE",

"step_mode": false,

"executions": [

{

"id": "499c4f79244349a8bfb021b748ebf55a",


"created_at": "2019-03-18T22:47:33.666677321Z",

"updated_at": "2019-03-18T22:56:22.074714071Z",






19



"status": "PASS",

"status_message": "No errors",

"results_url": "http://{cloudsure_app_vm_ip}/orion-

res/results/vicrncrbivetvcfk?profileId=3c21428de59c4298a748a330d6f582c4",

"start_time": "2019-03-18T22:47:34.082431449Z",

"end_time": "2019-03-18T22:56:22.074212401Z",

"testcase_results": {

"error_info": "No errors",

"events": {

"instantiate_network": {

"details": "Success",

"duration": 1.9866480827331543

},

"instantiate_stcv": {

"details": "Success",

"duration": 125.31659317016602

},

"terminate_network": {

"details": "N/A"

},

"terminate_stcv": {

"details": "N/A"

}

},

"overall_status": "PASSED",


res/results/vicrncrbivetvcfk?profileId=3c21428de59c4298a748a330d6f582c4",

"resources": {},

"scaling_down": {},

"scaling_from_level": {},

"scaling_in": {},

"scaling_out": {},

"scaling_to_level": {},

"scaling_up": {},

"steps": {

"1": {

"description": "Configures parameters for the NFVI Network

Benchmark Test.",

"duration": 2.327486038208008,

"name": "Configure NFVI Network Benchmark Test",

"status": "PASS"

},

"2": {

"description": "Collect basic information about the compute

host.",

"duration": 0.6601090431213379,

"name": "Collect Compute Host Metadata",

"status": "PASS"

},

"3": {

"description": "Create the test network(s) and subnet(s)",

"duration": 2.348832130432129,

"name": "Create Test Networks and Subnets",

"status": "PASS"

},

"4": {


20

"description": "Instantiates the STCv VMs",

"duration": 125.31698298454285,

"name": "Instantiate the STCv VMs",

"status": "PASS"

},

"5": {

"description": "Validate STCv VM instantiation state is

INSTANTIATED and state is STARTED",

"duration": 42.84990882873535,

"name": "Validate STCv VM is instantiated",

"status": "PASS"

},

"6": {

"description": "Collect meta data for the STCv VMs for use in the

test.",

"duration": 1.04921293258667,

"name": "Collect STCv VM Meta Data",

"status": "PASS"

},

"7": {

"description": "Use the meta data to select VMs to use in the test

based on the number of desired VMs and the distribution of VMs across NUMA

nodes.",

"duration": 24.980865955352783,

"name": "Select STCv VMs For Use in the Test",

"status": "PASS"

},

"8": {

"description": "Connect to the chassis and reserve ports.",

"duration": 41.37377381324768,

"name": "Connect to Chassis/Reserve Ports",

"status": "PASS"

},

"9": {

"description": "Configure traffic between the ports.",

"duration": 5.6873040199279785,

"name": "Configure Traffic",

"status": "PASS"

},

"10": {


"duration": 211.99264907836914,


"status": "PASS"

},

"11": {

"description": "Derive summary statistics, charts, and generate

results report.",

"duration": 0.00014495849609375,

"name": "Analyze Results",

"status": "PASS"

},

"12": {

"description": "Disconnect from the STC chassis to release the

ports.",

"duration": 18.84112000465393,

"name": "Disconnect From STC Chassis",

"status": "PASS"


21

},

"13": {

"description": "Delete instance and networks created for the

test.",

"duration": 17.081406116485596,

"name": "Clean up the VIM",

"status": "PASS"

}

},

"tc_duration": "0:08:43.664355",

"tc_end_time": "2019-03-18T22:56:17.738Z",

"tc_start_time": "2019-03-18T22:47:34.074Z",

"timestamps": {

"instantiate_network.END": 1552949264.202904,

"instantiate_network.START": 1552949262.216256,

"instantiate_stcv.END": 1552949391.550425,

"instantiate_stcv.START": 1552949266.233832

}

}

}

]

}

7. Analyze results

Review the test results through Spirent TestCenter IQ. From there, you can see summary tables

and time-series charts that help you characterize the components of your NFV ecosystem. Each

test case template comes with a unique set of results that are useful for identifying bottlenecks and

potential root causes of performance problems.

The Spirent TestCenter IQ result URL is contained within the execution both while a test is

running and after the test completes.

Live results URL

"live_results": {

"report_id": "http://{YOUR_CLOUDSURE_APP_VM_IP}/orion-


}

End of test results URL

"results_url": "http://{YOUR_CLOUDSURE_APP_VM_IP}/orion-

re/results/vicrncrbivetvcfk?profileId=3c21428de59c4298a748a330d6f582c4",

Refer to the Spirent TestCenter IQ API for more information about how to query result data.


22

8. Clean up results

You can permanently remove test results, if necessary. Whether you want to free up storage space

or remove invalid result runs, you can delete test case executions along with their associated

diagnostic logs and results database.

To delete a test case execution

Request:

curl -X DELETE \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/testcase-executions/

a223a37a17c64fffa3d6677526a6cade \





from cloudsure import ExecutionsApi





executions_api = ExecutionsApi()

execution =

executions_api.delete_testcase_execution("a223a37a17c64fffa3d6677526a6cade")


import requests



headers = {




}

r = requests.delete(

url=f"http://{cloudsure_app_vm_ip}/api/testcase-

executions/a223a37a17c64fffa3d6677526a6cade",

headers=headers,

)

Response:

HTTP/1.1 204 No Content


23

Import a test package

CloudSure includes a set of test packages authored by Spirent. If you choose to create your own

custom test package, you can import the test package into an existing CloudSure application

instance provided you purchased a Developer license. You can only see and run your custom test

package on that one specific application instance. Test package authoring is not within the scope

of this guide.

The test package that is being imported must be a compressed archive file (.tgz) and the test

package folder must be at the top-level of the archive.

To create a test package archive $ tar -zcvf my_new_pkg.tgz path_to_my_new_pkg/

A bearer token is required to import a test package. Refer to AION Authentication for information

about how to obtain an authorization token.

To import a test package

Request:

curl -X POST \

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/test-packages/x/import \


-H 'content-type: application/octet-stream' \

--data-binary @my_new_pkg.tgz


import requests



headers = {


"content-type": "application/octet-stream",

}

with open('my_new_pkg.tgz', 'rb') as f:

data = f.read()

r = requests.post(

url=f"http://{cloudsure_app_vm_ip}/api/test-packages/x/import",

data=data,

headers=headers,

)


Response:

{

"id": "my_example",

"name": "My Example Test Package",

"description": "Set of example test cases to demonstrate test package

import.",


24

"author": "Spirent",

"version": "0.1.0",

"license_ids": [

"Examples"

],

"testcase_templates": [

{

"id": "my_example.HELLO_WORLD",

"name": "Hello World",

"description": "A basic test package",

"dependencies": [

{

"type": "cloudsure",

"name": "Spirent CloudSure Framework",

"min_required_version": "1.0.0"

}

],

"test_package_id": "my_example"

}

]

}

API reference

The CloudSure application VM serves a Swagger-generated REST API document. Enter the

following URL in your browser:

http://{YOUR_CLOUDSURE_APP_VM_IP}/api/schema

Example scripts

The cloudsure-1.0.0-py2.py3-api-examples.tgz gzipped tarball includes Python code that

illustrates creating a project with one or more test cases and execution of projects or test cases

with the option to configure test input from a standalone JSON file.

To use the example scripts, download the CloudSure Python REST API Client Library from the

“Additional Downloads” drop-down menu under CloudSure in the AION downloads page under

Product Downloads.

To install the CloudSure Python client library: pip install cloudsure-1.0.0-py2.py3-none-any.whl

The cloudsure-1.0.0-py2.py3-api-examples file can be found in the same place as CloudSure

Python REST API Client Library.


25

Spirent Support

To obtain technical support for any Spirent Communications product, contact our Support

Services department using any of the following methods:

Americas

E-mail: [email protected]

Web: https://support.spirent.com

Toll Free: +1 800-SPIRENT (+1 800-774-7368) (North America)

Phone: +1 818-676-2616

Hours: Monday through Friday, 05:00 to 17:00 Pacific Time

Europe, Middle East, Africa



Phone: +33 (1) 6137 2270 (France)

Phone: +44 1803 546333 (UK)

Hours: Monday through Thursday, 09:00 to 18:00, 9:00 to 17:00 Friday, Paris Time

Asia Pacific



Phone: +86 (400) 810-9529 (toll-free mainland China only)

Phone: +86 (10) 8233 0033 (China)

Operating Hours: Monday through Friday, 09:00 to 18:00 Beijing Time

Company Address

Spirent Communications, Inc.

27349 Agoura Road

Calabasas, CA 91301

USA

The latest versions of user manuals, application notes, and software and firmware updates are available on the Spirent

Communications Customer Service Center website at https://support.spirent.com.

Information about Spirent Communications and its products and services can be found on the main company website at

https://www.spirent.com.

© 2022 All of the company names and/or brand names and/or product names referred to in this document, in particular, the name

“Spirent” and its logo device, are either registered trademarks or trademarks of Spirent plc and its subsidiaries, pending

registration in accordance with relevant national laws. All other registered trademarks or trademarks are the property of their

respective owners.

mailto:[email protected]

https://support.spirent.com/






https://www.spirent.com/

spirent cloudsure automation guide

Documents