Created by Jan Medvedwww.opendaylight.org

Getting Started with

Robot Framework

Carol Sanders, Brocade

Created by Jan Medvedwww.opendaylight.org

Robot Framework is a Keyword Driven automation framework using Python or Java as

it's backend.

Test cases are automated by writing steps using Robot framework keywords.

Provides a simple library API for creating customized test libraries.

Provides a command line interface and XML based outputs for integration into existing

build infrastructure (continuous integration systems).

Report files are created as a result of every pybot execution: report.html, log.html,

output.xml.

Implemented with Python

– Runs also on Jython (JVM) and IronPython (.NET)

– Can be extended natively using Python or Java

– Other languages supported via a remote interface

Open source

– Apache 2.0 license

– Active development and growing community

What is Robot Framework

2

Created by Jan Medvedwww.opendaylight.org

Test Suite Organization

3

Created by Jan Medvedwww.opendaylight.org

Integration CSIT Organization

4

git clone https://git.opendaylight.org/gerrit/integration cd- ./integration/test/

Two directoriesCSIT – contains the test suites, resource files containing keyword definitions particular to your

project tests, and testplans which are all needed to successfully run your test suite.

Tools – test scripts specifically written that test features

CSIT Directories (you will always put your files here)Libraries

Suites

Testplans

Variables

Created by Jan Medvedwww.opendaylight.org

Viewing Test Suites in Robot Framework Ride

5

Created by Jan Medvedwww.opendaylight.org

Collections of test cases are called test suites. Test suites are made of files and directories.

1. Now that you have cloned the integration repository, you have the integration test infrastructure needed to begin

creating your test suite.

2. Open ride on the command line – ride.py&

3. Open directory to Integration test suites directory -- ./integration/tests/csit/suites

4. Create a new directory. This where your test suite and associated files will be located. Use the name of your

project as the directory name.

5. Create a new test suite within the directory just created.

6. Include all elements described in Test Suite Contents: Settings, Keywords, Resource file(s), Variables,

Documentation, Tags, Log, Setup, Teardown

7. Follow best practices when writing test cases

Creating Robot Framework Tests Suites

6

Created by Jan Medvedwww.opendaylight.org

Integration Sample Test Suite

7

Created by Jan Medvedwww.opendaylight.org

Sample Test Case Format

8

*** Settings ***Documentation Test suite for RESTCONF inventory

Suite Setup Create Session session http://${CONTROLLER}:${RESTCONFPORT} auth=${AUTH}

headers=${HEADERS_XML}

Suite Teardown Delete All Sessions

Library Collections

Library ../../../libraries/RequestsLibrary.py

Library ../../../libraries/Common.py

Variables ../../../variables/Variables.py

Resource ../../../libraries/Utils.txt

*** Variables ***${REST_CONTEXT} /restconf/operational/opendaylight-inventory:nodes

@{node_list} openflow:1 openflow:2 openflow:3

*** Test Cases ***

Get list of nodes

[Documentation] Get the inventory

[Tags] restconf

Log ${start}

Wait Until Keyword Succeeds 30s 2s Ensure All Nodes Are In Response ${REST_CONTEXT} ${node_list}

Get nodeconnector for a node 1

[Documentation] Get the inventory for a node

[Tags] restconf

Log ${start}

${resp} Get session ${REST_CONTEXT}/node/openflow:1

Should Be Equal As Strings ${resp.status_code} 200

Should Contain ${resp.content} openflow:1:1

Should Contain ${resp.content} openflow:1:2

A test case file, for our purposes, contains the following default sections:

Settings, Variables, Test Cases, Setup and Teardown.Setup and Teardown usually defined in resources file or initialization file.

Created by Jan Medvedwww.opendaylight.org

Initialization File Format

9

*** Settings ***

Documentation Flow test suite for the OpenDaylight karaf-compatible feature set

Suite Setup Start Suite

Suite Teardown Stop Suite

Resource ../../../libraries/Utils.txt

*** Variables ***

*** Keywords ***

[Teardown]

This section is used to shutdown systems, close connections, and cleanup configurations.

Created by Jan Medvedwww.opendaylight.org

Test Suite Contents

10

Recommended Test Suite Contents Setup

Settings

Resource file

Keywords

Documentation

Log

Tags

Test case step(s)

Tear Down

Created by Jan Medvedwww.opendaylight.org

Test Suite Contents

11

A test case file can contain other configuration information to ease the processing and speed

results such as specifying variables, path to external keywords or python functions or other

configuration data.

When a test directory is executed, the files and directories it contains are processed recursively

as follows:

Files and directories with names starting with a dot (.) or an underscore (_) are ignored.

Directories with the name CVS are ignored (case-sensitive).

Files not having one of the recognized extensions (.html, .xhtml, .htm, .tsv, .txt, .rst, or .rest) are

ignored (case-insensitive).

Other files and directories are processed.

If a file or directory that is processed does not contain any test cases, it is silently ignored (a

message is written to the syslog) and the processing continues.

Created by Jan Medvedwww.opendaylight.org

12

Test execution is normally started using pybot, jybot or ipybot runner script.

These scripts are otherwise identical, but the first one executes tests using Python, the second using Jython, and the last one using IronPython.

Regardless of execution approach, the path (or paths) to the test data to be executed is given as an argument after the command.

Additionally, different command line options can be used to alter the test execution or generated outputs in some way.

Specifying test data to be executedRobot Framework test cases are created in files and directories, and they are executed by giving the path to the file or directory in question to the selected runner

script. The path can be absolute or, more commonly, relative to the directory where tests are executed from.

The given file or directory creates the top-level test suite, which gets its name, unless overridden with the --name option, from the file or directory name.

Example cli command to execute test cases / suitespybot test_cases.html | pybot path/to/my_tests/ | pybot c:\robot\tests.txt

It is also possible to give paths to several test case files or directories at once, separated with spaces. In this case, Robot Framework creates the top-level test suite

automatically, and the specified files and directories become its child test suites.

The name of the created test suite is derived from child suite names by concatenating them together with an ampersand (&) and spaces.

These automatically created names are often quite long and complicated. In most cases, it is thus better to use the --name option for overriding it:

pybot my_tests.html your_tests.html pybot --name Example path/to/tests/pattern_*.html

Using optionsRobot Framework provides a number of command line options that can be used to control how test cases are executed and what outputs are generated.

When options are used, they must always be given between the runner script and the data sources. For example:

pybot -L debug my_tests.txt pybot --include smoke --variable HOST:10.0.0.42 path/to/tests/

Test Suite Execution

Created by Jan Medvedwww.opendaylight.org

Short and long options

Options always have a long name, such as --name, and the most frequently needed options also have a short name, such as -N. In addition to that, long options can be shortened as long as they are unique. For example, --logle DEBUG works, while --

lo log.html does not, because the former matches only --loglevel, but the latter matches several options. Short and

shortened options are practical when executing test cases manually, but long options are recommended in start-up scripts,

because they are easier to understand.

The long option format is case-insensitive, which facilitates writing option names in an easy-to-read format. For example, --

SuiteStatLevel is equivalent to, but easier to read than --suitestatlevel.

Setting option values

Most of the options require a value, which is given after the option name. Both short and long options accept the value separated from the option name with a space, as in --include tag or -i tag. With long options, the separator can also

be the equals sign, for example --include=tag, and with short options the separator can be omitted, as in -itag.

Some options can be specified several times. For example, --variable VAR1:value --variable

VAR2:another sets two variables. If the options that take only one value are used several times, the value given last is

effective.

Simple patterns

Many command line options take arguments as simple patterns. These glob-like patterns are matched according to the

following rules:•* is a wildcard matching any string, even an empty string.

•? is a wildcard matching any single character.

•Unless noted otherwise, pattern matching is case, space, and underscore insensitive.

Examples:

--test Example* # Matches tests with name starting 'Example', case insensitively. --include f?? # Matches tests with

a tag that starts with 'f' or 'F' and is three characters long.

Test Suite Execution

13

Created by Jan Medvedwww.opendaylight.org

Test Suite Reporting

14

Robot outputs three files detailing the results of the test case / suite run

● Output.xml

Output files contain all the test execution results in machine readable XML

format. Log, report and xUnit files are typically generated based on them, and they can also be

combined and otherwise post-processed with Rebot.

● Report.html

Report files contain an overview of the test execution results in HTML format. They have statistics based

on tags and executed test suites, as well as a list of all executed test cases. When both reports and logs

are generated, the report has links to the log file for easy navigation to more detailed information. It is

easy to see the overall test execution status from report, because its background color is green, if

all critical tests pass, and bright red otherwise.

The command line option --report (-r) determines where report files are created. Similarly as log files,

reports are always created unless NONE is used as a value, and their default name is report.html.

● Log.html

Log files contain details about the executed test cases in HTML format. They have a hierarchical

structure showing test suite, test case and keyword details. Log files are needed nearly every time when

test results are to be investigated in detail. Even though log files also have statistics, reports are better

for getting an higher-level overview.

The command line option --log (-l) determines where log files are created. Unless the special

value NONE is used, log files are always created and their default name is log.html.

Created by Jan Medvedwww.opendaylight.org

Robot Test Suite Log File

Example Test Suite Report

15

Created by Jan Medvedwww.opendaylight.org

Summary

16

Test Suite and Test Case Automation Best Practices

● Document each test case. Explain the purpose of the test.

● Use the Log function to return useful information about keyword usage and test case runs.

● Only test one attribute of the feature in a test case instead of many attributes.

● Every test case should have a tag to enable/disable test case run.

● Use ‘wait until’ instead of sleep when waiting for a process to complete.

● Use Resource files if your keywords span more than one test case and when using user created

keywords.

● Use an initialization file ‘__init_.txt to start services or systems needed for the tests.

● It is better to use variables instead of static assignment. Make you test suite more portable and

dynamic.

● Keep your test case Clear and concise.

● Clean up the test environment after each test suite run.

● Setup and Teardown sections in test case can be used for this purpose.

● Define global variables in a variables file.

Created by Jan Medvedwww.opendaylight.org

Open Daylight Integration Reference

https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Test_Tools

Robot Integrate Dev Environment (RIDE)

https://code.google.com/p/robotframework-ride/

https://github.com/robotframework/RIDE/wiki

https://code.google.com/p/robotframework-ride/wiki/InstallationInstructions

Robot Guides

http://robotframework.org/

https://code.google.com/p/robotframework/

http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#executing-

test-cases

Extending Robot

http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#extending-

robot-framework

http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#test-data-

syntax

How to write good test cases:

https://code.google.com/p/robotframework/wiki/HowToWriteGoodTestCases

References

17