staf 在自動化測試上的延伸應用 -- tmstaf (trendmicro staf)

李志和 (Bevis Lee)Sr. Engineer, R&D Dept.

2012/06/10

STAF 在自動化測試上的延伸應用 –TMSTAF (TrendMicro STAF)

6/12/20121 | Copyright 2012 Trend Micro Inc.

Agenda

• Brief Introduction of STAF

• What‟s TMSTAF?

• TMSTAF vs. QA automation

• Q&A

6/12/2012 2| Copyright 2012 Trend Micro Inc.

Brief Introduction of STAF


• History

– Created at IBM Austin in 1998

– Open-sourced (on SourceForge) in 2001

• http://staf.sourceforge.net/

46/12/2012 | Copyright 2012 Trend Micro Inc.

http://staf.sourceforge.net/

• STAF runs as a daemon process, called STAFProc


What‟s TMSTAF?



TMSTAF = Trend Micro STAF

6/12/2012 8Confidential | Copyright 2012 Trend Micro Inc.

TMSTAF Architecture

File Structure



TMSTAF Features


Testware configurations can be customized

Test suite, test case defined in Python syntax

Cross-platform execution

Command line interface

Testware concept included:

Test process included: Setup, Test and Teardown

Execution log for tracing

XML test report w/ email notification

Testing Report


Email Notification of Testing Result


TMSTAF vs. QA‟s Automation


QA‟s Automation Process

• Automation construction

• Automation online


Test

cases

creation

Test cases

convert to

automation

scripts

Automation

script pre-

run and

debug

Integrate

with build

release

process

Build

quality

verification

Automation

environment

setup


Test Cases Creation

• Based on concept of „test suite‟, test case can be created for both „manual‟ and „automation‟ use



Automation

Environment Setup

• Automation environment can be deployed easily


Automation Environment

Python 2.6

TMSTAF

framework

STAF 3.x


Test Cases Convert to

Automation Scripts

• Convert test case to automation scripts structurally


Automation

Module

Automation

ModuleTest step

Test stepTest case

(automation

script)

Automation

Module


3 Weapons…

I - Scenario Mechanism


var1 = 111

var2 = 222

func = PythonFunction('moduleATester', 'testFunc1')

r = self.addTest(func, [var1,var2])

r.failUnlessEqual(0)

var1 = 333

var2 = 444


r = self.addTest(func, [var1,var2])

r.failUnlessEqual(0)

test0010.py

test0020.py


var1 = 111

var2 = 222

self.setScenario('_scenario01.py')

test0010.py

test0020.py

var1 = 333

var2 = 444

self.setScenario('_scenario01.py')


result = self.addTest(func, [var1,var2])

result.failUnlessEqual(0)

_scenario01.py

II - Setup, Teardown and Test procedures


Test Case

Setup steps

Test steps

Teardown steps


• Setup = addSetup()

• Test = addTest()

• Teardown = addTeardown()

III - Result Verification


• Execution result of each steps (includes setup and teardown) can be verified easily

Passed if ‘result’ = 0


Script Pre-run and

Debug


• More efficiency development of automation

Refinement

of script

Script

execution

Debugging

of script


• Script execution from command line interface


Run Target Case

Run Target Suite

“-h” Online Help

Easy to Integrate


• Execution log for tracing/debugging


Automation Online


Build Release Flow


1

2

3

4

5


Build Quality

Verification


Mail Notifications

Detail Testing Report



So, with TMSTAF,

you can…


Test

cases creation

Test cases

convert to

automation

scripts

Automation script

pre-run and debug

Automation

environment

setup

Test case can be used for both ‘manual’ and

‘automation’ use, no extra effort needs

Easy to setup automation environment for

development and execution

Implement test cases with structural and flexible

mechanism

More efficiency development and trouble-shooting

of automation

Integrate with

build release

process

Build quality

verification

Easy integrate with standard build release process

Useful information in testing report can help to

identify issue more quickly

Q&A


staf 在自動化測試上的延伸應用 -- tmstaf (trendmicro staf)

Technology