QAntiumAn automation and quality tests environment.June 2009José Manuel Marín Román – Lead Software Architect

2OPTENET

Table of contents

1. Objective: Products certification2. SQA team organization3. Test types4. Selenium / QAntium / TAPStudio5. SQA Intranet – Quality universe management6. Distributed computation framework7. Technical and technologies8. Some pictures

OPTENET 3

1. Objective: Products certification

OPTENET 4

1. Objective: Products certification

Decrease certification cycles Decrease no. of issues Regression tests Create a quality framework for

development

=> Increase the products quality

OPTENET 5

2. SQA Team organization

OPTENET 6

2. SQA Team organization

Two teams:

TAP – Test Automation Project. R&D Engineers.

SQAA – Test cases analysis and manual executions. QA specialists.

OPTENET 7

3. Test types

OPTENET 8

3. Test types

Features tests: End-to-End. Check results as final users and administrators would do.• Control and navigation via web• Mail sending• Firewall, p2p, ...

Stress and performance tests:• Subject the product to extreme and high

work load. For ISPs. Non-intrusive tets => Product doesn't

need to be changed for the tests.

OPTENET 9

4. Selenium / QAntium / TAPStudio

OPTENET 10

4. Selenium / QAntium / TAPStudio (I)

Selenium: Web automation.• We create a driver between selenium

and our tets.• We modify selenium IDE for firefox.• Still we need a programming and web

knowledgement (xpath, frames ...) Need a more specific framework:• Automation Kernel libraries.• Machines and processes communication

protocol. Selenium is insufficientinsufficient to our purposes.

OPTENET 11

4. Selenium / QAntium / TAPStudio (II)

QAntium: .Net control designed as real selenium alternative.• More ajax control over our web interface.• More powerful and useful interface to

automate tests (winform, no longer web)• Highly configurable.• Own element locators (id, xpath,

attributes...) => decrease impacts when interface changes• Navigation and actions recording and

playing => like a macros system.

OPTENET 12

4. Selenium / QAntium / TAPStudio (y III)

TAP Studio:• Application which integrates QAntium to

automate tests cases.• Actions in a time line, saved as metaformat

in a relational data base.• Code auto generation, in python, to

maintain backward compatibility.• Configurable actions with data dictionaries

binding.• No longer programming! => Anyone can

automate tests.

OPTENET 13

5. SQA Intranet – Quality universe management

OPTENET 14

5. SQA Intranet

Improve communication between teams and departments.

Unify processes using a common methodology.

Manual testing, automation and certification monitoring.

Webservices. Automation cluster control. Reports and statistics generation. Tasks assignation and tracking.

OPTENET 15

6. Distributed computation framework

OPTENET 16

6. Distrib. computation framewk. (I)

We need execute automatic tests as fast as possible.

Product snapshots, status controlling. Own communication protocol between

machines, based on remote object execution. Multiplatform.

Cluster:• Control Unit distributes testcases along

available nodes.• Each node virtualizes an environment with

the product and a client.

OPTENET 17

6. Distrib. computation framewk. (II)

More on cluster:• Selenium and QAntium support.• Cluster remote control.• Intranet communication and SOAP data

base access.• Parallelism at different levels.• Nodes auto setup depending on version to

test.• 99% automatic. 1% maintenance.• Computation power shared between

departments.

OPTENET 18

6. Distrib. computation framewk. (y III)

Stats:• > 1000 automatic testcases.• ~17 minutes average per case• ~50 nodes in cluster• Linear execution: +11 days• Cluster execution: ¡< 8 hours !

OPTENET 19

7. Technical and technologies

OPTENET 20

7. Technical and technologies (I)

Integration of technologies, languages, environment:• Linux, windows, MacOSx• Shell scripts, Python, C++, .Net (Asp.net,

C#), javascript.• Bugzilla, Mantis, Firefox, Internet Explorer• VMWare, virtual networks.• SMTP, HTTP, log parsing, Sockets ...

OPTENET 21

8. Pictures

OPTENET 22

8. Pictures

• Node virtualizes a product environment

OPTENET 23

8. Pictures

• Initial cluster configuration with 28 nodes

OPTENET 24

8. Pictures

• Test plans execution in cluster

OPTENET 25

8. Pictures

• A test plan detailed. 7 failed cases.

OPTENET 26

8. Pictures

TAPStudio. Automation tool

OPTENET 27

8. Pictures

TAPStudio. Automation tool

OPTENET 28

8. Pictures

TAPStudio. Automation tool (remote playing)

© TODOS LOS DERECHOS RESERVADOS

Thank you for your interest

José Manuel Marín Román

w w w . o p t e n e t . c o m

OPTENET SAN SEBASTIANParque Tecnológico de Miramon

Pº Mikeletegi 58 - 1ª Planta20009 San Sebastián SPAIN

Tel: (+34) 902 154 604Fax: (+34) 94 330 81 82

OPTENET MADRIDJosé Echegaray, 8 Edif.3 2ª Planta Modulo I Parque Empresarial Alvia28230 Las Rozas ( Madrid ) SPAIN

Tel: (+34) 902 154 604Fax: (+34) 91 357 54 33

OPTENET FRANCIA 32 rue Guy Moquet

92240 MalaKoff ( Paris )FRANCE

Tel: +33 (0) 173 03 90 60 Fax: +33 (0) 173 03 90 69

OPTENET MEXICOJosé María Ibarrarán, 47, 8º

Col San José Insurgentes03900 México D.F. MEXICO

Tel: +52 (55) 56 60 71 38Fax: +52 (55) 56 60 71 44

OPTENET ITALIA Tel: +(39)-355-1034943

OPTENET UK11 Rosemont RoadLondon , NW3 6NG UNITED KINGDOM

Tel: +44 (0)870 0990 322Fax: +44 (0)870 0990 323

OPTENET, INC.633 NE, 167 st suite 1001

33162 Miami (Florida)United States

Tel: +1 305 249 7505Fax: +1 305 249 7506

OPTENET BRASILAv. Paulista 2444 Cj 51Cep 01310-300 ParaísoSão Paulo – SP BrasilTel: +55 11 3151 4832Fax: +55 11 3214 0516

OPTENET COLOMBIACalle 104 A Nº 21-07

Bogotá, ColombiaTel: +57 (1) 6377284Fax: +57 (1) 6104937

OPTENET CHILETel: +56 99 647 05 31