![Page 1: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/1.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
ARC tools for revision and nightly functional tests
Jozef Cernak, Marek Kocan, Eva Cernakova
(P. J. Safarik University in Kosice, Kosice, Slovak Republic)
![Page 2: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/2.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Testing strategy in ARC• ARC-EMI testing tools
– Build testing• Nightly (every night)• Revision (after each commit of developers)
– Automatic functional testing of nightly builds• Future plans
– Increasing number of automatic functional tests– Automatic performance testing
• Conclusion
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 2
Outline
![Page 3: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/3.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Several roles in project: developers, testers and ARC release manager • Adopted tests:
– Unit tests(developers)– Build tests (testers)
• Revision (Kosice) • Nightly (Copenhagen)• EMI-RC (ETICS)
– Functional tests (testers, developers)• Manual • Automatic (Kosice)
– Regression testing (testers, developers)– Performance testing (testers)
• Manual• Automatic in development phase
– Large scale testing (in preparation phase)Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 3
Testing practice in ARC
![Page 4: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/4.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
ARC-EMI testing tools Relation between ARC and EMI testing tools
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 4
![Page 5: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/5.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Block diagram of automatic revision and Functional toolsSimple database server with PHP and Python scripts
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 5
![Page 6: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/6.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Infrastructure (local cluster)– 2 SUN 2200M2 servers (2x4GB RAM, 250GB
disk, AMD 64 bits)– 2 desktops (2x 1GB RAM, 500GB disk 32 bits)– 1 MAC mini (2GB RAM, 160GB, 32 bits)
• Software– MySQL, PHP, Python– Code is free
• Distributed data processing – SVN
Technical details
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 6
![Page 7: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/7.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Features– Supported are several platforms of OS:
• CentOS (32,64), • Debian(32,64), • and MAX OS (64)
– Detailed identification of errors during building process• Autogen• Configuration• make• make dist• Build
Revision automatic testing
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 7
![Page 8: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/8.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
– User friendly interface to define search criteria• Daily• Weakly• Monthly • Custom
– Revision number, etc (see next examples)
• Direct connection with SVN• Warning messages for developers if code is broken
– CCCC metric – Quick mechanisms to identify breaking event
…continued.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 8
![Page 9: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/9.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Example of www interface for automatic testing toolsHome page: http://arc-emi.grid.upjs.sk/ (Revision database is slow)
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 9
![Page 10: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/10.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Several criteria to search results of revision testing
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 10
![Page 11: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/11.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Example of the results of daily revisions
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 11
![Page 12: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/12.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Features– Automatic download code from SVN based on several criteria:
• trunk, • revision code,• nightly builds
– Automatic build– Automatic deployment– User friendly interface for:
• submitting proposals of test cases• to search the results of functional tests
– Tests are grouped into two main groups:• functional tests of server,• functional tests of client.
Automatic functional tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 12
![Page 13: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/13.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Interface to enter search criteria
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 13
![Page 14: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/14.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
The results of automatic functional tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 14
![Page 15: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/15.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• client_arcsub_to_arex_gridmap_simplejob_jsdl– Tries to submit a simple JSDL job to a grid-map secured A-REX service.
• client_arcsub_to_grid-manager_gridmap_simplejob_jsdl– Tries to submit a simple JSDL job to a grid-map secured ARC0 CE.
• client_arcsub_to_GIIS_simplejob_jsdl– Submission of a simple JSDL job to a GIIS service.
• client_arcsub_to_ISIS_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud.
• client_arcsub_random_broker_isis_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud using random broker.
• client_arcsub_random_broker_giis_simplejob_jsdl– Submission of a simple JSDL job to an GIIS cloud using random broker.
• client_arcsub_fastestqueue_broker_isis_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud using fastest queue broker.
• client_arcsub_fastestqueue_broker_giis_simplejob_jsdl– Submission of a simple JSDL job to a GIIS service using fastest queue broker.
• Failed– There was some error before we managed to start testing. The error could have been in
code retrieval, installation or somewhere else.
Functional test scenaria
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 15
![Page 16: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/16.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Requirements:– several servers and clients– Monitoring of several distributed processes:
• resources: – Memory– CPU– Network usage
• Performance– Scalability– Number of concurrent requests
• Reliability of services:– Ratio between successful tasks and total tasks.
Examples of tasks: job submission, file transfer and etc.
Automatic performance tests
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 16
![Page 17: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/17.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Proposed infrastructure:– Grid servers and clients, we plan to use
existing infrastructure– Communication layer • XML-RPC client-server architecture
– Central database server– Common Python library on each server and
client (for example start stop service)– Central control program
… continued.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 17
![Page 18: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/18.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Automatic testing covers:– Build process (ARC, ETICS),– Functional,– Performance tests (in progress).
• Utilization of automatic test tools increases probability to find weak part of the software. On the other hand, manual tests are important.
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 18
Conclusions
![Page 19: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/19.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
• Testing in ARC– http://wiki.nordugrid.org/index.php/Testing
• Revision tests– http://download.nordugrid.org/builds/– http://arc-emi.grid.upjs.sk/revisionTests.php
• Functional tests– http://arc-emi.grid.upjs.sk/functionalTests.php
• Code:– http://svn.nordugrid.org/trac/workarea/browser/
ARCTestScriptshttp://svn.nordugrid.org/trac/workarea/browser/ARCTestScripts
References
Use Insert Header & Footer to set this field
Use Insert Header & Footer to set this field 19
![Page 20: ARC tools for revision and nightly functional tests](https://reader033.vdocuments.mx/reader033/viewer/2022042822/56814023550346895dab834a/html5/thumbnails/20.jpg)
EMI I
NFS
O-R
I-261
611
EMI I
NFS
O-R
I-261
611
Thank you
Use Insert Header & Footer to set this field
20Use Insert Header & Footer to set this field
EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611