performance optimization in qtp execution over video automation testing

22
1 Performance Optimization In QTP Execution Over Video Automation Testing Speaker : Krishnesh Sasiyuthaman Nair Date : 10/05/2012

Upload: dooley

Post on 06-Feb-2016

57 views

Category:

Documents


4 download

DESCRIPTION

Performance Optimization In QTP Execution Over Video Automation Testing. Speaker : Krishnesh Sasiyuthaman Nair Date : 10/05/2012. Importance Of Performance Optimization. Prevent unnecessary wastage of test resources. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Performance Optimization In QTP Execution Over Video Automation Testing

1

Performance Optimization In QTP Execution Over Video Automation Testing

Speaker : Krishnesh Sasiyuthaman Nair Date : 10/05/2012

Page 2: Performance Optimization In QTP Execution Over Video Automation Testing

2

Importance Of Performance Optimization

• Prevent unnecessary wastage of test resources.• Get faster test results thus reducing cost in

automated testing.

Page 3: Performance Optimization In QTP Execution Over Video Automation Testing

3

Video Testing

• Involves different interlinked components like streaming servers, settop box racks, IR blasters etc.

• Need support from multiple applications and frameworks which provides OCRs, Image comparisons, SNMPs, trace processing etc

Page 4: Performance Optimization In QTP Execution Over Video Automation Testing

4

HPQC-QTP Framework In Video Testing

• Requirements and test cases will be available in QC Test Plan. QC provides a template in Test Plan for storing the test scripts, allows them to be added to QC Test Set, executes and saves result to QC test set.

• Provides feature to store defects in the defects area of QC.

Page 5: Performance Optimization In QTP Execution Over Video Automation Testing

5

QTP Test Execution From QC

Page 6: Performance Optimization In QTP Execution Over Video Automation Testing

6

Problems : QC-QTP Automation Development & Execution

• Interruptions during long execution due to QC timeouts and network issues.

• In the case of multi-location, its not possible to sync the local agile development version repository with mapping and execution of test cases in QC test plan.

Page 7: Performance Optimization In QTP Execution Over Video Automation Testing

7

Problems : QC-QTP Automation Development & Execution Continued

• Manual testers and other QC users will also be able to view the scripts and edit configuration files. This is not necessary.

• Network bandwidth utilization is too much during execution due to frequent connecting local machine QC client to QC server for transferring reports, initiating new runs etc as execution needs multiple active QC sessions.

Page 8: Performance Optimization In QTP Execution Over Video Automation Testing

8

Long Test Execution Cycle Time

In video testing each script will take around 10-15 minutes and a typical release testing will contain around 200-400 scripts , will be around 20-25 box types to validate. On a minimum there will be 4000 test runs and will take around 27 days if executed in one machine.

Page 9: Performance Optimization In QTP Execution Over Video Automation Testing

9

No Support For Agile Development & Coupled Execution

• There is no continuous integration available in QC-QTP default framework.

• In rigorous script development environment when multiple people works on the same library files, there are chances of bugs getting added to the impacted areas of code change and need to executed and validated frequently, with minimal effort, which is not possible currently.

Page 10: Performance Optimization In QTP Execution Over Video Automation Testing

10

Need For Common Local Subversion For Execution

• Since video automation framework using QTP is huge(contains lot of images, xmls , data tables, vbs file, csv files etc ), the execution need to be as local as possible because of multiple reasons.

• Its always an overhead in maintaining script and framework in two locations(subversion & QC) has wastage of time and space.

Page 11: Performance Optimization In QTP Execution Over Video Automation Testing

11

Network Bandwidth Usage Due To Active QC Sessions

As discussed in previous slide an execution can extend up to 27 days, which need to be shortened through parallel executions in multiple machines, say 10 machines it will come down to 2.7 days. But, this will need 10 active QC sessions polling frequently to client machines for execution triggering and result upload. Since framework itself is big, the file transfer between server to client will eat up lot of network bandwidth.

Page 12: Performance Optimization In QTP Execution Over Video Automation Testing

12

Solution : Performance Optimization In Framework Root causes for the problems mentioned in the

previous slides were found out and the converged causes are the following.

– Continuous integration support not available.– Execution need to be triggered locally with minimum

interaction with the QC server.

A test driver created with QTP AOM and QC OTA will be sufficient to solve the major bottle necks.

Page 13: Performance Optimization In QTP Execution Over Video Automation Testing

13

Continuous Integration In Script Development

Continuous integration involves integrating code early and often so as to avoid the pitfalls of ‘integration hell’. Aim is to reduce rework and reduce cost and time. There is a need to have version based execution and this support is added with the help of test driver.

Page 14: Performance Optimization In QTP Execution Over Video Automation Testing

14

Minimal Interaction With QC Server

With local execution framework, the long download times of script and framework files from QC is reduced along with reducing the interaction of QC during execution with local client executing machines. Also a configurable tool reboot time is introduced which removes the unexpected memory leaks associated with long duration executions.

Page 15: Performance Optimization In QTP Execution Over Video Automation Testing

15

Video Test Automation With Test Driver

Page 16: Performance Optimization In QTP Execution Over Video Automation Testing

16

Video Test Automation With Test Driver Continued

• Driver retrieves the details of test set created in QC.• Test lab added test cases with supporting files are

downloaded to local QTP execution machine will be the latest version or specific version in subversion.

• Execution trigger will be added to windows scheduler for daily runs.

• Execution results and logs will be available in QC Test Set on completion of execution.

Page 17: Performance Optimization In QTP Execution Over Video Automation Testing

17

Test Execution Duration Difference

Execution initiated Without Driver Execution Initiated With Driver

Total Scripts Duration Hrs

200 34

400 67

Total Scripts Duration Hrs

200 28

400 54

There is more than 10 hrs execution duration saving during execution. This is a definite advantage in video test automation space, since we can free up the hardware and software tools and resources blocked for this execution cycle & advantage is thus tremendous.

Page 18: Performance Optimization In QTP Execution Over Video Automation Testing

18

Script Level :Performance Optimization

• Each individual scripts need to be optimized separately for better results.

• There are a set of guidelines to be followed in the scripts and tool settings for better performance .

Page 19: Performance Optimization In QTP Execution Over Video Automation Testing

19

Conclusion

Performance optimized framework provides excellent foundation for automated testing in video test automation space. As manual execution is really tiring, the cost benefit a well optimized framework provides is really immense.

Page 20: Performance Optimization In QTP Execution Over Video Automation Testing

20

Question & Answers

Page 21: Performance Optimization In QTP Execution Over Video Automation Testing

21

Reference

• QC, QTP Documentation Library & HP ALM Open Test Architecture Reference.

• Tata Elxsi Automation Information Database.

Page 22: Performance Optimization In QTP Execution Over Video Automation Testing

22

Thank you

ITPB Road, Whitefield

Bangalore, India 560 048 Tel +91 80 22979123Fax +91 80 28411474

e-mail [email protected]

www.tataelxsi.com