Download - Bug zillatestopiajenkins

Automated Testing Environment by Bugzilla, Testopia and Jenkins

Chin Pin Chang

04/07/2014

1

Outline

• Bugzilla

• Testopia

• Jenkins

• Automated Testing Environment (Hypervisor Testing as an example)

– Automated Testing Process

– Example

• References

2

Bugzilla Introduction

• Bugzilla is a Defect Tracking System and it is server software designed to help you manage software development

3

Bugzilla Features

• Excellent security to protect confidentiality

• Optimized database structure for increased performance and scalability

• Integrated email capabilities

• Comprehensive permissions system

• Editable user profiles and comprehensive email preferences

• Proven under fire as Mozilla's bug tracking system

4

Bugzilla Requirements

• Hardware– CPU : 3 GHz

– RAM : 4 GB or more recommended

– Hard Disk : 50 GB of free space is large enough

• Operating System : Including Windows, Linux, and Mac OS X

• Software– Perl : 5.12

– Database Server : Bugzilla supports MySQL 5.5, PostgreSQL 8.3, Oracle and SQLite

– Web Server : Apache 2.2

5

MySQL Installation

Step 1. Update

6

Step 2. Install MySQL

Step 3. Set the password

MySQL Installation (cont.)

Step 4. Login MySQL

– user

– password

7

MySQL Configuration

Step 5. Create an user for Bugzilla

8

Step 6. List users

MySQL Configuration (cont.)

Step 7. Create database for Bugzilla

9

Apache Configuration

10

Step 1. Install Apache2

Step 2. Modify Apache2 configuration file

Bugzilla Installation

Step 1. Download Bugzilla : http://www.bugzilla.org/

11

Step 2. Decompress the file to /var/www/

Step 3. Modify folder name and switch to /var/www/bugzilla

http://www.bugzilla.org/

Bugzilla Installation (cont.)

Step 4. Execute checksetup.pl

12


13

Step 5. Install module


14

Step 6. Re-run checksetup.pl

Step 7. Need to modify localconfig


15

Step 8. Modify Bugzilla configuration

Step 9. Modify permission


16


Step 11. Set up user information


17

Step 12. Modify Apache configuration and restart

Apache2


18

Step 13. Installation complete

Testopia Introduction

• Testopia is a test case management extension for Bugzilla

• Testopia Requirements– Bugzilla

– MySQL or PostgreSQL

– Mozilla compatible browser

– Additional Perl Modules

• Testopia Architecture– Test Plans

– Test Cases

– Test Runs

19

Relationship between Bugzilla and Testopia

20

• Since Testopia is an extension to Bugzilla, it relies on many of the same objects that Bugzilla uses to track bugs

• Testopia is comprised of several objects that are interdependent, so they make managing the testing procedure possible

Relationship between Bugzilla and Testopia (cont.)

21

• Component– A Bugzilla component

– An attribute of a product


22

• Product– The product name belong to Bugzilla

23

• Milestone– A Bugzilla object

– Testopia builds are associated with milestones


24

• Test Plan– At the top of the Testopia hierarchy are Test Plans. Before you can do anything else in

Testopia, you need a Test Plan

– Test plan will serve as the storage point for all related Test Cases and Test Runs and it will act as the Dashboard for your testing



25

• Test Case– Test Cases are the heart of all testing– Test Cases spell out what steps to take while running a test and what outcomes to expect – Test Cases are associated with one or more Test Plans and with zero or more Test Runs


26

• Test Run– Once you have defined a set of Test Cases, you are ready to run through those tests in a

Test Run

– Each run is associated with a single Test Plan and can consist of any number of Test Cases from that plan


27

• Build– Software development is usually an iterative affair. In Testopia, each iteration is called a build

– In software testing, a string denoting the compiled results of a period of development

– Builds are often associated with milestones of the project. This is reflected in the build's relationship to Bugzilla's target milestones object


28

• Case Run– A Test Case-Run is the record of how a particular Test Case fared in a particular run for a given

Build in a given Environment


29

• Category– A property of a product that is used to classify Test Cases


30

• Environment– A list of the surrounding conditions that a test run is performed in

Testing Process of Testopia

31

The testing process in Testopia is as follows :

Step 1. Create a Product and

multiple Components

Step 2. Create a Test Plan

Step 3. Create Test Cases

Step 4. Create a Build

Testing Process of Testopia (cont.)

32

The testing process in Testopia is as follows :

Step 5. Create Environment

Step 6. Create a Test Run

Step 7. Execute the Test Run

Step 8. Produce a test report

on Dashboard

Testopia Installation

Step 1. Download Testopia– http://www.mozilla.org/projects/testopia/

33

Step 2. Decompress the file to Bugzilla folder

Step 3. Run checksetup.pl and install module

http://www.mozilla.org/projects/testopia/

Access to Testopia


34

Step 5. Installation complete

Jenkins Introduction

• Jenkins is an open source continuous integration tool written in Java

• Jenkins provides a web-based user interface

35

Jenkins Features

• Easy installation

• Easy configuration

• Change set support

• E-mail integration

• Test reporting

• Plugin support

36

Jenkins Installation

Step 2. Access to Jenkins

– http://localhost:8080

37

Step 1. Install Jenkins

Install Testopia plugin on Jenkins

38

Integrate Jenkins with Testopia

39

• This plug-in integrates Jenkins with Testopiaand generates reports on automated test execution

• With this plug-in you can manage your tests in Testopia, schedule and control in Jenkins

• It reads TAP test report formats, used to update Testopia test case's executions

Integrate Jenkins with Testopia(cont.)

40

• Testopia configuration section

Auto Test Process

Test Controller

Ubuntu 13.10

Bugzilla 4.4

Testopia

Jenkins 1.5

Shell scripts (expect)Shell scripts (expect)

Shell scripts (expect)

Step 1. Retrieve Test Case from Testopia

41

1.

Step 2. Invoke corresponding script to initialize the test environment

2. Auto testing environment for ITRI ARM hypervisor :

Auto Test Process (cont.)

Test Controller

Ubuntu 13.10

Bugzilla 4.4

Testopia

Jenkins 1.5



Ubuntu 12.04 LTS

ITRI ARM Hypervisor

Guest VM

Guest VM

Guest VM

Test Target

Step 3. Initialize the test environmentStep 4. Launch necessary Guest VMs

42

3.

4.


Test Controller

Ubuntu 13.10

Bugzilla 4.4

Testopia

Jenkins 1.5



Ubuntu 12.04 LTS

ITRI ARM Hypervisor

Guest VM

Guest VM

Guest VM

Test Target

Step 5. Execute testing tasks

43

5.


Test Controller

Ubuntu 13.10

Bugzilla 4.4

Testopia

Jenkins 1.5



Ubuntu 12.04 LTS

ITRI ARM Hypervisor

Guest VM

Guest VM

Guest VM

Test Target

Step 6. Generate and return test result (by TAP, Test Anything Protocol)

44

6.


Test Controller

Ubuntu 13.10

Bugzilla 4.4

Testopia

Jenkins 1.5



Step 7. Return test result to Jenkins

45

7.


Test Controller

Ubuntu 13.10

Bugzilla 4.4

Testopia

Jenkins 1.5



Step 8. Write corresponding result to Test Case in Testopia based on TAP

46

8.

Auto Test Example

Step 1. Add user on Bugzilla to execute auto testing

47

The process to create a Test Plan

48

• To create a new Test Plan in Testopia– Step 1. Click the "New Plan" link in the Bugzilla footer

– Step 2. Enter a name for this Test Plan

– Step 3. Select a product from the product list

– Step 4. Select a type for this plan

– Step 5. Select a product version

– Step 6. Type or paste your plan document into the Plan

Document Editor

– Step 7. Click the "Submit" button

Step 2. Create a new Test Plan

49

• Plan Name – Test Plan name

Step 2. Create a new Test Plan (cont.)

50

• Product– Bugzilla product name


51

• Product Type– The testing type for this Test Plan


52

• Product Version– The Bugzilla product version


53

• Plan Document– This is document that spells out the type of testing and testing methods

used for this plan

The process to create a Test Run

54

• To create a new Test Run in Testopia– Step 1. Click "Create a New Test Run" link on your Test Plan

or the New Run link in the footer

– Step 2. Select which Test Cases to include

– Step 3. Enter a Summary for this Test Run

– Step 4. Select a build from the Build list or type the name

for a new one

– Step 5. Select an environment from the Environment list


Step 3. Create a new Test Run

55

• Product Version– The Bugzilla product version

Step 3. Create a new Test Run (cont.)

56

• Run Manager– The user in charge of this Test Run


57

• Target Completion Rate– The expect completion rate of target


58

• Build– The default product build


59

• Environment– The environment in use for this Test Run


60

• Target Pass Rate– The expect pass rate of target


61

• Summary– A short description of this Test Run


62

• Notes– This field is a place to make notes and observations about this run

The process to create a Test Case

63

• To create a new Test Case in Testopia– Step 1. Click "Create a New Test Case" on the plan page or

New Case from the footer

– Step 2. Enter a short description of your Test Case in the

Summary field

– Step 3. Select a Category

– Step 4. Add a default tester

– Step 5. List the steps for testing in the Action field

– Step 6. Provide the expected outcomes in the Expected

Results field


Step 4. Create a new Test Case

64

• Summary– A short description of the Test Case

Step 4. Create a new Test Case (cont.)

65

• Default Tester– The person is assigned to this Test Case


66

• Alias– A unique string that uses to identify test result


67

• Priority– The level of testing


68

• Category– The product category


69

• Estimated Time– The estimated time that this Test Case should take to complete


70

• Bugs– You can attach bugs to your Test Cases


71

• Blocks– If this Test Case prevents others from being run their ID numbers will appear

here


72

• Status : Test Case status – PROPOSED : This test case is a work in progress and has not been reviewed for

accuracy. It is therefore not ready to be included in test runs– CONFIRMED : This test case has passed review and is ready to be included in future test

runs. Only test cases with this status can be included in new test runs– DISABLED : This test case is no longer applicable to current testing


73

• Add Tags– A user defined string used to classify Test Cases


74

• Requirements– The requirement number or URL to a document containing the requirement

this Test Case is designed to test


75

• Automated – Automatic Test Cases are run by a script


76

• Scripts – The name of the script that runs this Test Case


77

• Arguments – The specific arguments sent to the script


78

• Add to Run– Entering a run ID here will include this Test Case in a Test Run if it is not

already included


79

• Depends On– If this Test Case requires other Test Cases be run before this one, their case

numbers should appear here


80

• Action– This field lists the steps of testing


81

• Expected Results– This details the expected outcomes of a test

Process to Launch Test Script

82

Configure

1. Clean Environment2. Check Image3. Launch Test Script

Start Test Script

1. 3.

2.

1. Login to Host2. Start VM3. Ping Host4. Turn off VM5. Return Result

Testopia

Contents :1. Parameters2. Functions

4.

1 . Retrieve Test Script

Set up a new project

83

Step 5. Create a project on Jenkins to invoke Test

Script

Step 6. Set Testopia configuration section

84

• Testopia Version– The version name of Testopia

Step 6. Set Testopia configuration section (cont.)

85

• Test Run ID– The Test Run ID of Testopia


86

• Single Test Build Steps– The command to execute by single step


87

• Iterative Test Build Steps– The parameters of Testopia are used to execute by test cases

Step 7. Set Result configuration section

88

• Test Anything Protocol (TAP)

– TAP is a simple text-based interface between testing modules in a test harness

– It allows individual tests to communicate test results to the testing harness

TAP File Example

89

• The path of the file

• Success contents

• Fail contents

Launch The Project

90

Step 8. Start this project on Jenkins

Jenkins Console

91

Step 9. Monitor the shell scripts

Jenkins Result

92

Return Result to Testopia

93

Question

94

• Mail:[email protected]

References

95

• http://www.bugzilla.org/

• https://developer.mozilla.org/en-US/docs/Mozilla/Bugzilla/Testopia#Requirements

• http://blog.codylab.com/testcase-management-using-testopia/

• https://wiki.jenkins-ci.org/display/JENKINS/Testopia+Plugin

• http://testanything.org/

• https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins

Download - Bug zillatestopiajenkins

Top Related