Bug zillatestopiajenkins

Download Bug zillatestopiajenkins

Post on 17-Jul-2015

52 views

Category:

Engineering

1 download

Embed Size (px)

TRANSCRIPT

<p>Virtualization</p> <p>Automated Testing Environment by Bugzilla, Testopia and JenkinsChin Pin Chang04/07/201411</p> <p>OutlineBugzillaTestopiaJenkins Automated Testing Environment (Hypervisor Testing as an example)Automated Testing ProcessExampleReferences22</p> <p>Bugzilla IntroductionBugzilla is a Defect Tracking System and it is server software designed to help you manage software development3</p> <p>3</p> <p>Bugzilla FeaturesExcellent security to protect confidentialityOptimized database structure for increased performance and scalabilityIntegrated email capabilitiesComprehensive permissions systemEditable user profiles and comprehensive email preferencesProven under fire as Mozilla's bug tracking system44</p> <p>Bugzilla RequirementsHardwareCPU : 3 GHzRAM : 4 GB or more recommendedHard Disk : 50 GB of free space is large enoughOperating System : Including Windows, Linux, and Mac OS XSoftwarePerl : 5.12Database Server : Bugzilla supports MySQL 5.5, PostgreSQL 8.3, Oracle and SQLiteWeb Server : Apache 2.25</p> <p>5</p> <p>MySQL InstallationStep 1. Update</p> <p>6</p> <p>Step 2. Install MySQL</p> <p>Step 3. Set the password </p> <p>6</p> <p>MySQL Installation (cont.)Step 4. Login MySQLuserpassword</p> <p>7</p> <p>7</p> <p>MySQL ConfigurationStep 5. Create an user for Bugzilla</p> <p>8</p> <p>Step 6. List users</p> <p>8</p> <p>MySQL Configuration (cont.)Step 7. Create database for Bugzilla</p> <p>9</p> <p>9</p> <p>Apache Configuration 10</p> <p>Step 1. Install Apache2</p> <p>Step 2. Modify Apache2 configuration file10</p> <p>Bugzilla InstallationStep 1. Download Bugzilla : http://www.bugzilla.org/</p> <p>11Step 2. Decompress the file to /var/www/</p> <p>Step 3. Modify folder name and switch to /var/www/bugzilla</p> <p>11</p> <p>Bugzilla Installation (cont.)Step 4. Execute checksetup.pl12</p> <p>12</p> <p>Bugzilla Installation (cont.)13Step 5. Install module</p> <p>13</p> <p>Bugzilla Installation (cont.)14Step 6. Re-run checksetup.pl </p> <p>Step 7. Need to modify localconfig</p> <p>14</p> <p>Bugzilla Installation (cont.)15Step 8. Modify Bugzilla configuration</p> <p>Step 9. Modify permission</p> <p>15</p> <p>Bugzilla Installation (cont.)16Step 10. Re-run checksetup.pl</p> <p>Step 11. Set up user information</p> <p>16</p> <p>Bugzilla Installation (cont.)17Step 12. Modify Apache configuration and restart Apache2</p> <p>17</p> <p>Bugzilla Installation (cont.)18Step 13. Installation complete</p> <p>18</p> <p>Testopia IntroductionTestopia is a test case management extension for Bugzilla Testopia RequirementsBugzillaMySQL or PostgreSQLMozilla compatible browserAdditional Perl ModulesTestopia ArchitectureTest PlansTest CasesTest Runs</p> <p>1919</p> <p>Relationship between Bugzilla and Testopia20</p> <p>Since Testopia is an extension to Bugzilla, it relies on many of the same objects that Bugzilla uses to track bugsTestopia is comprised of several objects that are interdependent, so they make managing the testing procedure possible20</p> <p>Relationship between Bugzilla and Testopia (cont.)21</p> <p>ComponentA Bugzilla component An attribute of a product</p> <p>21</p> <p>Relationship between Bugzilla and Testopia (cont.)22</p> <p>ProductThe product name belong to Bugzilla22</p> <p>23</p> <p>MilestoneA Bugzilla objectTestopia builds are associated with milestonesRelationship between Bugzilla and Testopia (cont.)23</p> <p>24</p> <p>Test PlanAt the top of the Testopia hierarchy are Test Plans. Before you can do anything else in Testopia, you need a Test Plan Test plan will serve as the storage point for all related Test Cases and Test Runs and it will act as the Dashboard for your testingRelationship between Bugzilla and Testopia (cont.)24</p> <p>Relationship between Bugzilla and Testopia (cont.)25</p> <p>Test CaseTest Cases are the heart of all testing Test Cases spell out what steps to take while running a test and what outcomes to expect Test Cases are associated with one or more Test Plans and with zero or more Test Runs25</p> <p>Relationship between Bugzilla and Testopia (cont.)26</p> <p>Test RunOnce you have defined a set of Test Cases, you are ready to run through those tests in a Test Run Each run is associated with a single Test Plan and can consist of any number of Test Cases from that plan26</p> <p>Relationship between Bugzilla and Testopia (cont.)27</p> <p>BuildSoftware development is usually an iterative affair. In Testopia, each iteration is called a buildIn software testing, a string denoting the compiled results of a period of developmentBuilds are often associated with milestones of the project. This is reflected in the build's relationship to Bugzilla's target milestones object27</p> <p>Relationship between Bugzilla and Testopia (cont.)28</p> <p>Case RunA Test Case-Run is the record of how a particular Test Case fared in a particular run for a given Build in a given Environment28</p> <p>Relationship between Bugzilla and Testopia (cont.)29</p> <p>CategoryA property of a product that is used to classify Test Cases29</p> <p>Relationship between Bugzilla and Testopia (cont.)30</p> <p>EnvironmentA list of the surrounding conditions that a test run is performed in30</p> <p>Testing Process of Testopia 31</p> <p>The testing process in Testopia is as follows :Step 1. Create a Product and multiple ComponentsStep 2. Create a Test PlanStep 3. Create Test CasesStep 4. Create a Build</p> <p>31</p> <p>Testing Process of Testopia (cont.) 32</p> <p>The testing process in Testopia is as follows :Step 5. Create EnvironmentStep 6. Create a Test RunStep 7. Execute the Test RunStep 8. Produce a test report on Dashboard</p> <p>32</p> <p>Testopia InstallationStep 1. Download Testopia http://www.mozilla.org/projects/testopia/33Step 2. Decompress the file to Bugzilla folderStep 3. Run checksetup.pl and install module</p> <p>33</p> <p>Access to TestopiaStep 4. Re-run checksetup.pl34</p> <p>Step 5. Installation complete</p> <p>34</p> <p>Jenkins IntroductionJenkinsis anopen source continuous integration tool written inJavaJenkins provides a web-based user interface35</p> <p>35</p> <p>Jenkins FeaturesEasy installationEasy configurationChange set supportE-mail integrationTest reportingPlugin support3636</p> <p>Jenkins InstallationStep 2. Access to Jenkinshttp://localhost:808037</p> <p>Step 1. Install Jenkins</p> <p>37</p> <p>Install Testopia plugin on Jenkins38</p> <p>38</p> <p>Integrate Jenkins with Testopia39This plug-in integratesJenkinswith Testopia and generates reports on automated test executionWith this plug-in you can manage your tests in Testopia, schedule and control in JenkinsIt readsTAPtest reportformats, used to update Testopia test case's executions39</p> <p>Integrate Jenkins with Testopia(cont.)40Testopia configuration section</p> <p>40</p> <p>Auto Test ProcessTest ControllerUbuntu 13.10Bugzilla 4.4TestopiaJenkins 1.5Shell scripts (expect)Shell scripts (expect)Shell scripts (expect)Step 1. Retrieve Test Case from Testopia</p> <p>411. Step 2. Invoke corresponding script to initialize the test environment2. Auto testing environment for ITRI ARM hypervisor : 41</p> <p>Auto Test Process (cont.)Test ControllerUbuntu 13.10Bugzilla 4.4TestopiaJenkins 1.5Shell scripts (expect)Shell scripts (expect)Shell scripts (expect)Ubuntu 12.04 LTSITRI ARM HypervisorGuest VMGuest VMGuest VMTest TargetStep 3. Initialize the test environmentStep 4. Launch necessary Guest VMs423. 4. 42</p> <p>Auto Test Process (cont.)Test ControllerUbuntu 13.10Bugzilla 4.4TestopiaJenkins 1.5Shell scripts (expect)Shell scripts (expect)Shell scripts (expect)Ubuntu 12.04 LTSITRI ARM HypervisorGuest VMGuest VMGuest VMTest TargetStep 5. Execute testing tasks435. 43</p> <p>Auto Test Process (cont.)Test ControllerUbuntu 13.10Bugzilla 4.4TestopiaJenkins 1.5Shell scripts (expect)Shell scripts (expect)Shell scripts (expect)Ubuntu 12.04 LTSITRI ARM HypervisorGuest VMGuest VMGuest VMTest TargetStep 6. Generate and return test result (by TAP, Test Anything Protocol)446. 44</p> <p>Auto Test Process (cont.)Test ControllerUbuntu 13.10Bugzilla 4.4TestopiaJenkins 1.5Shell scripts (expect)Shell scripts (expect)Shell scripts (expect)Step 7. Return test result to Jenkins</p> <p>457. 45</p> <p>Auto Test Process (cont.)Test ControllerUbuntu 13.10Bugzilla 4.4TestopiaJenkins 1.5Shell scripts (expect)Shell scripts (expect)Shell scripts (expect)Step 8. Write corresponding result to Test Case in Testopia based on TAP</p> <p>468. 46</p> <p>Auto Test ExampleStep 1. Add user on Bugzilla to execute auto testing47</p> <p>47</p> <p>The process to create a Test Plan48To create a new Test Plan in TestopiaStep 1. Click the "New Plan" link in the Bugzilla footerStep 2. Enter a name for this Test PlanStep 3. Select a product from the product listStep 4. Select a type for this plan Step 5. Select a product versionStep 6. Type or paste your plan document into the Plan Document EditorStep 7. Click the "Submit" button48</p> <p>Step 2. Create a new Test Plan49</p> <p>Plan Name Test Plan name49</p> <p>Step 2. Create a new Test Plan (cont.)50</p> <p>Product Bugzilla product name50</p> <p>Step 2. Create a new Test Plan (cont.)51</p> <p>Product TypeThe testing type for this Test Plan 51</p> <p>Step 2. Create a new Test Plan (cont.)52</p> <p>Product VersionThe Bugzilla product version 52</p> <p>Step 2. Create a new Test Plan (cont.)53</p> <p>Plan DocumentThis is document that spells out the type of testing and testing methods used for this plan53</p> <p>The process to create a Test Run54To create a new Test Run in TestopiaStep 1. Click "Create a New Test Run" link on your Test Plan or the New Run link in the footerStep 2. Select which Test Cases to include Step 3. Enter a Summary for this Test RunStep 4. Select a build from the Build list or type the name for a new one Step 5. Select an environment from the Environment list Step 6. Click the "Submit" button54</p> <p>Step 3. Create a new Test Run 55Product VersionThe Bugzilla product version55</p> <p>Step 3. Create a new Test Run (cont.) 56Run ManagerThe user in charge of this Test Run</p> <p>56</p> <p>Step 3. Create a new Test Run (cont.) 57Target Completion RateThe expect completion rate of target57</p> <p>Step 3. Create a new Test Run (cont.) 58BuildThe default product build58</p> <p>Step 3. Create a new Test Run (cont.) 59EnvironmentThe environment in use for this Test Run59</p> <p>Step 3. Create a new Test Run (cont.) 60Target Pass RateThe expect pass rate of target60</p> <p>Step 3. Create a new Test Run (cont.) 61SummaryA short description of this Test Run61</p> <p>Step 3. Create a new Test Run (cont.) 62NotesThis field is a place to make notes and observations about this run62</p> <p>The process to create a Test Case63To create a new Test Case in TestopiaStep 1. Click "Create a New Test Case" on the plan page or New Case from the footerStep 2. Enter a short description of your Test Case in the Summary fieldStep 3. Select a CategoryStep 4. Add a default tester Step 5. List the steps for testing in the Action field Step 6. Provide the expected outcomes in the Expected Results fieldStep 7. Click the "Submit" button63</p> <p>Step 4. Create a new Test Case64SummaryA short description of the Test Case</p> <p>64</p> <p>Step 4. Create a new Test Case (cont.)65Default TesterThe person is assigned to this Test Case</p> <p>65</p> <p>Step 4. Create a new Test Case (cont.)66AliasA unique string that uses to identify test result</p> <p>66</p> <p>Step 4. Create a new Test Case (cont.)67Priority The level of testing</p> <p>67</p> <p>Step 4. Create a new Test Case (cont.)68CategoryThe product category</p> <p>68</p> <p>Step 4. Create a new Test Case (cont.)69Estimated TimeThe estimated time that this Test Case should take to complete</p> <p>69</p> <p>Step 4. Create a new Test Case (cont.)70BugsYou can attach bugs to your Test Cases</p> <p>70</p> <p>Step 4. Create a new Test Case (cont.)71BlocksIf this Test Case prevents others from being run their ID numbers will appear here</p> <p>71</p> <p>Step 4. Create a new Test Case (cont.)72Status : Test Case status PROPOSED : This test case is a work in progress and has not been reviewed for accuracy. It is therefore not ready to be included in test runsCONFIRMED : This test case has passed review and is ready to be included in future test runs. Only test cases with this status can be included in new test runsDISABLED : This test case is no longer applicable to current testing</p> <p>72</p> <p>Step 4. Create a new Test Case (cont.)73Add TagsA user defined string used to classify Test Cases</p> <p>73</p> <p>Step 4. Create a new Test Case (cont.)74RequirementsThe requirement number or URL to a document containing the requirement this Test Case is designed to test</p> <p>74</p> <p>Step 4. Create a new Test Case (cont.)75Automated Automatic Test Cases are run by a script</p> <p>75</p> <p>Step 4. Create a new Test Case (cont.)76Scripts The name of the script that runs this Test Case</p> <p>76</p> <p>Step 4. Create a new Test Case (cont.)77Arguments The specific arguments sent to the script</p> <p>77</p> <p>Step 4. Create a new Test Case (cont.)78Add to RunEntering a run ID here will include this Test Case in a Test Run if it is not already included</p> <p>78</p> <p>Step 4. Create a new Test Case (cont.)79Depends On If this Test Case requires other Test Cases be run before this one, their case numbers should appear here</p> <p>79</p> <p>Step 4. Create a new Test Case (cont.)80Action This field lists the steps of testing</p> <p>80</p> <p>Step 4. Create a new Test Case (cont.)81Expected ResultsThis details the expected outcomes of a test81</p> <p>Process to Launch Test Script82Configure1. Clean Environment2. Check Image3. Launch Test ScriptStartTest Script1. 3. 2. 1. Login to Host2. Start VM3. Ping Host4. Turn off VM5. Return ResultTestopiaContents :1. Parameters2. Functions4. 1 . Retrieve Test Script82</p> <p>Set up a new project83</p> <p>Step 5. Create a project on Jenkins to invoke Test Script83</p> <p>Step 6. Set Testopia configuration section84</p> <p>Testopia VersionThe version name of Testopia84</p> <p>Step 6. Set Testopia configuration section (cont.)85</p> <p>Test Run IDThe Test Run ID of Testopia85</p> <p>Step 6. Set Testopia configuration section (cont.)86</p> <p>Single Test Build StepsThe command to execute by single step86</p> <p>Step 6. Set Testopia configuration section (cont.)87</p> <p>Iterative Test Build StepsThe parameters of Testopia are used to execute by test cases87</p> <p>Step 7. Set Result configuration section88</p> <p>Test Anything Protocol (TAP)TAP is a simple text-based interface between testing modules in a test harnessIt allows individual tests to communicate test results to the testing harness88</p> <p>TAP File Example89The path of the fileSuccess contents</p> <p>Fail contents</p> <p>89</p> <p>Launch The Project90</p> <p>Step 8. Start this project on Jenkins90</p> <p>Jenkins Console 91</p> <p>Step 9. Monitor the shell scripts91</p> <p>Jenkins Result92</p> <p>92</p> <p>Return Result to Testopia93</p> <p>93</p> <p>Question94</p> <p>Mail:walkerchang@iii.org.tw94</p> <p>References95http://www.bugzilla.org/https://developer.mozilla.org/en-US/docs/Mozilla/Bugzilla/Testopia#Requirementshttp://blog.codylab.com/testcase-management-using-testopia/https://wiki.jenkins-ci.org/display/JENKINS/Testopia+Pluginhttp://testanything.org/https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins95</p>