rqm best practices
TRANSCRIPT
-
8/17/2019 RQM Best Practices
1/52
© 2015 IBM Corporation
IBM Rational QualityManager:
Tips, Tricks and BestPractices
John Nason – RQM Chief Programmer
Sachin Patel – RQM Dev Lead
-
8/17/2019 RQM Best Practices
2/52
22
IBM Rational Quality Manager (RQM)Tips, Tricks and Best Practices
Intro To RQM
Sticking Points
Best Practices: SetupBest Practices: Planning
Best Practices: Execution
Advanced Topics: DevOps
Advanced Topics:Command Line Test
Adapter
-
8/17/2019 RQM Best Practices
3/52
Intro to RQM
-
8/17/2019 RQM Best Practices
4/52
4
Rational Quality Manager: Central Hub for QualityManagement
Rational Quality Manager
Test Planning
Test Construction
Test Execution
Team Collaboration
Reporting & Dashboard
RequirementsManagement
DefectManagement
Rational DNG
Rational DOORSRational RequisitePro
Rational Team Concert
Rational ClearQuest
FunctionalTesting Performance
Testing Integration TestingandTest Virtualization
TestData
Security andCompliance
Rational Functional Tester
Worksoft Certify RationalPerformanceTester
RationalTest Workbench
AppScan
Optim
-
8/17/2019 RQM Best Practices
5/52
5
Rational Quality Manager: A Closer Look
Test Planning
Comprehensive test plan
Shared objectives
Scope, Timeline, Resources
Risk assessment
Test Construction Requirement driven testing
Test environments coverage
Manual test authoring
Test lab management
Test Execution
Manual test execution
Use test automation tools
Record test results
Submit & track defects
Reporting & Dashboard Status and progress tracking
Customizable live dashboard
Real-time metrics and reports
Compliance and quality audit
Team Collaboration Process enactment and enforcement
Review and approval
Task management
Rapid team member on-boarding
-
8/17/2019 RQM Best Practices
6/52
Sticking Points
-
8/17/2019 RQM Best Practices
7/52
Sticking Point: Execution Records
• New users, especially those coming from other test
management tools, are often confused about the Test ExecutionRecord (TCERs & TSERs)
• Execution Records associate the following with a test:
• Who (owner)
• What (test case or suite)
• When (iteration/milestone)
• Where (Plan & Environment)
• Testers “run” TCERs or TSERs, not test cases or suites
7
-
8/17/2019 RQM Best Practices
8/52
Sticking Point: Execution Records
• Reporting generally is against the “last result” of a given TCER
• Progress tracking views (more later) work against TCERs
• Tip: from the ER view of a test plan use “Duplicate this ExecutionRecord to (this test plan | another test plan)” to quickly planregression testing, rerun failed tests, or those that you did not get to
8
-
8/17/2019 RQM Best Practices
9/52
Sticking Point: Execution Records
• A Test Suite Execution Record tracks running a test suite
• Test Suites are generally used for:
− Regression buckets
− Test scenarios with interdependencies
• Test Suite Execution Records are composed of Test CaseExecution Records
− Note: running a test standalone with the same iteration,plan, test case, and environment “counts” the same as a
test that's part of a suite with the same characteristic Last result is what's used in reporting
• Tip: plan your suites at the same time as your cases for accurateprogress tracking!
9
-
8/17/2019 RQM Best Practices
10/52
Best Practices:Setup
-
8/17/2019 RQM Best Practices
11/52
11
Best Practices: SetupCustomizing the Menu
You can customize the menu to only show items relevant to yourorganization. Find this in Admin > Manage This Project Area > MenuOrganization.
-
8/17/2019 RQM Best Practices
12/52
12
Best Practices: SetupCustomizing the MenuMenu after preceding customization
Menu choices reduced
Drop down choices reduced
-
8/17/2019 RQM Best Practices
13/52
13
Best Practices: Setup Artifact Templates Customize sections present Add new sections Add default content to sections, e.g. “How To”, best practices, etc. Tip: Replace your wiki with self-documenting process in the tool!
-
8/17/2019 RQM Best Practices
14/52
14
Best Practices: SetupCustomizing Permissions
Permissions can be viewed andconfigured by Role or Operation.
-
8/17/2019 RQM Best Practices
15/52
15
• Each artifact type comes with a predefined set of state transitions,which can be customized
• Each artifact supports a set of pre-defined state groups, which canbe used to write process enforcement against
• Each state will then map to each state group
• Transition between states can be customized too
• Can assign process advisers on each state that act aspre-conditions or follow-up actions
Draft
Readyfor Review
Approved
Retired
Readyfor review
Approve
Retire
Reject
Returnto Approve
Reopen
Tailor integrated process to fit your needs
Best Practices: SetupCustom Workflows
-
8/17/2019 RQM Best Practices
16/52
16
Best Practices: SetupProcess Advisors
- Process Advisors provide fine grained control over product behavior andenforce organizational best practices.- Require approvals before state transitions- Require tests to be in a specific state before running them- Configure (extensible) e-Signatures- Automatically lock and unlock test artifacts
-
8/17/2019 RQM Best Practices
17/52
17
• Ability to extract a process template from an existing QM project area
• A QM process template includes:
– Roles and permissions
– Test artifacts workflow customization
– Test artifacts templates
– QM project properties and preferences such as custom attributes, categoriesor quality objectives
• Process template can be reused to quickly create new QM project area
Reuse best practices and quickly setup new projects
Best Practices: SetupProcess Templates
-
8/17/2019 RQM Best Practices
18/52
Best Practices:Planning & Construction
-
8/17/2019 RQM Best Practices
19/52
1919
Test Planning and ConstructionTest Case and Test Suite Categories
RQM now provides functionality to allow a more
comprehensive parent child relationship between categories
Sub categoriescan also beassociated tomultiple parentsfor re-use
-
8/17/2019 RQM Best Practices
20/52
2020
Test Planning and ConstructionTest Case and Test Suite Categories
Pre-defined filter queries based on test case categories.
-
8/17/2019 RQM Best Practices
21/52
2121
Saved queries canbe promoted to themenu
Test Planning and ConstructionTest Case and Test Suite Categories
-
8/17/2019 RQM Best Practices
22/52
2222
Test Planning and ConstructionTest Case and Test Suite Categories
Test Case and Suite categories are visible in Execution Record tables
Saved execution recordqueries can be promotedto the dashboard.
Product quality and defectslisted per test case wherefound.
Group by capabilityavailable. Filtering byProduct and/or capability
-
8/17/2019 RQM Best Practices
23/52
2323
Test Casecategoryfiltersavailable
Test Planning and ConstructionTest Case and Test Suite Categories
Detailedpointprogressreport
-
8/17/2019 RQM Best Practices
24/52
2424
Report suggestions:Test Case Coverage by TCER(Live)
TCER Listing (Live) – with parameters for test casecategory type and name to gather data
Execution Status by Owner using TCER Count (Live)Test Cases (Live) – parameters of Test Case Type andName used to gather data
Test Cases by Team (Live) - parameters of Test Case
Type and Name used to gather data
Test Planning and ConstructionTest Case and Test Suite Categories
-
8/17/2019 RQM Best Practices
25/52
Best Practices:Execution
-
8/17/2019 RQM Best Practices
26/52
2626
Topics
Tester best practices
Test lead bestpractices
Test Execution
-
8/17/2019 RQM Best Practices
27/52
2727
Test Execution – Tester best practicesPerforming test execution tasks
As a tester, whattest case executionrecords do I need torun or resume?
-
8/17/2019 RQM Best Practices
28/52
2828
Test Execution – Tester best practicesMonitoring tester execution progress of a suite
A test suite is created, approved and ready for test. It has been scheduled for aniteration. Application is deployed – Tanuj gets started with his testing. Stops for lunch.Restarts for the afternoon after the staff meeting.Pauses for the day. X TCERs completed with 4 defects entered.
Let’s show the best way for Tanuj to get started the next dayLet’s show how Tanuj arranges his personal dashboard to track his progressHow can he decide what TCER to execute nextWhat is status of defectsWhat reports is test lead Tammy running to track progress
-
8/17/2019 RQM Best Practices
29/52
2929
Test Execution – Tester best practicesPerforming test execution tasks
Enable testers to use dashboard viewlets as their execution to do lists by:Creating shared execution record queries that leverage the Current User filtervalue for the Owner column and the Last Result column. Examples:
My Not Run test case execution recordsMy In Progress/Paused test case execution records
Using the Execution Record(s) widget to expose each test case execution
record query to a tester tab in the project dashboard.
Content seen by Tester 1: Content seen by Tester 2:
Total count returned by dashboardviewlet when viewed by Tester 1.
Total count returned by dashboardviewlet when viewed by Tester 2.
-
8/17/2019 RQM Best Practices
30/52
3030
Test Execution – Tester best practicesResuming execution of a suite
Test can be resumed from the testsuite execution record
Resumesuiteexecutionfrom theexecutionconsole oradashboardquery
-
8/17/2019 RQM Best Practices
31/52
3131
Test Execution – Tester best practicesMonitoring tester execution progress of a suite
Option to create TCERs when aTSER is created.
-
8/17/2019 RQM Best Practices
32/52
3232
Test Execution – Test lead bestpracticesMonitoring execution progress
As a test lead, howcan I monitor
execution progressand determinewhere defects arebeing found by myteam?
T t E ti
-
8/17/2019 RQM Best Practices
33/52
3333
Test ExecutionMonitoring execution progress
Tammy, our test lead, needs to create some shared tables and reports to monitorProduct application quality across the iteration as test progresses.She want to know which high priority test cases are passing, failing or blocked andwants to be able to see how the defects entered are progressing to resolved so theycan be verified by her team.What is the best way she can do that?
She will gather the specific test cases she needs by using a pre-defined filter ontest case categories in the execution record table. Gets a lot of records returned!She will then narrow that down by determining the iteration she wants – can bespecific or ‘current iteration’Then she groups by priority or perhaps Owner.She adds a column for defects so she can see resolution status and link to themquicklyShe can drag/drop columns for quick visual editing.What reports can she run to track progress
-
8/17/2019 RQM Best Practices
34/52
3434
Test ExecutionMonitoring Execution Progress
Use test Case Categoriesto narrow down the testcase execution records
Narrowed – but stillmore work to filterneeded.
-
8/17/2019 RQM Best Practices
35/52
3535
Test ExecutionMonitoring Execution Progress
Group by Priority to seeprogress bars
Column for defects tosee resolution statusand link to them quickly
Saved query for re-useand sharing
-
8/17/2019 RQM Best Practices
36/52
3636
Report suggestions:
Execution and defects by owner* - Displays the status ofexecution records and their associated defects for eachowner. Data Warehouse report.
Defect list* - Lists defects associated with the selected TestPlan(s) and iterations. Data warehouse report.
Test ExecutionMonitoring Execution Progress
For the teamDashboard, a set ofquery links.Reports can be addedto widgets on the
dashboard also
-
8/17/2019 RQM Best Practices
37/52
Advanced Topic:
DevOps
-
8/17/2019 RQM Best Practices
38/52
Running tests as part of your build:a key to devOps success
• The RQM Execution Tool allows for “headless” (unattended) execution ofautomated tests
• Two options:
– Run a standalone java program
– Embed execution directly as an Ant task
• Standard pattern:
– Build
– Provision (uDeploy, Chef, etc)
– Test (automated tests via RQM Execution Tool)
• Tip : use execution variables to parameterize your test
• https://jazz.net/wiki/bin/view/Main/RQMExecutionTool
38
-
8/17/2019 RQM Best Practices
39/52
39
Develop Build (multiple per day)Manual Test (daily but not
completed in a day)
FunctionTest
PerformanceTest
System Test
BuildUnitTest
ALMIntegration
Test
Develop
Staging (daily)
Production
(end of eachsprint)
StagingEnvironment
ProductionEnvironmentIntegration
Test
Rational
Jazz Build
Rational
Collaborative
LifecycleManagement
JUnit
IBM Workload
Deployer
IBM UrbanCode
Deploy
CSPF
IBM UrbanCode DeployIBM Workload
Deployer
IBM UrbanCode Deploy
devOps in ALM
-
8/17/2019 RQM Best Practices
40/52
40
Existing integrations with test automation tools
Quality Manager
Managing mobile applications testingacross a global handset testenvironment
Accelerating test sandbox creationthrough service virtualization
Quick Test ProfessionalLoadRunner
Embedded SW Testing
Rational Integration Tester(GreenHat)
Rational Functional Tester
Rational Performance Tester
Rational Service Tester for SOA
Quality
Worksoft Certify
Rational AppScan Tester Edition
Rational Test RealTimeTelecom Test Automation
VeriStand & TestStand
Selenium Junit tests
-
8/17/2019 RQM Best Practices
41/52
41
Selenium Junit test execution adapter
• Selenium primarily automates web applications for testing purposes
• Support for executing JUnit tests that call the Selenium 2 APIs to drive webbrowsers
• Gather the JUnit results and present them in a test case result
Support developer testing of Web UI
-
8/17/2019 RQM Best Practices
42/52
Advanced Topic:
Command Line Adapter
-
8/17/2019 RQM Best Practices
43/52
Command Line Test Adapter
• Offers a “low barrier to entry” for most of your custom, home
grown, or unsupported testing tools
• Run any test program from the shell and:
• Capture verdict via return code
• Capture standard out and standard error as the execution log that'suploaded to RQM
• Tip: GREAT article - https://jazz.net/library/article/809
43
-
8/17/2019 RQM Best Practices
44/52
Command Line Test Adapter
• Tip 1: you can map return codes from the command line script to
RQM verdicts using the commandline.properties file
Note: using “*” maps any undefined return to that verdict.
44
-
8/17/2019 RQM Best Practices
45/52
Command Line Test Adapter
• Tip 2: there is a set of OOTB execution variables that include
most details of the test being run, and you can define customexecution variables
– Passed via the ENV block to the test being run
– Use Project execution variables to define common vars andenumerated values to minimize error and promote reuse!
45
-
8/17/2019 RQM Best Practices
46/52
Command Line Test Adapter
Did you know you can track progress?
Use com.ibm.rqm.commandline.executionproperties to map afile where you write progress to, and it's visible in the RQM webUI. Cool!
You can periodically update an integer from 0-99 to showpercentage complete
You can add a status message
You can even embed a clickable link, e.g. if your automation
harness has a management console, etc.
46
-
8/17/2019 RQM Best Practices
47/52
Command Line Test Adapter
• Need a bigger boat?
− You can write your own test automation integration usingthe OSLC Automation specification for RQM
More flexibility, but higher cost of entry compared to thecommand lined adapter
− You can provide custom XHTML to be embedded intoresults, giving more UI flexibility
− https://jazz.net/wiki/bin/view/Main/RQMTestAutomationAdapterAPI
47
-
8/17/2019 RQM Best Practices
48/52
Questions?
-
8/17/2019 RQM Best Practices
49/52
Where are we in the Solution EXPO?
49
Business Process &
Decision Management
Application Infrastructure
& Integration
Mobile Enterprise
Cloud Business
Applications
Asset Management &
Internet of Things
Cloud Infrastructure & IT
Optimization
DevOps & Continuous
Engineering
Security Intelligence
T1 T2
T1 – IBM RTW (integration testing)IBM RTVS (Service Virtualization)
T2 - IBM RTW (functional,performance, & mobile testing)IBM RQM (quality management)
Notices and Disclaimers
-
8/17/2019 RQM Best Practices
50/52
Notices and DisclaimersCopyright © 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced ortransmitted in any form without written permission from IBM.
U.S. Government Users Restricted Rights - Use, duplication or d isclosure restricted by GSA ADP Schedule Contract w ith
IBM.
Information in these presentations (including information relating to products that have not yet been announced by IBM) has beenreviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBMshall have no responsibility to update this information. THIS document is distributed "AS IS" without any warranty, either expressor implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, lossof data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the termsand conditions of the agreements under which they are provided.
Any statements regarding IBM's future d irect ion, intent or product p lans are subject to change or wi thdrawal wi thout
notice.
Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples arepresented as illustrations of how those customers have used IBM products and the results they may have achieved. Actualperformance, cost, savings or other results in other operating environments may vary.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such products,programs or services available in all countries in which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared by independent session speakers, and do notnecessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neitherintended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.
It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legalcounsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’sbusiness and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice orrepresent or warrant that its services or products will ensure that the customer is in compliance with any law.
Notices and Disclaimers (con’t)
-
8/17/2019 RQM Best Practices
51/52
Notices and Disclaimers (con t)
Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products in connection with thispublication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM
products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of thoseproducts. IBM does not warrant the quality of any third-party products, or the ability of any such third-party productsto interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but notlimited to, the implied warranties of merchantability and fitness for a particular purpose.
The provision of the information contained herein is not intended to, and does not, grant any right or license under anyIBM patents, copyrights, trademarks or other intellectual property right.
• IBM, the IBM logo, ibm.com, Bluemix, Blueworks Live, CICS, Clearcase, DOORS®, Enterprise Document
Management System™, Global Business Services ®, Global Technology Services ®, Information on Demand,ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™,PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®,pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, SoDA, SPSS, StoredIQ, Tivoli®, Trusteer®,urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of InternationalBusiness Machines Corporation, registered in many jurisdictions worldwide. Other product and service namesmight be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at"Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtml
-
8/17/2019 RQM Best Practices
52/52
Thank You
Your Feedback isImportant!
Access the InterConnect 2015Conference CONNECT AttendeePortal to complete your sessionsurveys from your smartphone,
laptop or conference kiosk.