Software EngineeringLecture # 22

Quality Concepts

• Stages of Software Testing

• Requirement Traceability Matrix

Stages of Testing

Unit testing/Component Testing/Module testing[2,3,6]

Integration Testing

System testing

Acceptance testing

Stages of Testing

• Unit Testing: Testing the individual components of the system.Involves the design of test cases that validate that the internalprogram logic of a given unit/component is functioning properly, andthat program inputs produce valid outputs.

• Integration Testing: Concerned with testing the system as it isintegrated from its components. Should focus on Interface testingwhere the interactions between sub-systems and components aretested.

Stages of Testing

• System Testing: Testing the system as a whole to validate that it meetsits specification and the objectives of its users. The testing of acomplete system prior to delivery. The purpose of system testing is toidentify defects that will only surface when a complete system isassembled. System testing includes testing of performance, security,startup and recovery from failure modes.

• Acceptance Testing: User Acceptance Testing is a phase of softwaredevelopment in which the software is tested in the "real world" bythe intended audience.

Stages of Testing• Acceptance Testing includes:

• Alpha Testing

• Beta Testing

• Alpha testing – The alpha phase is the software prototype stage when thesoftware is first able to run. It will not have all the intended functionality, but itwill have core functions and will be able to accept inputs and generate outputs.An alpha test usually takes place in the developer's offices on a separate system.

• Beta Testing - The beta phase of software design exposes a new product, whichhas just emerged from in-house (alpha) testing, to a large number of real people,real hardware, and real usage. Beta testing is not a method of getting freesoftware long-term, because the software expires shortly after the testing period.

Requirement Traceability Matrix (RTM)

• A requirements traceability matrix is a document that traces and maps userrequirements [requirement Ids from requirement specification document]with the test case ids.

• Purpose is to make sure that all the requirements are covered in test casesso that while testing no functionality can be missed.

• This document is prepared to make the clients satisfy that the coveragedone is complete

• This document consists of Requirement/Base line doc Ref No., Testcase/Condition, and Defects/Bug id. Using this document the person cantrack the Requirement based on the Defect id

Types of RTM

• Forward Traceability – Mapping of Requirements to Test cases

• Backward Traceability – Mapping of Test Cases to Requirements

• Bi-Directional Traceability - A Good Traceability matrix is thereferences from test cases to basis documentation and vice versa.

Types of RTM• Forward Traceability - This matrix is used to check whether the project progresses

in the desired direction and for the right product. It makes sure that eachrequirement is applied to the product and that each requirement is testedthoroughly. It maps requirements to test cases.

• Backward Traceability - It is used to ensure whether the current product remainson the right track. The purpose behind this type of traceability is to verify that weare not expanding the scope of the project by adding code, design elements, testor other work that is not specified in the requirements. It maps test cases torequirements.

• Bi-Directional Traceability contains both Forward & Backward Traceability. Thistraceability matrix ensures that all requirements are covered by test cases andvice versa.

Usefulness of RTM

• How can I certify that the final software product meets thecustomer’s needs?

• Now we can only make sure requirements are captured in the testcases by traceability matrix.

Benefits of using RTM

• Make obvious to the client that the software is being developed asper the requirements.

• To make sure that all requirements included in the test cases

• To make sure that developers are not creating features that no onehas requested

• Easy to identify the missing functionalities.

• If there is a change request for a requirement, then we can easily findout which test cases need to update.

Disadvantages of NOT using RTM

• Poor or unknown test coverage, more defects found in production

• It will lead to miss some bugs in earlier test cycles which may arise inlater test cycles. Then a lot of discussions arguments with otherteams and managers before release.

• Difficult project planning and tracking, misunderstandings betweendifferent teams over project

Steps to create RTM

• Make use of Microsoft Excel to create RTM

• Defining Columns (Requirement Id, Requirement description, TestCase Id)

• You can make RTM more effective by adding more columns

RTM Format

Creating RTM - Example

• Testers start writing their test scenarios/objectives and eventually the test cases based on some input documents – Business requirements document, Functional Specifications document and Technical design document (optional).

Business Requirement Document (BRD)

Functional Specification Document (FSD)• The below is the Functional Specifications document (FSD) based on the

interpretation of the Business requirements document (BRD) and the adaptationof it to computer applications.

Creating test cases for RTM

• Based upon the BRD and FSD, test scenarios are generated

Creating RTM

• Such a document establishes a trace between, the BRD to FSD andeventually to the test scenarios. By creating a document like this, wecan make sure every aspect of the initial requirements have beentaken into consideration by the testing team for creating their testsuites.

Creating RTM

• You can leave it this way. However, in order to make it more readable, I prefer including the section names. This will enhance understanding when this document is shared with the client or any other teams. The outcome is as below:

Creating RTM

• For each test scenario that you came up with, you are going to have at least 1 or more test cases. So, include another column when you get there and write the test case IDs as shows below:

Creating RTM

• At this stage, the Traceability Matrix can be used to find gaps. For example, in the above Traceability Matrix you see that there are no test cases written for FSD section 1.2.

• As a general rule, any empty spaces in the Traceability Matrix are potential areas for investigation. So a gap like this can mean one of the two things:

• The test team has somehow missed considering the “Existing user” functionality.

• The “Existing user” functionality has been deferred to later or removed from the application’s functionality requirements. In this case, the RTM shows an inconsistency in the FSD or BRD – which means that an update on FSD and/or BRD documents should be done.

• If it is scenario 1, it will indicate the places where test team needs to work some more to ensure 100% coverage.

• In scenarios 2, RTM not just shows gaps it points to incorrect documentation that needs immediate correction.

• Let us now expand the RTM to include test case execution status and defects.

Creating RTMThis version of the Traceability Matrix is generally prepared during or after test execution:

Creating RTM

• The following are the important points to note about the Traceability Matrix:

• 1) The execution status is also displayed. During execution, it gives a consolidated snapshot of how work is progressing.

• 2) Defects: When this column is used to establish the backward traceability we can tell that the “New user” functionality is the most flawed. Instead of reporting that so and so test cases failed, RTM provides a transparency back to the business requirement that has most defects thus show casing the Quality in terms of what the client desires.

• 3) As a further step, you can color code the defect ID to represent their states. For example, defect ID in red can mean it is still Open, in green can mean it is closed. When this is done, the RTM works as a health check report displaying the status of the defects corresponding to a certain BRD or FSD functionality is being open or closed.

• 4) If there is a technical design document or use cases or any other artifacts that you would like to track you can always expand the above created document to suit your needs by adding additional columns.

Example

• Let's understand the concept of Requirement Traceability Matrixthrough a banking project. On the basis of Business RequirementDocument (BRD) and Technical Requirement Document (TRD), testersstart writing test cases.

• Lets suppose, the following table is our Business RequirementDocument or BRD for banking project. Here the scenario is that thecustomer should be able to login to xyz banking website with thecorrect password and user id while manager should be able to loginto the website through customer login page.

Business Requirement Document

Technical Requirement Document

• The below table is our Technical Requirement Document

• Note: QA teams do not document the BRD and TRD. Also some companies use FunctionRequirement Documents (FRD) which are similar to Technical Requirement Documentbut the process of creating traceability matrix remains the same.

• Let's Go Ahead and create RTM

Creating RTM

• Step 1: Our Test Case is

• "Verify Login, when correct ID and Password is entered, it should login successfully"

Creating RTM

• Step 2: Identify the Technical Requirement that this test case is verifying. For our test case, the technical requirement is T94 is being verified.

Creating RTM

• Step 3: Note this Technical Requirement (T94) in the Test Case.

Creating RTM

• Step 4: Identify the Business Requirement for which this TR (Technical Requirement-T94) is defined

Creating RTM

• Step 5: Note the BR (Business Requirement) in Test Case

Creating RTM

• Step 6: Do above for all Test Cases. Later Extract the First 3 Columns from your Test Suite. RTM is Ready!

•

Advantage of Requirement Traceability Matrix

• It confirms 100% test coverage

• It highlights any missing requirements or document inconsistencies

• It shows the overall defects or execution status with a focus on business requirements

• It helps in analyzing or estimating the impact on the QA team's work with respect to revisiting or re-working on the test cases