quality and change management - macquarie...
TRANSCRIPT
Chapter 9 Quality and Change Management
© Pearson Education Limited 2007
MACIASZEK, L.A. (2007): Requirements Analysis and System Design, 3rd ed.
Addison Wesley, Harlow EnglandISBN 978-0-321-44036-5
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 22
TopicsTopics
Quality management
• Quality assurance
• Quality control
Change management
• Change requests
• Traceability
1. Quality management
Quality management is part of an overall software process management, along such other activities as
people, risk and change management.Quality management divides into quality assurance
and quality control.
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 44
Main conceptsMain conceptsTesting is not just the debugging of programs – it is part of quality management: • Quality assurance is about proactive ways of building
quality into a software system• Quality control is about (mostly reactive) ways of testing
the quality of a software system Test driven development builds quality into a software system by an outright demand that a test code has to be written before the application codeand that the application must pass the test to be quality assuredChange management is a fundamental aspect of the overall project management Traceability underlies testing and change management
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 55
Quality managementQuality managementPart of an overall software process managementalong with:• people management, • risk management and • change management.
To be performed in parallel with and in addition to project management (scheduling, budget estimation, tracking project progress). • should have its own budget and schedule • one of its tasks should be to ensure quality in project
management per se• the actions and outcomes of quality and change
management may involve changes to the project schedule and budget baselines, also known as the performance measurement baselines
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 66
Software qualitiesSoftware qualitiescorrectnessreliabilityrobustnessperformanceusabilityunderstandabilitymaintainability (repair ability)scalability (evolvability)reusabilityportabilityinteroperabilityproductivitytimelinessvisibility
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 77
Quality assuranceQuality assuranceSoftware Quality Assurance (SQA) team
Techniques:
• Checklists
• Reviews
–– WalkthroughsWalkthroughs
–– InspectionsInspections
• Audits
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 88
ChecklistsChecklistsA predefined list of to-do points that need to
be scrupulously checked off in the
development process.
Processes differ from project to project
checklists cannot be fixed for ever
• Also, the “baseline” checklists need to be
modified to account for new IT technologies and
changes in IT development paradigms.
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 99
WalkthroughsWalkthroughsA type of formal brainstorm review that can be conducted in any development phaseA friendly meeting of developers, carefully planned and with clear objectives, an agenda, duration, and membershipA few days prior to the meeting, the participants are handed the materials to be reviewedDuring the meeting the problems need to be pinpointed but no solutions attemptedAcknowledged problems are entered on a walkthrough issues listThe list is used by the developer to make corrections to the reviewed software product or process A follow-up walkthrough may be necessary
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1010
InspectionsInspectionsLike the walkthrough, an inspection is a friendly meeting but done under close supervision by the project managementThe purpose is also to identify defects, validate that they are in fact defects, record them, and schedule when and by whom they have tobe fixedUnlike the walkthroughs, inspections are conducted less frequently, may target only selected and critical issues, and are more formal and more rigorousAn informational meeting usually takes place one week before theinspection meetingDuring the meeting, the defects are identified, recorded and numberedImmediately after the meeting, the moderator prepares the defect log – recorded in a change management toolThe developer is normally requested to resolve the defects quickly and record the resolution in the change management toolThe moderator – in consultation with the project manager – submits the development module to the Software Quality Assurance (SQA) group in the organization.
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1111
AuditsAuditsA quality assurance process that is modeled on and similar in required resources to traditional accounting audits. Positioned within the overall software process management: • It places risk management in its view. • It relates to the strategic importance of IT to the enterprise and
addresses IT governance. • It looks at the alignment of the audited project with IT
investments in the context of the enterprise mission and business objectives.
Differences between an audit and other QA techniques:• the producer of the audited product or process is usually a team,
not a person,• an audit can be performed without the presence of the producer,• audits make extensive use of checklists and interview and less
so of reviews,• audits can be external, i.e. conducted by auditors external to the
organization,• an audit can last from a day to a week or more (but is always
restricted to strictly defined scope and objectives).
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1212
Test driven developmentTest driven developmentThe idea is to write test cases and scripts as well as test programs before the application code (the unit under test) is developed (designed and programmed)• application code is written as a response to a test code
and the test code can be used to test the application code as soon as it is available
Advantages• allows to clarify user requirements (and the use case
specifications) before the programmer writes the first line of the application code
• drives in fact the software development, not just the software verification
• supported by testing frameworks, such as JUnit
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1313
JUnitJUnit testingtesting
TestCase
run(TestResult)runTest()setUp()tearDown()
TestResultXXXTest
Test
run(TestResult)
TestSuite
run(TestResult)addTest(Test)
public class XXXTest extends TestCase { public XXXTest(String testName) { super(testName); } public void testSomething() { ... } public void testSomethingElse() { ... } public void runTest() { testSomething(); testSomethingElse(); }}
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1414
Testing types and development phasesTesting types and development phases
Integrationtest
Unittest
Unittest
Functionaltest
Constrainttest
Acceptancetest
Installationtest
Designspecifications
Analysisspecifications
Systemconstraints
Customerrequirements
Userenvironment
Systemdeployed
Reu
sabl
e co
mpo
nent
s
Programming
Performance,Security,Scalabilty,etc.
Integrationtest
Unittest
Unittest
Functionaltest
Constrainttest
Acceptancetest
Installationtest
Designspecifications
Analysisspecifications
Systemconstraints
Customerrequirements
Userenvironment
Systemdeployed
Reu
sabl
e co
mpo
nent
s
Programming
Performance,Security,Scalabilty,etc.
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1515
Test conceptsTest concepts
Business Use Case Document
Feature 1Feature 2
…
Use Case Documents
Use Case Req 1Use Case Req 2
…
Test Plan Document
Test Case 1Test Case 2
…
Test Case Documents
Test Req 1Test Req 2
…
Defects Document
Defect 1Defect 2
…
Enhancements Document
Enhancement 1Enhancement 2
…
Business Use Case Document
Feature 1Feature 2
…
Use Case Documents
Use Case Req 1Use Case Req 2
…
Test Plan Document
Test Case 1Test Case 2
…
Test Case Documents
Test Req 1Test Req 2
…
Defects Document
Defect 1Defect 2
…
Enhancements Document
Enhancement 1Enhancement 2
…
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1616
Test case documentTest case document
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1717
Test environmentTest environment
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1818
Testing system servicesTesting system servicesInformal testing
Methodical testing
• Non-execution-based (formal reviews)
–– WalkthroughsWalkthroughs
–– InspectionsInspections
• Execution-based
–– Testing to specsTesting to specs
–– Testing to codeTesting to code
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 1919
Testing to specs Testing to specs Execution-based test type• applies to executable software products, not to documents or
models• also known as
–– blackblack--box testingbox testing–– functional testingfunctional testing–– input/output driven testinginput/output driven testing
Test module treated as a black box that takes some input and produces some output (no need to understand the program logic or computational algorithms)Requires that the test requirements are derived from the use case requirements, and then identified and documented in a separate test plan and test case documentsTest scenarios can be recorded in a capture-playback tool and used repeatedly for regression testingLikely to discover defects normally difficult to catch by other means, such as missing functionality
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2020
Testing to specsTesting to specs
[1735]
Input set Ie
Output set Oe
Test Unit(as black box)
Inputs causinganomalous behavior
Outputs which revealthe presence of defects
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2121
Feasibility of testing to specsFeasibility of testing to specsExample 1:Consider price calculation based on two factors: 5
types of commission and 7 types of discount →
35 test cases
Example 2:
Consider 20 factors, each taking on 4 values → 420
or 1.1 × 1012 test cases
Conclusion:
Combinatorial explosion[1816]
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2222
Testing to codeTesting to codeExecution-based testingAlso known as• white-box testing• glass-box testing• logic-driven testing• path-oriented testing
Starts with the careful analysis of the program’s algorithmsTest cases are derived to exercise the code – i.e. to guarantee that all possible execution paths in the program are verified• the test data are specially contrived to exercise the code
Can be supported by the capture-playback tools and used then for regression testing• playback scripts need to be written by the programmer rather
than generated by the toolLike all other forms of execution-based testing, testing to code cannot be exhaustive
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2323
Testing to codeTesting to code
[1735]
Test inputs
Test Unitas white box
Test outputs
TestDerives
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2424
Feasibility of testing to codeFeasibility of testing to codeCombinatorial explosion
It is possible to test every path without detecting every fault.A path can be tested only if it is present.Weakening white-box testing:• exercising both, and only, the true branch and the false
branch of all conditional statements• execute every statement ( and that’s it)
[1816]
loop <= 18 times
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2525
Techniques of testing to specsTechniques of testing to specsEquivalence testing• suppose the range 1 through 16,000 is an
equivalence class• test cases:
–– less than 1 (error message)less than 1 (error message)–– from 1 through 16,000 (correct)from 1 through 16,000 (correct)–– more than 16,000 (error message)more than 16,000 (error message)
Boundary value analysis:–– 00–– 11–– 22–– 723723–– 15,99915,999–– 16,00016,000–– 16,00116,001[1816]
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2626
Techniques of testing to codeTechniques of testing to code
Structural testing: • statement coverage
• branch coverage
• path coverage
• all-definition-use-path coverage
Complexity metrics:• lines of code
• cyclomatic complexity[1816]
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2727
Approaches to integration testingApproaches to integration testingBottom-up testing• requires drivers (routines that call a particular
component and pass a test case to it)
Top-down testing• requires stubs (routines to simulate the activity
of the missing component; they answer the calling sequence and pass back output data that lets the testing process continue)
Big-bang integration
Sandwich integration[2110]
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2828
Testing system constraints Testing system constraints Predominantly execution-basedIncludes such issues as:• user interface testing• database testing• authorization testing• performance testing• stress testing• failover testing• configuration testing• installation testing
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 2929
Graphical user interface testing Graphical user interface testing Is the window modal or modeless? Which should it be?Is a visual distinction made between the required and optional fields?Are any fields missing?Are there any spelling mistakes in titles, labels, prompt names, etc.?Are command buttons (OK, Cancel, Save, Clear, etc.) used consistently across all dialog boxes?Is it possible always to abort the current operation (including the delete operation)?Are all static fields protected from editing by users? If the application can change the static text, is this being done correctly?Do the sizes of edit boxes correspond to ranges of values that they take?Are the values entered into edit boxes validated by the client program?Are the values in drop-down lists populated correctly from the database?Are edit masks used in entry fields as specified?...
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3030
Database testingDatabase testingVerify that the transaction executes as expected with correct input. Is the system’s feedback to the UI correct? Is the database content correct after the transaction?Verify that the transaction executes as expected with incorrect input. Is the system’s feedback to the UI correct? Is the database content correct after the transaction?Abort the transaction before it finishes. Is the system’s feedback to the UI correct? Is the database content correct after the transaction?Run the same transaction concurrently in many processes. Deliberately make one transaction hold a lock on a data resourceneeded by other transactions. Are the users getting understandable explanations from the system? Is the database content correct after the transactions have terminated?Extract every client SQL statement from the client program and execute it interactively on the database. Are the results as expected and the same as when the SQL is executed from the program?...
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3131
Authorization testingAuthorization testingThe user interface of the program should be able to configure itself dynamically to correspond to the authorization level of the current user (authenticated by the user id and password)Server permissions (privileges):• to access individual server objects (tables, views, columns,
stored procedures, etc.)• to execute SQL statements (select, update, insert, delete,
etc.)Permissions for a user may be assigned at a user level or at a group levelMost DBMSs support also the role levelAn authorization database may need to be set up alongside the application database to store and manipulate the client and server permissions
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3232
Testing of other constraintsTesting of other constraintsperformance testing• transaction speed and throughput• peak loads
stress testing• to break the system when abnormal demands are placed on it • frequently coupled with performance testing
failover testing• system’s response to a variety of hardware, network or software
malfunctions• closely related to the DBMS recovery procedures
configuration testing• how the system operates on various software and hardware
configurations. installation testing• extends the configuration testing• verifies that the system operates properly on every platform
installed
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3333
Review Quiz 9.1Review Quiz 9.11. Which technique of quality assurance is also
known as non-execution-based testing?
2. Which quality assurance technique produces a
defect log to act on?
3. What is a popular testing framework for Java
applications?
4. How are test cases realized (specified)?
5. Which kind of testing can discover missing
functionality?
2. Change management
Change management is the underlying aspect of overall project management.
Change requests must be documented and the impact of each change on development artifacts tracked and retested after the change has been
realized.
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3535
Change and quality managementChange and quality management
Use Case Requirements
Test Requirements
Quality management
Defects
Model Artifacts
Change Requests
Enhancements
Use Case Requirements
Test Requirements
Quality management
Defects
Model Artifacts
Change Requests
Enhancements
Change Management
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3636
Managing changeManaging change
Details of selected defect
Workspace browser with predefined queries, charts and reports
List of defects
Possible actions(state changes)
Details of selected defect
Workspace browser with predefined queries, charts and reports
List of defects
Possible actions(state changes)
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3737
Submitting change request Submitting change request
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3838
Defect managementDefect management
Allowed actions for submitted defectAllowed actions for submitted defect
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 3939
Keeping track of change requestsKeeping track of change requests
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4040
TraceabilityTraceabilityThere is a significant cost to the project associated with the traceability, testing and change management • the cost-benefit analysis should determine the scope and
depth of project traceability–– as a minimum, the traceability should be maintained between as a minimum, the traceability should be maintained between
the use case requirements and defectsthe use case requirements and defects
What can be traced?• Features are linked to test cases and to use case
requirements in the use case documents• Test requirements in the test case documents can be
traced back to test cases and use case requirements• Test requirements are linked to defects and enhancements
are traced to use case requirements
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4141
System features to use cases and use case requirementsSystem features to use cases and use case requirements
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4242
Test plans to test cases and test requirementsTest plans to test cases and test requirements
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4343
UML diagrams to documentsUML diagrams to documents
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4444
UML diagrams to requirementsUML diagrams to requirements
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4545
Use case requirements to test requirementsUse case requirements to test requirements
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4646
Test requirements to defectsTest requirements to defects
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4747
Use case requirements to enhancementsUse case requirements to enhancements
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4848
Review Quiz 9.2Review Quiz 9.21. How does an activity of managing a
change originate?
2. What technique is used to determine the
scope and depth of project traceability?
3. What tools can automate test scripts?
©© Pearson Education 2007Pearson Education 2007 Chapter 9 (Maciaszek Chapter 9 (Maciaszek -- RASD 3/e)RASD 3/e) 4949
SummarySummaryTesting and change management span the development lifecycleTesting has two dimensions• It is a reactive (post factum) activity when used as a quality
control mechanism • It can, however, be a very proactive quality assurance activity
when used within the framework of the test driven developmentTesting and change management assume that the traceability links between system artifacts exist and have been properly maintained during the developmentTesting divides into the testing of system services and the testing of system constraintsTest requirements are identified in the test case documents and linked to the use case requirements in the use case documentsA change request is normally either a defect or an enhancement