software anomalies test planning - department of · pdf fileagenda • iso/iec/ieee 29119...
TRANSCRIPT
![Page 2: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/2.jpg)
Agenda• ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies
2
![Page 3: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/3.jpg)
3
Organizationaltestprocess
ISO/IEC/IEEE, “ISO/IEC/IEEE international standard for software and systems engineering – software testing part 2: Test processes,” ISO/IEC/IEEE 29119-2:2013(E), pp. 1–68, Sept 2013.
Testmanagementprocesses
Testplanningprocess
Testmonitoringandcontrolprocess
Testcompletionprocess
Dynamictestprocesses
Testdesign&implementationprocess
Testenvironmentsetup&maintenanceprocess
Testexecutionprocess
Testincidentreportingprocess
![Page 4: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/4.jpg)
4
Test planningUnderstandcontext
Organisetestplandevelopment
Iden5fyandanalyserisks
Iden5fyriskmi5ga5onapproaches
Designteststrategy
Determinestaffingandscheduling
Recordtestplan
Gainconsensusontestplan
Communicatetestplanandmake
available
Start
Testplan
scope
Testplandevelopmentschedule
Analyzedrisks
Mi5ga5onapproaches
Teststrategy
Scheduling,staffingprofile
DraHtestplan
Approvedtestplan
![Page 5: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/5.jpg)
5
Organizationaltestprocess
ISO/IEC/IEEE, “ISO/IEC/IEEE international standard for software and systems engineering – software testing part 2: Test processes,” ISO/IEC/IEEE 29119-2:2013(E), pp. 1–68, Sept 2013.
Testmanagementprocesses
Testplanningprocess
Testmonitoringandcontrolprocess
Testcompletionprocess
Dynamictestprocesses
Testdesign&implementationprocess
Testenvironmentsetup&maintenanceprocess
Testexecutionprocess
Testincidentreportingprocess
![Page 6: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/6.jpg)
Test design and implementation process
6
Iden%fyfeaturesets
Derivetestcondi%ons
Derivetestcoverageitems
Derivetestcases
Assembletestsets
Derivetestprocedures
Start
Featuresets
Testcondi%ons
Testcoverageitems
Testcases
Testsets
Testdesignspecifica%on
Testcasespecifica%on
Testcasespecifica%on
Testprocedures
andtestscripts
![Page 7: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/7.jpg)
Example“The system shall accept insurance applicants over the age of 18 and under the age of 80 years on the day of application based on their input age in whole years; all others shall be rejected. Accepted applicants of 70 and over shall receive a warning that in the event of a claim they shall pay an excess of $1000.”
7
![Page 8: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/8.jpg)
Derive test conditions• Completion criterion? • Valid input? • Invalid input? • What if we note the following: 40 <= Age <= 55 results in a discount
message (unspecified in the description). How do we handle that?
8
![Page 9: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/9.jpg)
Derive test coverage items• Equivalence class items to cover
9
![Page 10: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/10.jpg)
Derive test cases• Select representatives from each class to achieve 100% equivalence class
coverage
10
![Page 11: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/11.jpg)
Assemble test sets• What can be automated? • What must be manually tested?
11
![Page 12: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/12.jpg)
Derive test procedures• Ordering of test cases based on exposure/dependencies • Traceability
12
![Page 13: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/13.jpg)
Software anomaliesHow to classify anomalies according to ISO/IEC/IEEE 1044
ISO/IEC/IEEE,“IEEEstandardclassificationforsoftwareanomalies,”IEEEStd1044-2009(RevisionofIEEEStd1044-1993)
![Page 14: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/14.jpg)
Error(Mistake)
Fault(Defect,Bug)
Failure
maycause
Incident(Symptom)
maybeobservedas
Test
Testcaseexercises
mayinduce
Remember this?
![Page 15: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/15.jpg)
Defect - FaultFaults are executed defects. Some defects are detected before they become faults
![Page 16: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/16.jpg)
Attribute Definition
DefectID
Description Whatismissing,wrong,orunnecessary?
Status Currentstatewithindefectreportlifecycle.
Asset Thesoftwareasset(product,component,module,etc.)containingthedefect.
Artifact Thespecificsoftwareworkproductcontainingthedefect(specification,design,code).
Versiondetected
Versioncorrected
Priority Rankingforprocessingassignedbytheorganization
Severity Thehighestfailureimpactthatthedefectcould(ordid)cause,asdeterminedby(fromtheperspectiveof)theorganizationresponsibleforsoftwareengineering.
Probability
Effect Theclassofrequirementthatisimpactedbyafailurecausedbyadefect.
Type Isthedefectduetocode,data,configuration?
Mode Omissionoffeature?Misrepresentationofspec?Unnecessaryfeature?
Defect attributes
…
![Page 17: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/17.jpg)
Example 1 Attribute Value Definition
Effect Functionality Actualorpotentialcauseoffailuretocorrectlyperformarequiredfunction(orimplementationofafunctionthatisnotrequired),includinganydefectaffectingdataintegrity.
Effect Usability Actualorpotentialcauseoffailuretomeetusability(easeofuse)requirements.
Effect Security Actualorpotentialcauseoffailuretomeetsecurityrequirements,suchasthoseforauthentication,authorization,privacy/confidentiality,accountability(e.g.,audittrailoreventlogging),andsoon.
Effect Serviceability Actualorpotentialcauseoffailuretomeetrequirementsforreliability,maintainability,orsupportability(e.g.,complexdesign,undocumentedcode,ambiguousorincompleteerrorlogging,etc.).
![Page 18: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/18.jpg)
Example 1
Attribute Value Definition
Type Data Defectindatadefinition,initialization,mapping,access,oruse,asfoundinamodel,specification,orimplementation.Examples:Variablenotassigned,initialvalueorflagnotset,incorrectdatatype…
Type Interface Defectinspecificationorimplementationofaninterface(e.g.,betweenuserandmachine,betweentwointernalsoftwaremodules,betweensoftwaremoduleanddatabase,betweeninternalandexternalsoftwarecomponents,betweensoftwareandhardware,etc.).Examples:Incorrectmoduleinterfacedesignorimplementation,incorrectreportlayout(designorimplementation),incorrectorinsufficientparameterspassed
![Page 19: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/19.jpg)
Attribute DefinitionFailureID Uniqueidentifierforthefailure.Status Currentstatewithinfailurereportlifecycle.Title Briefdescriptionofthefailureforsummaryreportingpurposes.Description Fulldescriptionoftheanomalousbehaviorandtheconditionsunderwhichitoccurred,
includingthesequenceofeventsand/oruseractionsthatprecededthefailure.
Analysis Finalresultsofcausalanalysisonconclusionoffailureinvestigation.Environment Identificationoftheoperatingenvironmentinwhichthefailurewasobserved.
Configuration Configurationdetailsincludingrelevantproductandversionidentifiers.
Testreference Identificationofthespecifictestbeingconducted(ifany)whenthefailureoccurred.
Incidentreference
Identificationoftheassociatedincidentifthefailurereportwasprecipitatedbyaservicedeskorhelpdeskcall/contact.
Defectreference
Identificationofthedefectassertedtobethecauseofthefailure.
Failure attributes
…
![Page 20: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/20.jpg)
Case 1
“Sue calls service desk and reports she cannot log in to timesheet system because the password field is missing from the login screen.” — In this example, Sue has a problem in that she cannot log in, caused by a failure wherein the password field did not appear on the login screen, which was in turn caused by a defect inserted during coding of the Login.asp artifact.
![Page 21: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/21.jpg)
Failure/Defect Attribute ValueFailure FailureID 1Failure Status OpenFailure Title MissingpasswordfieldFailure Description
Failure AnalysisFailure Environment
Defect Asset
Defect Artifact
Defect Severity
Defect Probability
Defect Effect
Defect Type
Defect Mode
Case 1
![Page 22: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/22.jpg)
Case 2
“Joe calls service desk and reports he cannot log in to timesheet system because the password field is missing from the login screen.”
Duplicate: several failures can indicate the same fault
![Page 23: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/23.jpg)
Case 3
“During a peer review for software requirements for a new financial management system, Alice discovers that values are in the requirements as thousands of dollars instead of as millions of dollars.”
![Page 24: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/24.jpg)
Failure/Defect Attribute ValueDefect DefectID 2Defect Status ClosedDefect Description
Defect Asset
Defect Artifact
Defect Severity
Defect Probability
Defect Effect
Defect Type
Defect Mode
Case 3
![Page 25: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/25.jpg)
Case 4
“Company A’s battery ran out of power because there was no low-power warning. The design of a security system monitoring system did not include a warning for low battery power, despite the fact that this feature was specified in the requirements.”
![Page 26: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/26.jpg)
Failure/Defect Attribute ValueFailure FailureID 3Failure Status OpenFailure Title MissinglowpowerbatteryalertFailure Description
Failure AnalysisFailure Environment
Defect Asset
Defect Artifact
Defect Severity
Defect Probability
Defect Effect
Defect Type
Defect Mode
Case 4
![Page 27: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/27.jpg)
SEI Risk Identification TaxonomyClass Element Attribute
ProductEngineering
Requirements
Stability
Completeness
Clarity
Validity
Feasibility
Precedent
Scale
DesignFunctionality
Difficulty
![Page 28: software anomalies test planning - Department of · PDF fileAgenda • ISO/IEC/IEEE 29119 - Test planning • ISO/IEC/IEEE 1044 - Software anomalies 2](https://reader030.vdocuments.mx/reader030/viewer/2022012319/5aacc1327f8b9a2e088d5f06/html5/thumbnails/28.jpg)
So, how does that relate to me as a tester?
IfIamconcernedabout… ThenIwanttoemphasize
Requirementstability Traceability(testplanning)
Incompleterequirements Exploratorytesting
Unclearrequirements Decisiontables/statetransitiontesting
Designdifficulty Control-flowtesting
… …