requirements engineering - quality assurance

Requirements Engineering:

Quality Assurance

CECS 542

Dr.BirgitPenzenstadler1

Photocredit:NathanDumlao,Unsplash

Dr.BirgitPenzenstadler 2

LearningGoals

•  Founda@onsofqualityassurance– QualitycriteriaforRE– Construc@veandanaly@calQualityAssurance

•  QAforArtefacts•  TechniquesforQualityAssurance


Context Layer

System Layer

Requirements Layer

Stakeholder Model Objectives & Goals

Constraints & Rules

!

!

!

!

!

Data Model

EA

A AE

System Vision

Functional Hierarchy

Architecture Overview

System

Function ModelFun 1

Fun 2

Component Model

C C

Data Model

EA

A AE

Behaviour Model

Business Case

Deployment Requirements

System Constraints

Domain Model

Service ModelUsage Model

Quality Requirements

Risk List

Project Scope

Process Requirements

Glossary

Glossary

Glossary

Overview:QualityAssurance

•  Mo@va@onandTerminology•  QualityofRequirementsDocuments•  PrinciplesofQualityAssurance•  Techniquesforconstruc@veQA•  Techniquesforanaly@calQA


KRayker,stock.xchng

16

11

9

1

7

7

5

3

3

1

5

1

4

3

0

2

1

1

0

31

22

22

20

16

13

13

12

11

11

9

9

8

8

5

3

3

2

2

0 5 10 15 20 25 30 35

Incomplete/hiddenreqs.

Movingtargets

Timeboxing

[email protected]@ons

Underspecifiedreqs.

Communica@onflawstocustomer

Inconsistentreqs.

Communica@onflawsinteam

Missingtraceability

Goldpla@ng

Unclearnon-func@onalreqs.

Terminologicalproblems

Insufficientsupportbycustomer

Unclearresponsibili@es

Vola@ledomain

Weakaccesstocustomerneeds

Insufficientsupportbyprojectlead

Technicallyunfeasiblereqs.

[email protected]

Overallfrequency

Causeforprojectfail

Recap:Challengesinprac@ce

5

Requirementsareo.en...•  Incomplete•  Notagreedupon•  Contradictory•  Notmeasurable•  Unfounded•  Irreproducible

Mendezetal.NamingthePaininRequirementsEngineering–TheNaPiRE-Report

Oneofthemanychallenges...

6

TerminologyinthecontextofqualityassuranceinREQualitydefect•  Incorrect(invalid)requirement:Requirementthatdoesnotreflectthe

inten@onofthestakeholder(inthesenseof„validity“)•  Qualitydefect:Requirementthatcanbevalid,buthasqualita@vedefects,

e.g.missingmeasurability,lowunderstandability,contradictory,...•  Interrela@onofthosetwo:

–  Incorrectrequirementsareojenhiddenduetoqualitydefects–  Correctnessofrequirementsojenviewedasqualitycriterium

Valida7onandVerifica7on•  Valida@on:Checkofrequirementw.r.t.correctness(it‘savalid

requirement,meaningitrepresentstheinten@onofthestakeholder)•  Verifica@on:Checkofsystemw.r.t.fulfillmentofrequirements•  BotharepartofQA


QualityAssuranceinRequirementsEngineering

•  Def.QAinRE:Applica@onofsystema@cmeasuresforiden@fyingqualitydefectsandassuringthequalityoftherequirementsspecifica@ons.

à Checkofqualitycriteria,e.g.:§  Correctness§  Completeness§  Consistency§  Traceability§  ...(seefollowingslides)

à Theexamina@oncanbeconductedconstruc@veoranaly@calusingaformalprocedure.

8




KRayker,stock.xchng

Recap:Requirements,documents,artefacts,repositories

•  Arequirementisademandedcharacteris@cofasystemorprocess.

•  Wedis@nguish–  Syntac@crepresenta@on:Text,table,diagram,formula–  Seman@crepresenta@on:Content–informalorformal

•  Arequirementsdocumentorartefact–  Containsanumberofrequirements–  Hasastructure

•  Delimita@on:Arequirementsrepository(databaseforrequirements)servesforstoringlargesetsofrequirements(andrequirementsartefacts)

10

QAofrequirementsdocuments

•  Qualityofrequirementsdocumentsiscrucialforprojectsuccess.àWhyisthat?Whatisbasedonthem?

•  WeneedspecificproceduresforQA•  Relevanceofthequalitycriterianeedstobedeterminedby

thefurtheruseofthedocuments.

UCIwinter2014 Dr.BirgitPenzenstadler 11

Focus of quality assurance in RE

Perspec@vesinQAinRE•  Wedis@nguishthequalityof–  Requirementsdocuments/artefacts–  Setsofrequirements/statements–  Individualrequirements–  Systems

12

Context Layer

System Layer

Requirements Layer


Constraints & Rules

!

!

!

!

!

Data Model

EA

A AE

System Vision



System

Function ModelFun 1

Fun 2

Component Model

C C

Data Model

EA

A AE

Behaviour Model

Business Case


System Constraints

Domain Model



Risk List

Project Scope


Glossary

Glossary

Glossary




KRayker,stock.xchng

Analytical QS

Dependingonthequalitycriteria,responsibleforcheckingare:•  Projectteammemberswithdomainknowledgeduringelabora@onofthe

requirements,e.g.„correctness“àthisiscalledconstruc@veQA•  External/neutralqualityresponsibleswhoperformchecks,e.g.

„traceability“and„understandability“àthisiscalledanaly@calQAàWhichmeasurescanyouthinkofforperformingeitherofthese?

Constructive QS

Principleofconstruc@veandanaly@calQA

14

Context Layer

System Layer

Requirements Layer


Constraints & Rules

!

!

!

!

!

Data Model

EA

A AE

System Vision



System

Function ModelFun 1

Fun 2

Component Model

C C

Data Model

EA

A AE

Behaviour Model

Business Case


System Constraints

Domain Model



Risk List

Project Scope


Glossary

Glossary

Glossary

Projectteam

Qualityresponsible

Classifica@onofQA

QA

Construc@ve

Processstandards

Qualitycriteria

Linguis@crules

Programmingguidelines

Namingconven@ons

Structuringconven@ons

Analy@cal

Analyzing

Metrics

Anomalyanalysis

Graphics

Tes@ng

Dynamictests

Review/inspec@on

Autom.sta@canalysis

Verifying

Formalverifica@on

ModelcheckingDr.BirgitPenzenstadler 15

Note:thisisageneralclassifica@onofQA,andnotallofitappliestoQAwithinRE.


•  Mo@va@onandTerminology•  QualityofRequirementsDocuments•  PrinciplesofQualityAssurance•  Techniquesforconstruc@veQA–  ReferenceModels– QualitycriteriaaccordingtoIEEE830–  Linguis@cs– Guidelines

•  Techniquesforanaly@calQA


KRayker,stock.xchng

ReferenceModels

•  AMDiRE•  IEEE830•  Cockburntemplate•  UML2standard


Construc@veQAinRE:[email protected]•  Completeness:Systema@callyrunthrough

allcases(Youcanonlyfindincompleteness,notguaranteecompleteness!)

•  Consistency:Relatealltoonesystemmodel•  Unambiguity:Checkphrasing•  Correctness:Valida@on•  Structuredness:Examinestructure•  Traceability:Arerequirementssufficientlylinked?•  Changeability:Canexpectedchangesinrequirementsbe

conductedefficiently?•  Understandability:Checkformula@ons•  Agreedupon:Checkwithstakeholders

18

Checklist:Ques@onsandCriteriafor...

•  Completeness:•  Consistency:•  Unambiguity:•  Correctness:•  Structuredness:•  Traceability:•  Changeability:•  Understandability:•  Agreedupon:

19

Checklist:Ques@onsandCriteriaforaspreadsheetwithrequirements

•  Completeness:Readthrough,aretheres@llopenques@ons?Leadengineer,stakeholder,customer

•  Consistency:Checkforrequirementsconflicts•  Unambiguity:languagecheck,letsomeoneelsereaditwhetheritcouldbe

misunderstood•  Correctness:checkwithstakeholder•  Structuredness:adherestotemplate/outline,hasabreakdownthat

makessense•  Traceability:linksbetweenrequirementsinspreadsheet,acrossversions

ofspreadsheet•  Changeability:checkdependenciesbetweenrequirements(highdegreeof

dependenciesmeanslowchangeability)•  Understandability:letsomeoneelsereaditwhetherit‘seasyto

understand•  Agreedupon:checkwithstakeholder

20

Linguis@csinRE•  Classifica@onoflinguis@cqualitydefects

–  lexical/ontological(whatdoes„green“mean?)–  syntac@c(“Isawthemanonthehillwithatelescope”)–  seman@c(“Allpersonshaveauniquena@onalinsurancenumber“)

–  pragma@c(“Thetrucksshalltreattheroadsbeforetheyfreeze“)–  weakphrases:(“assoonaspossible“)–  Omissionorgeneraliza@on

•  Syntaxpaperns–  [when?][underwhatcondi@ons?]THESYSTEMSHALL|SHOULD|WILL<process><thingtobeprocessed>[<processdetail>*]

21

Exercise:Improvephrasing

1. Thesystemshallrespondasfastaspossible.

2. Studentstake10coursespersemester.Studentstake1000coursespersemester.

3. Shortlybeforetheduedatethemediumisextended,unlesssomebodyelsereservedit.


Phrasing:Do‘sandDon‘ts1.  Thesystemshallrespondasfastaspossible.

In90%ofallcases,thesystemshallrespondtoallquerieswithin3s.

2.  Studentstake10coursespersemester.Studentstake1000coursespersemester.Everystudenttakes10coursespersemester.

3.  Shortlybeforetheduedatethemediumisextended,unlesssomebodyelsereservedit.Threedaysbeforetheduedate,thesystemcheckswhetherthemediumhasbeenreserved.Ifnotreserved,thesystemextendsthelendingperiod.

23

Guidelines&Checklists

Howwouldyouwriteaguidelineandachecklistfor…?Teamup!•  StakeholderModel•  GoalModel•  SystemVision•  UsageModel•  Non-func@onalRequirements



•  Mo@va@onandTerminology•  QualityofRequirementsDocuments•  PrinciplesofQualityAssurance•  Techniquesforconstruc@veQA•  Techniquesforanaly@calQA– Checklists– QualityGates– FaganInspec@ons


KRayker,stock.xchng

ExemplaryChecklistaccordingtoLamsweerde(1/2)

•  Generalcheck:Mustbeclear"what,who,when,where“

•  Defect-basedcriteria(asinIEEE830):–  ContradicJon–  Inadequacy–  Unmeasurability–  Unfeasibility–  Poorstructuring–  ...

•  Quality-specificcriteria:–  IsthereanyunspecifiedresponseinthisoperaJontonot

receivinganexpectedinputvalue,orreceivingittooearlyortoolate?

–  DoesthelogicalORoftheinputcondiJononthisoperaJonformatautology?

–  ... 26

ExemplaryChecklistaccordingtoLamsweerde(2/2)

•  Domain—specificcriteria:typicalissuesintheparJculardomain

•  Content-relatedcriteria:–  Templates

•  allfieldsfilled•  idenJfieruserconsistently•  statementtypecorrect•  ...

– Graphicalnota@ons•  dataflowconsistent•  ERDiagramdeclaraJon•  ...

–  Formalspecifica@ons 27

QualityGates

Specificmilestoneinasojwareprojectthatchecks•  Content:The„usual“qualitycriteria:Completeness,consistency,...

•  Documenta@on:Compliancewithformat,understandability,unambiguity,...

•  Accordance:Everyrequirementagreedupon,conflictsresolved,...

28

Role Model Process Model

Project Scope defined

System Specificationaccepted

Business Analyst

RequirementsEngineer

System Architect

ArchitectureOverviewdefined

RequirementsSpecificationaccepted

System Visiondefined

ContextSpecificationaccepted

Context Specification

System Specification

Requirements Specification

FaganInspec@on

ThetermFaganinspecJonreferstoastructuredprocessoftryingtofinddefectsindevelopmentdocuments.Itincludesthefollowingphases:•  Planning:Moderatorplansreviewprocess•  Overview:Authordescribesthebackgroundof

thedocumentunderinspec@on•  Prepara@on:Everyreviewerexaminesthe

documentinordertofinddefects.•  Inspec@onmee@ng:Aspecificreaderwalks

throughthedocumentchapterbychapter,andtheinspectorspointoutfounddefects.

•  Adapta@ons:Theauthorofthedocumentcorrectsthefounddefectsaccordingtotheac@onplanagreeduponinthemee@ng.

•  Follow-upcontrol:Theinspectorscheckwhetherthedefectswerefixedcorrectly. 29

Planning

Overview

Preparation

Inspection Meeting

Adaptation

Follow-up Control

IEEE730–2014:IEEEStandardforSojwareQualityAssuranceProcesses


Take-away:QA•  Defini@ons– QualityAssurance– QualityDefect

•  Construc@veQA– Guidelinesandcriteria–  Referencemodels

•  Analy@calQA– Qualitygates–  Faganinspec@on–  Checklists

•  IEEE730StdforSQADr.BirgitPenzenstadler 35

Context Layer

System Layer

Requirements Layer


Constraints & Rules

!

!

!

!

!

Data Model

EA

A AE

System Vision



System

Function ModelFun 1

Fun 2

Component Model

C C

Data Model

EA

A AE

Behaviour Model

Business Case


System Constraints

Domain Model



Risk List

Project Scope


Glossary

Glossary

Glossary


Timeforabreak

RequirementsNego@a@on


requirements engineering - quality assurance

Software