ambiguous requirements – translating the message from c-level to implementation

Ambiguous Requirements – Translating the message from C-level to implementationIIBA Presentation, 24/04/2014, Salford, Manchester

Llyr Wyn Jones – Senior Programmer / Consultant – Grid-Tools Ltd.

Introduction

• Llyr Wyn Jones• Senior Programmer, Grid-Tools Ltd.• Graduated 2010 from the University of St. Andrews – Masters in Mathematics

(First Class)• Certified Requirements-Based Testing (RBT) Practitioner• Main roles: Programming; Software Design; Requirements and Test Case

Design Consultancy• Specialist subjects: Test case design, requirement specifications, cryptography• Contributor to Professional Tester Magazine, as well as writing white papers

for Grid-Tools Ltd.

Motivation – Why am I here?

• Using Requirement-Based Testing (RBT) methods on software projects, discovered that, by the time a project gets to the testing phase, realizing cost/quality/time projections depends on what has gone before (i.e. it’s too late to do anything about it!)• The majority of project overruns are caused by poor and/or ambiguous

requirements.• C-level executives do not speak the same language as a database coder (for

example).• As a business analyst, it is critical to be able to write requirements that can be

understood by both the user (e.g. CIO) and the person(s) implementing it (e.g. the programmers and testers).

Cost Projection via Unambigious Requirements

• The white paper “Cost, Complexity and Coverage” was written to address the problem of calculating project cost by considering the number of testable requirements in a software project.• Built on the work of Peter B. Wilson (“Sizing Software With Testable

Requirements”).• Using a mathematical model of requirements (see Appendix), shown how

ambiguities can increase the variance of cost projections (i.e. the margin of error increases in proportion with the “quality” of requirements)• You may ask yourself: What has this got to do with C-level visions?

Clarity of Vision via Unambiguous Requirements

• The same mathematical model can be employed to show how ambiguities increase the margin of error as it travels through the project lifecycle.• We focus on software projects, but it applies to ALL projects – for non-software

projects, if we define a “defect” as “an instance where the implementation differs from the high-level requirement”, then the problem is reduced to that of a software project.• Throughout, we use the above definition of a defect (otherwise referred to as a

‘bug’).

Example: A typical development process

Business Analyst Programmer TesterUser

The User Knows what they want

The Analyst specifies what that is

The Programmer writes the code

The Tester tests the program

The further the visions diverge The greater the problems

Clarity and Vision during development


There are less bugs and the product is delivered faster

The closer the vision means the user gets a quality product

Clarity and Vision during development


Verify & Clarify Clear Logic Test Cases

Agile-Designer

Design Tests

Design Data

• Minimum• Optimal

• Find it• Make it The scope for ambiguity is

much less

The product is of higher quality and is delivered faster

Vision Convergence – Unambiguous Requirements

• The key to ‘converging the vision’ is making sure that the requirements are:• Unambiguous• Can be understood by everyone reading them.

• The first can be achieved by putting the requirements through a formal “ambiguity review” process, which involves analyzing the language used and making sure that the requirements can be interpreted in one and only one way.• The second is a bit more difficult to achieve, but can be done by sub-dividing

requirements such that:• Each sub-division level is more granular than the last.• Sub-requirements add up to the higher-level requirement.• Each sub-requirement is unambiguous.• Each sub-requirement can be understood by the appropriate person.

Examples of Ambiguities

• Dangling Else• All “If” clauses must have an “else”• Non-exhaustive case statements must have an “else”

• Ambiguity of Reference• Consistently use explicit variable names.

• Scope of Action• Use tables, explicit lists.

• Omissions• Structure and reviews improve completeness

• Ambiguous Logical Operators• Use restricted set• Use brackets


• Negation• Use brackets• Put negatives last in list

• Ambiguous Statements• Use explicit variable names• Use restricted set of qualifiers

• Random Organisation• Sequence set by “If, then, else” statements.

• Built-In Assumptions• Use restricted set of terms• Factor in environmental constraints• Add a glossary of terms


• Ambiguous Precedence Relationships• Decision making very structured

• Implicit Cases• Else clauses qualified with description

• Etc.• All variables explicit• All processes explicit

• i.e. versus e.g.• Correct usage defined

• Temporal Ambiguity• Timing explicit

• Boundary Ambiguity• Boundaries explicit

Sub-dividing Requirements

• Using the classical model of software requirements, outline the high-level requirements:

Highest Level Requirements

Account Management

Security

Payments

Integration With Orders System


• For each high-level requirement, sub-divide into lower-level requirements:

Highest Level Requirements Next Level

Account Management Add Account

Delete Account

….

Security Password security

Payment details security

…


• Continue (usually three or four levels of subdivision suffice) until the entire requirement has been fully specified:

Highest Level Next Level Lowest Level

Account Management Add Account Customer First Name

Customer Last Name

Customer Address

The Importance of Sub-division

• Sub-division allows for the business analyst to write a requirement in many different ways – so that the requirement can be understood by everyone involved from conception to implementation.• Granularity increases along the way – this eases the process of passing the vision

down the hierarchy (not unlike an organizational hierarchy)• For each requirement, it is critical that it’s constituent parts sum up to the whole:

this ensures that they say the same thing.• Ambiguity is strongly discouraged at every step: each ambiguity results in a

divergence from the vision.• Each level of granularity needs to be understood by the appropriate person – else

the whole exercise is a waste.

The Bottom Line – Ambiguities Results in Divergence

• Using the mathematical model used for cost projection, ambiguities result in increased margins of error.• Using the same reasoning, it can be shown that ambiguities directly results in

divergence from the high-level vision.• By the time the implementation is reviewed by the C-Level executive, it is too

late, and correcting the ‘defects’ will take time and money.• Delivering quality projects, on time and to specification, requires eliminating this

divergence – hence, better and less ambiguous requirements are the order of the day.

Thank you for listening

Cost Projection Mathematical Model

The User knows what they wantThe Analyst specifies what it isThe Programmer writes the codeThe Tester tests the solution


• For this analysis, we model ‘requirement sets’ simply as sets of requirements as understood by the particular group (e.g. the user has his set of interpretations, marked in red, while the analyst has his set of interpretations, marked in violet, and the intersection is what is understood commonly by the two). • We will be introducing the Hausdorff metric to measure the differences between

the interpretations of each group (basically, treating the sets as geometric spaces and then using the Hausdorff metric to measure distance between the spaces, as we would do a geometric space).


Let IU, IB, IP and IT denote the ‘requirement set’ or ‘interpretation’ of the user, BA, programmer and tester, respectively:Let

(i.e. the set of all interpretations)Then, for X, Y Θ, the Hausdorff metric is defined by:∈

Intuitively, this can be thought of as the maximum distance required to travel from subspace X to subspace Y given any arbitrary point.


Since X, Y are finite:

i.e. a restriction of dH to finite subspaces, since in finite subspaces:

For the entire space Θ, rework can be expressed as the effort required to minimize dH (ideally, to zero) for each pair of subspaces.


Define the work ωX,Y to be the amount of work it will take to minimize the Hausdorff metric between X and Y. It can be expressed as:

For some arbitrary work constant cX,Y

Let PΘ be the set of all pairs in Θ under equivalence (i.e. the direct square of Θ minus duplicates, so (X,Y) Θ and ∈ (Y,X) Θ are equivalent). Basically ∈ PΘ is equivalent to the set of equivalence classes of the direct square of Θ under the relation (X,Y) = (Y,X). For this example:


So the total work ΩΘ can be expressed as:

This forms the basis of the assertion that the re-work factor is proportional to the distances between each interpretation. This re-work factor will need to be factored into the variability of the overall cost – in our discussions, the standard deviations which we have been talking about are proportional to the re-work costs. In particular, given a single requirement R, the variability is proportional to:

ambiguous requirements – translating the message from c-level to implementation

Software

highlevel requirements

quality of requirements

unambigious requirements

subdividing requirements

project cost

nonsoftware projects

restricted set use brackets

clarity of vision