lecture 17 “does every inspection need a meeting?” for0383 software quality assurance...

18
Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 06/17/22 1 Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection Need a Meeting? Proceedings of the first ACM symposium on the Foundations of Sofware Engineering, SIGSOFT’93, pp 107- 114. ©ACM n = 4

Upload: emil-barnett

Post on 12-Jan-2016

230 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

Lecture 17“Does every inspection need a meeting?”

FOR0383 Software Quality Assurance

04/21/23 1Dr Andy Brooks

Source: Votta, L G (1993) Does Every Inspection Need a Meeting? Proceedings of the first ACM symposium on the Foundations of Sofware Engineering, SIGSOFT’93, pp 107-114. ©ACM

n = 4

Page 2: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 2

The survey data

Reason Authors & Managers (29)

Synergy 79% (23)

Education 62% (18)

Schedule 48% (14)

Competition 34% (10)

Requirement 28% (8)

Qu: Why do you think inspection meetings are held?Responses coded from answers.All other reasons given occurred with a frequency of less than 20%.

Always provide data about what a % means.

Page 3: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 3

Top five reasons why meetings are held.

1. SynergyThe idea is that working together as team produces a better performance than the sum of the performance of individual team members. Synergy has been clearly demonstrated in many areas. Planting a tree as an individual is difficult, but not if you work in pairs.

However, synergy has not been clearly demonstrated for inspection meetings.

“If this effect occurred in the software inspection process, we would see many inspection faults found only by holding a meeting.”

samvirkni

Page 4: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 4

2. Education

The idea is that new employees learn from experienced reviewers how the software development process works, and what makes a good reviewer.

However, this might be “the blind leading the blind”. Sending employees on proper training courses about inspections might well be more efficient and effective.

Top five reasons why meetings are held.

Page 5: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 5

3. Schedule deadlines

People have a planned meeting to work towards.

Scheduling events such as inspection meetings helps with the time management of a project.

Top five reasons why meetings are held.

Page 6: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 6

4. Competition

Employees make their contribution to defect finding in public and so can earn the respect of their peer group. Employees try to improve their inspection performance.

However, managers can easily destroy teamwork by rewarding individual performance.

“Why should I devote a lot of time to inspecting, if it is always Joe who gets the salary bonus?”.

Top five reasons why meetings are held.

Page 7: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 7

5. Requirement

Holding a meeting to collect individual reviewer defects is required by the inspection process.

Having a well-defined process helps with the management of a project.

Top five reasons why meetings are held.

Page 8: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

Figure 1. Typical Time Line for Inspection Process ©ACM

21.04.23 Dr Andy Brooks 8

“Typical values in a large software development for a feature of 600 NCSL are 10, 10, and 10 working days for Tinspection, Tcollection, and Trepair, respectively.”

L G Votta

Page 9: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 9

The development interval

• The longer money is borrowed, the more difficult it is to realise a profit.

• You get less and less responsive to customer needs and marketing conditions as you take longer and longer to develop something.

• Waiting for the collection meeting (Tcollection) to happen can be very costly.– hazard cost of being late to market– carrying cost if the author is not working on the product– rework cost if the author continues working on the product only

to have this work invalidated by the results of the inspection meeting

TinspectionTcollectionTrepair TinspectionTcollectionTrepair TinspectionTcollectionTrepair..................

Page 10: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 10

When can n people have a meeting?• Suppose

– each day comprises 4 two-hour slots

– m is the average number of meetings per day

– p is the probability a two hour slot is occupied by a meeting (= m/4)

– q is the probability that a two hour slot is unoccupied (= 1-p)

– assume p is the same for each person and independent

• The probability of any slot being available for n people is:

• The average number of days that elapse before the meeting occurs is:

nm

41

14

14

1n

m

n = 5

Page 11: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 11

© ACM

Keep the number of meetings to a minimum and the number attending small.Exception: architectural design meetings for a large system.

Average number of meetings per day.

Page 12: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 12

Is there enough synergy?

Each point represents the number of faults undetected in individual preparation divided by the total number recorded at an inspection meeting.

4% average

© ACM

Always provide information about what an error bar means.

actual measurements

Page 13: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 13

The severity of a defect is important.

• Even if the inspection meeting yields only another 4%, the meeting will be invaluable if you find a showstopper.– A showstopper is a defect that will cause a system crash in the field

(such as null pointer dereference).

• Holding inspection meetings is, however, unnecessary if only additional minor defects are found.– This is especially true if static analysis tools can detect all the minor

defects (violations of coding conventions, for example).

• Inspection data needs to be recorded and analysed to help make decisions about the value of holding an inspection meeting.

• Common sense can also be applied:– “...this part of the requirements, design or code is so important that

we should have an inspection meeting.”

Page 14: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 14

False positives and lost defects.

• During individual preparation, inspectors can soemtimes report issues that are not defects (false positives).– An inspection meeting usually eliminates most of the

false positives.

• During the inspection meeting, some issues found in individual preparation or at the meeting do not get recorded (lost defects).– Lost defects typically represent a small percentage of

the total. (L G Votta quotes a figure of 4%.)• ... the benefit from synergy was measured at 4%.

Page 15: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 15

What are the alternatives to holding an inspection meeting?

Deposition Meetings• The moderator and author meet with single reviewers. (3

person meetings.)• There is no reading activity: a deposition meeting collects

the defects found from the reviewer.• The moderator acts as the recorder.• The moderator and author have to attend more meetings.• There is no meeting schedule delay as depositions can

be taken in a day.

Page 16: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 16

What are the alternatives to holding an inspection meeting?

Direct deposition hand over• The moderator collects all the individual logs.• The moderator can simply hand over the logs to

the author.• Or, the moderator creates a master defect report

which is then handed over to the author.

Page 17: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

• The author and reviewers never physically meet and can even be located in different countries.

• Reviewers can use e-mail to report defects.• Reviewers can use general or special purpose

groupware to report defects.

21.04.23 Dr Andy Brooks 17

What are the alternatives to holding an inspection meeting?

Correspondence

IBIS - Internet Based Inspection Systemhttp://sourceforge.net/projects/ibis/

Adobe Acrobat 9http://www.adobe.com/products/acrobat/

Page 18: Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection

21.04.23 Dr Andy Brooks 18

Inspection data needs to be recorded and analysed to help make decisions about the value of holding an inspection meeting.

What is the synergy rate?Are showstoppers being found?What is the lost defect rate?How big is the meeting schedule delay?

Common sense can be applied: “...this part of the requirements, design or code is so important that we should have an inspection meeting.”

almenn skynsemi