founda’ons of soware engineeringckaestne/17313/2017/20170907-metrics.pdf · visual studio since...
TRANSCRIPT
![Page 1: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/1.jpg)
Founda'onsofSo,wareEngineering
Lecture4:MeasurementClaireLeGoues
1
![Page 2: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/2.jpg)
Administrivia
• HW1aduetonight– Rememberteampolicy
• Reminderre:quizzes/readings
2
![Page 3: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/3.jpg)
LearningGoals• Usemeasurementsasadecisiontooltoreduceuncertainty
• Understanddifficultyofmeasurement;discussvalidityofmeasurements
• ExamplesofmetricsforsoIwarequaliKesandprocess
• UnderstandlimitaKonsanddangersofdecisionsandincenKvesbasedonmeasurements
3
![Page 4: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/4.jpg)
AboutYouCouldexplainCycloma'cComplexity
No
Vaguely
Yes
4
![Page 5: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/5.jpg)
CaseStudy:TheMaintainabilityIndex
5
![Page 6: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/6.jpg)
VisualStudiosince2007
“MaintainabilityIndexcalculatesanindexvaluebetween0and100thatrepresentstherelaKveeaseofmaintainingthecode.AhighvaluemeansbeGermaintainability.ColorcodedraKngscanbeusedtoquicklyidenKfytroublespotsinyourcode.Agreenra'ngisbetween20and100andindicatesthatthecodehasgoodmaintainability.Ayellowra'ngisbetween10and19andindicatesthatthecodeismoderatelymaintainable.Aredra'ngisara'ngbetween0and9andindicateslowmaintainability.”
6
![Page 7: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/7.jpg)
• Indexbetween0and100represenKngtherelaKveeaseofmaintainingthecode.
• HigherisbeZer.Colorcodedbynumber:– Green:between20and100– Yellow:between10and19– Red:between0and9.FromVisualStudio,since2007
7
![Page 8: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/8.jpg)
Designra'onal(fromMSDNblog)
• "WenoKcedthatascodetendedtoward0itwasclearlyhardtomaintaincodeandthedifferencebetweencodeat0andsomenegaKvevaluewasnotuseful."
• "Thedesirewasthatiftheindexshowedredthenwewouldbesayingwithahighdegreeofconfidencethattherewasanissuewiththecode."
hZp://blogs.msdn.com/b/codeanalysis/archive/2007/11/20/maintainability-index-range-and-meaning.aspx8
![Page 9: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/9.jpg)
TheIndex
MaintainabilityIndex=MAX(0,(171– 5.2*log(HalsteadVolume)– 0.23*(CyclomaKcComplexity)– 16.2*log(LinesofCode) )*100/171)
9
![Page 10: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/10.jpg)
LinesofCode• Easytomeasure
>wc–lfile1file2…
LOC projects
450 ExpressionEvaluator
2.000 Sudoku,FuncKonalGraphLibrary
40.000 OpenVPN
80-100.000 BerkeleyDB,SQLlight
150-300.000 Apache,HyperSQL,Busybox,Emacs,Vim,ArgoUML
500-800.000 gimp,glibc,mplayer,php,SVN
1.600.000 gcc
6.000.000 Linux,FreeBSD
45.000.000 WindowsXP10
![Page 11: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/11.jpg)
NormalizingLinesofCode• Ignorecommentsandemptylines• Ignorelines<2characters• PreZyprintsourcecodefirst• Countstatements(logicallinesofcode)
for(i=0;i<100;i+=1)print("hello");/*Howmanylinesofcodeisthis?*/
/*Howmanylinesofcodeisthis?*/for(
i=0;i<100;i+=1
){print("hello");
}11
![Page 12: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/12.jpg)
Normaliza'onperLanguageLanguage Statementfactor
(produc'vity)Linefactor
C 1 1
C++ 2.5 1
Fortran 2 0.8
Java 2.5 1.5
Perl 6 6
Smalltalk 6 6.25
Python 6.5
Source:hZp://www.codinghorror.com/blog/2005/08/are-all-programming-languages-the-same.htmlu.a.
12
![Page 13: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/13.jpg)
HalsteadVolume
• IntroducedbyMauriceHowardHalsteadin1977
• HalsteadVolume=numberofoperators/operands*log2(numberofdisKnct operators/operands)
• Approximatessizeofelementsandvocabulary
13
![Page 14: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/14.jpg)
HalsteadVolume-Example
main(){inta,b,c,avg;scanf("%d%d%d",&a,&b,&c);avg=(a+b+c)/3;print("avg=%d",avg);
}
Operators/Operands:main,(),{},int,a,b,c,avg,scanf,(),"…",&,a,&,b,&,c,avg,=,a,+,b,+,c,(),/,3,
print,(),"…",avg14
![Page 15: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/15.jpg)
Cycloma'cComplexity• ProposedbyMcCabe1976• Basedoncontrolflowgraph,measureslinearlyindependentpathsthroughaprogram– ~=numberofdecisions– Numberoftestcasesneededtoachievebranchcoverage
if(c1){f1();
}else{f2();
}if(c2){
f3();}else{
f4();}
M=edgesofCFG–nodesofCFG+2*endpoints15
![Page 16: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/16.jpg)
Origins• 1992PaperattheInternaKonalConferenceonSoIwareMaintenancebyPaulOmanandJackHagemeister
• DevelopersratedanumberofHPsystemsinCandPascal
• StaKsKcalregressionanalysistofindkeyfactorsamong40metrics
COM=percentageofcomments16
![Page 17: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/17.jpg)
Thoughts?• MetricseemsaZracKve• Easytocompute• OIenseemstomatchintuiKon
• Parametersseemalmostarbitrary,calibratedinsinglesmallstudycode(fewdevelopers,unclearstaKsKcalsignificance)
• Allmetricsrelatedtosize:justmeasurelinesofcode?
• Original1992C/PascalprogramspotenKallyquitedifferentfromJava/JS/C#code
hZp://avandeursen.com/2014/08/29/think-twice-before-using-the-maintainability-index/17
![Page 18: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/18.jpg)
Evalua'ngMetrics• ForeverymetricanswerthefollowingquesKons:
– Whatisthepurposeofthemeasure?– Whatisthescopeofthemeasure?– WhataZributearewetryingtomeasure?– WhatisthenaturalscaleoftheaZributewearetryingtomeasure?– WhatisthenaturalvariabilityoftheaZribute?– Whatisthemetric(measurementfuncKon)?Whatmeasuringinstrumentdo
weuse?– Whatisthenaturalscaleforthemetric?– Whatisthenaturalmeasurementerrorforthisinstrument?– WhatistherelaKonshipoftheaZributetothemetric?(constructvalidity)– Whatarethenaturalandforeseeablesideeffectsofusingthisinstrument?
Furtherreading:KanerandBond.So,wareEngineeringMetrics:WhatDoTheyMeasureandHowDoWeKnow?METRICS200418
![Page 19: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/19.jpg)
MeasurementforDecisionMakinginSo,wareDevelopment
19
![Page 20: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/20.jpg)
WhatisMeasurement?• Measurementistheempirical,objecKveassignmentofnumbers,accordingtoarulederivedfromamodelortheory,toaZributesofobjectsoreventswiththeintentofdescribingthem.–Craner,Bond,“SoIwareEngineeringMetrics:WhatDoTheyMeasureandHowDoWeKnow?”
• AquanKtaKvelyexpressedreducKonofuncertaintybasedononeormoreobservaKons.–Hubbard,“HowtoMeasureAnything…”
20
![Page 21: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/21.jpg)
So,warequalitymetric
IEEE1061says:“Aso&warequalitymetricisafunc5on
whoseinputsareso&waredataandwhoseoutputisasinglenumericalvaluethatcan
beinterpretedasthedegreetowhichso&wareprocessesagivena<ributethat
affectsitsquality.”
21
![Page 22: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/22.jpg)
MeasurementforDecisionMaking
• Fundproject?• MoretesKng?• Fastenough?Secureenough?• Codequalitysufficient?• Whichfeaturetofocuson?• Developerbonus?• TimeandcostesKmaKon?PredicKonsreliable?
22
![Page 23: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/23.jpg)
Whatso,warequali'esdowecareabout?(examples)• Scalability• Security• Extensibility• DocumentaKon• Performance• Consistency• Portability
• Installability• Maintainability• FuncKonality(e.g.,dataintegrity)
• Availability• Easeofuse
23
![Page 24: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/24.jpg)
Whatprocessquali'esdowecareabout?(examples)• On-Kmerelease• Developmentspeed• MeeKngefficiency• Conformancetoprocesses
• Timespentonrework• ReliabilityofpredicKons
• Fairnessindecisionmaking
• Measure5me,costs,ac5ons,resources,andqualityofworkpackages;comparewithpredic5ons
• Useinforma5onfromissuetrackers,communica5onnetworks,teamstructures,etc
• …
24
![Page 25: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/25.jpg)
Trendanalyses
25
![Page 26: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/26.jpg)
Benchmark-BasedMetrics
• Monitormanyprojectsormanymodules,gettypicalvaluesformetrics
• ReportdeviaKons
hZps://semmle.com/insights/26
![Page 27: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/27.jpg)
MeasurementisDifficult
27
![Page 28: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/28.jpg)
28
![Page 29: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/29.jpg)
Measurementsvalidity• Construct–Arewemeasuringwhatweintendedtomeasure?
• PredicKve–TheextenttowhichthemeasurementcanbeusedtoexplainsomeothercharacterisKcoftheenKtybeingmeasured
• Externalvalidity–ConcernsthegeneralizaKonofthefindingstocontextsandenvironments,otherthantheonestudied
29
![Page 30: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/30.jpg)
Everythingismeasurable1. IfXissomethingwecareabout,thenX,bydefiniKon,
mustbedetectable.– Howcouldwecareaboutthingslike“quality,”“risk,”“security,”or“publicimage”ifthesethingsweretotallyundetectable,directlyorindirectly?
– IfwehavereasontocareaboutsomeunknownquanKty,itisbecausewethinkitcorrespondstodesirableorundesirableresultsinsomeway.
2. IfXisdetectable,thenitmustbedetectableinsomeamount.
– Ifyoucanobserveathingatall,youcanobservemoreofitorlessofit
3. Ifwecanobserveitinsomeamount,thenitmustbemeasurable.
D.Hubbard,HowtoMeasureAnything,2010
30
![Page 31: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/31.jpg)
31
![Page 32: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/32.jpg)
Thestreetlighteffect• AknownobservaKonalbias.
• Peopletendtolookforsomethingonlywhereit’seasiesttodoso.– Ifyoudropyourkeysatnight,you’lltendtolookforitunderstreetlights.
32
![Page 33: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/33.jpg)
33
![Page 34: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/34.jpg)
Whatcouldpossiblygowrong?• BadstaKsKcs:Abasicmisunderstandingofmeasurementtheoryandwhatisbeingmeasured.
• Baddecisions:Theincorrectuseofmeasurementdata,leadingtounintendedsideeffects.
• BadincenKves:Disregardforthehumanfactors,orhowtheculturalchangeoftakingmeasurementswillaffectpeople.
34
![Page 35: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/35.jpg)
Lies,damnedlies,and…
• In1995,theUKCommiZeeonSafetyofMedicinesissuedthefollowingwarning:"third-generaKonoralcontracepKvepillsincreasedtheriskofpotenKallylife-threateningbloodclotsinthelegsorlungstwofold--thatis,by100percent”
35
![Page 36: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/36.jpg)
…sta's'cs• “…ofevery7,000womenwhotooktheearlier,second-generaKonoralcontracepKvepills,aboutonehadathrombosis;thisnumberincreasedtotwoamongwomenwhotookthird-generaKonpills…”
• “…Theabsoluteriskincreasewasonlyonein7,000,whereastherela'veincrease(amongwomenwhodevelopedbloodclots)wasindeed100percent.”
36
![Page 37: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/37.jpg)
Understandingyourdata
37
![Page 38: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/38.jpg)
Measurementscales• Scale:thetypeofdatabeingmeasured.• Thescaledictateswhatsortsofanalysis/arithmeKcislegiKmateormeaningful.
• YouropKonsare:– Nominal:categories– Ordinal:order,butnomagnitude.– Interval:order,magnitude,butnozero.– RaKo:Order,magnitude,andzero.– Absolute:specialcaseofraKo.
38
![Page 39: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/39.jpg)
Summaryofscales
39
![Page 40: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/40.jpg)
Nominal/categoricalscale• EnKKesclassifiedwithrespecttoacertainaZribute.
CategoriesarejointlyexhausKveandmutuallyexclusive.– Noimpliedorderbetweencategories!
• Categoriescanberepresentedbylabelsornumbers;however,theydonotrepresentamagnitude,arithme5copera5onhavenomeaning.
• CanbecomparedforidenKtyordisKncKon,andmeasurementscanbeobtainedbycounKngthefrequenciesineachcategory.Datacanalsobeaggregated.
En'ty AGribute Categories
ApplicaKon Purpose E-commerce,CRM,Finance
ApplicaKon Language Java,Python,C++,C#
Fault Source assignment,checking,algorithm,funcKon,interface,Kming
40
![Page 41: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/41.jpg)
Ordinalscale• Orderedcategories:mapsameasuredaZributetoanordered
setofvalues,butnoinformaKonaboutthemagnitudeofthedifferencesbetweenelements.
• Measurementscanberepresentedbylabelsornumbers,BUT:ifnumbersareused,theydonotrepresentamagnitude.– Honestly,trynottodothat.IteliminatestemptaKon.
• Youcannot:add,subtract,performaverages,etc(arithmeKcoperaKonsareout).
• Youcan:comparewithoperators(like“lessthan”or“greaterthan”),createranksforthepurposesofrankcorrelaKons(Spearman’scoefficient,Kendall’sτ).
En'ty AGribute Values
ApplicaKon Complexity VeryLow,Low,Average,High,VeryHigh
Fault Severity 1–CosmeKc,2–Moderate,3–Major,4–CriKcal
41
![Page 42: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/42.jpg)
Intervalscale• Hasorder(likeordinalscale)andmagnitude.
– TheintervalsbetweentwoconsecuKveintegersrepresentequalamountsoftheaZributebeingmeasured.
• DoesNOThaveazero:0isanarbitrarypoint,anddoesn’tcorrespondtotheabsenceofaquanKty.
• MostarithmeKc(addiKon,subtracKon)isOK,asaremeananddispersionmeasurements,asarePearsoncorrelaKons.RaKosarenotmeaningful.– Ex:Thetemperatureyesterdaywas64oF,andtodayis32oF.Istodaytwiceascoldasyesterday?
• Incrementalvariables(quanKtyasoftoday–quanKtyatanearlierKme)andpreferencesarecommonlymeasuredinintervalscales.
42
![Page 43: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/43.jpg)
Ra'oscale• AnintervalscalethathasatruezerothatactuallyrepresentstheabsenceofthequanKtybeingmeasured.
• AllarithmeKcismeaningful.• Absolutescaleisaspecialcase,measurementsimplymadebycounKngthenumberofelementsintheobject.– Takestheform“numberofoccurrencesofXintheenKty.”
En'ty AGribute Values
Project Effort Realnumbers
SoIware Complexity CyclomaKccomplexity
43
![Page 44: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/44.jpg)
• ForcausaKon– Provideatheory(fromdomainknowledge,independentofdata)
– ShowcorrelaKon– Demonstrateabilitytopredictnewcases(replicate/validate)
hZp://xkcd.com/552/44
![Page 45: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/45.jpg)
45
![Page 46: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/46.jpg)
Confoundingvariables
– IfyoulookonlyatthecoffeeconsumpKon→cancerrelaKonship,youcangetverymisleadingresults
– Smokingisaconfounder
CoffeeconsumpKon
Smoking
Cancer
AssociaKons
CausalrelaKonship
46
![Page 47: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/47.jpg)
EffectofClassSizeontheValidityofObject-orientedMetrics
KhaledElEmam,SaidaBenlarbi,andNishithGoel,September1999
Onlyfour,outoftwenty-fourcommonlyusedobject-orientedmetrics,wereactuallyusefulinpredicKngthequalityofasoIwaremodulewhentheeffectofthemodulesizewasaccountedfor.
47
![Page 48: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/48.jpg)
48
![Page 49: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/49.jpg)
TheMcNamaraFallacy• Thereseemstobeageneralmisunderstandingtotheeffect
thatamathemaKcalmodelcannotbeundertakenunKleveryconstantandfuncKonalrelaKonshipisknowntohighaccuracy.ThisoIenleadstotheomissionofadmiZedlyhighlysignificantfactors(mostofthe“intangibles”influencesondecisions)becausetheseareunmeasuredorunmeasurable.Toomitsuchvariablesisequivalenttosayingthattheyhavezeroeffect...Probablytheonlyvalueknowntobewrong…– J.W.Forrester,IndustrialDynamics,TheMITPress,1961
49
![Page 50: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/50.jpg)
McNamarafallacy
1. Measurewhatevercanbeeasilymeasured.
2. Disregardthatwhichcannotbemeasuredeasily.
3. Presumethatwhichcannotbemeasuredeasilyisnotimportant.
4. Presumethatwhichcannotbemeasuredeasilydoesnotexist.
hZps://chronotopeblog.com/2015/04/04/the-mcnamara-fallacy-and-the-problem-with-numbers-in-educaKon/50
![Page 51: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/51.jpg)
DefectDensity• Defectdensity=Knownbugs/lineofcode• Systemspoilage=Kmetofixpost-releasedefects/
totalsystemdevelopmentKme• Post-releasevspre-release• Whatcountedasdefect?Severity?Relevance?• Whatsizemetricused?• Whatqualityassurancemechanismsused?
• LiZlereferencedatapubliclyavailable;typically2-10defects/1000linesofcode
51
![Page 52: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/52.jpg)
MeasuringUsability
52
![Page 53: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/53.jpg)
Measurementstrategies
• Automatedmeasuresoncoderepositories
• Useorcollectprocessdata• Instrumentprogram(e.g.,in-fieldcrashreports)
• Surveys,interviews,controlledexperiments,expertjudgment
• StaKsKcalanalysisofsample53
![Page 54: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/54.jpg)
MetricsandIncen'ves
54
![Page 55: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/55.jpg)
hZp://dilbert.com/strips/comic/1995-11-13/
55
![Page 56: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/56.jpg)
56
![Page 57: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/57.jpg)
Produc'vityMetrics
• Linesofcodeperday?– Industryaverage10-50lines/day– Debugging+reworkca.50%ofKme
• FuncKon/object/applicaKonpointspermonth
• Bugsfixed?• Milestonesreached?
57
![Page 58: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/58.jpg)
StackRanking
58
![Page 59: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/59.jpg)
Incen'vizingProduc'vity
• Whathappenswhendeveloperbonusesarebasedon– Linesofcodeperday– AmountofdocumentaKonwriZen– Lownumberofreportedbugsintheircode– Lownumberofopenbugsintheircode– Highnumberoffixedbugs– AccuracyofKmeesKmates
59
![Page 60: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/60.jpg)
AutonomyMasteryPurpose
CanexKnguishintrinsicmoKvaKonCandiminishperformance
CancrushcreaKvityCancrowdoutgoodbehavior
CanencouragecheaKng,shortcuts,andunethicalbehaviorCanbecomeaddicKve
Canfostershort-termthinking60
![Page 61: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/61.jpg)
Tempta'onofSo,wareMetrics
61
![Page 62: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/62.jpg)
So,wareQualityMetrics
• IEEE1061definiKon:“AsoIwarequalitymetricisafuncKonwhoseinputsaresoIwaredataandwhoseoutputisasinglenumericalvaluethatcanbeinterpretedasthedegreetowhichsoIwareprocessesagivenaZributethataffectsitsquality.”
• MetricshavebeenproposedformanyqualityaZributes;maydefineownmetrics
62
![Page 63: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/63.jpg)
ExternalaGributes:MeasuringQuality
McCall model has 41 metrics to measure 23 quality criteria from 11 factors
63
![Page 64: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/64.jpg)
Decomposi'onofMetrics
Maintainability
Correctability
Testability
Expandability
Faultscount
DegreeoftesKng
Effort
Changecounts
ClosureKmeIsolate/fixKmeFaultrateStatementcoverageTestplancompletenessResourcepredicKonEffortexpenditureChangeeffortChangesizeChangerate
64
![Page 65: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/65.jpg)
Object-OrientedMetrics
• NumberofMethodsperClass• DepthofInheritanceTree• NumberofChildClasses• CouplingbetweenObjectClasses• CallstoMethodsinUnrelatedClasses• …
65
![Page 66: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/66.jpg)
Otherqualitymetrics?
• Commentdensity• Testcoverage• Componentbalance(systembreakdownopKmalityandcomponentsizeuniformity)
• Codechurn(numberoflinesadded,removed,changedinafile)
• …66
![Page 67: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/67.jpg)
Warning• MostsoIwaremetricsarecontroversial
– Usuallyonlyplausibilityarguments,rarelyrigorouslyvalidated– CyclomaKccomplexitywasrepeatedlyrefutedandissKllused– “SimilartotheaZemptofmeasuringtheintelligenceofapersonin
termsoftheweightorcircumferenceofthebrain”
• Usecarefully!• Codesizedominatesmanymetrics• Avoidclaimsabouthumanfactors(e.g.,readability)and
quality,unlessvalidated• Calibratemetricsinprojecthistoryandotherprojects• Metricscanbegamed;yougetwhatyoumeasure
67
![Page 68: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/68.jpg)
(Some)strategies• Metricstrackedusingtoolsandprocesses(processmetricslikeKme,orcodemetricslikedefectsinabugdatabase).
• Expertassessmentorhuman-SubjectExperiments(controlledexperiments,talk-aloudprotocols).
• MiningsoIwarerepositories,defectdatabases,especiallyfortrendanalysisordefectpredicKon.– Somesuccesse.g.,asreportedbyMicrosoIResearch
• Benchmarking(especiallyforperformance).
68
![Page 69: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/69.jpg)
Factorsinasuccessfulmeasurementprogram1. SetsolidmeasurementobjecKvesandplans.2. Makemeasurementpartoftheprocess.3. Gainathoroughunderstandingof
measurement.4. Focusonculturalissues.5. Createasafeenvironmenttocollectand
reporttruedata.6. CulKvateapredisposiKontochange.7. Developacomplementarysuiteofmeasures.
CarolA.DekkersandPatriciaA.McQuaid,“TheDangersofUsingSoIwareMetricsto(Mis)Manage”,2002.69
![Page 70: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/70.jpg)
Kaner’sques'onswhenchoosingametric1. Whatisthepurposeofthis
measure?2. Whatisthescopeofthis
measure?3. WhataZributeareyoutryingto
measure?4. WhatistheaZribute’snatural
scale?5. WhatistheaZribute’snatural
variability?6. Whatinstrumentareyouusing
tomeasuretheaZribute,andwhatreadingdoyoutakefromtheinstrument?
CemKanerandWalterP.Bond.“SoIwareEngineeringMetrics:WhatDoTheyMeasureandHowDoWeKnow?”2004
7. Whatistheinstrument’snaturalscale?
8. Whatisthereading’snaturalvariability(normallycalledmeasurementerror)?
9. WhatistheaZribute’srelaKonshiptotheinstrument?
10. Whatarethenaturalandforeseeablesideeffectsofusingthisinstrument?
70
![Page 71: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/71.jpg)
Summary• Measurementisdifficultbutimportantfordecisionmaking
• SoIwaremetricsareeasytomeasurebuthardtointerpret,validityoIennotestablished
• Manymetricsexist,oIencomposed,pickordesignsuitablemetricsifneeded
• Carefulinuse:monitoringvsincenKves• Strategiesbeyondmetrics
71
![Page 72: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/72.jpg)
FurtherReadingonMetrics• Sommerville.SoIwareEngineering.EdiKon7/8,SecKons26.1,27.5,and28.3
• Hubbard.Howtomeasureanything:Findingthevalueofintangiblesinbusiness.JohnWiley&Sons,2014.Chapter3
• KanerandBond.SoIwareEngineeringMetrics:WhatDoTheyMeasureandHowDoWeKnow?METRICS2004
• FentonandPfleeger.SoIwareMetrics:Arigorous&pracKcalapproach.ThomsonPublishing1997
72
![Page 73: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/73.jpg)
Microso,Survey(2014)
• "SupposeyoucouldworkwithateamofdatascienKstsanddataanalysistswhospecializeinstudyinghowsoIwareisdeveloped.PleaselistuptofivequesKonsyouwouldlikethemtoanswer.Whydoyouwanttoknow?Whatwouldyoudowiththeanswers?"AndrewBegelandThomasZimmermann."Analyzethis!145quesKonsfordatascienKstsinsoIwareengineering."ICSE.2014.
73
![Page 74: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/74.jpg)
TopQues'ons• HowdouserstypicallyusemyapplicaKon?• WhatpartsofasoIwareproductaremostusedand/orlovedbycustomers?
• HoweffecKvearethequalitygateswerunatcheckin?
• HowcanweimprovecollaboraKonandsharingbetweenteams?
• Whatarebestkeyperformanceindicators(KPIs)formonitoringservices?
• Whatistheimpactofacodechangeorrequirementschangetotheprojectandtests?
74
![Page 75: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/75.jpg)
TopQues'ons• WhatistheimpactoftoolsonproducKvity?• HowdoIavoidreinvenKngthewheelbysharingand/orsearchingforcode?
• WhatarethecommonpaZernsofexecuKoninmyapplicaKon?
• Howwelldoestestcoveragecorrespondtoactualcodeusagebyourcustomers?
• WhatkindsofmistakesdodevelopersmakeintheirsoIware?Whichonesarethemostcommon?
• WhatareeffecKvemetricsforshipquality?
75
![Page 76: Founda’ons of Soware Engineeringckaestne/17313/2017/20170907-metrics.pdf · Visual Studio since 2007 “Maintainability Index calculates an index value between 0 and 100 that represents](https://reader033.vdocuments.mx/reader033/viewer/2022060212/5f04f11f7e708231d4107944/html5/thumbnails/76.jpg)
BoGomQues'ons• Whichindividualmeasurescorrelatewithemployee
producKvity(e.g.,employeeage,tenure,engineeringskills,educaKon,promoKonvelocity,IQ)?
• WhichcodingmeasurescorrelatewithemployeeproducKvity(e.g.,linesofcode,KmeittaketobuildthesoIware,aparKculartoolset,pairprogramming,numberofhoursofcodingperday,language)?
• Whatmetricscanbeusedtocompareemployees?• HowcanwemeasuretheproducKvityofaMicrosoI
employee?• Isthenumberofbugsagoodmeasureofdeveloper
effecKveness?• CanIgenerate100%testcoverage?
76