A  FUNCTIONAL  SOFTWARE  MEASUREMENT  APPROACH  BRIDGING  THE  GAP  BETWEEN  PROBLEM  AND  SOLUTION  DOMAINS

by  Dr.Erdir  Ungan,  Prof.Dr.  Onur  Demirörs

Problem

•   Granularity    •   Issues  in  Parametric  Es3ma3on  Methods  •   Issues  in  Benchmarking  •   Reliability  Issues  in  FSM  Measurements  •   Issues  Regarding  Measurement  Procedure  

Anything  you  try  to  quan.fy  can  be  divided  into  any  number  of  "anythings,"  or  become  the  thing  -­‐  the  unit  -­‐  itself.  And  what  is  any  number,  itself,  but  just  another  unit  of  measurement?    

What  is  a  'six'  but  two  'threes',  or  three  'twos'...half  a  'twelve',  or  just  six  'ones'  -­‐  which  are  what?  

-­‐  F.L.  Vanderson  

Issues  in  Parametric  EsJmaJon  Methods

•   Most  es3ma3on  methods  suggest    ◦  predic3ng  soEware  size    ◦  Measure  PD  size  and  convert  it  to  SD  size  based  on  some  historical  data.  ◦  ….Huge  varience.  

• The  process  of  developing  a  solu3on  to  a  problem  is  ambiguous  and  soE.  Therefore,  es3ma3on  methods  in  PD  need    subjec3ve  input  ◦  This  is  one  of  the  points  where  the  reliability  and  repeatability  of  these  methods  are  ques3oned.  

Issues  in  Benchmarking

•   Problems  in  measurement  results  and  granularity  also  affects  the  quality  of  data  in  benchmarking  data  sets.  

•   Özcan  Top  and  Yilmaz  concluded  that,  those  sets  lack  structural  informa3on  about  the  projects  

•   We  cannot  deduct  informa3on  about  the  abstrac3on  level  of  measurements  in  those  data  sets  

•   Comparing  data  from  varying  abstrac3on  levels  will  result  in  erroneous  benchmarking  

SeparaJon  of  Problem  and  SoluJon  Domains

The  difference  between  theory  and  prac2ce  equals  your   inep2tude.. .

Anonymous

What  vs.  How

◦  How,  is  the  process  of  breaking  down  a  larger  What  to  smaller  What’s.  

◦ What1  ◦  What  1.1  ◦  What  1.2  

◦  What  1.2.1  ◦  What  1.2.2  

◦  What  1.2.3  ◦  What  1.2.3.1  ◦  .  ◦  .  

◦  One  of  these  transi3ons  in  the  W-­‐H  chain  will  pose  as  a  boundary  between  Problem  and  Solu3on  domains  based  on  the  defini3on  of  the  system  at  hand.    

What  vs.  How •   Increase  Profit    •   …  •   Decrease  Costs  •   Increase  Sales  ◦  ...  ◦  Increase  Adver3sement  ◦  Establish  an  online  store  

◦  …  ◦  Get  a  Server  ◦  Build  an  e-­‐commerce  site  

◦  Design  Web  Site  ◦  Test  Web  site  ◦  Develop  Web  Site  

◦  Develop  Interface  ◦  Develop  Client  Side  SoEware  ◦  Develop  Server  Side  SoEware  

•   Develop  Database  •   Develop  Data  Abstrac3on  Layer  •   Develop  Business  Layer  

◦  Develop  Admin  Module  ◦  Develop  Stock  Module  ◦  Develop  Customer  Module  

◦  Develop  membership  func3ons  ◦  Develop  CRM  func3ons  

◦  Develop  Customer  Class  ◦  Develop  Purchases  Class  

◦  Develop  add  purchase  method  ◦  Develop  get  purchasename  method  

PD  &  SD  in  SoQware  Engineering

Mutual  Independence  of  PD  &  SD

• Although  common  sense  and  experience  dictates  that  PD  and  SD  are  two  separate  domains.  We  needed  to  demonstrate  this  phenomenon.    

• We  conducted  two  case  studies  ◦  First  Study:  We  inves3gated  the  correla3on  of  problem  and  solu3on  domain  sizes  with  the  problem  and  solu3on  domain  effort  

◦  Second  Study:    We  conducted  a  case  study  where  a  single  set  of  requirements  were  to  be  developed  using  two  different  implementa3on  approaches  

• Both  studies  supported  our  sugges3ons  about  problem  and  solu3on  domain  separa3on  

Mutual  Independence  of  PD  &  SD

    Bytes  /  CFP       Std.  Dev.   R2  Comfort  &  Convenience  Type  Components   7.6   0.99  

Display  &  Indica3on  Type  Components   21.6   0.992  

For  single  organiza3on’s  data  set,  varia3on  and  correla3on  in  Physical  size  (Bytes)  to  

COSMIC  size  is  given  in  

(Gencel,  Heldal,  and  Lind  2009)  

    SLOC  /  CFP       Std.  Dev.   R2  Comfort  &  Convenience  Type  Components   8.2   0.4855  

    15.4   0.417  

For  single  organiza3on’s  data  set,  varia3on  and  correla3on  in  SLOC  sizes  to  COSMIC  

Context

Context

◦ We  define  Gaps  1-­‐2-­‐3-­‐5  and  6  as  imperfec3ons  and  therefore  evitable.    

◦ However,  we  believe,  ideally  there  is:  ◦  One  correct  and  exact  defini3on  of  the  real  life  problem.  ◦  One  correct  set  of  requirements  that  describe  the  problem.  Conforming  to  a  requirements  standard.  

◦  One  correct  set  of  specifica3ons  that  breaks  down  the  requirements  to  lower  level  specifica3ons.  

◦  One  best  implementa3on  of  the  design  based  on  the  technology  and  implementa3on  method  used.  

◦  One  best  and  unique  soEware  build  for  the  implementa3on.  Through  an  ideal  compiler.      

Context

•   However,  Gap  4,  the  gap  between  specifica3ons  and  design  is  different  in  nature  from  these  gaps.  “There  is  no  one  correct  solu3on  to  a  problem.”    

•   Engineering  solu3ons  are  guided  by  many  principles,  design  palerns  and  rules.  Nevertheless,  there  is  always  a  subjec3ve,  crea3ve  element  in  crea3ng  solu3ons  to  problems.  Therefore  this  gap  is  essen3ally  inevitable.    

•   The  transforma3on  between  specifica3ons  and  design  is  actually  where  we  define  the  border  between  the  problem  domain  and  solu3on  domain  is.    

Context

SoluJon  Approach

SoluJon  Approach

•   U3lize  a  Common  Concept  For  PD  &  SD  •   Es3ma3on  Accuracy  is  Based  on  Level  of  Detail  •   Incorpora3ng  Decomposi3on  Level  (Granularity)  Into  Measurement  •   Defining  an  Absolute  Reference  Point    

•   Automated  Measurement  •   Minimizing  Interpreta3ons  and  Assump3ons    

Proposed  Method

• We  propose  a  two  dimensional  measurement:  ◦ Data  movement  ◦ Decomposi3on  Level  

Therefore,  we  define  each  aspect  of  a  measurement  method  in  these  two  dimensions.  

Conceptual  Modeling

Increasing  Decomposi3on    Decreasing  Tier  #  

Tier  •   Each  decomposi3on  of  the  system  will  result  in  a  new  Tier  of  system  defini3on.  Each  increase  in  the  3er  number  represents  one  higher  level  of  decomposi3on  traversed  in  the  descrip3on  of  the  system.    

•   Each  3er  consists  of  structural  en33es  communica3ng  with  same  3er  level  of  en33es.    

Note:  One  set  of  objects  in  a  3er  can  be  present  in  a  higher  level  if  the  entry  and  exit  point  of  data  movements  between  them  and  other  objects  does  not  change  on  their  end.  That  is,  certain  object  may  belong  to  more  than  one  3er  at  the  same  3me.  

Measurement  Method

Etalon  for  Func=onal  Size  Abran  states  that,    • “The  key  concept  of  func3onality  at  the  highest  level  of  commonality  that  is  present  in  all  soEware  was  iden3fied  as  the  ‘data  movement’.  This  data  movement  concept  was  then  assigned  to  the  metrology  concept  of  a  size  unit.”  • The  base  unit  of  decomposi3on  level  is  defined  as  the  level  of  a  single  method  within  a  single  class.    • The  base  unit  of  decomposi3on  level  is  defined  as  Tier  0.  Tier  0  is  the  level  of  decomposi3on  in  which  further  level  of  decomposi3on  will  be  irrelevant  of  the  measurement  objec3ves  of  DMP  method.  (eg.  Effort  correla3on)  

Conceptual  Modeling

•   We  chose  sequence  diagram  as  the  main  soEware  model  for  the  empirical  world  to  measure  func3onal  size  alribute.  As:    

◦ Possible  and  automated  round  trip  engineering.  Code  –  Seq.Diagam  &  Seq.Diagram  –  Code.  Many  CASE  tools  and  IDEs.    

◦ Can  be  defined  in  each  composi3on  level.    ◦ Are  being  developed  for  business  level  behavior  and  design  level  behavior.  That  is,  they  can  be  u3lized  in  both  problem  and  solu3on  domains.  

◦  Incorporate  more  informa3on  about  a  system  than  other  UML  diagrams.  informa3on  about  the  structures  in  a  system  as  well  as  their  rela3ons  and  system  behavior.    

Measurement  Method

Opera=onal  View  for  Func=onal  Size  

• In  DMP  measurement  method,  sequence  diagram  elements  which  represent  the  data  movements  are  taken  into  account  and  other  elements  are  used  for  suppor3ng  informa3on.    

• Sequence  Diagram  elements  that  represent  data  movement  are:  ◦  Synchronous  message  ◦  Asynchronous  message  ◦  Crea3on  message  ◦  Destruc3on  message  ◦  Self  message  ◦  Found  message  

Mapping  Phase  

Iden=fying  Scenarios    

Based  on  the  decomposi3on  level  (Tier  Value)  for  the  measurement,  defini3on  of  the  func3onal  processes  and  their  triggering  entries  change.  

1.  The  triggering  entry  of  the  func3onal  process  must  be  visible  in  the  system  model  defined  at  this  level.    

2.  The  structural  en3ty  receiving  the  triggering  entry  must  be  visible  in  the  system  model  defined  at  this  level.  

3.  The  output  of  the  process  (Exit  or  Write)  must  be  visible  in  the  system  model  defined  at  this  level.  

Measurement  Tool

•   Developed  as  a  master  project  with  Yalın  Meriç  supervised  by  Onur  Demirörs,  Erdir  Ungan  (2013)    The  automated  XMI  genera3on  process  was  as  below:    1.  Select  programming  language.  2.  Select  folder  where  source  code  files  are  present.  3.  Automa3cally  generate  all  sequence  diagrams  for  imported  source  

codes.  4.  Create  and  save  XMI  file.  5.  Count  data  movements  for  each  method.  6.  Assign  objects  to  3er  level(s)  7.  Enter  3er  level  8.  Consolidate  DMs  in  that  level  and  measure.  

11  Data  Movements  

A1

A3

A2

B1 C1

C2

D1

D2

A1

A3

A2

B1 C1

C2

D1

D2

A

B

C

D

7  Data  Movements

2  Data  Movements

System

ValidaJon

•   Theore3cal  Valida3on  

•   Empirical  Valida3on  

ValidaJon

•   Theore3cal  Valida3on  ◦ Rice  Cooker  Case  ◦ Lavazza  et  al.  and  Levesque  et  al.  separately  measured  the  system  with  sequence  diagrams.    

◦ They  reached  different  measurement  conclusions.    ◦ They  modelled  the  func3onality  differently.  

No. Functional Process

Triggering Event Data Movement Description

Data Group DM CFP

1 Control Cooking Lamp

Tick (elapsed) Tick (elapsed) TimedEvents E 4

Cooking Mode Cooking Mode R

Cooking Status CookingLamp X

Cooking Status WarningLamp X

2 5 sec. signal management (control heater)

Every 5s signal Signal5 TimedEvents E 4 GetTargetTemp CookingState R ReadTemp TemperatureSens

or E

HeaterOn or

Heater command X

3 set target temperature

Signal30 Signal30 TimedEvents E 3

GetMode CookingMode R

SetTargetTemp CookingState W

4 Select Cooking Mode

Selected Cooking Mode

Selected Cooking Mode

E 2

Selected Cooking Mode

W

TOTAL 13

Rice Cooker COSMIC Measurement Results According to Levesque et. al.  

No. Functional Process Triggering Event Data Movement Description

Data Group DM CFP

1 Control Cooking Lamp

Tick (elapsed) Tick (elapsed) TimedEvents E 2

On CookingLamp X

2 5 sec. signal management (control heater)

Signal5 Signal5 TimedEvents E 4 GetTargetTemp CookingState R ReadTemp TemperatureSensor E HeaterOn or

Heater command X

3 30 sec. signal management (set target temperature)

Signal30 Signal30 TimedEvents E 5

GetMode CookingMode R

Tick(elapsed) TimedEvents E

GetCookingTemp CookingSpecs R

SetTargetTemp CookingState W

TOTAL 11

Rice Cooker COSMIC Measurement Results According to Lavazza  

• Levesque  Iden3fied  a  func3onal  process  as  “Select  cooking  mode”  

• It  complies  with  the  defini3on  of  func3onal  process.  

• However,  it  is  one  decomposi3on  level  lower  than  the  others.    

• It  should  have  been  a  sub  process  of  “set  target  temperature”.  

• That  is  :  ◦  The  size  of  the  rice  cooker  is    ◦  11  DMP  in  Tier  1  ◦  13  DMP  in  Tier  0  

Emprical  ValidaJon  -­‐  Findings

DMP  had  beOer  correla=on  with  effort  than  COSMIC  but  worse  than  LOC    

DMP  vs.  Effort  R2  =    0,89  

COSMIC  vs.  Effort  R2  =  0,62  

LOC  vs.  Effort  R2  =  0,92  

Example

•   Upon  receiving  X  from  A,  the  system  shall  read  X  from  B  and  compare  their  values.  If  X  is  greater  than  the  Balance,  system  shall  send  Z  to  C.  •   COSMIC  Size:  E  R  X  :  3  

• Upon  receiving  the  Amount_to_be_Withdrawn  from  the  Customer,  the  system  shall  read  Current_Balance  from  Customer's  Account  and  compare  their  values.    

• If  the  Amount_to_be_Withdrawn  is  greater,  system  shall  send  Warning  SMS  #213  to  Customer's  Cell  Phone.    

• COSMIC  Size:  3    

• Tier:  7  

Example

• Upon  receiving  WebService_Withdrawal  from  System_ATM,  the  system  shall  read  Current  Balance  from  Customer's  Account  and  compare  their  values.    

• If  the  Amount_to_be_Withdrawn  is  greater,  system  shall  send  WebService_BalanceNotEnough  to  CRM.Module.            

• COSMIC  Size:  3  

• Tier:  5  

• Upon  receiving  WebService_Withdrawal  from  System_ATM,  the  system  shall  WebService.CustomerAccount.    

Upon  response,  system  will  read  Current  Balance  from  the  response  and  compare  with  The  Amount_to_be_Withdrawn.    

If  The  Amount_to_be_Withdrawn  is  greater,  system  shall  send  WebService_BalanceNotEnough  to  CRM.Module.      

• COSMIC  Size:  5  

• Tier:  4    

Example

• Upon  receiving  CurrentHeat  from  Thermometer,  the  system  shall  read  RequiredHeat  from  ReqHeatRegister  and  compare  their  values.    

• If  CurrentHeat  is  greater  than  RequiredHeat,  system  shall  send  TurnoffHeater  Signal  to  the  Heater.  

• COSMIC  Size:  3    

Example

• If  we  are  measuring  the  embedded  soEware  on  a  microcontroller  level  of  measurement  can  well  be  Tier  :  0    

• If  the  system  is  comprised  of  a  higher  level  soEware  running  on  a  computer  and  communicate  with  intelligent  sensors  and  heater  through  certain  interfaces:  ◦  If  the  boundary  is  only  the  soEware  running  on  the  computer,  and  see  sensors  as  black  boxes:  COSMIC  Size:  3,  Tier:0  

◦  If  the  boundary  includes  the  sensors  and  interfaces:  COSMIC  Size:  3,  Tier:  3+  

Example

ContribuJons

Streamlining  the  Measurement  Concepts  

• We  defined  a  measurement  approach  based  on  a  concept  that  is  common  in  both  problem  and  solu3on  domains;  data  movement.    

• This  streamlines  measurement  of  both  project  and  product  alributes  in  two  domains.    

• Improves  conversion  of  units,  es3ma3ons,  approxima3ons  and  normaliza3on  of  several  size  defini3ons  and  values.      

Improvements  in  Reliability  of  Measurement  Results

• Backfiring  concepts  of  FSM  from  the  source  code  elimina3ng  the  reliability  issues  in  Tier  0.  (product  measurement)  

• We  traverse  through  higher  levels  u3lizing  metadata  for  the  lowest  level.  Only  point  of  human  interpreta3on  is  in  the  genera3on  of  this  metadata  which  relates  lower  level  concepts  to  higher  level  ones.    

• This  mi3gates  measurement  errors  as  there  is  less  room  leE  for  interpreta3on  and  renders  errors  recoverable  by  fixing  the  metadata.    

BeWer  Benchmarking

• Having  decomposi3on  level  incorporated  into  measurement  results  makes  the  scope  and  abstrac3on  of  the  measured  soEware  product  visible.      

• Comparing  measurement  methods  on  the  same  level  of  decomposi3on  will  greatly  improve  the  accuracy  of  benchmarking  studies.  

BeWer  EsJmaJons

•   Instead  of  using  conversion  factors  between  two  different  size  measurements  that  are  essen3ally  unrelated,  we  define  an  es3ma3on  method  which  rely  on  the  same  concepts  that  the  measurement  method  does.    

•   Es3ma3ons  become  less  prone  to  gaps  between  domains  and  project  phases.        

•     By  measuring  in  lower  levels  of  decomposi3on,  DMP  method  measures  data  manipula=ons  which  would  otherwise  be  leE  out  in  higher  levels  into  measurement.  This  also  increases  the  accuracy  of  es3ma3ons.    

EsJmaJon  Approach

•  N  –  Op3mal  Decomposi3on  Level  (Tier  #)  

•  Expected  Accuracy    

BeWer  Measurement  of  SoQware  Changes

• By  DMP  method,  it  is  possible  to  measure  specifica3ons  defined  in  levels  lower  than  func3onal  user  requirements.    

• This  makes  measuring  lower  level  so\ware  changes  more  accurate  than  exis3ng  FSM  methods.    

• Iden3fying  the  3er  level  of  change  requests  also  improves  the  change  management  processes  as  the  scope  and  impact  of  the  change  can  be  beler  analyzed.      

Further  Studies

• The  same  approach  can  be  applied  to  system  specifica3on  level  or  business  process  defini3ons.    

• Another  PhD  thesis  is  being  conducted  in  our  research  group  on  sizing  business  process  models  based  on  func3onality.    ◦  It  is  possible  to  extend  the  problem-­‐solu3on  flow  to  the  level  of  business  processes.    

◦  This  would  be  a  great  opportunity  to  build  a  streamlined  measurement  and  es3ma3on  framework  spanning  the  whole  soEware  engineering  process.