indwelling and outflowing – applying the jidoka technique
Post on 26-May-2015
506 Views
Preview:
TRANSCRIPT
Collabera Collabera
Indwelling and Outflowing – Applying the JIDOKA Technique
Moses & Smitha
Collabera solutions
Bangalore
1
Collabera
2
Collabera
3
Collabera
4
Collabera
5
Collabera
6
Controllable Input Lead
Non Controllable Output
Lag
6
Collabera
Requirements structure
Test case adequacy Code
structure
Structural application profile JIDOKA
7
Collabera
8
Collabera
9
Collabera
10
Collabera
11
Collabera
Population : 20.5 M Richest city in India, Per-capita income in 2009 was 486,000 (US$10,692) ( 3 times the national average) 6% of India’s GDP & Highest GDP of any City in South/West/Central Asia Commercial and entertainment capital of India & one of the world's top 10 centers of commerce 70% of capital transactions to India's economy Home to premier financial, scientific & nuclear institutions A potpourri of many communities and cultures.
12
Collabera
Figuring out
13
Requirements
Architecture
Design & Coding
Unit Testing
System Testing C
ust
om
er
Cu
sto
me
r
? ? ? ? ?
Requirements Review defects
Architecture review defects
Code Review defects
Unit Testing defects
System testing defects
Effort
Schedule
Quality Control Metrics
Project Management
Engineering Metrics
Collabera
14
Collabera
15
Pre-SIT SIT UAT
Cost of fixing 3.5 18 80
0
20
40
60
80H
ou
rs f
or
fixi
ng
Cost of fixing defect
Collabera
JIDOKA-Step By Step
Requirements structure
Test case adequacy Code
structure
Structural application profile
16
Collabera
CASE Study : Correlating use case to SIT
Use Case
Volatility
Detailing
Verification
Complexity
# of change after first baseline
Level 1 Basic & alternate flows Level 2 Additional details
Level 1 No SME reviews Level 2 SME reviews
As defined in UCP template Based on # of transactions
SIT defects Mapped to each use case
UAT defects Mapped to each use case
Production defects
Mapped to each use case
Functional Defects
α
17
Collabera
JIDOKA in Requirements
“Can I say my requirements are at a level “ X” which signifies the quality of my requirements?” “Can I set thresholds and monitor my requirements that enables me to do course corrections?”
18
Collabera
Gold – Silver - Bronze Rating
Basic Flow Available, documented and reviewed.
Alternate scenarios added and reviewed
Special considerations of non functional and interface aspects documented and reviewed. Validated through prototypes
19
Collabera
Requirements Qualification matrix
Planning & Tracking iterative assessment (Score Card)
Business Requirements assessment (Score Card)
Functional Requirements assessment (Score Card)
Non Functional Requirements assessment (Score Card)
Requirement documentation assessment (Score Card)
Requirement Phase Design Phase Proposal / SOW / Discovery 20
Collabera
TS N TS 2 TS 1
Requirements structure- A deeper insight
Category 1
Category 2
Category 3
Category 4
Category 5
80% 60% 20%
# of use cases and # of reports
% Use cases with Alternate / exceptional flows
% use cases without business rules
% use cases with Open points
% Use cases with screens/Prototypes % of use case with QOS criteria
21
Collabera
Figured out - Requirements
22
Requirements
Architecture
Design & Coding
Unit Testing
System Testing C
ust
om
er
Cu
sto
me
r
% UC with Alt flows % UC with Open points % Ucs without business rules % Use cases with screens/Prototype
? ? ? ?
Requirements Review
Architecture review
Code Review Unit Testing System testing
Effort
Schedule
Quality Control Metrics
Project Management
Engineering Metrics
Collabera
JIDOKA-Step By Step
Requirements structure
Test case adequacy Code
structure
Structural application profile
23
Collabera JIDOKA - Architecture
Attributes Baseline Arch
Candidate Arch 1
Candidate Arch 2
Candidate Arch n
Map all Stakeholder Concerns
Identify Architecturally Significant Requirements
Architecture Strategy & Style
Architecturally Significant Decisions
Architectural Views
Architectural Analysis & Evaluation
Number of ASRs
% coverage of BR to ASRs
# of architectural styles/patterns
Ratio of ASRs covered through structured decision making techniques ( to tradeoffs, risks & sensitivities )
Completeness of architectural views
Quality attribute ranking
24
Collabera
Figured out - Architecture
25
Requirements
Architecture
Design & Coding
Unit Testing
System Testing C
ust
om
er
Cu
sto
me
r
% UC with Alt flows % UC with Open points % Ucs with more than 2 changes % Use cases with screens/Prototype
% coverage of BR to ASRs # of arch styles /patterns Ratio of ASD t o tradeoffs, r isks & sensitivities
? ? ?
Requirements Review
Architecture review
Code Review Unit Testing System testing
Effort Variance
Schedule Variance
Quality Control Metrics
Project Management
Engineering Metrics
Collabera
JIDOKA-Step By Step
Requirements structure
Test case adequacy Code
structure
Structural application profile
26
Collabera
Cyclomatic complexity
%Branch Statements
Depth of Inheritance
Calls/method
Methods / Class
Class Coupling
Maintainability Index Boolean Expression
complexity
Class Data Abstraction coupling
Class Fan Out Complexity
Npath Complexity
Java NCSS
% comment Lines Density of comment lines
Public undocumented API
Duplicated lines
Duplicated blocks
Number of children
Afferent couplings
Lack of cohesion methods
Package dependencies to cut
Violations
Complexity distribution by method
Complexity distribution by class
File dependencies to cut
The problem of plenty
27
Collabera
II
Slide 28
% Branches Block Depth
Cyclomatic Complexity
Correlating SIT with engineering metrics
% Branches Block Depth
Cyclomatic Complexity 28
Collabera
Case Study
92%
4%
2% 1% 1%
Number of Class vs SIT defects
8% of classes is contributing to 100% of SIT defects
Study from European university
Key parameters measured
Cyclomatic complexity No. of methods No. of Calls LOC/method
Our performance is above UCL on
LOC/method
29
Collabera
Sample Java Class and Code toxicity analysis
30
Detailed study of the data shows most of the SIT & AT defects are occurring amongst the top four Java classes shown in the sample data above which have highly toxic code with high method length
Correlates to AT & SIT defects
Collabera
2.3.10
2.3.9.2 2.3.6.2
2.3.53.2
2.3.52.2
2.3.5.2 2.3.4.2 2.3.36.2
2.3.35.2
2.3.34.2
2.3.33.2
2.3.32.2 2.3.30.2 2.3.20.2 2.3.19.2
2.3.10.2
2.3.7.2
2.3.50.2
2.3.49.2
2.3.20.2
2.3.13.2 2.3.44.2 2.3.43.2 2.3.42.2 2.3.41.2
2.3.40.2
2.3.39.2
2.3.38.2
2.3.37.2
2.3.11.2
2.3.12.2
2.3.27.2
2.3.2.2
2.3.1.2
2.3.48.2
2.3.47.2
2.3.46.2
2.3.45.2
2.3.26.2
2.3.25.2
2.3.23.2
2.3.22.2
2.3.18.2
2.3.17.2
2.3.16.2
2.3.15.2
UC complexity vs. max code complexity
31
Collabera
32
Collabera
Unit test measures
33
Unit Test coverage Test LOC Vs SLOC Test Pass percentage
Collabera
Figured out - Coding
34
Requirements
Architecture
Design & Coding
Unit Testing
System Testing C
ust
om
er
Cu
sto
me
r
% UC with Alt flows % UC with Open points % Ucs with more than 2 changes % Use cases with screens/Prototype
% coverage of BR to ASRs # of arch styles /patterns Ratio of ASD t o tradeoffs, r isks & sensitivities
?
Requirements Review
Architecture review
Code Review Unit Testing System testing
Effort
Schedule
Quality Control Metrics
Project Management
Engineering Metrics
Cyclomatic complexity Methods / Class Calls Per Method % Branch Statements
Test Coverage Test LOC/SLOC Test cases failed per build
Collabera
JIDOKA-Step By Step
Requirements structure
Test case adequacy Code
structure
Structural application profile
35
Collabera
Variant -1
Application TC TV TI Others
1 Application 1 24 24 9 6
2 Application 2 36 54 27 9
3 Application 3 6 6 6 3
4 Application 4 9 12 0 6
Before
After
No Application TC TV TI Others
1 Application 1 30 45 15 18
2 Application 2 45 75 36 24
3 Application 3 9 9 6 12
4 Application 3 12 15 6 15
TC TV TI Others 1 1.25 1.25 2.3
36
Collabera
Variant – 2
% Navigation test cases
% Presentation test cases
% Behavioral test cases
% UI test cases
% Accessibility test cases
% Security test cases
37
Collabera
Variant - 3
Scenario Level
• % of end to end business scenarios
• % of alternate business scenarios
• % of untestable scenarios
Test case Level
• # of test cases per use case
• % of positive test cases
• % of negative test cases (exceptional cases, abnormalities)
• % of interface test cases
• % of UI test cases
• % of non functional test cases
• % of requirements covered through test cases
38
Collabera
JIDOKAA in software
39
Requirements
Architecture
Design & Coding
Unit Testing
System Testing C
ust
om
er
Cu
sto
me
r
% UC with Alt flows % UC with Open points % Ucs with more than 2 changes % Use cases with screens/Prototype
% coverage of BR to ASRs # of arch styles /patterns Ratio of ASD t o tradeoffs, r isks & sensitivities
% of end toend scenarios % of untestable scenarios % of positive test cases % of negative test cases
Requirements Review
Architecture review
Code Review Unit Testing System testing
Effort
Schedule
Quality Control Metrics
Project Management
Engineering Metrics
Cyclomatic complexity Methods / Class Calls Per Method % Branch Statements
Test Coverage Test LOC/SLOC Test cases failed per build
Collabera
Shifting to the better - I
40
Collabera
Improved Pre-SIT Detection
Pre_SIT Defects %
Dec 2009 Last 1 year
70 %
49 %
100% of development and enhancement were above 2011 pre-SIT target Most development and enhancements had slipped defects ratio less than 2%
III
41
Collabera
Shift in defect distribution
Requirements & Design review
Code Review Unit Test System Test Post Release
Pre Post
42
Collabera
What Did We See….
Overall Satisfaction Quality On time
CSAT performance
Mar-10 Sep-10 Mar-112009 2010 2011
43
Collabera
Qualitative Benefits
These metrics and dashboards are easier to understand
Improve ability to isolate and deal with the defective ones
Improved risk management and transparency
44
Collabera
Thank you
45
Mosesraj R - (mosesrajr@collabera.com)
Smitha T – (smithat@collabera.com)
top related