cycle time mathematics

Cycle Time Mathematics

Presented at KLRAT 2013 [email protected]

Commercial in confidence2

Conclusions

• Forecasting using cycle time is proving useful• Cycle time follows a Weibull/Lognormal shape

– We can estimate the actual distribution with just a minimum and a maximum guess (initially)

• Controlling the shape of the distribution (narrowing) improves predictability

• Proposing a way of identifying what risks and delays will have the greatest impact…

Prediction Intervals

Actual Maximum

Actual Minimum

1

3

2

Q. What is the chance of the 4th sample being between the range seen after the first three samples? (no duplicates, uniform distribution, picked at random)

4

Actual Maximum

Actual Minimum

1

3

2

4

Highest sample

Lowest sample


?

?

?

?

Actual Maximum

Actual Minimum

1

3

2

4

25% chance higher than highest seen

25% lower than highest and higher than second highest

25% higher than lowest and lower than second lowest

25% lower than lowest seen

Highest sample

Lowest sample


A. 50%

% = (1 - (1 / n – 1)) * 100

Actual Maximum

Actual Minimum

1

3

2

12

5% chance higher than highest seen

5% lower than lowest seen

Highest sample

Lowest sample


?

?

A. 90%

% = (1 - (1 / n – 1)) * 100

4

5

6

7

8

9

10

11

# Prior Samples Prediction Next Sample Within Prior Sample Range

3 50%4 67%5 75%6 80%7 83%8 86%9 88%

10 89%11 90%12 91%13 92%15 93%17 94%20 95%

Halved the risk with 3 samples

Monte Carlo


If a measurement changes over time or is different each time you measure it, it is a

DISTRIBUTION

Can’t apply normal mathematical operators….


Cycle Time Forecasting

1. Easy to Capture Metric

3. Follows Known Distribution Pattern

2. Forecasting using Historical Data

(even a little)

4. Forecast at Project or Feature

or Story Level

Sum Random Numbers

25112943342631452227

31436545

87

34735448

1912242721

39

202329

187410295Sum:

…..

Historical Story Lead Time Trend

Days To Complete

Basic Cycle Time Forecast Monte Carlo Process

1. Gather historical story lead-times2. Build a set of random numbers based on pattern3. Sum a random number for each remaining story to build a single outcome4. Repeat many times to find the likelihood (odds) to build a pattern of likelihood outcomes

𝑇𝑜𝑡𝑎𝑙𝐷𝑎𝑦𝑠❑=𝑆𝑢𝑚 (𝑆𝑡𝑜𝑟𝑦𝑛×𝑅𝑎𝑛𝑑𝑜𝑚𝑛)

𝐸𝑓𝑓𝑜𝑟𝑡

How I Quantify Lead Time Reduction

Lead Time # Stories / Year Throughput Benefit

Current 6 0%

10% Decrease 7 17% More

20% Decrease 8 33% More

When no ROI is easily discerned (maintenance teams?)

Even a small decrease in Cycle Time has a increased impact on throughput over a year..

Cycle Time

Forecast Date Forecast Cost

Cash flow to EOY14(cost saving + revenue)

Benefit

Current 15-Jul-2014 $1,000,000 $0 + $60,000 = $60,000 0%10% Decrease 27-May-2014 $912,500 $87,500 + $90,000 =

$177,500296% Better

20% Decrease 04-Apr-2014 $820,000 $120,000 + $145,000 =

$265,000442% Better

Month Revenue

April $30,000

May $25,000

June $20,000

July $20,000

Aug-Dec $10,000

Revenue Estimates for product:

How I Quantify Cycle Time Reduction

Even a small decrease in Cycle Time can have huge a benefit on

cash flow over time.

This example shows the cost of missing a seasonal uptick in summer sales (revenue estimates shown to the left). Cost per work day is calculated at $2,500 day.

Shape and Impact of Cycle-Time & Scope


Probability Density Function

Histogram Gamma (3P) Lognormal Rayleigh Weibull

x1301201101009080706050403020100-10

f(x)

0.32

0.28

0.24

0.2

0.16

0.12

0.08

0.04

0

Note: Histogram from actual data



Histogram Weibull

x1301201101009080706050403020100-10

f(x)

0.32

0.28

0.24

0.2

0.16

0.12

0.08

0.04

0


Likelihood…

Total Story Lead Time

30 days

Story / Feature Inception5 Days

Waiting in Backlog25 days

System Regression Testing & Staging 5 Days

Waiting for Release Window5 Days

“Active Development”30 days

Pre Work

30 days

Post Work

10 days


Binary PermutationsRisk1

Risk2

Risk3

Risk 4

Risk 5

Risk 6

Risk 7

Risk 8

Risk 9

Risk 10

No No No No No No No No No No

No No No No No No No No No Yes

No No No No No No No No Yes No

No No No No No No No No Yes Yes

No No No No No No No Yes No No

… … … … … …

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

Why Weibull

• Now for some Math – I know, I’m excited too!

• Simple Model• All units of work between 1 and 3 days• A unit of work can be a task, story, feature, project• Base Scope of 50 units of work – Always Normal• 5 Delays / Risks, each with

– 25% Likelihood of occurring– 10 units of work (same as 20% scope increase each)

Normal, or it will be after a few

thousand more simulations

Base + 1 Delay

Base + 2 Delays

Base + 3 Delays

Base + 4 Delays

Base + 5 Delays



Histogram Weibull

x1201101009080706050403020100

f(x)

0.28

0.24

0.2

0.16

0.12

0.08

0.04

0

Scale – How Wide in Range. Related to the

Upper Bound. *Rough* Guess: (High – Low) / 4

Shape – How Fat the distribution. 1.5 is a good starting point.

Location – The Lower Bound

KEY POINT: WITH JUST MIN AND MAX THE CURVE CAN BE INFERRED

Changing Cycle Time Shape



Weibull (1.2; 25)

x8880726456484032241680

f(x)

0.028

0.024

0.02

0.016

0.012

0.008

0.004

0


Weibull (1.5; 30)

x8880726456484032241680

f(x)

0.026

0.024

0.022

0.02

0.018

0.016

0.014

0.012

0.01

0.008

0.006

0.004

0.002

0

Teams following this curve WILL be able to predict more

predictability because forecast range will be tighter

When forecasting, the wider the curve, the MORE higher

value numbers will occur


Binary PermutationsRisk1

Risk2

Risk3

Risk 4

Risk 5

Risk 6

Risk 7

Risk 8

Risk 9

Risk 10

No No No No No No No No No No

No No No No No No No No No Yes

No No No No No No No No Yes No

No No No No No No No No Yes Yes

No No No No No No No Yes No No

… … … … … …

Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

EVERY RISK OR DELAY YOU CAN REMOVE REDUCE COMBINATIONS BY 2n


Order of Priority for Improvement

• Prioritized list of blockers and delay states• Balanced to include most frequent & biggest delay• Order Risks and Delays by weighted impact

Impact = Frequency Risk or Delay “Type” x Duration

• This will remove the most combinations of delay and shrink the area of our distribution giving biggest benefit of predictability

Mining Cycle Time Data


Mining / Testing Cycle Time Data

Capture Data

Scatter Plot

Histogram

Fit Shape


Cycle Time Capture Practices

• Clearly understand from where and to where• Capture begin and end date; compute the

number of days in-between• Does cycle time include defect fixing?• Are there multiple types of work in the same

cycle time data– Stories– Defects– Classes of Service


Things that go wrong…

• Zero values• Repetitive (erroneous) values• Batching of updates• Include/exclude weekends• Project team A staff raided impacting their

cycle time (Team A up, team B’s down)• Work complexity changes• Team skill changes



Histogram Weibull

x120100806040200

f(x)

0.36

0.32

0.28

0.24

0.2

0.16

0.12

0.08

0.04

0


Histogram Weibull

x160140120100806040200

f(x)

0.22

0.2

0.18

0.16

0.14

0.12

0.1

0.08

0.06

0.04

0.02

0

Many low values. Often zero or values below what makes sense. Check the most frequent low values.

Multiple modes. In this case two overlapping Weibulls. Often due to multiple classes of service, or most often - defects versus stories.



Histogram Weibull

x1201101009080706050403020100

f(x)

0.28

0.24

0.2

0.16

0.12

0.08

0.04

0

Scale – How Wide in Range. Related to the

Upper Bound

Shape – How Fat the distribution.

Location – The Lower Bound

Estimating Distributions using

Historical Data


What Distribution To Use...

• No Data at All, or Less than < 11 Samples (why 11?)– Uniform Range with Boundaries Guessed (safest)– Weibull Range with Boundaries Guessed (likely)

• 11 to 50 Samples– Uniform Range with Boundaries at 5th and 95th CI– Weibull Range with Boundaries at 5th and 95th CI– Bootstapping (Random Sampling with Replacement)

• More than 100 Samples– Use historical data at random without replacement– Curve Fitting


Sampling at Random Strategies

• If you pick what samples to use, you bias the prediction…

• Strategies for proper random sampling –– Use something you know is random (dice, darts)– Pick two groups using your chosen technique and

compute your prediction separately and compare– Don’t pre-filter to remove “outliers”– Don’t sort the data, in fact randomize more if

possible


Concurrent WIP Sample : Find the smallest and the biggest or take at least 11 samples to be 90% sure of range

Estimating Concurrent Effort from Cumulative Flow Chart

20% 40%

60% 80%


Scope Creep Over TimeLook at the rate new scope is added over time

cycle time mathematics

Documents