Granular Metrics,Granular Metrics, Feedback and ExperimentationFeedback and Experimentation

Arnoldo Hax Alfred P. Sloan Professor of Management

Contributions of the Delta ModelContributions of the Delta Model

Opening theOpening the The BestThe Best Three Strategic Options:Three Strategic Options:Product doesProduct doesmindset to newmindset to new zz Best ProductBest ProductThe TriangleThe Triangle not always winnot always winstrategicstrategic

zz Total Customer SolutionsTotal Customer Solutionspositionpositionzz SySystem Lockstem Lock--inin

LinkingLinkingStrategy withStrategy with

AdaptiveAdaptive ExecutionExecutionProcessesProcesses

Execution isExecution is not thenot the problem,problem,linking tolinking to strategy isstrategy is

Execution is captured through ThreeExecution is captured through ThreeAdaptive Processes:Adaptive Processes:zz Operational EffectivenessOperational Effectivenesszz Customer TargetingCustomer Targetingzz IInnnovnovationationwhose roles need to change to achievewhose roles need to change to achievedifferent Strategic Positionsdifferent Strategic Positions

MeasuringMeasuring GoodGood successsuccess financials dofinancials do

AggregateAggregate not alwaysnot alwaysMetricsMetrics lead to goodlead to good

resultsresults

Aggregate performance metrics need toAggregate performance metrics need toreflect each of the Adaptive Processesreflect each of the Adaptive Processes and their role based upon the strategicand their role based upon the strategicpositionpositionzz Product performanceProduct performancezz Customer performanceCustomer performance

DiscoveringDiscoveringGranularGranular performanceperformanceMetrics andMetrics and driversdriversFeedbackFeedback

Managing byManaging by averagesaveragesleads to belowleads to below averageaverageperformanceperformance

zz Complementor performanceComplementor performanceBusiness is nonBusiness is non--linear. Performance,linear. Performance, particularly bonding, is concentrated.particularly bonding, is concentrated. Granular metrics allow us to focus onGranular metrics allow us to focus on underlying performance drivers, tounderlying performance drivers, to detect variability, to explain, to learn, anddetect variability, to explain, to learn, and to act.to act.

The Delta Model and Granular MetricsThe Delta Model and Granular Metrics

Select Performance Indicators

Explain Variability

• Identify performance drivers

• Correlate drivers with variability

Detect Variability

• Drill down to detailed segmentation

• Isolate variability

• Define program • Define structured tests • Rollout

• Hypothesize cause and effect

• Evaluation

Act from Variability

Learn from Variability

Drivers of variability for selected performance indicatorsDrivers of variability for selected performance indicatorsDrivers of variabilityPerformance indicatorsPerformance indicators Drivers of variability

Product cost and qualityProduct cost and quality zz ScaleScalezz DensityDensity--e.g. concentrae.g. concentrattion of serion of servviciceezz LLoocatiocationnzz Labor productivityLabor productivity-- practices andpractices and trainintraininggzz EquipmeEquipmennt product producttivivityity-- desigdesignnzz Process designProcess designzz AnAnd sod so oonn

Customer profitCustomer profit zz Customer sizeCustomer sizezz Customer revenueCustomer revenuezz TenureTenurezz Acquisition costAcquisition costzz ChanChannel mixnel mixzz Customer care supCustomer care supppoortrtzz Customer investments inCustomer investments in relatiorelationnshshiippsszz AnAnd sod so oonnzz Size of relevSize of relevaant complementor productsnt complementor productsComplementor contributionComplementor contributionzz Complementor investment in bComplementor investment in buusinesssinesszz Relative size in customer value chainRelative size in customer value chain

n t customer economicszz Complementor contributiComplementor contributioon too customer economics zz ExclusivityExclusivity of relaof relationshiptionshipzz AnAnd sod so oonn

Business segment economicBusiness segment economic zz ROI (return oROI (return onn investment)investment) –– profitabilityprofitabilityvaluevalue zz RiskRisk-- volatilityvolatility and covariancand covariancee

zz Option valueOption valuezz Investment baseInvestment basezz CashflowCashflowss

Experimentation as the basis for effective changeExperimentation as the basis for effective change

LargeLarge

Size ofSize of ChangeChange

SmallSmall

The middle road:The middle road: lower returns, orlower returns, or

unacceptable whenunacceptable whenfirst moverfirst mover

advantage is highadvantage is high

Unacceptable risk,Unacceptable risk,as a starting point.as a starting point.Highly desirable asHighly desirable as

endpointendpoint

Ineffective: lowerIneffective: lower returns, orreturns, or

unacceptable whenunacceptable whenfirst moverfirst mover

advantage is highadvantage is high

Testing: theTesting: therelevant area forrelevant area for

experimentsexperimentsleading to largeleading to large

changechange

SlowSlow FastFastSpeed ofSpeed of ChangeChange

Cost DeCost De--Averaging:Averaging:Using Granular Metrics ToUsing Granular Metrics To

Drive PerformanceDrive Performance

Most Expensive

$100

$1,000

Least Expensive 0

20 %

of c

osts

% of orders

40

60

80

100

20 40 60 80 100Orders

2. Individual Order Cost

Cos

t/ord

er

Switching Billing Marketing Etc.Order fulfillment

1. The Value Chain of the Local Business Data Circuit

Average Cost = $395/order

3. The 80-20 Cost Effect

Transmission

THE BEHAVIOR OF COST - THE CASE OF BUSINESS DATA SERVICES

Figure by MIT OCW.

The drivers of the cost of orderThe drivers of the cost of order--fulfillmentfulfillment

Cost of order fulfillment

EquipmentLocation of work centers

Activities of order fulfillment

Manpower productivity

1 -

}

/

/

THE BEHAVIOR OF COST BY LOCATION

1,000

70

Hea

dcou

nt re

quire

d to

pro

cess

1,

000

orde

rs p

er m

onth

2,000 5,000

Chicago

Competitor 1 ­San Francisco

Kansas City

Competitor 2 ­San Jose

Competitor 2 Indianapolis

Competitor 3 ­Jacksonville

Competitor 1 ­Denver

New York Tulsa Charlotte

Competitor 1 - Chicago Competitor 1 - Houston

39% gap

Orders Month

Order Fulfillment

12,000 27,000 60,000

Client performance curve

Company Competitors

Scale = 72% Log log graph

Best-in-class performance

Figure by MIT OCW.

()

()

THE BEHAVIOR OF COST BY FIVE ACTIVITIES IN THE VALUE CHAIN

Order

Facilities available

Access available Pass test

Valid design

Access available

Access not available

Access not available

Facilities not available

Automatic switch map

Fast facilities test

Pass access test

Customer ready

$100 per access leg

10x cost difference

1 M

ILL

ION

PO

SSIB

LE

OR

DE

R P

AT

HS

$2,000 per access leg

Skin test

Invalid design

Manual switch map

Fail facilities test

Fail access test

Customer not ready

'Defect-free Path'

'Defect-prone Path'

Figure by MIT OCW.

The behavior of cost by type of defectsThe behavior of cost by type of defectsClientClient

Business Data Service Orders

Business Data Service Orders

Business Data Service Orders

Defect-free

Average

71%71% $100$100 5 Days5 Days 18%18%

5.0%5.0% $1,800$1,800 45 Days45 Days 22%22%

24.0%24.0% $900$900 25 Days25 Days 60%60%

1 $395 12.3 Days

ordersorderse cyclee cycle

timetimeCost/Cost/ orderorder costcost

Focus ofFocus of programprogram

developmendevelopmentt

AveragAverag% of% of TotalTotal

0

0 0

A

(%)

(%)

/

B C D E F G H I J K L M N

2

4

6

8

( )

25 100

80

100

Craftsmen

Cum

ulat

ive

cost

Cumulative craftsmen

Craftsmen Efficiency

The Need for Granular Metrics

Tota

l hou

rscl

eare

d fa

ult

S:1

10

Fault not fixed correctly & repeats Pair throw: reroute around fault Request assistance Refer fault to someone else specialist

Fix fault

THE BEHAVIOR OF COST BY INDIVIDUAL WORKERS

Figure by MIT OCW.

0 0

0 0

j

THE BEHAVIOR OF COST BY TYPE OF EQUIPMENT

20

20

40

60

80

100

40

% of Junctions

Line Fault Concentration

% o

f Lin

e Fa

ults

60 80 100 20

20

40

60

80

100

40

% of Equipment

Records Errors Concentration

% o

f Rec

ords

Err

ors

60 80 100

100% of demand pair reclamations are concentrated in 17% of unctions

100% of records errors are concentrated in 19% of equipment

Figure by MIT OCW.

/

/

----

a b

c

/

----

OPERATIONAL EFFECTIVENESS: ADAPTIVE FEEDBACK USING GRANULAR METRICS

Cost Drivers

Individual Performance

Select Associates

Best Practice Sessions

Supervisor

Benchmarks

Supervisor 'Supervisor Coaching'

Individual Reviews

Standard

Standard

Supervisor Review District Center Reviews

Director Process Reviews

Ind.

Total

Actual Target

a b

Cos

tUni

t

Supervisor

Total

Actual Target

Figure by MIT OCW.

Profit DeProfit De--Averaging:Averaging:Using Granular Metrics ToUsing Granular Metrics To

Drive PerformanceDrive Performance

0

PROFIT MARGIN CONTRIBUTION

THE NEED FOR GRANULAR METRICS

-7x

-6x

-5x

-4x

Prof

it M

argi

n

-3x

-2x

-x

Cumulative percentage of customers 1x

2x

3x

85% of Profits

46% of Profits

40% of Profits

21% of Profits

10 20 30 40

35

50 60 70 80 90 100

Figure by MIT OCW.

The drivers of customer profitability in the credit card businesThe drivers of customer profitability in the credit card businesss

Profit by customer

Months of tenure

Revenue per customer

Number of customers

Balance level per month

$( )

/

/

$( )

()

()

VA R I A B I L I T Y O F P R O F I T S B Y U S A G E L E V E L S

20Mon

thly

Pro

fit

Cus

tom

er

$10-$100 $100-$200

Revenue Customer

$200-$400 $400+

10$0

$10 $20 $30 $40 $50 $60 $70

75th Percentile + 1.5* Box Size 'Upper limit of connected data'

75th Percentile

Median

25th Percentile

25th Percentile -1.5* Box Size 'Lower limit of connected data'

Figure by MIT OCW.

()

0

/

0

FURTHER EXAMINATION OF PROFIT DRIVERS

5MM

15%

22% 18%

10MM 20MM

Number of Customers

Company Scale Customer Share Customer Tenure

Prof

it M

argi

n R

OS

30MM $1000 -50 -40 -30 -20 -10

10 20 30

$2000 $3000 $4000

Balance Level $ Month

10 20 30 -50 -40 -30 -20 -10

10 20 30

Months of Tenure

Figure by MIT OCW.

T H E E F F E C T S O F C U S TO M E R TA R G E T I N G

150,000 0

x

3.5x

Tailor offers to individual customers (i.e., increase response & profit with differential offers tailored to customer's preferences)

Offer one product to customers, based on both response & profitability (i.e., acquire customers likely to have good volume/ margin/lifetime characteristics)

Offer one product to those who will respond (i.e., maximize response for a given offer, assuming average economics)

Offer one product to everyone Untargeted solicitations yield a negative profit

5x

Cum

ulat

ive

Prof

it

Cumulative Number of Customers Solicited

300,000

}}}

}}}

}}

Figure by MIT OCW.

CustomersCustomers’’ targeting behaviorstargeting behaviors

BehaviorsBehaviors Typical competitorsTypical competitors Capital OneCapital Onezz OfferOffer 5,000 products5,000 products75 products75 productszz Test/yTest/yearear 3030 15,00015,000zz Service tierService tier 44 1313zz KeKeyy metricmetric ROE, DelinquencyROE, Delinquency NPVNPVzz TargetsTargets High response likelihoodHigh response likelihood High NPV potentialHigh NPV potentialzz Solicitations/ySolicitations/yearear 50 million50 million 400 million400 millionzz Segmentation frequencSegmentation frequencyy BiannualBiannual OngoingOngoingzz Risk orientationRisk orientation Risk averseRisk averse Price for riskPrice for riskzz Data sourcesData sources 8080 50,00050,000zz Staff analyStaff analysststs 2020 150150

Capital OneCapital One-- achieving a Total Customer Solutions throughachieving a Total Customer Solutions through Customer TargetingCustomer Targeting

FurtherFurtherimprovementimprovement

Variation 2

Variation 1

Test cells

Select Offers

Offer Creation

•• IntereInterest rast ratestes•• FeesFees•• PromotionsPromotions

MassMassmarketingmarketing

Screen for profitability

•• Full lifeFull life cycle customercycle customer profitabilityprofitability

Variation 2

Variation 1

Core offer

Collect customer response

Success metricsSuccess metrics

Typical competitorTypical competitor Capital OneCapital One

zz Customer NPVCustomer NPVzz Acquisition costAcquisition costzz ActivationActivationzz PercenPercent witht with revorevolvinglving bbaalanlancesceszz Fee income per accountFee income per accountzz ChargeCharge--off per accountoff per accountzz TenureTenure

$62$62$77$7758%58%25%25%$55$55$71$71

4 years4 years

$427$427$67$6779%79%40%40%$73$73$54$54

6 years6 years

0

1) 2) 3) 4)

5)

0

5

i)

)

i)

)

i) )

i) )

% of Equity Investment

100% of value

created

6 BUs contribute 100% of value created 16 more BUs contribute another 42% of value created Those 22 BUs are worth $ 15.5 billion, on an equity investment of only $4.2 billion 44% of new capital investment and 53% of R&D spending over the next three years is going to BUs earning economic profits 56% of new capital investment and 47% of R&D spending are going to BUs that will be generating economic losses

-40

-20

-5

10

15

20

60

% o

f Tot

al C

ompa

ny V

alue

Cre

ated

25

-21% of value created -21% of value created 42% more value created

Value Created by Business Unit

24% of new capital 21% of R&D

ii

20% of new capital 32% of R&D ii

36% of new capital 25% of R&D ii

50 75 100

SOURCES OF VALUE CREATION

20% of new capital 22% of R&D ii

Figure by MIT OCW.