devops days tel aviv 2013: what exactly is anti-fragile in devops? - asher sterkin
TRANSCRIPT
![Page 1: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/1.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 1
What is Exactly Anti-Fragile in DevOps? Asher Sterkin
Distinguished Engineer, SPVSS, Cisco Video Systems, Israel
September 30, 2013
![Page 2: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/2.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 2
![Page 3: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/3.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
![Page 4: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/4.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4
Antifragile
Some things benefit from shocks…
volatility, randomness, disorder,
and stressors and love adventure,
risk, and uncertainty… there is no
word for the exact opposite of
fragile. Let’s call it antifragile. Nassim N. Taleb, “Antifragile. Things that gain from
disorder”
![Page 5: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/5.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
The Book and Reactions
I think this concept is incredibly
powerful when applied to systems
and organizational architecture.
Jez Humble, “On Antifragility in Systems and
Organizational Structure”
![Page 6: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/6.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
The Book and Reactions
The Netflix cloud architecture is
anti-fragile… The Netflix culture is
anti-fragile… Getting stronger
through failure is the basis of anti-
fragility. Avoiding failure at all costs
… makes you brittle and
vulnerable...
Adrian Cockcroft, “Looking back at 2012 with
pointers to 2013”
![Page 7: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/7.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7
The Book and Reactions
If the idea is nice and neat,
however, the book that houses it
is just the opposite. It is a big,
baggy, sprawling mess.
David Runciman, review of the book in Guardian
November 21, 2012
![Page 8: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/8.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
Larger Body of Knowledge
• Complex Adaptive Systems
• Highly-Optimized Tolerance
• Technology Development Cycle
• Disruptive Innovations
• Product Development Flow
• Lean Start Up
![Page 9: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/9.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
For Today • De-fragilization
• Skin in the Game
• Barbell
• Asymmetric Pay-off
![Page 10: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/10.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
DevOps Areas
OPS DEV
Area 1: extend delivery to
production
Area 2: extend operations
feedback to project
Area 3: embed project knowledge into operations
Area 4: embed operations knowledge into project
Patrick Debois: “Codifying devops practices”
![Page 11: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/11.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 11
De-Fragilization
![Page 12: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/12.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
“Beauty plus pity-that is the closest we can get to a definition of art. Where there is beauty there is pity for the simple reason that beauty must die: beauty always dies, the manner dies with the matter, the world dies with the individual.”
V. Nabokov, “Lecture on Metamorphosis”
![Page 13: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/13.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
DevOps Areas
OPS DEV
Area 1: extend delivery to
production
Area 2: extend operations
feedback to project
Area 3: embed project knowledge into operations
Area 4: embed operations knowledge into project
![Page 14: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/14.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
Continuous Delivery
... to exert a constant stress on
your delivery and deployment
process to reduce its fragility so
that releasing becomes a
boring, low-risk activity.
Jez Humble, “On Antifragility in Systems and
Organizational Structure”
![Page 15: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/15.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 15
Large batches increase cycle time
Large batches increase variability in flow
John Allspaw: “Ops Meta-Metrics”,
slides 103-109
![Page 16: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/16.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 16
Reducing batch size accelerates feedback
Reducing batch size reduces overhead
Reducing batch size reduces risk
![Page 17: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/17.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17
Batch size and Bottlenecks Reduce batch
size before you
attack
bottlenecks
![Page 18: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/18.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18
DevOps Areas
OPS DEV
Area 1: extend delivery to
production
Area 2: extend operations
feedback to project
Area 3: embed project knowledge into operations
Area 4: embed operations knowledge into project
![Page 19: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/19.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19
“Resilience through Failure”
![Page 20: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/20.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20
![Page 21: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/21.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
![Page 22: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/22.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
Resilience to failure is a
lofty goal. It enables a
system to survive and
withstand failure. There's an
even higher peak to strive
for, however: making the
system stronger and better
with each failure.
A. Tseitlin, “The Antifragile Organization”
Culture of Continuous Learning
![Page 23: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/23.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
E. Deming, “Out of Crisis”
“Confusing common causes with special causes will only make things worse
![Page 24: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/24.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
Resilience, Reliability, Robustness
Normal Operation
MTBF
Failure!
MTTD MTTR
Max Downtime
Normal Operation
MTBF
Max Data
Lost
![Page 25: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/25.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25
It’s me
“Obsessive protection of the system against extremely rare events makes it more fragile. Resilience comes before the last percentiles of reliability.
![Page 26: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/26.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 26
![Page 27: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/27.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27
DevOps Areas
OPS DEV
Area 1: extend delivery to
production
Area 2: extend operations
feedback to project
Area 3: embed project knowledge into operations
Area 4: embed operations knowledge into project
![Page 28: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/28.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28
Give ‘em pagers!
![Page 29: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/29.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29
“DevOps: architects, developers, QA/QC, system engineers, and IT cooperate to maximize the company value
It’s me again
![Page 30: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/30.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 30
Barbell
![Page 31: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/31.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31
DevOps Areas
OPS DEV
Area 1: extend delivery to
production
Area 2: extend operations
feedback to project
Area 3: embed project knowledge into operations
Area 4: embed operations knowledge into project
![Page 32: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/32.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32 John Alsspaw, “Dev and Ops Collaboration”
![Page 33: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/33.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
![Page 34: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/34.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 34
Developers
• Responding to outages, takes on-call
• Alerting systems thresholding, design
• Architecture design and review
• Building metrics collection
• Application configuration
• Shipping public-facing code
• Responding to outages, takes on-call
• Alerting systems thresholding, design
• Architecture design and review
• Building metrics collection
• Application configuration
• Infrastructure buildout/management
Operations
John Allspaw, “Reply to NoOps @ Netflix”
![Page 35: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/35.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
DevOps Areas
OPS DEV
Area 1: extend delivery to
production
Area 2: extend operations
feedback to project
Area 3: embed project knowledge into operations
Area 4: embed operations knowledge into project No, not really
![Page 36: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/36.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36
Every analogy has its limit.
One just needs to learn where
to stop.
![Page 37: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/37.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37
N. Taleb, “Antifragile”
“The first step to anti-fragility consists in decreasing downside…This brings us to the solution in the form of barbell … Away from Golden Middle.
![Page 38: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/38.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38
Away from golden Middle
Risk
Aversion
Risk
Loving
![Page 39: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/39.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 39
Niek Bartholomeus,
“DevOps For Dinosaurs”
![Page 40: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/40.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 40 Jeremy Edberg, “DevOps at Netflix”
Risk Aversion Risk Loving
![Page 41: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/41.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41
N. Taleb, “Antifragile”
“The downside/lost should be known and protected, not probability
![Page 42: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/42.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 42
It’s me
“ You never know where from your next outage or cyber attack will come from and when. The maximal downtime and maximal data lost should be known and guaranteed regardless of probabilities.
![Page 43: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/43.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43
Asymmetric Pay-Offs
x
Gain
/Loss f
(x)
Pain
Gain
![Page 44: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/44.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 44
DevOps Areas
OPS DEV
Area 1: extend delivery to
production
Area 2: extend operations
feedback to project
Area 3: embed project knowledge into operations
Area 4: embed operations knowledge into project
![Page 45: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/45.jpg)
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 45
![Page 46: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/46.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 46
N. Taleb
“Never be sucker. Period!
![Page 47: DevOps Days Tel Aviv 2013: What exactly is anti-fragile in DevOps? - Asher Sterkin](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554a10c6b4c9055c598b4b55/html5/thumbnails/47.jpg)
Thanks