ops happen: improve security without getting in the way
TRANSCRIPT
Ops Happens: Improve Security Without Getting in the Way
February 29, 2016 ● San FranciscoDamon Edwards @damonedwards
Damon Edwards
Operational Improvement
DevOps Consulting
Tools
Damon Edwards
Operational Improvement
DevOps Consulting
Tools
Community
The Shared Plight of Ops and Security
OPS &
SEC“Go faster!” “Open it up!”
“Be more secure!” “Be more reliable!”
Deployment dominates the conversation2013
Deployment. Deployment. Continuous Delivery. Deployment. Deployment. Continuous Deployment. Deployment. CI/CD. Deployment. Deployment. Deployment. PaaS. Deployment. IaaS. Deployment. Deployment. Infrastructure as Code. Deployment. Deployment.
Deployment. Deployment. Containers. Containers. Deployment. Deployment. Deployment. Docker Deployment. Docker. CaaS. Deployment. Docker. Docker. Docker. Docker. Mesos. Deployment. Kubernetes. Deployment. Microservices. Deployment. Deployment. Docker.
2016
What this sounds like to enterprise Ops & Sec
“What we always give you, but more of it… and a lot more frequently”
“What we always give you, but more of it… and a lot more frequently”
What this sounds like to enterprise Ops & Sec
“Shift Left” to avoid disaster (a.k.a “DevOps 101”)
Writing / Running Automated Tests Writing / Exercising Deploy Automation Running Security Scanning Tools
“Shift Left” to avoid disaster (a.k.a “DevOps 101”)
Writing / Running Automated Tests Writing / Exercising Deploy Automation Running Security Scanning Tools
Deploy. Deploy. Deploy.
“Shift Left” to avoid disaster (a.k.a “DevOps 101”)
But guess what...
Sh*t happens
But guess what...
Sh*t happensOperations
How do you “shift left” incident response?
How do you “shift left” incident response?
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
How do you “shift left” incident response?
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
How do you “shift left” incident response?
But...
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
How do you “shift left” incident response?
But...How do you safely and securely give out access?
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
How do you “shift left” incident response?
But...How do you safely and securely give out access?How do you enable the experts to contribute remediations?
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
How do you “shift left” incident response?
But...How do you safely and securely give out access?How do you enable the experts to contribute remediations?How do you give visibility into operations?
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
How do you “shift left” incident response?
But...How do you safely and securely give out access?How do you enable the experts to contribute remediations?How do you give visibility into operations?How do you do postmortems days/weeks/months later?
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
Design pattern we’ve seen developing in the community...
Shift Left Step 1: Establish a Secure Ops Portal
Shift Left Step 2: Establish a SDLC for Ops Procedures
Shift Left Step 3: Connect with Enterprise Management Systems
Shift Left Step 4: Make Compliance Really HappyWho created the procedure? Who reviewed it? Who? When? Where? Approval trail?
Pay for it with ROI outside of Security
Mark Maun
Jody Mulkey
Ticketmaster’s “Support at the Edge” model • Empowered support teams with self-service ops tasks • Automated Ops procedures written/vetted by the delivery teams • Expanded who could take action, but ops remained in full control of
the policy
Pay for it with ROI outside of Security
Mark Maun
Jody Mulkey
Ticketmaster’s “Support at the Edge” model • Empowered support teams with self-service ops tasks • Automated Ops procedures written/vetted by the delivery teams • Expanded who could take action, but ops remained in full control of
the policy
Sources: https://www.youtube.com/watch?v=_hr4KiB19bQ http://rundeck.org/stories/mark_maun.html
• Removed multiple days of effort from throughout the lifecycle • Reduced escalations by 30% - 40% and overall support incident
costs by 55% • Reduced mean time to repair (MTTR) by 50% - 150%
Want to talk more about “shift left” and operations?
@alexhonor [email protected]
My colleague who thinks a lot about these solutions
A word from today’s organizers…
A word from today’s organizers…
A word from today’s organizers…