devops (start walking in the same direction) by ops
TRANSCRIPT
![Page 1: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/1.jpg)
DEVOPSSTART WALKING IN THE SAME DIRECTION
![Page 2: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/2.jpg)
WHO I AM?
Demis Rizzotto
System Engineer,DevOps Linux Engineer @Exxoss
Currently consultant @Lampiris
![Page 3: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/3.jpg)
THE PROBLEM OF DOWNWARD SPIRAL
![Page 4: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/4.jpg)
DIAMETRICALLY OPPOSED GOALS AND INCENTIVES
DEV▸ Respond to the rapidly
changing competitive landscape
OPS▸ Provide stable, reliable, and
secure service to the customer
THE CORE, CHRONIC CONFLICT WHEN ORGANIZATIONAL MEASUREMENTS AND INCENTIVES ACROSS DIFFERENT SILOS PREVENT THE
ACHIEVEMENT OF GLOBAL, ORGANIZATIONAL GOALS
![Page 5: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/5.jpg)
FRUSTRATION
▸Unnecessary process steps
▸Unnecessary rework
▸Unnecessary features
▸Waiting for someone else to act on an action or task
▸Waiting for an approval
▸Waiting for an environment
▸Creating a ticket for a manual task
▸Reporting to management by manually updating spreadsheets
▸Manual status reporting
▸…
![Page 6: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/6.jpg)
RESULTS
▸Fragile application are prone to failure
▸Ever increasing backlog and technical debt
▸Urgent security rework and remediation
▸More urgent projects into the queue
▸Big deployments very difficult to diagnose when fail
![Page 7: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/7.jpg)
DEVOPS
More features quicker Stability
![Page 8: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/8.jpg)
DEVOPS
DEV OPS
FACILITATE AND AUTOMATE DEPLOYMENT
HELP DEV TEAM BECOME AUTONOMUS
FACILITATE DIAGNOSTIC
GET DEVS INTERESTED IN « PROD STUFF »
GIVE ACCESS TO LOGS
GIVE ACCESS TO MONITORING
OPTIMIZE LOGS
DEFINE METRICS
QUALITY TESTS
MONITOR YOUR APPS
![Page 9: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/9.jpg)
DEV IN DEVOPS
![Page 10: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/10.jpg)
DEV IN DEVOPS
IMPROVEMENTS
▸ The ability to accurately track dependencies
▸ Services that are resilient and degrade gracefully
▸ The ability to archive data to manage the size of the production data set
▸ The ability to easily search and understand log messages across services
▸ The ability to trace requests from users through multiple services
▸ Simple, centralized runtime configuration using feature flags, …
![Page 11: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/11.jpg)
DEV IN DEVOPS
▸ Make Dev responsible for their own services
▸ Integrate Dev into the incident management processes
▸ Have Dev cross-train Ops
▸ Embed Ops knowledge and capabilities into Dev
▸ Design for IT Operations
EMBED IT OPERATIONS INTO DEVELOPMENT
![Page 12: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/12.jpg)
OPS IN DEVOPS
![Page 13: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/13.jpg)
OPS IN DEVOPS
IMPROVEMENTS
▸ Self service
▸ Infrastructure as code
▸ Standardize infrastructure
▸ Make all infrastructure data visible
▸ Make application info data visible
▸ Modify the incident resolution process and blameless post-mortems
▸ Monitor the health of the deployment pipelines
![Page 14: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/14.jpg)
OPS IN DEVOPS
▸ Single “repository of truth” containing both the code and environments
▸ Create the one-step Dev, Test and Production environment build process
▸ Extend the deployment pipeline processes into production
EXTEND DEVELOPMENT INTO IT OPERATIONS
![Page 15: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/15.jpg)
WE ARE ALL DEVOPS
WE ARE ALL DEVOPS
▸ Stop think . Do
▸ Share responsability
▸ It’s Never Someone Else’s Problem
▸ Dedicated Designated Ops in every team
![Page 16: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/16.jpg)
▸ Different type of devops, but devops is possible for all type and size of companies
▸ Iterate!
▸ Identify bottleneck and remediate
▸ Define small target and share results
DEVOPS IS FOR ALL
DEVOPS IS FOR ALL
![Page 17: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/17.jpg)
DEVOPS IS FOR ALL
▸ AUTONOMUS TEAMS
▸ Trust but verify
▸ Local autonomy, but global consistency
▸ SELF SERVICE
▸ How ? API, web portal, infra as code , AWS accounts, …
▸ AUTOMATE
▸ If you are tired to do it manually. Automate it.
▸ REDUCE NON-VALUE ADDED
▸ If isn't your business it's maybe better don't do it your self
KEY OF SUCCESS : REDUCE FRUSTRATION
![Page 18: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/18.jpg)
INFRASTRUCTURE
![Page 19: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/19.jpg)
ON DEMAND ( IAAS )
EASIER TO REBUILD THAN TO REPAIR
INFRASTUCTURE
![Page 20: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/20.jpg)
INFRASTRUCTURE
IMMUTABLE INFRASTRUCTURE▸ Automate the setup and deployment for every part and
every layer of your infrastructure.
▸ Never change any part of your system once it is deployed. If you need to change it, deploy a new system.
AVOID HISTORY
![Page 21: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/21.jpg)
INFRASTRUCTURE
INFRASTRUCTURE AS CODE
Versionning and code review
Easier track changes
Bootstrap your infrastructure from scratch
Automate
![Page 22: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/22.jpg)
CI/CD
![Page 23: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/23.jpg)
TESTING
![Page 24: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/24.jpg)
TESTING
CONTINUOUS TESTING
▸ Automated tests
▸ code quality
▸ Unit tests
▸ Stress tests
▸ Don’t forget to test infrastructure (load banacer, recovery etc.)
▸ Extend you build pipeline
![Page 25: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/25.jpg)
SECURITY
![Page 26: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/26.jpg)
SECURITY
INFORMATION SECURITY AS EVERYONE’S JOB, EVERY DAY
▸ Updates
▸ Use secrets management
▸ Integrate security into our CI/CD pipeline
▸ Check code libraries and their recommended configurations
▸ Dependency scanning
▸ Static analysis
▸ Dynamic analysis (ex: OWASP Zed Attack Proxy + Selenium)
![Page 27: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/27.jpg)
SECURITY
SECURITY METRICS MONITORING
▸ Successful and unsuccessful user logins
▸ User password resets
▸ User email address resets
▸ Database syntax error
▸ User credit card changes
![Page 28: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/28.jpg)
DEPLOYMENTS
![Page 29: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/29.jpg)
DEPLOYMENTS
DEPLOYMENTS ARE ROUTINE AND PREDICTABLE
No stress . It’s a routine.
On business day when everyone is already in the office and without our customers even noticing
![Page 30: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/30.jpg)
DEPLOYMENTS
HOW▸ Work in small batches and delivering quickly and
frequently
▸ Blue green deployment
▸ Feature toggle
▸ Canary testing
▸ Dark launch
▸ Automated deployments
![Page 31: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/31.jpg)
MONITORING
![Page 32: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/32.jpg)
ALERTING
▸ Mesure service health not only server health (Health pattern)
▸ Reducing Alert Fatigue
▸ Cut alerts that aren’t actionable
▸ Adjust thresholds
▸ Consolidate related alerts
▸ Give alerts relevant names & descriptions
▸ Make sure the right people are getting alerts
▸ Multiple notification rules and choose right channel
▸ Use incident management tools(Pagerduty, Victoryops,..)
![Page 33: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/33.jpg)
MONITORING
IF IT MOVES, TRACK IT
ANOMALY DETECTIONTRENDS
![Page 34: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/34.jpg)
APM
End user monitoring
Application Performance Management
![Page 35: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/35.jpg)
DELIVERY PIPELINE METRICS - TEAM LEVEL DASHBOARD
![Page 36: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/36.jpg)
LOGS
![Page 37: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/37.jpg)
LOGS
▸ Set your log severity right
▸ Remember you will not be the only one reading these logs
▸ Track your communication with other systems
▸ Log exceptions correctly
▸ Use an ID to track your events
▸ Normalize yours logs
▸ Separate and Centralize your Log Data
![Page 38: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/38.jpg)
LOGS
![Page 39: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/39.jpg)
DOCUMENT, MEASURE, COMMUNICATE AND COLLABORATE
![Page 40: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/40.jpg)
DOCUMENT
WRITING EFFECTIVE DOCUMENTATION
▸ Treat documentation like a requirement
▸ Put the Information in the Most Appropriate Place
▸ *Keep your documentation close to the code
▸ Keep documentation just simple enough, but not too simple
▸ Avoid creating large volumes of documentation of questionable detail which become obsolete shortly after they are written
![Page 41: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/41.jpg)
MEASURE
MEASURE PROGRESS
▸ KPIs
▸ Deployment frequency
▸ Deployment speed
▸ Failure rate
▸ Time from code to prod
▸ Share goals
▸ Increase the visibility of work
![Page 42: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/42.jpg)
COMMUNICATE
▸ Find better ways to communicante
▸ Promote day to day direct communication
▸ Coffee machine meetings
▸ Chat (Slack, HipChat, Team,…)
▸ Use video (Slack, Skype, Hangout,..)
▸ Quality over quantity
▸ Post Mortem meetings
▸ Bootcamps,…
IN DEVOPS CULTURE, COMMUNICATION AND COLLABORATION ARE KEY
![Page 43: Devops (start walking in the same direction) by ops](https://reader033.vdocuments.mx/reader033/viewer/2022051006/58ecd7a11a28ab47018b46f5/html5/thumbnails/43.jpg)
COLLABORATE
TIPS FOR BETTER COLLABORATION
▸ Change the seating and rotate people through development
▸ Improve the environment for collaboration (Breakout areas, whiteboards, nice meeting rooms, a decent coffee machine)
▸ Cross domain workshops for analyze your process
▸ Block time for process improvement work (lean)
▸ Run lunch and learn / brown bag sessions
▸ Put a persistent chat room in place