linux tag 2013 data center automation with yadt
DESCRIPTION
In this talk we will talk about the design decisions behind YADT and how it helps reduce the complexity of your datacenter infrastructrure while at the same time enabling you to fully automate almost all of it. YADT - an Augmented Deployment Tool is a next generation data center deployment and system management tool. It consists of two central components: Firstly, a central management of dependencies between services, systems and software packages, and secondly a deployment based on packages for both software and configuration management. YADT is developed at and used by ImmobilienScout24 as an Open Source project. The software makes it easy to deploy your software, configuration and operating system updates. It also provides a service-oriented abstraction layer which makes dependencies explicit and comprehensible. YADT will update packages and restart the services in the correct order within your data center. It understands the traditional dependencies between packages as well as dependencies between services, even over the network.TRANSCRIPT
![Page 1: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/1.jpg)
Data Center Automationwith YADT
Berlin | 23.05.2013 | Schlomo SchapiroSystems Architect, Open Source Evangelist
License: http://creativecommons.org/licenses/by-nc-nd/3.0/
![Page 2: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/2.jpg)
Slide 2 | Data Center Automation with YADT | Schlomo Schapiro
www.ImmobilienScout24.de
>2 billion PI per month
2 data center with ~1400 VM
total of ~600 employees
~30 crossfunctional IT teams
~160 in IT
15 years in business
part of Deutsche Telekom
![Page 3: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/3.jpg)
Slide 3 | Data Center Automation with YADT | Schlomo Schapiro
![Page 4: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/4.jpg)
Slide 4 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT
![Page 5: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/5.jpg)
Slide 5 | Data Center Automation with YADT | Schlomo Schapiro
The Big Picture – Continuous Delivery Platform
BusinessDecisionTo GoLive
All Humans are on the Same Side
BuildConfigDeployTestSystems-Management
Automation
Scale OutData Centers
![Page 6: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/6.jpg)
Slide 6 | Data Center Automation with YADT | Schlomo Schapiro
1st task: Reduce Systemic Complexity
File Management
Systems Management
![Page 7: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/7.jpg)
Slide 7 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
![Page 8: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/8.jpg)
Slide 8 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
Libraries
Applications
Configuration
Operating System
100 %
![Page 9: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/9.jpg)
Slide 9 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QA
YUMRepos
![Page 10: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/10.jpg)
Slide 10 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
Interface
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QA
YUMRepos
![Page 11: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/11.jpg)
Slide 11 | Data Center Automation with YADT | Schlomo Schapiro
PROD
DEV
InfrastructureBuild
ApplicationBuild QA
YUMRepos
Configuration Build – Pre-built Config Content
ConfigurationBuild
![Page 12: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/12.jpg)
Slide 12 | Data Center Automation with YADT | Schlomo Schapiro
Configuration Build – Pre-built Config Content
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
![Page 13: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/13.jpg)
Slide 13 | Data Center Automation with YADT | Schlomo Schapiro
TemplatingVariables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
![Page 14: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/14.jpg)
Slide 14 | Data Center Automation with YADT | Schlomo Schapiro
we use1 RPM
per host
TemplatingVariables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
![Page 15: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/15.jpg)
Slide 15 | Data Center Automation with YADT | Schlomo Schapiro
Sharedresponsability
Ops, Dev, QA ...
we use1 RPM
per host
TemplatingVariables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
![Page 16: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/16.jpg)
Slide 16 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Continuous Release Delivery
PROD
DEV
InfrastructureBuild
QA
YUMRepos
ConfigurationBuild
ApplicationBuild
![Page 17: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/17.jpg)
Slide 17 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Package Promotion Pipeline
YUMRepo
TestedRepo
YUMRepo
ReleasedRepo
YUMRepo
ComittedRepo
DEV Pre-PROD PROD
![Page 18: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/18.jpg)
Slide 18 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Package Promotion Pipeline
YUMRepo
TestedRepo
YUMRepo
ReleasedRepo
YUMRepo
ComittedRepo
DEV Pre-PROD PROD
CI
Deploy&
Test
Auto-promote release if tests OK
Deploy&
Test
Business decision
to Go Live with
tested versions
Deploy&
Test
![Page 19: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/19.jpg)
Slide 19 | Data Center Automation with YADT | Schlomo Schapiro
File Management through many YUM repositories
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QAYUMRepo
YUMRepoYUM
RepoYUMRepo
![Page 20: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/20.jpg)
Slide 20 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT
![Page 21: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/21.jpg)
Slide 21 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Unified Dependency Tree
Web Server
HTTPD
Tomcat
App Server
Load Balancer
![Page 22: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/22.jpg)
Slide 22 | Data Center Automation with YADT | Schlomo Schapiro
HTTPD
Tomcat
web-app
httpd
config
tomcat
YADT – Unified Dependency Tree
initscript
initscript
![Page 23: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/23.jpg)
Slide 23 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
![Page 24: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/24.jpg)
Slide 24 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
![Page 25: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/25.jpg)
Slide 25 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
1st
chun
k
![Page 26: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/26.jpg)
Slide 26 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
1st
chun
k
2nd chunk
![Page 27: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/27.jpg)
Slide 27 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT
Reduce Systemic Complexity
Keep It Simple
![Page 28: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/28.jpg)
Slide 28 | Data Center Automation with YADT | Schlomo Schapiro
Automation is our Continuous Delivery Platform
BusinessDecisionTo GoLive
All Humans are on the Same Side
BuildConfigDeployTestSystems-Management
Automation
Scale OutData Centers
![Page 29: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/29.jpg)
Slide 29 | Data Center Automation with YADT | Schlomo Schapiro
DevOps
We build it togetheras one team.
We understandit completely.
And it is fully automated.
![Page 30: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/30.jpg)
Slide 30 | Data Center Automation with YADT | Schlomo Schapiro
www.yadt-project.org
Stand 7.1C145
![Page 31: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/31.jpg)
Slide 31 | Data Center Automation with YADT | Schlomo Schapirogo.schapiro.org/slides
![Page 32: Linux tag 2013 Data Center Automation with YADT](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554baaaab4c905b3618b53aa/html5/thumbnails/32.jpg)
Slide 32 | Data Center Automation with YADT | Schlomo Schapiro
Kontakt:Immobilien Scout GmbHAndreasstraße 1010243 Berlin
Fon: +49 30 243 01-1229 Email: [email protected]: www.immobilienscout24.de
Thank you very much!Please contact me for further questions and discussions.
All images are either public domain, used in appropriate context or taken from openclipart.org