"how to build powerful ci / cd based on gitlab and docker", aleksandr matkovskiy &...
TRANSCRIPT
![Page 1: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/1.jpg)
Continuous Integration & Delivery
To Infinity and Beyond…
How to build powerful CI / CD based on GitLab and Docker
![Page 2: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/2.jpg)
Vladislav Anikin Aleksandr MatkovskiyTeam Leader & Software architect in OrbitLift a Provectus company
Head of IT Department in Provectus
Say Hello!
Уже не Dev :( Вряд ли OPs :)
![Page 3: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/3.jpg)
OrbitLift Department
Before• 3 Developers • 1 QA (manual) • 1 DevOps (part time) • 1 Team LeadService model
Now• 6 Developers • 1 QA (automation) • 1 DevOps (part time) • 1 Team LeadProduct model
+ +
Jenkins GitLab WebHooks
+ +
GitLabDocker Aleksandr
![Page 4: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/4.jpg)
Jenkins + GitLab + WebHooks
✓ Price - it’s free :) ✓ Кастомизация и поддержка плагинов ✓ Устоявшееся CI решение на рынке ✓ Выполняет все необходимые задачи ✓ Интегрируется с Docker
- Price of Setup - it’s NOT free - Ужасный UI / UX - Where is my car memory, dude! - Время на конфигурацию / кастомизацию - Иногда доводит до слез
Pros Cons
How It Works
commit hook build trigger wait…………..
![Page 5: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/5.jpg)
Service Product Model & Git Flow
• Reduce routine costs • Semantic Versioning 2.0.0 • Unit Test & Automation • Clean & Simple QA phase • Ability to Revert • Ability to Revert of Revert
![Page 6: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/6.jpg)
Requirements
![Page 7: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/7.jpg)
Requirements• Поддержка веток: master, hotfix, development, feature, rc • Полная пересборка свежих изменений (commits) • Подержка Tests & Coverage steps • Возможность очистки dev dependencies (housekeeping) • Доступность любого success build с заданным TTL • Fast, Clean, Secure delivery to destination • Удобство управления процессами (pipelines) • Plug & Play features • Lightweight
![Page 8: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/8.jpg)
CI Concept
![Page 9: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/9.jpg)
CI & CD Concept
![Page 10: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/10.jpg)
Как «Я» понял ТЗ от Dev Team
• Проект на PHP • Один репозиторий • Конечное количество веток
• Хранение билдов в абстрактном хранилище
• Возможность развернуть любой билд на одной из площадок (QA, Dev, RC)
• Привязка ветки к площадке для разворачивания
![Page 11: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/11.jpg)
Реали …• 2 различных репозитория • 2 разных проекта: Backend - PHP+nodejs, Frontend - Angular
• Не должно быть ограничений по количеству веток
• Разное окружение на build и execute серверах
• Нет жесткой привязки к веткам
![Page 12: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/12.jpg)
Technology
![Page 13: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/13.jpg)
Why GitLab
• With more than 100,000 organizations self-hosting GitLab • Docker private registry
Source: https://about.gitlab.com/2017/06/29/whats-next-for-gitlab-ci/
![Page 14: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/14.jpg)
executorsGitLab Runner executors:
✓ Shell ✓ Docker ✓ Docker Machine (auto-scaling) ✓ Parallels ✓ VirtualBox ✓ SSH ✓ Kubernetes
![Page 15: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/15.jpg)
Why • Минимальные накладные расходы
(containers vs virtualization) • Изолировання среда • Скажи НЕТ vendor lock • Сдаться облаку - НЕТ • Небывалая простота доставки
![Page 16: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/16.jpg)
Why
![Page 17: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/17.jpg)
Because
![Page 18: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/18.jpg)
Implementation of the concept
![Page 19: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/19.jpg)
Логическая схема Build Server
![Page 20: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/20.jpg)
Логическая схема Execute Server
![Page 21: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/21.jpg)
Capabilities CI/CD ✓ Не зависит от проекта/языка. ✓ Передача переменных Project -> CI/CD через Secret variables (GitLab) ✓ Гибкое управление процессом развертывания ✓ Подготовка окружения для сборки и тестирования ✓ Разное окружение на Execute и Build серверах ✓ Сборка по расписанию
![Page 22: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/22.jpg)
GitLab CI Logic Scheme and PipelineGitLab CI Logical scheme
Pipeline on Feature branch
Pipeline on Develop branch
![Page 23: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/23.jpg)
Cleanup
• Инструкция «пока» НЕ хранится в репозитории проекта
• Dev команда сама решает, как прибраться
• Индивидуальный набор инструкция для каждого проекта
![Page 24: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/24.jpg)
Save build in storage
• Разделение прав - только на запись и только на чтение - AWS IAM • Уникальное имя архива по шаблону _имяВетки-хешКоммита_ • Шифрование архива (Опция) • Перенос старых билдов из S3 в Glacier (Опция)
![Page 25: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/25.jpg)
Требования к CI / CD✓ GitLab v10+ ✓ Storage for builds ✓ Compute capacity : 1vCPU, 1GB RAM, 5GB Storage and more ✓ Software requirements: Docker, Docker-compose, GitLab-Runner on servers
We use:
✓ GitLab - 10.4.1 ✓ Storage - AWS S3 (+AWS Glacier in future) ✓ Compute capacity: AWS EC2 - t2.micro ✓ Software requirements:
✓ Ubuntu 16.04 ✓ Docker version 17.09.0-ce ✓ docker-compose version 1.17.0
![Page 26: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/26.jpg)
SummaryPipelines history
![Page 27: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/27.jpg)
SummaryRun custom pipeline on-demand
![Page 28: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/28.jpg)
SummaryFull control and stop on error
![Page 29: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/29.jpg)
SummarySupporting of Tests & Coverage
![Page 30: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/30.jpg)
SummaryContinuous Delivery and Multiple Destinations
![Page 31: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/31.jpg)
SummaryNightly Builds is our everything
![Page 32: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/32.jpg)
Vision of IPWe can use it anywhere!
etc…
![Page 33: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/33.jpg)
Bugs and Problem
1. Выполнение нескольких Pipeline одновременно. Привет ночным сборкам
2. Миграция cleanup инструкции в репозиторий проекта
3. Сделать CD для CI/CD
![Page 34: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/34.jpg)
Achievements
У меня локально работает Продуктивная атмосфера Счастливые QA
![Page 35: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/35.jpg)
Jenkins VS GitLab
Jenkins GitLab CIAverage Build Time6m 27s 0m 31s
excluding unit tests excluding unit tests
Memory Consumption~1,3Gb per build ~100Mb
AWS Instance (minimal requirement)m3.medium t2.micro
![Page 36: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/36.jpg)
Where is my money?~ 21 build per day
6m 27s 0m 31s$15.65 per hour
(DOU Dec 2017)
$34,27 per day $2,66 per day
$685,4 per month $53,2 per month
$8464,69 per year $657,02 per year
![Page 37: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/37.jpg)
Where is my money?
AWS
m3.medium t2.micro$0.077 per hour
$1.84 per day $0.27 per day
$55.44 per month $8.35 per month
$674.52 per year $101.6 per year
$0.0116 per hour
![Page 38: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/38.jpg)
What’s next?
Revert deployments
Browser Performance Tests
SAST (Static App Security Tests)
Zero Downtime Deployments
![Page 39: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/39.jpg)
Почему все получилось?
![Page 40: "How to build powerful CI / CD based on GitLab and Docker", Aleksandr Matkovskiy & Vladislav Anikin](https://reader034.vdocuments.mx/reader034/viewer/2022051318/5aac106e7f8b9adb278b46a7/html5/thumbnails/40.jpg)
Vladislav Anikin Aleksandr MatkovskiyTeam Leader & Software architect in Orbitlift a Provectus company
Head of IT Department in Provectus
Thanks for patience
email: [email protected] skype: just__a__man1
email: [email protected] skype: matkovskiy.aleksandr