@gitenterprise#DevoxxPL
Platinum Sponsors:
Speed-up your Continuous Delivery
with BigData
Luca Milanesio
GerritForge
@gitenterprise#DevoxxPL
• Founded in the UK
• HQ in London, Offices in
the UK and Italy
• Committed to
OpenSource
About GerritForge
@gitenterprise#DevoxxPL
• Co-founder and
Director of GerritForge
• over 20 years in Agile
Development and ALM
• Contributor to many
OpenSource projects
• Mind and hands of
@gitenterprise and
@gerritreview
About Luca Milanesio
@gitenterprise#DevoxxPL
NOT a lecture on Continuous Delivery
NOT a lecture on Code Review
NOT a lecture on BigData
What NOT to expect?
@gitenterprise#DevoxxPL
Continuous Delivery Pipeline
Push Build
Unit Tests
Push Build
Unit Tests
Automated
Tests
Push Build
Unit Tests
Automated
Tests
Beta Users
Tests
User's feedback Publish to all Users
Failed
Failed
@gitenterprise#DevoxxPL
Code Review & CD Validation
A1
refs/for/master
master
C2 C2*
A1 C2(local)
masterC2*
push
Verified: -1
(build failed)
push
Verified: +1
(build OK)
Reviewed: +2
CI Build: OK
C2*
CI Build: OK
amend
@gitenterprise#DevoxxPL
Git/Gerrit(Code Review)
Jenkins (Continuous
Integration & Delivery)
Docker(Release and Service
Packaging)
Mesos &
Marathon(Delivery and Resource
Manangement)
Components
@gitenterprise#DevoxxPL
We use micro-services
• 100+ Components
• 200+ Repositories
• 300+ People and 100,000+ of review comments
• Cross-components [implicit] dependencies
• Lots of build chains
• 10,000+ of builds
Now feel the pain :-(
@gitenterprise#DevoxxPL
"It was working until 5
mins ago … then
someone changed
something"
Rings any
bell?
Problem #1 : Ripple effect
@gitenterprise#DevoxxPL
In order to deliver Feature ABC
I NEED
Ver. N of Service-A with
Ver. M of Service-B and
Ver. K of Service C
Problem #2 : Dependencies
@gitenterprise#DevoxxPL
"Where can I find an environment
to
run ALL my Micro-Services
integration tests?"
Problem #3 : Resources
@gitenterprise#DevoxxPL
Release small changes is GOOD
Release often every change is GOOD
BUTHow long it takes the "Big Picture" and End-to-End?
Where is my bottleneck?
Problem #4 : Cycle time
@gitenterprise#DevoxxPL
Micro-Services are
distributed
Builds are distributed
Deployments are
distributed
Tackling THIS
have you ever felt
like drowning in
your logs?
Problem #5 : Lots of logs
@gitenterprise#DevoxxPL
Problems #1 + #2 + #3 + #4 + #5 =
lead to your daily
Micro-Services mental meltdown
Problem #6 : all together
@gitenterprise#DevoxxPL
• Collect all review events
• Collect all logs
• Channel them to a
central store
• Crunch and Crunch
continuously
• Never delete
• Process, inspect and
learn
BigData to the rescue
@gitenterprise#DevoxxPL
•Problem #1 – Ripple effect?
•Problem #2 – Dependencies?
•Problem #3 – Resources?
•Problem #4 – Cycle Time?
•Problem #5 – Logs aggregation?
•Problem #6 – Avoid mental meltdown?
Did we solve the problems?
@gitenterprise#DevoxxPL
• Collect more data (System, JVM, Review
scores)
• Real-time statistics
• Events correlation
• Feedback loop to Code-Review
• Production data harvest
• More?
Future directions