rise of the machines - automate your development
TRANSCRIPT
Rise of the machinesAutomate your development
@svenpet
Disclaimer
Scripts Chef, Puppet, etc better build pipelines
Disclaimer
Ideas through examples
Rise of the machines
ROBOTSare coming to our day to day lives
Rise of the machines
IN Software Development
Sven PetersAtlassian Evangelist
@svenpetsvenpet.com
Automation in Software Developmentmy personal story
build scripts (make)
1996JUnit
2001
2004
ant -> later maven
2005
Eclipse (code generation)
Hudson
2006
2009
Puppet
1998
GUI builder2013
Docker
Build & TestsDeployments
Small scripts
What are we automating right now?
What are we automating right now?
Code Reviews
Stand Ups
Ops
Support
Merging
Engineering Health
Bots everywhere
Coding bots Ops bots Service bots Doc botsReport botsTest bots
Welcome… orI’m new
Add ons
PostgresScala
IntelliJ
Git
JDK
Happy installing & setup… for the next 8 hours
Agent CharlieMeet
Download
Install
Configure
Agent CharlieMeet
Download
Install
Configure
Agent CharlieMeet
Puppet, Shell & RubyWorks with OSX & Ubuntu
Atlassian Plugin SDK
Bitbucket SSH Keys
Git
Mercurial
HipChat
HomeBrew
IntelliJ IDEA
JDK
Maven
PostgreSQL
Stash Keys
and much more…
Without Agent Charlie
With Agent Charlie
Coding botsAutomate your Coding phase
27’’ is not enough
Chat
EmailRepo
manager
IDE
Command
line
Issue
Tracker
Issue tracker workflow
TODO IN PROGRESS IN REVIEW DONE
BRANCH CODE REFACTOR PULL MERGE
Developer workflow
BRANCH CODE REFACTOR PULL MERGE
What you should do
TODO IN PROGRESS IN REVIEW DONE
BRANCH CODE REFACTOR PULL MERGE
What you do
TODO IN PROGRESS IN REVIEW DONE
Keep the team updated!
Smart commitsmention issue_key and #trigger the issue tracker
Github JIRA YouTrack Trello Bitbucket
Control the issue by commits
TODO IN PROGRESS IN REVIEW DONE
JRA-566 #start-Progress
JRA-566 #start-review
fixes #566
No tool switching
Robot is just receiving
commands
TODO IN PROGRESS IN REVIEW DONE
branchcommit
Controlled by action
pullcreate review
request
mergeclose review
Our workflow
TODO IN PROGRESS IN REVIEW DONE
branch pull merge
branch pull merge
master
JRA-554-bugfix
branch pull merge
master
commits code directly to the master branch!
branch pull merge
master
branch pull merge
master
JRA-554-bugfix
branch pull merge
master
JRA-554-bugfix
bad surprisesh*t happens, but can be avoided
bad surprisesh*t doesn’t happen that often
no
branch pull merge
don’t branch from a red build
master
JRA-554-bugfix
branch pull merge
master
JRA-554-bugfix
invite the best reviewers
Recent comitters or author
auto suggest reviewer
branch pull merge
minimum 2 approvals
master
JRA-554-bugfix
branch pull merge
unapprove reviewers when code changes
master
JRA-554-bugfix
The need to update older versions
master
Version 1.1
Version 1.0
Support older versions
master
Version 1.1
Version 1.0
JRA-234-hotfix
Support older versions
master
Version 1.1
Version 1.0
JRA-234-bugfix
Support older versions
boring!
Huhu I can do it!
master
Version 1.1
Version 1.0
JRA-234-bugfix
auto merges
Test botsSMARTER Test Automation
&:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6; cursor: pointer;
}
&:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6; cursor: pointer;
}
Static code analysis can catch this!Findbugs - Checkstyle - PMD
Listen to what Freud tells you about
source codeyour
Freud Bot
&:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6;
cursor: pointer; }
You should use a variable instead for . I suggest that you use a variable of similar color: #366CA6
@globalHeaderPrimaryActionFallbackColor @globalHeaderTopColor @primaryLinkColor
Freud Botfor Pull Requests
Freud Bot
less review work makes suggestions
We are trying hard to write awesome code
packagecom.miguelcatalan.materialsearchview;
importandroid.app.Activity;
importandroid.widget.ListView;
importandroid.widget.RelativeLayout;
importandroid.widget.TextView;
importjava.lang.reflect.Field;
importjava.util.List;
/**
*@authorMiguelCatalanBañuls
*/
publicclassMaterialSearchViewextendsFrameLayoutimplementsFilter.FilterListener{
privateMenuItemmMenuItem;
privatebooleanmIsSearchOpen=false;
publicMaterialSearchView(Contextcontext){
this(context,null);
}
publicMaterialSearchView(Contextcontext,AttributeSetattrs){
this(context,attrs,0);
@OverridepublicvoidonTextChanged(CharSequences,intstart,intbefore,intcount){
mUserQuery=s;
startFilter(s);
MaterialSearchView.this.onTextChanged(s);
}
}
LOC: 32
MyAwesomeClass
One month later
packagecom.miguelcatalan.materialsearchview;
importandroid.app.Activity;
importandroid.content.Context;
importandroid.content.Intent;
importandroid.content.pm.PackageManager;
importandroid.content.pm.ResolveInfo;
importandroid.content.res.TypedArray;
importandroid.graphics.Rect;
importandroid.graphics.drawable.Drawable;
importandroid.os.Build;
importandroid.os.Parcel;
importandroid.os.Parcelable;
importandroid.speech.RecognizerIntent;
importandroid.text.Editable;
importandroid.text.TextUtils;
importandroid.text.TextWatcher;
importandroid.util.AttributeSet;
importandroid.util.Log;
importandroid.view.KeyEvent;
importandroid.view.LayoutInflater;
importandroid.view.MenuItem;
importandroid.view.View;
importandroid.view.inputmethod.InputMethodManager;
importandroid.widget.AdapterView;
importandroid.widget.EditText;
importandroid.widget.Filter;
importandroid.widget.Filterable;
importandroid.widget.FrameLayout;
importandroid.widget.ImageButton;
importandroid.widget.ListAdapter;
importandroid.widget.ListView;
importandroid.widget.RelativeLayout;
importandroid.widget.TextView;
importcom.miguelcatalan.materialsearchview.utils.AnimationUtil;
…one month later
LOC: counting….
MyAwesomeClass
//RequestFocus
mSearchSrcTextView.setText(null);
mSearchSrcTextView.requestFocus();
if(animate){
AnimationUtil.fadeInView(mSearchLayout,AnimationUtil.ANIMATION_DURATION_MEDIUM,newAnimationUtil.AnimationListener(){
@Override
publicbooleanonAnimationStart(Viewview){
returnfalse;
}
@Override
publicbooleanonAnimationEnd(Viewview){
if(mSearchViewListener!=null){
mSearchViewListener.onSearchViewShown();
}
returnfalse;
}
@Override
publicbooleanonAnimationCancel(Viewview){
returnfalse;
}
});
}else{
mSearchLayout.setVisibility(VISIBLE);
if(mSearchViewListener!=null){
mSearchViewListener.onSearchViewShown();
}
}
mIsSearchOpen=true;
}
/**
*Closesearchview.
*/
publicvoidcloseSearch(){
…one month later
LOC: 986
MyAwesomeClass
Code over time
86%
20
1045
27
test coverage
deprecated methods
old unit tests (JUnit 3)
compiler warnings
Code over time
83% test coverage
23 deprecated methods
1136 old unit tests (JUnit 3)
36 compiler warnings
Code over time
75% test coverage
32 deprecated methods
1289 old unit tests (JUnit 3)
45 compiler warnings
Developers are trying to solve problems
Developers don’t write bad code on purpose
Calling
Dr. Codemeasure constantly the engineering health
measure constantly the engineering health
Calling
Dr. Code
jira.stats.tests.junit.3.count
No more JUnit 3 tests!JUnit 3 JUnit 4
jira.stats.bundled.jars.in.plugins
Inspect all the things
jira.stats.tests.junit.3.count
Hall of Shame!You’ll enter the
JS not minified
API compatibility :(
New JUnit3 test
Caught by Dr Code?
Hall of Shame!You’ll enter the
Wallboard - so everyone sees it!
Hall of Fame!You’ll enter the
Fixed a matrix?
publicabstractclassTask{
publicfinalvoidexecuteWith(Callbackcallback){
execute();
if(callback!=null){
callback.call();
}
}
publicabstractvoidexecute();
}
publicabstractclassTask{
publicfinalvoidexecuteWith(Callbackcallback){
execute();
if(callback!=null){
callback.call(20);
}
}
publicabstractvoidexecute();
}
Old version New version
publicabstractclassTask{
publicfinalvoidexecuteWith(Callbackcallback){
execute();
if(callback!=null){
callback.call();
}
}
publicabstractvoidexecute();
}
publicabstractclassTask{
publicfinalvoidexecuteWith(Callbackcallback){
execute();
if(callback!=null){
callback.call(20);
}
}
publicabstractvoidexecute();
}
Old version New version
Old version New version
Old version New version
Hard to spot?
Image Compare Bot
Compare images
(on build)
Alert on differences
(build warning)Easy reporting
(build report)
Our functional tests ran for
h ursso we ran them parallel
Test block 1
Test block 2
Test block 3
Test block 4
Test block 5
statically sized test blocks
waiting for the longest test block
HallelujahFunctional Test Balancer
T1
T 2
T 3
T4
T5
T6
T7
T8
T9
T10
T11
T 12
T13
T14
Hallelujah
build status
build status
build status
Flaky Test Detector
Flaky Test Detector
failing test
rerun test
successful test
test in quarantaine
continue build
create issue
1888 flaky tests detected
817 quarantined tests
fixed
Flaky Test Detector
no log file searching
forces fixing flaky tests
no manual reruns
no more ‘this one again’ moments
OPS botsAutomated Operations
Ops nowproblem
ops
Slow feedback loop
devprob
Faster feedback loop
devprob
Faster feedback loopknows which team -> service responsibility
Faster feedback loop
But that’s a lot of configuration!
Faster feedback loop
add team in deployment descriptor
add connection to
pager duty
Own your codeend-to-end
Prepared for the release?
WAIT…
Release Notes?
New Feature
Improvement
Bugfix
Do you know which features really ship?
Release Branch
Merged JRA-345
Commit messages
+Issue
information
=Release notes
Release Notes Bot
Label in GitHub Issues
Label in GitHub Issues
New features
Improvements
Fixed bugs
Issue Description as
Release NotesHuman readable
issue descriptions
Think about customer value
No surprises
Service botsAutomate your Service
Support Developer
scanning log filesask for log file system knowledge
pause work
Support Developer
scanning log filesask for log file system knowledge
Support Hercules
scanning log filesask for log file system knowledge
Teaching the robot
Hercules get’s smarter
suggest read
update
{ Well f*%k meTHE ROBOT
was right! }
Your boss has a really important task for your team
Everyone is working from home today
by
With chat you can reach everyone immediately
All these questions!
How do I download the Sprint report?
How do I set up a build for my
feature branch?
What Java versions do we
support?
!askI’ll look it up for you
Self service for chat
!ask the knowledge base first!
REPORT botsManual Reporting sucks
Stand UpAutomating
s ?
too longsetup VC calls
timezones remind people
Stand UpProblems
Never miss a stand up
Music - Start at 09:29:00
Crontab!
Video - Open at 09:30
No video set up time
Stand up report bots
colocated
distributed
different timezones
Status update
Team update
Stand up bot
Exception
Build failure
New issue
Deployment
Chat Botseverywhere
ΩΩΩΩ
Builds
ΩΩΩΩ
Deployments
ΩΩΩΩ
Exceptions
Chat Botseverywhere
Fast feedback loops Team knowledge Start discussions
The build failed
New deployment
The build failed
New deployment
The build failed
New deployment
New important
issue
New deployment
New important
issue
New deployment
New important
issue
New deployment
Get the big picture
Get the big picture
Every team has one or more
http://
atlasb
oard.bitb
ucket.org
/
Every team has one or more
http://
atlasb
oard.bitb
ucket.org
/
see what’s going on point directly
All kind of boards
AdvocatesDeveloper
Support
Bots everywhere
Coding bots Ops bots Service bots Doc botsReport botsTest bots
Reinvent the wheel?
Lots of unmaintained projects
Writing automation code
Very specific problem
No. 1 Problem
10%for better engineering healthwork time
Technical debt
10%for better engineering healthwork time
Technical debt
is serious fun!Automation
Go - Ruby - PHP - PythonLanguages
Puppet - Chef - Ansible - DockerTechnologies
Fast feedbackSuccess
Your ProblemsMotivation
From scripts on a developer hard disc
to professional automation
Source ControlAdd code to
Code ReviewsDo
for every change
TestsWrite
Deployment ScriptsHave
Keep your automated systemsunder control
Keep your automated systemsunder control
Lots of different places
Instances for automation services
cost centreow
ner
serv
ice
We use labelsfor better overview
Micros <our own PaaS>
Docker
Java
Go
Python
micros service:create <service-name> Service descriptor
Owner
PagerDuty
Cost Centre
MicrosSimple to create & register
Were are we heading?
Better customer insides
better conclusions
collecting more data
Were are we heading?
Faster dev feedback loops
less time updating & searching
integrated tools
Were are we heading?
Better Quality
deeper code analysis
smarter tests
Were are we heading?
Building software
Parts can be automated
craftis a
Creating customer valueis an
art
Hard to automate
Rise of the machines
…Skynet became self aware
@svenpetsvenpet.com/talks
I’ll be back…find slides here
Photo
Credits
“VW Golf TDI Clean Diesel was 2010 8983" by Mariordo Mario Roberto Duran Ortiz - Own work. Licensed under CC BY-SA 3.0 via Commons - https://commons.wikimedia.org/wiki/
Backyard Series by syauqee mohamad "http://www.flickr.com/photos/7901820@N07/6945260735"
Tesla Model S by Chrishmt0423 from flickr
Burnt & Abandoned Computer "http://www.flickr.com/photos/99649389@N02/13262802714"