everything-as-code - a polyglot journey
TRANSCRIPT
Everything-as-codeA polyglot journey.
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 1
About me
Mario-Leander ReimerChief Technologist, QAware [email protected]
twitter://@LeanderReimerhttp://github.com/lreimerhttp://speakerdeck.com/lreimerhttp://www.qaware.de
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 2
Which language or technology do real programmers use?
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 4
My #FirstSevenLanguages
• Pascal
• Basic
• C / C++
• Assembler
• PHP
• Java
• C#
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 5
My #LastSevenLanguages
• Java
• Groovy
• TypeScript
• Ruby
• Kotlin
• Scala
• Python
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 6
There is no unanimous opinion ...
• http://spectrum.ieee.org/computing/software/the-2015-top-ten-programming-languages
• http://spectrum.ieee.org/computing/software/the-2016-top-programming-languages
• https://www.sitepoint.com/whats-best-programming-language-learn-2015/
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 10
There is no best language!
Every language is strong in a specific domain.
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 11
Real programmers are polyglot!
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 12
The IDE is our workbench.
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 13
open fun everythingAsCode() : Boolean {
everytingIsMadeFromCode() && everythingIsMadeByCode()
}
val softwareIndustrialization = everythingAsCode()
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 14
Definition of Software Industrialization
• This has nothing to do with cheap labor!
• Automation of repetitive and laborious tasks
• Better software quality through standardized, streamlined tool chain
• Well integrated tool chain leads to a higher productivity and happiness of your team
• Better cost efficiency and competitiveness
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 15
Quest for a polyglot project archetype
• Which languages are used for specific domains in our projects?
• Which tools are used for Setup, Build, Code, Test, CI, Infrastructure, Documentation?
• What are the dos and don'ts of using a specific language or technology?
+ some wishful greenfield thinking!
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 16
The polyglot journey begins.
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 17
Lightweight Developer Provisioning
• Use a build tool for the automated creation and update of a software development environment
• Software packages are expressed as dependencies
• Gradle tasks and Groovy are used instead of shell scripting
• Everything is version controlled just like ordinary source code
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 19
plugins { id 'de.qaware.seu.as.code.base' version '2.4.0' }
import static de.qaware.seu.as.code.plugins.base.Platform.isMac
seuAsCode { seuHome = { if (isMac()) '/Volumes/Everything-as-code' else 'Y:' } projectName = 'Everything-as-code'}
dependencies { // list of software dependencies ... software 'org.groovy-lang:groovy:2.4.7' software 'org.scala-lang:scala:2.11.8' software 'org.jruby:jruby:9.1.4.0'}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 20
Why Gradle beats Maven.
• Very flexible. Gradle can build everything.
• Polyglot builds are supported easily.
• Succinct build scripts. Default conventions over configuration.
• Incremental builds, reduced build times.
• New features: Kotlin support, Composite Builds, ...
• Frequent releases. Mature and stable.
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 22
apply plugin: 'application'apply plugin: 'war'apply plugin: 'kotlin'apply plugin: 'groovy'
repositories { jcenter() }
dependencies { providedCompile 'fish.payara.extras:payara-micro:4.1.1.163' // and many more ...}
task everythingAsCode() << { println 'Everything-as-code @JavaOne2016.'}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 23
There is nothing wrong with Java as primary language!
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 25
But Kotlin is a serious alternative worth considering
as primary language.
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 26
But why Kotlin? And not Scala, Clojure, ...
• Easy to learn for a Java developer.
• Null Safety!
• Loads of other useful language features.
• Small library size.
• Good IDE support.
• Wishful greenfield thinking.
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 27
@JsonIgnoreProperties(ignoreUnknown = true)data class Book(val title: String, val isbn: String, val author: String) { }
@ApplicationScopedopen class Bookshelf { private val books = listOf(Book("The Hitchhiker's Guide to the Galaxy", "0345391802")) open fun byIsbn(isbn: String): Book? = books.find { it.isbn == isbn }}
@Path("books")@Produces(MediaType.APPLICATION_JSON)open class BookResource @Inject constructor(private val bookshelf: Bookshelf) { @GET @Path("/{isbn}") open fun byIsbn(@PathParam("isbn") isbn: String): Response { val book = bookshelf.byIsbn(isbn) return if (book != null) Response.ok(book).build() else Response.status(Status.NOT_FOUND).build() }}
@ApplicationPath("api")class BookstoreAPI : Application() { override fun getClasses() = hashSetOf(JacksonFeature::class.java, BookResource::class.java)}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 28
Groovy and Spock for general testing
class BookshelfSpec extends Specification { @Subject def bookshelf = new Bookshelf()
@Unroll def "Find book #title by ISBN #isbn"() { when: 'we search a book by ISBN' def book = bookshelf.byIsbn(isbn)
then: 'the title and author are correct' book?.title == title book?.author == author
where: isbn || title | author "0345391802" || "The Hitchhiker's Guide to the Galaxy" | "Douglas Adams" "0345391829" || "Life, the Universe and Everything" | "Douglas Adams" }}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 30
Scala and Gatling for load testing
class BooksPerformanceTest extends Simulation { val conf = http.baseURL("http://localhost:18080").acceptHeader("application/json")
val feeder = csv("books.csv").random
val scn = scenario("Book Search") .exec(http("Get all books").get("/api/books")) .during(30 seconds) { feed(feeder) .exec(http("Get book by title ${Title}").get("/api/books?title=${Title}")) .pause(1 second) .exec(http("Get book with ISBN ${ISBN}").get("/api/books/${ISBN}")) }
setUp(scn.inject(atOnceUsers(10), rampUsers(50) over (30 seconds))) .assertions(global.responseTime.max.lessThan(5000)) .protocols(conf)}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 31
Pipeline-as-code
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 32
Use Gradle and Docker to start Jenkins
task createJenkinsHome() { mkdir("${System.getProperty('user.home')}/Jenkins")}
task createJenkins(type: Exec, group: 'jenkinsci', dependsOn: createJenkinsHome, description: 'Create Jenkins') { commandLine 'docker', 'run', '--name', 'jenkinsci', '-p', '8080:8080', '-p', '50000:50000', '-v', "${System.getProperty('user.home')}/Jenkins:/var/jenkins_home", 'jenkinsci/jenkins'}
task startJenkins(type: Exec, group: 'jenkinsci', description: 'Start Jenkins') { commandLine 'docker', 'start', 'jenkinsci'}
task stopJenkins(type: Exec, group: 'jenkinsci', description: 'Stop Jenkins') { commandLine 'docker', 'stop', 'jenkinsci'}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 33
Add Jenkinsfile and define pipeline#!/usr/bin/env groovy
node {
stage 'Checkout SCM'
checkout scm
stage 'Build/Analyse/Test'
sh './gradlew clean build'
archiveUnitTestResults()
archiveDistributions()
stage 'Dockerize'
sh './gradlew buildDockerImage'
stage 'Generate Documentation'
sh './gradlew asciidoctor'
}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 34
Infrastructure-as-code
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 35
Docker, Docker, Docker, ...
FROM qaware-oss-docker-registry.bintray.io/base/debian8-jre8MAINTAINER M.-Leander Reimer <[email protected]>
RUN mkdir -p /appADD build/distributions/everything-as-code-1.0.0.tar /app
WORKDIR /app/everything-as-code-1.0.0RUN chmod 755 bin/everything-as-code
EXPOSE 18080
CMD ./bin/everything-as-code
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 36
Vagrant and Ruby for local VM setup
require 'yaml'
$setup = <<SCRIPT
sudo apt-add-repository ppa:ansible/ansible
sudo apt-get update
sudo apt-get install -y ansible sshpass
SCRIPT
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/trusty32"
settings = YAML.load_file 'src/vagrant/vagrant.yml'
config.vm.provider "virtualbox" do |vb|
vb.name = settings['vm']['name']
vb.gui = false
vb.memory = "512"
end
config.vm.provision "shell", inline: $setup
end
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 37
Provisioning with Ansible (and Python)
---
# file: jenkinsci.yml
- hosts: jenkinsci
remote_user: root
tasks:
- debug: msg="Creating a Jenkins pipeline job on {{ inventory_hostname }}"
- jenkins_job:
name: Everything-as-code Pipeline
config: "{{ lookup('file', 'templates/pipeline-job.xml') }}"
url: "http://{{ inventory_hostname }}"
user: admin
password: admin
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 38
Cluster orchestration with K8S---apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: everything-as-codespec: replicas: 1 template: metadata: labels: tier: backend spec: containers: - name: everything-as-code image: "qaware-oss-docker-registry.bintray.io/lreimer/everything-as-code:1.0.0" ports: - containerPort: 18080 env: - name: PORT value: 18080
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 39
Cluster orchestration with DC/OS{ "id": "everything-as-code", "instances": 1, "cpus": 0.25, "mem": 256, "container": { "type": "DOCKER", "docker": { "image": "qaware-oss-docker-registry.bintray.io/lreimer/everything-as-code:1.0.0", "network": "BRIDGE", "portMappings": [ { "hostPort": 18080, "containerPort": 18080, "protocol": "tcp", "servicePort": 18080 } ] } }, "env": { "PORT": 18080 }}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 40
Documentation-as-code
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 41
Yes, we need documentation!
• And no, the source code is not enough.
• Writing technical docs with Word is ! " #
• Documentation should be located next to the source code: change code, change docs.
• It should be easy, quick and fun to write.
• Support for code, images, UML diagrams, ...
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 42
AsciidoctorJ and Gradle to the rescue
plugins { id "org.asciidoctor.convert" version "1.5.3" }
asciidoctorj { version = '1.5.4.1' }
asciidoctor {
sourceDir 'src/docs/architecture'
resources {
from('src/docs/architecture') {
include 'images/**/*.png'
include 'images/**/*.jpg'
}
}
backends 'html5'
options doctype: 'article'
attributes 'source-highlighter': 'coderay'
}
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 43
// Example architecture documentation using arc42 (https://arc42.github.io):imagesdir: ./images
= image:qaware-logo.png[QAware GmbH,2016] Everything-as-code:toc-title: Table of Contents:toc:
[[section-introduction-and-goals]]== Introduction and Goals
The introduction to the architecture documentation should list the driving forcesthat software architects must consider in their decisions.
=== Requirements Overview
=== Quality Goals
=== Stakeholders
<<<<include::02_architecture_constraints.adoc[]
// further includes for the remaining sections
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 44
Presentation-as-code
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 45
These slides are written in Markdown.
---## [fit] These slides are written in Markdown.
- This is for real programmers! :smiley:- Several open source projects available- Use HTML and JavaScript alternatively.
---
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 46
What's the takeaway from this
journey?
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 47
Beware of the abstractions!
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 48
Everyone needs to do his homework.
• Developers: be polyglot, keep learning!
• Architects: choose the right language or tool for the job!
• Project Managers: give your techies freedom!
• Universities: teach polyglotism!
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 49
Have some fun and create your own polyglot project
archetype.
| JavaOne 2016 | Everything-as-code -> { created with ❤ and ☕ by @LeanderReimer } 50