jenkins user conference jenkins user conference new york, may 17 2012 #jenkinsconf yale build and...

36
Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Upload: lizbeth-cynthia-barton

Post on 16-Dec-2015

221 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Yale Build and Deploy

E Camden FisherYale University ITS

Page 2: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Who am I?

E Camden Fisher

Technical Lead

Unix Infrastructure and Virtualization

[email protected]

@fishnix

Page 3: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

What is Jenkins?

Jenkins is an application and a framework that manages and monitors the execution of

repeated tasks.

Page 4: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Who Uses Jenkins @Yale?

Page 5: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Overview

Before Jenkins After Jenkins• "fire and forget", consistent!• Testing is automated!• Code coverage is easy!• Bugs caught early and often!• Devs worry about code!• Change control in the right

places for deployments!• Rapid progress! Agility!• Identical artifact per

environment!• Identical configuration per

environments!• Deployments are “click”

easy• Integration Nirvana

• Building is slow, error prone• Testing is onerous• Code coverage is onerous• Bugs caught later• Devs worry about servers• No change control for

deployments• Slow progress• Different artifact per

environment• Inconsistent configuration

per environment• Deployments are "hard"• Integration Hell

Page 6: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Let’s Not Forget Continuous Integration!

Maintain a Single Source RepositoryAutomate the BuildMake Your Build Self-TestingEvery Commit Should Build the Mainline on an Integration MachineKeep the Build FastTest in a Clone of the Production EnvironmentMake it Easy for Anyone to Get the Latest ExecutableEveryone can see what's happeningAutomate DeploymentAvoid "Integration Hell!!”

Page 7: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Why did we chose Jenkins?

Easy!

Extensible

Scalable

Flexible

Open Source

Supported!

Page 8: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Yale History Lesson

Application lifecycle is a progression

Source Code Management

Maven and Artifactory

Building and Testing with Jenkins

Container Configurable artifacts

Runtime Configurable Containers

Managed deployments with Jenkins

Page 9: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

SCM

Page 10: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

MVN

Maven

Project Object Model (POM)

Simplifies dependency resolution ("oops I forgot

that .jar!")

Makes the build process easy and uniform

Artifactory (Maven Repository)

Where do I put my built artifacts?

Makes it easy for everyone to get the latest build!

Page 11: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Builds: Before Jenkins

We have a single source repo + a place

to store built artifacts, but…..

– Builds still take a long time

– Testing takes longer!

– Code coverage take even longer!

– Build environments are not standardized

– Mistakes are caught later, hard to debug

Page 12: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Builds: Enter Jenkins

SCM commits automatically kick off a build

Testing and code coverage is automated and is run on

every commit.

Broken builds immediately notify the team and the

committer

Tests run in a Clone of the Production Environment!

Everyone can easily see what's happening!

Developers can get back to coding instead of building

and testing.

Releases are quick and easy

Page 13: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS
Page 14: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Deployment: The problem.

Configuration must be different per environmentChoice between loss of control or loss of agility– Often devs configure the container– Either devs can edit deployables or they can't

If they can... things are bad– Code is deployed, edited and removed w/out Change

Control– Moving quickly… "I'll clean it up later!"– Dev environments quickly diverge from Production– Security is compromised

If they can't... things are worseChange requests are "slow”Systems groups must do everything

Page 15: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Deployments: How do we fix them?

Standardize the process!

Promote SANE change control

Normalize deployables/artifacts

Single location for app configuration data

Standardize the container

Externalize what makes an environment unique and special

Page 16: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Yale Application Installer Plugin

Page 17: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Container Configurable Artifacts

Artifacts were built with embedded configuration.– ie. datasources, CAS service endpoints, etc

– Artifacts are different per environment! What!?

– "Oops, I forgot to update that parameter!"

Externalization of configuration parameters– deployable XML

– Apps self configure with JNDI

Now we have Container Configurable artifacts– The SAME artifact migrates between environments

– XML configuration data (stored as build parameters in Jenkins) is all that differentiates environments

Page 18: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Yale Standard Java Container

JBoss EAP (5.1 now)

Minimal customization to externalize configuration into runtime

Packaged in RPM

Configuration Management to install + manage what’s “special”

Meets 100% of use cases (so far)

Runtime configurable container is key!

Page 19: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Yale Standard Java Container

$JBOSS_HOME/deploy/jbossweb.sar/server.xml

<!-- Connector for SSL 20100624 ECF --><Connector protocol="HTTP/1.1" SSLEnabled="true" port="${jboss.bind.httpsport}" address="${jboss.bind.address}” secure="true" clientAuth="false" scheme="https” proxyName="${jboss.proxyname}" proxyPort="${jboss.proxyport}” SSLCertificateFile="${jboss.server.home.dir}/conf/server.crt" SSLCertificateKeyFile="${jboss.server.home.dir}/conf/server.pem” URIEncoding="UTF-8” SSLProtocol="TLS”>

/etc/init.d/jboss_nodexx

# JBoss variablesexport JBOSS_OPTS=“-Djboss.bind.httpport=${HTTPPORT} \ -Djboss.bind.httpsport=${HTTPSPORT} ${ADDLJBOPTS}”

${JBOSS_HOME}/bin/run.sh ${JBOSS_OPTS} -c ${JBOSS_NODE} -g ${JBOSS_PARTITION} -b ${JBOSS_BIND} -u ${CLSTR_ADDR}

Page 20: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS
Page 21: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Page 22: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Page 23: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Page 24: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Page 25: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Page 26: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Page 27: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Deployments: The result.

Consistency!

Jenkins writes configuration XML: $human_error--

Eliminate shells, and elevated privileges on servers

Container is managed by infrastructure with the O/S

Developers can deploy to DEV at will

Empowers developers to GTD

Puts gates at appropriate places

Changes to the Jenkins jobs, containers + deploys to Test/Prod require change control

Frees Systems folks to work on more interesting things

Standard Container + Cloudy IaaS + Container Configurable artifacts = Vendor-lock-in-less PaaS!

Page 28: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Drupal Continuous Deployment

Yale has a large shared Drupal

infrastructure

Shared = needs change control

Migration process looks similar to Java

Some end users want to edit themes!

Jenkins to the rescue!

Page 29: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS
Page 30: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS
Page 31: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS
Page 32: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS
Page 33: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

What’s New?

Workflow integration through web servicesBuild + Release of Apache Servicemix bundles and “features”Spawning and Destroying Servicemix child instancesDeployment of Features and OSGI bundles to Apache ServicemixDeploying Drupal 7– git + pull

Page 34: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Other Languages

Php– Code coverage and unit testing available

Ruby– Automated unit testing– Automated code coverage– Automated deployment coming soon

.Net– Build, unit test, archive creation

Page 35: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

The Future

RBAC + folders– delegate responsibility to other systems

groups

More ruby– unit testing and code coverage– deployments

Enterprise Service BusCentralized SSH mangagement– the tools are better now

Managing/Provisioning with cfg mgmtCloud deployments

Page 36: Jenkins User Conference Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS

Jenkins User ConferenceJenkins User ConferenceNew York, May 17 2012 #jenkinsconf New York, May 17 2012 #jenkinsconf

Questions?

E Camden Fisher

[email protected]

@fishnix