Overview of the Automated Build & Deployment Process

Johnita BeasleyTuesday, April 29, 2008

Overview of Deployment Tiers

Software builds and deploymentsperformed by the QA Team

Software builds and deploymentsperformed by the Systems Team

Software builds and deploymentsperformed by the Systems Team

Inside the NIH Firewall

Outside the NIH Firewall

DEV

QA

STAGE

PROD

Build/Deployment Process (Current)

1. Generate & Test the Ant build script.

AntBuild Script

DEV

2. Test the Ant build scriptand Deployment on a Production-like server

Desktop

3. Generate Build & Deployment Instructions for QA

Build Request &DeploymentInstructions

QA

4. Test the Build &Deployment

STAGE

PROD

5. Tests app. Sends Build & Deployment To Systems Team

Build Request &DeploymentInstructions

6. Deploys ApplicationBased on Build & Deployment Instructions

6A

6B

Guiding Principles (Automated)

One process to rule them all. The same build and deployment process will be used on all tiers, without exception. This enables ongoing validation and will reveal problems more quickly. In a broader sense, this principle also means that all teams will use the same approach.

Simplicity. Where multiple alternatives existed, the team favored the one that seemed simpler and more direct.

Standardization. This involved standardizing on a directory structure, creating a master set of environment properties, and designing a standard build script. This approach benefits the Center as a whole, because developers who work on more than one project, and open source customers who implement more than one of our products locally, only have to learn one way of doing things.

Tool-neutrality.Build scripts must be executable from the command line in a properly configured environment. The Center uses AnthillPro as a Build Management Server, but open source users of our software may not have access to it. Therefore, we will not inject any dependencies on AnthillPro into our process.

Guiding Principles (Automated)

Tier-neutral builds.

The output of the build step should be a tier-neutral deployment unit, and a set of external property files,

which implies that a build script should not have tier specific build targets.

Appropriate error handling.

Build scripts should adopt a ‘fail early’ approach to detecting and reporting errors. Database scripts should

perform extensive error checking and should be able to rollback to the previous state if any error is encountered during execution. Development teams are responsible for providing for both confirmation of a successful deployment, and rollback of failed deployments.

Central Build Server

Used as a ‘reference’ build machine

Supports isolation of build and deployment configuration parameters

Configured with the necessary external dependencies

Anthill Pro build management server will fulfill build and deployment requests on all tiers

Training will be provided to teams in use of the Anthill Pro tool

Hot Deployments

NCICB uses shared application containers

Applications must be hot-deployable to be considered eligible for deployment at arbitrary time without disrupting operation of other applications

Applications that are NOT hot-deployable (require a container restart) will only be deployed during regularly scheduled deployment windows.

Master set of tier-specific properties

Systems team will provide to all developers a standard set of environment properties and their appropriate values.

This file will describe the deployment environment to which their applications must conform.

A version of this file will be maintained on each tier and the appropriate version will be supplied to the build process

All tier-specific properties will be included in the master file. Ensures all projects will use a consistent naming scheme.

Directory Structure Standardization

Influenced by the Eclipse web application defaults (since Eclipse is the tool of choice for most development teams)

Detailed description provided in the Automated Build and Deployment Handbook

Build Script Standardization

Two primary build targets recommended are:– build-all: creates the (tier-neutral) deployment unit, and copies all

output to the deployment-artifacts directory– deploy: first checks to see that the deployment-artifacts directory

exists and the deployment unit is there before proceeding with the deployment

These targets are the primary interface points for all users of the script

There should be no direct link between these targets

Standard for reading build property files

A build script using this approach can be invoked in one of two ways: a. >ant build-all

– Load build properties from a local file called build.properties b. >ant -DPROP.FILE=/path/to/my.properties build-all

– Overloads the default value of PROP.FILE and loads properties from the specified file The benefits of this approach are:

– The build script can be run with an arbitrary set of parameters– Allows the Systems Team to maintain protected versions of properties files for

STAGE & PROD– Keeps all tier-specific properties in easily accessible place

<property name="PROP.FILE" location="build.properties" /><property file="${PROP.FILE}" />

Naming Conventions

The file prefix “template.” indicates build and deploy substitution variables are present.The expanded file will have the same file name without the “template” prefix, e.g.template.web.xml will be used to create web.xml after the substitution variables areexpanded, using Ant's filtering capabilities.

Property names should be all UPPER CASE for external build.xml properties and alllower case for internal build.xml properties. This is to make it easier to determine where aproperty’s definition can be found.

Properties that are directory or path references should include “.DIR” or “.dir” in theirnames.

Properties that refer to files should include “.FILE” or “.file” in their names.

Properties that refer to environment variables should use “ENV.” in their names.

Target names begin with a dash (‘‘) for internal build.xml targets.

In addition to the conventions outlined here, the project team has adopted the recommendations of The Elements of Ant Style, available at http://wiki.apache.org/ant/TheElementsOfAntStyle.

Property validation

Each build script is responsible for validating it’s properties

For most cases, it will only be possible to validate that the property actually has a value

Validating of meaningful values should be performed if possible – Example: Verifying the correct versions of Java and Ant are being used

Both the JAVA_HOME and ANT_HOME environment variables will be available during script execution and should be checked for the appropriate values.

If any validation step fails, an appropriate error message should be generated and the script halted immediately.

Property Validation Example

The 1st segment of the following code snippet validates all properties, and sets the check.props property if a required property is not set. It is at the top of the script and immediately evaluated before any targets are invoked.

In the 2nd segment, the ‘–prop-check’ target is a dependent of every other target and will cause the script to fail if check.props is set.

<condition property="check.props"> <and> <isset property="PROPERTY1" /> <isset property="PROPERTY2" /> <isset property= ... /> </and></condition>

<target name="-prop-check" unless="check.props"> <fail>Failure Message</fail></target>

Summary of Build Step

Build script reads a tier-specific set of properties passed in on the command line.

Build script verifies that all required properties have values. If possible, the script may also check to ensure that the properties have meaningful values. This is the required approach for all teams implementing auto deployment.

Build script creates tier-neutral build artifacts, and copies them all to the deployment artifacts subdirectory.

Summary of Deploy Step

Based on the tier-specific properties selected at deploy time, the script should copy the deployment units to the specified directory on the deployment machine.

After the files are successfully copied over, the build script should move them to their final destination on the target machine. This can be accomplished this by using the Ant SSHEXEC task to execute remote commands on the target. The build machine has been configured with SSH keys to enable appropriate access on the target machine.

Example: Execution of Remote Commands

The example code is reusable as is, without modification, to any project that follows the recommended directory structure.

<scp todir="${SCP.USER}@${SCP.HOST}:${SCP.PATH}" port="${SCP.PORT}" passphrase="${SCP.PASSPHRASE}" keyfile="${SCP.KEYFILE}" trust="true" verbose="true"> <fileset dir="${artifacts.dir}" includes="**/*" /></scp>

Responsibilities

Systems/SCM Team– Maintains a master file of properties (provided to development

teams) that describes the build and deployment environment– Provides implementation support for teams automating their build

scripts through all tiers– Providing access to the Anthill Build Server

Development Teams– Writing and maintaining build scripts– Addressing build failures

Related Documents and References

Deployment Handbook Automated Build and Deployment Handbook

You can get these documents from:http://gforge.nci.nih.gov/docman/index.php?group_id=27&selected_doc_group_id=84&language_id=1

Questions