Packaging tool options

Len Bass

Deployment pipeline

2

Pre-commit tests

X

promote to normal

production

Build Image and Perform Integration

tests

UAT / staging /

performance tests

Deploy to production

Commit

...

...

Pre-commit tests

Commit

Developers

• Developers finish a code segment

• Developers commit their code to a continuous integration (CI) server

• The CI runs various functional tests

• If the tests are passed, the constructed system is passed to a staging

area for other tests after which it is placed in production

This talk

3

Pre-commit tests

X

promote to normal

production

Build Image and Perform Integration

tests

UAT / staging /

performance tests

Deploy to production

Commit

...

...

Pre-commit tests

Commit

Developers

• Developers should all be using the same software (including versions)

• The platform on which they are developing should be the same as the

CI platform and the production platform

• Virtual machines are one portion of a solution.

• Various tools exist to construction of common platform for developers.

This talk explores some of them.

Virtual machine at execution time

© Len Bass 2015 4

Messages

through IP

address

Virtual Machine

including

machine image

Environment

Configuration

parameters

Virtual machine is instructed how to interact with its

environment through the configuration parameters

• URLs of external services

• Any other parameter that affects the behavior of

the machine image

The build process produces this virtual machine

• The goals of the build process are

• Repeatable

• Invokable by different members of a development team

• Efficient both in human terms and in machine terms

© Len Bass 2015 5

Four issues:

1. How is the virtual machine created?

2. How is the configuration parameter file specified?

3. How is the machine image created?

4. How is the machine image loaded into the virtual machine?

• Different tools provide different solutions to these questions.

© Len Bass 2015 6

Creating a virtual machine

• The virtual machine exists on a platform – VirtualBox or AWS in our case

• The virtual machine is created within the platform through an API call to the platform.

• The API call specifies

• Hardware specifications' of virtual machine

• Initial software contents

• AWS requires many more parameters than VirtualBox – credentials, security settings.

© Len Bass 2015 7

Tools with respect to creating virtual machine

• Vagrant has a concept of back end provider that includes VirtualBox and AWS. It creates the necessary VMs.

• Chef, Puppet, Ansible assume that the VM already exists.

• Docker assumes a virtual machine exists on which a portion of it runs but has a library with some pre-supplied VMs.

© Len Bass 2015 8

How is the configuration parameter file specified?

• All of the tools have mechanisms for specifying configuration parameters in their specification file.

• Vagrant has a “Vagrantfile”

• Docker has a “Dockerfile”

• Other tools have things called “cookbooks” or “playbooks”

• How these parameters are presented to the executable image depends on the executable image. A common means is through command line parameters.

© Len Bass 2015 9

Layered system as an example

© Len Bass 2015 10

Building the layered system

• System is compiled, linked together and an executable image is created.

• In Chef, Puppet and Ansible, this executable image is stored on a disk in a directory.

• Vagrant builds its image inside of the VM

• Docker we will discuss in a minute © Len Bass 2015 11

Instantiating a Virtual Machine

© Len Bass 2015 12

• Executable image (machine image) is copied into a bare

metal instance to create a virtual machine. There may be

some software in the VM to orchestrate the reception of

the image.

• The virtual machine has an IP address and so can receive

messages.

• The virtual machine interacts with its environment via

credentials, configurations parameters and must know

how to do this

Bare metal instance

Messages

through IP

Environment

specifications

Executable

image

How long does this take?

• Sending 1K bytes over 1 Gbps network takes .01 ms

• A machine image may be 8GB or more.

• Transferring 8GB over 1Gbps network takes at least 64 seconds (bits in Gbps, bytes in GB)

• This number will vary based on speed of network and size of machine image but we are talking about multiple minutes to instantiate a virtual machine with a fully loaded system.

© Len Bass 2015 13

How can we speed this up?

• The machine image we create does not have to fill up the RAM of the target computer.

• Reducing the size of the machine image will speed up the instantiation time of a virtual machine.

© Len Bass 2015 14

Can we do better?

• Suppose each layer is built into a separate disk file

• Combine these files into the VM

• These separate disk files are called containers.

© Len Bass 2015 15

Application bundle

Services

Service Registry

Bare metal instance

Containers • Combining these files in the VM requires special software.

• On the VM is a container manager

• On the build side is a container builder

• Now the instantiation process consists of loading multiple containers.

• Suppose further that some software (such as the OS and the service registry) is shared across multiple applications.

• Then once a VM is loaded with one application, loading other applications becomes much faster since only the app container must be loaded.

© Len Bass 2015 16

Docker

• Docker is the most common container system today.

• The Linux version is built on LXC which was the original container system.

• Docker maintains a repository with common containers such as Linux.

© Len Bass 2015 17

Summary

Tool Software residing on the created VM?

Specification of parameters

VM – all of nothing

Vagrant No Vagrantfile Yes

Chef, Puppet Yes Playbooks or cookbooks

Yes

Ansible No Playbooks Yes

Docker Yes Dockerfile No

© Len Bass 2015 18