fabric
TRANSCRIPT
fabricControl your linux nodes with python
What is it?
• Command-line tool for deployment automation and system administration as a ssh wrapper
• Executes to local or remote
✓ local: not a ssh connection. uses python subprocess module.
✓ run or sudo: when executing against remote, it can execute as the current user or as current user with sudo rights.
What is it?
• Simply specify python functions in fabfile.py
• Install it
✓ pip install fabric
✓ Once installed, fab command is available in your system path.
✓ Start writing your python function in fabfile.py
automation
• Daily tasks• Server setup• System update and maintenance• New project creation• Deployment processes
get started
• Download and install virtualbox https://www.virtualbox.org
• Download and install vagrant http://www.vagrantup.com
cd ~/VirtualBox\ VMsmkdir debsqueeze64cd debsqueeze64vagrant init debsqueeze64 http://www.emken.biz/vagrant-boxes/debsqueeze64.boxvagrant up
get started
• This downloads and creates an image of debian 64-bit on your computer.
• We will use fabric on our local machine to interact with this debian virtual machine
• See http://www.vagrantbox.es to download other prepared boxes.
Python Virtualenv
• Create an isolated python project environment
Mac OS X: package management with port or brew
sudo port -v install virtualenv_select py27-virtualenv sudo port -v install py27-virtualenvwrappersudo port select --set virtualenv virtualenv27
(optional)
Python Virtualenv
• Edit your ~/.bash_profile export WORKON_HOME=$HOME/.virtualenvsexport PROJECT_HOME=$HOME/worksource `which virtualenvwrapper.sh`
• $HOME -- this refers to your user home or “~” (i.e., on my computer, “~” means “/Users/calvin” directory)
(optional)
Python Virtualenv
• Create our python virtualenv for our fabric tutorial
mkvirtualenv --distribute --no-site-packages learnfabriccd $PROJECT_HOMEgit clone git://github.com/calvinchengx/learnfabric.gitcd learnfabricpip install -r requirements.txt
(optional)
learn fabric
• Once we have our debian virtual machine and optionally our python virtualenv ready, we begin
git clone git://github.com/calvinchengx/learnfabric.gitcd learnfabricpip install -r requirements.txtcp -rf ~/VirtualBox\ VMs/debsqueeze64/.vagrant .
• This installs fabric and installs python-vagrant (a wrapper around vagrant commands so we can programmatically control our vagrant instance)
learn fabric
• Once we have our debian virtual machine and optionally our python virtualenv ready, we begin
git clone git://github.com/calvinchengx/learnfabric.gitcd learnfabricpip install -r requirements.txtcp -rf ~/VirtualBox\ VMs/debsqueeze64/.vagrant .
• This installs fabric and installs python-vagrant (a wrapper around vagrant commands so we can programmatically control our vagrant instance)
fabric functionfab mytask
[[email protected]:2222] Executing task 'mytask'[[email protected]:2222] run: echo $USER[[email protected]:2222] out: vagrant[[email protected]:2222] out:
Done.Disconnecting from [email protected]:2222... done.
the code
import vagrantfrom fabric.api import env, task, runv = vagrant.Vagrant()v.up()env.hosts = [v.user_hostname_port()]env.key_filename = v.keyfile()env.disable_known_hosts = True # useful when vagrant box ip changes.# our first fabric function@taskdef mytask(): run('echo $USER')
fabric core
• local() runs a command locally, via python subprocess
• run() runs a command remotely, via ssh• sudo() runs a command remotely as sudo, via
ssh• put() copies a file from local to remote, via ssh• get() copies a file from remote to local, via ssh
fabric AUth
• All authentication is ssh-based• SSH keys help us avoid typing in passwords -
place fabric ssh user (env.user)’s public key in remote node’s authorized_keys
• Avoid directly using root user• Give your fabric ssh user sudo rights instead
(env.user)
fabric config• from fabric.api import env
• fabric environment is simply a dictionary containing host information, roles, user (env.user); and
• any other custom information you would like to include.
• $HOME/.fabricrc allows custom configuration, e.g.
user = ssh_user_name where “ssh_user_name” is any value you so desire to pass in to env.user when a fab function runs.
Fabric ROLEDEFS# code
from fabric.api import envenv.roledefs = { ‘web’: [‘100.0.0.1’, ‘100.0.0.2’], ‘db’: [‘100.0.0.3’], ‘media’: [‘100.0.0.4’]}
# command line
fab -R web mytask
Fabric host(S)# code
from fabric.api import envenv.roledefs = { ‘web’: [‘100.0.0.1’, ‘100.0.0.2’], ‘db’: [‘100.0.0.3’], ‘media’: [‘100.0.0.4’]}
# command line
fab -H 100.0.0.1,100.0.0.2 mytask
map roles to tasks# codefrom fabric.api import envfrom fabric.decorators import rolesenv.roledefs = { ‘web’: [‘100.0.0.1’, ‘100.0.0.2’], ‘db’: [‘100.0.0.3’], ‘media’: [‘100.0.0.4’]}
@roles(‘web’)def mytask(): run(‘uptime’)
# command line
fab mytask # already mapped to -R web
handling failures• Do not abort upon failure; just give a warning
# codefrom fabric.context_managers import settings
def mytask(): with settings(warn_only=True): run(‘rm /var/www/proj/releases/current’) run(‘ln -s /var/www/proj/releases/deployed
/var/www/proj/releases/current’)
Lazy sysadmin• A generic command
# code
@roles(‘all’) def invoke(command): “””
Invoke an arbitrary command “””
sudo(command)
# command line
fab invoke:“aptitude update” fab invoke:”aptitude upgrade”
parallel execution• A generic command
# code
@roles(‘all’) def invoke(command): “””
Invoke an arbitrary command “””
sudo(command)
# command line
fab -P invoke:“aptitude update” fab -P invoke:”aptitude upgrade”
python web app
Example deploy@task(default=True)@set_target_envdef deploy(email=False): """ `fab -R all deploy` or fab -H mysite.com deploy`. Execute a deployment to the given groups of hosts or host """ if not chk_req(): return if git_branch_check() or test_host_check(): manage_release('Deployment start') git_archive_and_upload_tar() pip_requirements() collectstatic(deploy=True) symlink_current() webserver() migrate_db() # post-deployment tasks manage_release('Deployment end') _releases_cleanup() email_on_success(trigger=email)# command line
fab -R all deploy
quick refErence
• fab -H host1.com task• fab -H host1.com task:arg1,arg2• fab -P -H localhost, host1.com, host2.com task• fab -R web task # roles defined by
env.roledefs• fab -R web task:arg1,arg2• fab -P -R web task• fab -l• more options explained via fab --help
devops
• Bridging the gap between code implementation by developers and deployment to staging and/or live servers by sysadmin
• Automation leads to continuous integration (e.g. Jenkins CI Server) and continuous delivery
• Support heterogeneous development environment in combination with vagrant (developers test against vagrant, which has same distro as staging and production hosts)
referenceS• My reusable fabric functions
https://github.com/fabric-colors/fabric-colorshttps://fabric-colors.readthedocs.org/en/latest/index.html
• Code that accompanies this set of slideshttps://github.com/calvinchengx/learnfabric
• Fabric code and documentationhttps://github.com/fabric/fabrichttp://fabric.readthedocs.org
• Multiple vagrant boxes for testing your server confighttp://awaseroot.wordpress.com/2012/05/06/script-for-adding-multiple-vagrant-boxes/
q&A
• Get in touch @calvinchengx
• http://calvinx.com