development environments with vagrant and ansible · daniel groves

6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves

https://danielgroves.net/notebook/2014/05/developmentenvironments/ 1/17

VagrantisoneofthosetechnologieswhichIoncedidn'tthinkIneeded,butaftergivingitagoitquicklybecameanindispensablepartofmydailyworkflow.Withoutit,workingonServerObserverwouldhavebeenmuchmoredifficult.ThesamecaneasilybeappliedtomanyotherprojectsIhaveworkedonaspartofmydegree.

Vagrantitselfisonlypartofthesolution,ithandlesthegenerationandbasicimagingofvirtualmachinestoformisolateddevelopmentenvironments.However,Vagrantdoesn'thandletheprovisioningofthesevirtualmachines.It'sstilluptoustohandletheinstallationandconfigurationofthepackageswerequiretowork.Thankfully,Vagrantiseasilyintegrateswith

DevelopmentnvironmentwithVagrantandAnile

Pulihed:22Ma2014Tag:development,vagrant,pthon

HowtoueVagrantandAniletogethertouildapowerful,flexileandportaledevelopment

environment



existingprovisionersincludingChef,Puppet,andmyprovisionerofchoiceAnsible.

Onceavirtualmachinehasbeengeneratedandaprovisionerhasbeendevelopeditisveryfasttostartadevelopmentenvironmentonabrand-newcomputerwithouthavingtoworryaboutfinding,installing,andconfiguringmultipleprojectdependencies.Anydevelopercanpickupaprojectandstartworkingonitwithlittlemanualwork,inthesameenvironmentaseveryoneelse,usingtheirfavouritetext-editorontheirlocalcomputer.

ThisarticlewilltakeyouthroughthestagesofsettingupaVagrantconfigurationforanewvirtualmachine,andhowtoprovisionthiswithsoftwareusingtwocustomAnsibleplaybooks.Thiswillbeaveryhands-ontutorialwhereI'drecommendtryingthingsforyourself.ItisbackedbyarepositoryonGitHubwithacompletecopyofalloftheresourcesusedinthistutorial.

Inthistutorialwe'llcreateaDjangodevelopmentenvironment,usingaPostgreSQLbackend.Despitethis,thistutorialcaneasilybeadaptedforothersetups,suchastheNginx/MySQL/PHP-FPMsetupinthephpbranch.

PreparationIfyouhaven'tusedtheTerminalbeforeonyourcomputer,Iwouldsuggestyoufamiliariseyourselfwiththebasics.Thistutorialisveryterminal-heavy,itwouldbewiseforyoutoknowwhatyou'redoing.Ataveryminimumyou'llneedtoknowhowtonavigatethefilesystemonyouroperatingsystem.CommandsgivenhereshouldbethesameforMacandLinux.

Makesureyou'veinstalledthelatestversionofVagrantforyour



platform(version1.5.4attimeofwriting),aswellasAnsible(version1.6.1attimeofwriting).Ifyoudon'talreadyuseityou'llalsoneedtoinstallVirtualbox,althoughitispossibletouseVagrantwithVMwareFusionorHyper-V.Oncethisisdonewe'rereadytostartbuildingourfirstdevelopmentenvironment.

Vagrantisdesignedtobecommitteddirectlyintoyourprojectssourcecontrol,alongwithanyprovisionersused.Thisiscertainlysomethingthatisadvised,asitmakestheuseoftheenvironmentassimpleaspossible,andalsohelpstodocumentwhatthesystemdependenciesareandwhentheychange.Vagrantwillcreatesomefileswhichyoudonotwanttocommitthough,justastheuseofvirtualenvlaterinthistutorialwill.Thefollowing.gitignorefilewillstopanyofthesefilesfrombeingcommittedbyaccident.

#Vagrant.vagrant/

#Pythonbin/build/include/lib/local/.pyc

GettingtartedCreateadirectoryonyoulocalfilesystemtoworkfromandnavigateintothisfromtheterminal.Feelfreetoinitiatesomeversioncontrolsoftwarewithinthedirectory,beforerunningvagrantinit.Thiswillgenerateabasicconfigurationfileforusto



workfrom,includingsomeextensivecommentingwhichshouldhelpyouunderstandwhatisgoingon.Forthisusecasethevastmajorityofthisisnotneeded,sowe'llremovethisoncewe'vegainedabasicunderstandingofwhatisrequired.

Vagrantdoesn'tusethestandardimagesmostoperatingsystemdistributorsprovide.Insteaditusesa'vagrantbox',whichwillcontainmultiplepre-configuredpackagesaswellastheoperatingsystem.Anyonecangenerateabox,andVagrantCloudalreadyhasmanytochoosefromwithvariouspre-installedpackages.ItendtoworkfromastandardUbuntuPrecise64box,asthisistheoperatingsystemthatmostofmyserversrun.Tomakeuseofthis,we'llsetconfig.vm.boxto'hashicorp/precise64'.

Whilewe'readdingthistotheconfigurationwe'llforwardaportfromourlocalcomputertothePythondevelopmentserverandconfigureasharedfolderwhichwillallowustoworkonthecode-baseonourlocalcomputerandhaveVagrantsynchronisechangesintothedevelopmentenvironment.ChangeyourVagrantfileasfollows:

#*mode:ruby*#vi:setft=ruby:

#VagrantfileAPI/syntaxversion.Don'ttouchunlessyouknowwhatyou'redoing!VAGRANTFILE_API_VERSION="2"

Vagrant.configure(VAGRANTFILE_API_VERSION)do|config|#AllVagrantconfigurationisdonehere.Themostcommonconfiguration#optionsaredocumentedandcommentedbelow.Foracompletereference,#pleaseseetheonlinedocumentationatvagrantup.com.

#EveryVagrantvirtualenvironmentrequiresaboxtobuildoffof.



config.vm.box="hashicorp/precise64"

#Createaforwardedportmappingwhichallowsaccesstoaspecificport#withinthemachinefromaportonthehostmachine.Intheexamplebelow,#accessing"localhost:8080"willaccessport80ontheguestmachine.config.vm.network"forwarded_port",guest:8080,host:8080

#Createapublicnetwork,whichgenerallymatchedtobridgednetwork.#Bridgednetworksmakethemachineappearasanotherphysicaldeviceon#yournetwork.config.vm.network"public_network"

#Iftrue,thenanySSHconnectionsmadewillenableagentforwarding.#Defaultvalue:falseconfig.ssh.forward_agent=true

#ShareanadditionalfoldertotheguestVM.Thefirstargumentis#thepathonthehosttotheactualfolder.Thesecondargumentis#thepathontheguesttomountthefolder.Andtheoptionalthird#argumentisasetofnonrequiredoptions.config.vm.synced_folder"./app","/var/www/djangoapp"end

You'llnoticethattheconfig.vm.synced_folderlinemaps./apponourlocalfilesystemto/var/www/djangoappontheremotefile-system.You'llneedtomakethe./appdirectorywithinthesamedirectoryasyourVagrantfilenow,oradjustthispathtosuit.

Onthecommandline,runvagrantup.AtthispointVagrantwilldownloadtheUbuntuboxifitisn'talreadyonyoursystem,makeacloneforthespecificvirtualmachine,andbootthevirtualmachinewiththegivenconfiguration.Ifneededitispossibletochangethedefaultemulatedhardware(defaultof



512MBmemoryandasingle-coreCPU)orhaveoneVagrantfilelaunchmultiplevirtualmachines.

ProviioningVagrantAtthispointwehaveadevelopmentvirtualmachine,butitlacksthedependenciesofourproject;thisiswheretheprovisionercomesin.Inordertogetstartedweneedtoset-outabasicdirectorystructurecontainsthefilesandfoldersrequiredbyAnsible.Initially,thismayappeartobeoverkillbutgettingeverythingorganisednowmakeslifeeasieratalaterdate.Wecanusetheterminaltotakeafewshortcutsandquicklybuildtherequireddirectorystructure.

mkdirpprovision/{setup,deploy}/{tasks,vars,handlers}touchprovision/{setup,deploy}/{tasks,vars,handlers}/main.ymltouchprovision/vagrant.yml

Theresultingfile-structureshouldbeasfollows:

app/provision/setup/tasks/main.ymlvars/main.ymlhandlers/main.ymldeploy/tasks/main.ymlvars/main.yml



handlers/main.ymlvagrant.ymlVagrantfile.gitignore

TheideahereistospilttheAnsibleconfigurationupintomultipleplaybookstohelpkeepeverythingorganisedandasreusableaspossible.Thesetupdirectorywillhavetheconfigurationfortheoperatingsystem,whilethedeploywillcontaineverythingrequiredforthevirtualenv.

Now,let'stellVagrantabouttheprovisioners.AddthefollowingjustinsidetheclosingendstatementintheVagrantfile:

#Installrequiredsoftware,dependenciesandconfigurationsonthe#virtualmachineusingaprovisioner.config.vm.provision"ansible"do|ansible|ansible.playbook="provision/vagrant.yml"end

Then,addthefollowingtoprovision/vagrant.yml:

hosts:allroles:setupdeploy

Now,whenanewvirtualmachineisstartedwithVagrantitwillautomaticallyrunthevagrant.ymlwithAnsible,whichwillruneachoftheplaybooksinturn.

IntallingApplicationThissectionwillfocusonthesetupplaybook.Thefirstthingweneedtodoisinstallthesystemdependencieswerequire.Add



thefollowingtoprovision/setup/tasks/main.yml:

name:Installpackagessudo:yesapt:pkg={{item}}state=installedupdate_cache=yeswith_items:#Databasepostgresqllibpqdev#RequiredforAnsibletointeractwithpostgrespythonpsycopg2#RequiredforAnsibletointeractwithpostgres

#PythonDevpythondevpythonsetuptoolspythonvirtualenv

Thiswillupdatethepackagecachewithapt,andthenensureeachoftheprogramsareinstalledalongsidetheirdependencies.You'llnoticetwodependencieshaveacommentnexttothemdetailingthey'rerequiredbyAnsible.ThesetwoprogramsallowAnsibletointeractdirectlywithPostgreSQL,allowingustoautomateitssetup.

Earlieronwebootedourvirtualmachine,butitwasnotprovisionedasthishadn'tbeendevelopedatthetime.Ratherthanthrowingawayandrebuildingthevirtualmachinewecanforcevagranttoprovisionthevirtualmachinebyrunningvagrantprovision.Thefinallinesoftheoutputshouldlooklikethis:

PLAYRECAP********************************************************************default:ok=2changed=1unreachable=0failed=0

ConfiguringPotgreQLAtthispointwe'remakinggoodprogress,howeverweneedto



configurePostgreSQLsothatourDjangoprojectcanuseit.Thefirstthingthatneedsdoingistheauthenticationconfigurationsolocaluserscanusepasswordauthentication.Todothisafilecalledpg_hba.confneedsreplacing,Iwillnotexplainthisinanydetailhere,howeverthefileneededisintheGitHubrepository.Downloadacopyandadda'files'directorywithinthesetupplaybookandplacethefileinthere,namedpg_hba.conf.

Addthefollowingtotheprovision/setup/tasks/main.ymltocopythenewconfigurationontotheserver:

name:Allowpasswordauthenticationforlocalsocketuserssudo:yescopy:src=pg_hba.confdest=/etc/postgresql/9.1/main/pg_hba.confforce=yesnotify:RestartPostgres

You'llnoticethisstatementendswithanotifyaction.Thiscallsahandlerwiththegivennameintheeventthatthetaskchangesanythingonthesystem.Inthiscaseshouldthepg_hba.conffilechange,it'llrestartPostgreSQLtoloadthenewconfiguration.Wedoneedtowritethishandlerthough,soaddthefollowingtoprovision/setup/handlers/main.yml.

name:RestartPostgressudo:yesservice:name=postgresqlstate=restarted

NowwehavereconfiguredPostgreSQLtoallowlocalusertologinweneedtocreateauserandadatabase.First,we'lladdsomevariableswhichcontainthedetailswewanttouseforthenewuser.Addthefollowingtoprovision/setup/vars/main.yml:

db_name:django_appdb_user:django



db_password:sdjgh34iutwefhfgbqkj3

Now,wecanusetheAnsiblePostgreSQLmoduletoconfigureauserandadatabase.Addthefollowingtoprovision/setup/tasks/main.yml:

name:CreateDatabasesudo:yessudo_user:postgrespostgresql_db:name={{db_name}}

name:CreateUsersudo:yessudo_user:postgrespostgresql_user:name={{db_user}}password={{db_password}}state=presentrole_attr_flags=NOSUPERUSER,CREATEDB

name:ProvideuserwithDBpermissionssudo:yessudo_user:postgrespostgresql_user:user={{db_user}}db={{db_name}}priv=ALL

Thesudo_userlinestellAnsiblewhatusertorunacommandas,andsincePostgreSQLwillonlyallowuserstologinfromthepostgresaccountbydefaultwetellAnsibletorunthecommandsasthisuser.

Withthissectioncompletewehavenowsuccessfullyprovisionedavirtualmachinewithallofthesystemdependenciesfortheproject.Toforcevagranttore-provisionthevirtualmachineandsoloadthelatestchangesrunvagrantprovision.

WorkingwithVirtualenvNowwe'remakinggoodprogress,butit'stimetomoveonto



provisioningthesoftwaredeploymentdependencies.Fromhereonwe'llbeworkingwiththedeployplaybook.Thefirstthingtoconsideriswe'llbeusingpiptoinstallaseriesofpythonmoduleswithinaplaybook.Todothisweneedarequirementsfilewhichwillbepassedtopip.Let'sinstallthelatestversionsofDjangoandSouth,aswellaspsycopg2sothatDjangocantalktothePostgreSQLdatabase.Createafilecalledrequirements.txtwithintheappdirectory,andaddthefollowing:

DjangoSouthpsycopg2

Now,we'lltellAnsibletocreateavirtualenv,andthentoinstalltherequiredmodules.Addthefollowingtoprovision/deploy/tasks/main.yml:

name:SetupVirtualenvpip:virtualenv={{virtualenv_path}}requirements={{virtualenv_path}}/requirements.txt

You'llnoticetheuseofthevirtualenv_pathvariable,let'saddthisintheprovision/deploy/vars/main.ymlfile:

virtualenv_path:/var/www/djangoapp

Nowrunvagrantprovisionagaintosetupthepythonvirtualenvironment,thenwecanstartworkingwithDjango.

WorkingwithDjangoOncetheprovisionerhasfinishedrunningthat'sprettymuchitfortheinitialsetup,howeverwecouldtakethisfurther.Let'ssetupabasicDjangoapplicationwithSouthandthenuseansibletosetupthedatabaseonthevirtualmachinecreation.RunthefollowingtoSSHintothevirtualmachine,activatethevirtualenvironmentandcreateanewDjangoapplication.

DANILGROV HOWNAV



vagrantsshcd/var/www/djangoappsourcebin/activatedjangoadmin.pystartprojectvagranttest

Thiswillhavecreatedtheinitialapp,butnowweneedtoupdatethesettings.pyfilesoDjangocantalktoPostgreSQL.Findthefollowinglinesinapp/vagranttest/vagranttest/settings.py:

#Database#https://docs.djangoproject.com/en/1.6/ref/settings/#databases

DATABASES={'default':{'ENGINE':'django.db.backends.sqlite3','NAME':os.path.join(BASE_DIR,'db.sqlite3'),}}

Andreplacethemwith:

#Database#https://docs.djangoproject.com/en/1.6/ref/settings/#databases

DATABASES={'default':{'ENGINE':'django.db.backends.postgresql_psycopg2','NAME':'django_app','USER':'django','PASSWORD':'sdjgh34iutwefhfgbqkj3','HOST':'localhost',}}



AndaddSouthformigrationssupport:

#Applicationdefinition

INSTALLED_APPS=('django.contrib.admin','django.contrib.auth','django.contrib.contenttypes','django.contrib.sessions','django.contrib.messages','django.contrib.staticfiles','South',)

Now,let'smakeAnsiblesyncourdatabaseforus.

name:Djangosyncdbdjango_manage:command=syncdbapp_path={{virtualenv_path}}/vagranttestvirtualenv={{virtualenv_path}}name:Djangomigratedjango_manage:command=migrateapp_path={{virtualenv_path}}/vagranttestvirtualenv={{virtualenv_path}}

Nowprovisionthevirtualmachineonefinaltimetoensurethatitrunsthroughproperly.

PuttingverthingtoUeThat'sallthereistoit.Nowanydevelopersworkingontheprojectsimplyclonetherepositoryandrun:

vagrantupvagrantsshcd/var/www/djangoappsourcebin/activatepythonvagranttest/manage.pyrunserver0.0.0.0:8080



Oncethey'vedonethisthey'llhaveanidenticalenvironmenttoeveryoneelse,willbeabletocontinueworkingintheirfavouriteeditor,andwillbeabletoaccesstheprojectbysimplyaccessinghttp://localhost:8080inabrowserontheirlocalmachine.

ThereareafewVagrantcommandsworthknowing:vagrantup:Bootsavirtualmachinevagrantprovision:Forcetheprovisionerstoberunagainavirtualmachine.Usefulforupdatingtheconfigurationofexistingvirtualmachines.vagranthalt:Shutdownavirtualmachine.vagrantsuspend:Sleepavirtualmachine.vagrantdestroy:Deleteavirtualmachinefromyoursystem.

It'salsoworthnotingthatyoushouldalwaysadd.vagrant/totheignorefileforyoursourcecontrol.Otherwiseyouwillbecommittingentirevirtualmachinestoyourproject,whichisthelastthingyouwanttodo.

ummarVagrantprovidesaflexiblewayofallowingmultipledeveloperstoworkoncomplexprojectswithlittlemanualsetuptimeorcomplexity,andwithoutlitteringtheirsystemwithprojectdependencies.Itisawell-designedplatformwhichallowscomplexinfrastructuretobereplicatedonalocalcomputerwitheasebyasmanydevelopersasrequired.

Ansiblepresentsapowerfulwayofprovisioningserversandvirtualmachineswithaminimalleaningcurve.ThisarticlehasonlytouchedonthebasicsofwhatAnsiblecando,coveringthebasicsofkeepingyourconfigurationorganised,andhowtosetupandconfigurebasicsoftwarepackages.



BetweenVagrantandAnsibleyouhaveeverythingyouneedtocreateportabledevelopmentenvironmentwhichcaneasilybecommittedintosourcecontrolwiththerestofyourprojectforanydevelopertouse.

ThesourceforthisexampleprojectisavailableonGitHub.

Gotomethingou'dliketoa?FindmeonTwitter,ordropmeanemail.



MoreLikeThiroadFallWildCampAtripouttoroadFall,Dartmoor,tophotographthewaterfallwithadramaticackdrop

ikepackingatTeignheadFarmRuinAikepackingtripacroDartmoortowildcampandTeignheadFarmRuin.

PeterTavandFurTor

4Comments DanielGrovesWebDesign Login1

Share SortbyBest

Jointhediscussion

Reply

pageworthy 2monthsagoThankssomuchforcreatingthistutorial.IthelpedmeatonandmylocaldevelopmentisnowinVagrant!

Reply

DanielGroves 2monthsagoadmin >pageworthyGladyoufoundithelpful.Feelfreetoshoutifyou'vegotanyquestions.

Reply

Chris 9monthsagoGreatwriteup.Helpedmealot.Onethingthough,it'snotrecommendedtoaddyourpythoncodetotheWebserversdocumentroot(/var/www/),asthatopensupforotherstobeabletoreadit.

Reply

DanielGroves 9monthsagoadmin >ChrisThanksChris.

GenerallyforaproductionenvironmentIcreateanewuserfortheapplication,andruneverythingfortheparticularapplicationunderthatuser.ThereasonIdidn'tdothatinthisarticlewaspurelytokeepthingsassimpleaspossibleforthereader.

Thankyouforpickinguponthis,Ireallyappreciatetheinputofothers.

Recommend

Share

Share

Share

Share



ackpackingtoFurToronDartmoorforthePhotograph.

ThendofPhoto365Attheendof2014IcommittedtomakinganattemptonaPhoto365challenge.Itdidn'tgotoplan,andhere'whitwan'tafailure.

lewhereontheWe

PoweredJekllandhotedonDigitalOcean.

Copright20102015DanielGrove.Forlicencedetail,eetheGitHurepoitor.

development environments with vagrant and ansible · daniel groves

Documents