automated deployment with phing
DESCRIPTION
Website deployment is a tedious and intricate task that lends itself to human error (oops, did I forget to update the DB schema?). Using Phing in conjunction with deployment techniques can greatly reduce the risk of human error and increase productivity. The presentation will cover using Phing to sync files, run tasks, migrate databases, target configuration, and other deployment techniques.TRANSCRIPT
Automated Deployment With Phing - Daniel Cousineau
1
Automated Deployment With Phing
Or: How I Will Replace You With A Very Small Shell Script
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
2
Who
• Daniel Cousineau• Senior Software Applications Developer
Texas A&M UniversityDivision of Student AffairsDepartment of IT
• http://www.toosweettobesour.com/• @dcousineau
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
3
THE PROBLEM
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
4
Human beings make mistakes
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
5
We aren’t as accurate each repetition
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
6
Machines don’t have this problem
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
7
They do the same thing every time (supposedly)
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
8
At Any Given Time
• New Code• New Configuration• New Database Schema• New Static Files
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
9
A Lot To Remember
• Did you remember to upload ALL new files?• Did you remember to update your DB?• Did you remember to correct your config?
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
10
Even Worse
• Did you clear your caches?• Did you delete that old file/plugin?• In the upload process, was your configuration
overwritten?• Did you upload ALL the changed files?• When did you last upload?
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
11
THE SOLUTION
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
12
Automation!
• Build scripts!– We are programmers after all…
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
13ZendCon '09
Don’t Do More Work Than You
Have To!
Automated Deployment With Phing - Daniel Cousineau
14ZendCon '09
Work Hard At Being Lazy!
Automated Deployment With Phing - Daniel Cousineau
15
What is Automation?
• Automated deployment means a single command– Locks your live site– Uploads changed files– Clears caches and temporary files– Updates the database schema– Runs other cron tasks– Unlocks your live site
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
16
Why Do We Automate?
• Deployment is tricky• Repetition degrades quality– She sells sea shells by the sea shore
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
17
When Is Automation Used?
• All the time!– Staging– Live
• Probably best to use it on your dev box too!
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
18
THE BASICS
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
19
Tools of the Trade
• Build System– Phing– Apache ANT– Capastrano– Plain PHP or BASH or BAT Files
• File Transfer– Rsync (*NIX Environments)– Xcopy (Windows Environments)
• Configuration Management• Database Migration
– DBDeploy– Doctrine
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
20
Phing Philosophy
• Build scripts contains "Targets"– Targets should be small and specialized.– Example Targets:• clean
– Clear temporary and cached files
• copy– Copy files to their intended destination
• migrate– Upgrade the database schema
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
21
Phing Philosophy
• Targets can have dependencies– Target "live" can depend on clean, copy, and
migrate– Meaning, when we run the "live" target, it first
runs clean, copy, then migrate• And any tasks they depend on
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
22
Installing Phing
• pear channel-discover pear.phing.info• pear install phing/Phing
– Want all the little dependencies?• pear config-set preferred_state alpha• pear install –alldeps phing/Phing• pear config-set preferred_state stable
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
23
Running Phing
• $> phing –v– Lists Phing version
• Phing expects to find a file "build.xml" in the current directory– build.xml defines the targets– You can use the "-f <filename>" flag to
specify an alternate build file like• $> phing -f build-live.xml
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
24
Syntax
• XML– If you don’t already know it, you have bigger
problems• Variables– ${variablename}– Conventions• Psuedo-Namespaces using periods• ${namespace.variable.name}
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
25
Basic Conventions
• build.xml– Central repository of your top-level tasks
• build-*.xml– Can be included into build.xml. – Usually for grouping by target (dev, staging, live)
or task (migrate, test, etc.)• build.properties– Technically an INI file that contains variables to be
included by build files.
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
26
Basic build.xml
ZendCon '09
<?xml version="1.0" encoding="UTF-8"?><project name="ZendCon" default="default" basedir="."> <property file="build.properties" /> <target name="default"> <echo message="Howdy World!" /> </target> </project>
Automated Deployment With Phing - Daniel Cousineau
27
Basic build.properties
ZendCon '09
source.directory = /src
## Database Configurationdb.user = usernamedb.pass = passworddb.host = localhostdb.name = database
Automated Deployment With Phing - Daniel Cousineau
28
FILE TRANSFER
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
29
Techniques• Built in Phing Copy Task
– Cross Platform– Slow– Not over network
• Version Control Checkout/Update– Syncs deleted files– User created files usually ignored
• Rsync– Great for *NIX environments.– Almost guaranteed to be installed on
• Linux, BSD, Mac OSX• Can be installed on Windows
• Xcopy– Good for Windows environments– Faster than Phing Copy– Not over network
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
30
Pitfalls
• Cleanup Deleted Source Files– Usually only a problem when you have * include
patterns
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
31
Pitfalls
• User created files are a HUGE pitfall– /htdocs/images/upload
• Sync programs usually will delete content your deployment machine doesn’t have
• What if user upload are mixed in with files you want to manage?
– Solutions• Keep user created content completely separated. The
further up the file tree and consolidated the better.• Separate static file server, Amazon S3, or other style
CDNs
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
32
Executing External Tools
• Nearly all file transfer tools will be external commands
• For this we need the Exec task
ZendCon '09
<exec command="cp file1 file2" />
Automated Deployment With Phing - Daniel Cousineau
33
Rsync
• So many different options you’re best off finding your own tutorials or reading MAN pages
• rsync -aCvz-e 'ssh -i /path/to/ssh-key'${project.basedir}/srcuser@domain: ${staging.deployment.directory}/
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
34
Rsync Options Of Note
• -a archive-mode– Recursive copy, preserve everything, etc.
• -C cvs-exclude– Ignore .cvs, .svn, and other version control artifacts
• -v verbose– Know EXACTLY what’s going on
• -z compress– Compress information transmitted
• -e alternative remote shell– Use a custom SSH command (namely use key-pair)
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
35
Xcopy
• Best when your live deployment process involves copying to a network share on Windows machines
• xcopy${project.basedir}${source.directory}\*${staging.deployment.directory}/D /I /Y /S /E/Exclude:${project.basedir}\staging.deployment.exclude
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
36
Xcopy Options Of Note
• /d– Update only those files that are different (timestamps)
• /i– Creates target directories if they don’t exist
• /y– Do not prompt
• /s– Copy all directories and sub-directories…
• /e– …Even if the directories are empty
• /exclude– Exclude files based on glob patterns
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
37
Xcopy Exclude File Example
ZendCon '09
\tmp\\uploads\
Automated Deployment With Phing - Daniel Cousineau
38
CONFIGURATION MANAGEMENT
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
39
Techniques
• Check incoming domain– www.domain.com means load production
configuration– localhost/domain.com means load
development configuration• Check for an environment file– Contents of DOCUMENT_ROOT/.env determine
configuration• Copy over during deployment
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
40
Check Incoming Domain
• Pros– No chance for error
• Cons– Detection is a weak link• Especially when development, staging, and live have
different web servers (e.g. Apache to IIS)
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
41
Check for an Environment File
• Pros– Consistent regardless of Server or OS changes– Easy to repair• Quick FTP or SSH
– Easy Testing• Just swap file contents
• Cons– Forget to copy changes?
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
42
Copy Over During Deployment
• Pros– Images and CSS now manageable– Reduce redundancy– Simple to repurpose regular push command
• Cons– Difficult to manage– Forget to copy changes?
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
43
Which one to choose?
• Depends on your application architecture• Depends on your environment• You’ll know what’s best for your project– I use a combination of Environment File and Copy
Over During Deployment
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
44
DATABASE MIGRATION
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
45
Database Deployment Philosophy
• “Versions” often referred to as “Deltas”• Each Delta has an UP and a DOWN script– UP makes the changes– DOWN reverts the changes
• Each change should be a non-destructive schema change– Unless of course you need data alteration
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
46
DBDeploy Philosophy
• Each delta is 2 native SQL scripts– Separated by --//@undo
• Database version is stored in a special table– Version X was applied at TIMESTAMP…– Beware corruption!
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
47
DBDeploy Under The Hood
• Connect to the database, read from changelog table, get the most current revision installed
• Retrieve all delta files newer that the current revision– Revision numbers are based off of alphabetic
sorting, name your files accordingly• Pull all the “UP” changes and concatenate
them to a migration script• Up to YOU to run the migration scriptZendCon '09
Automated Deployment With Phing - Daniel Cousineau
48
Installing DBDeploy
• Is Phing already installed? Great! So is DBDeploy…• Create the changelog table– Track the current version of your DB
ZendCon '09
CREATE TABLE changelog ( change_number BIGINT NOT NULL, delta_set VARCHAR(10) NOT NULL, start_dt TIMESTAMP NOT NULL, complete_dt TIMESTAMP NULL, applied_by VARCHAR(100) NOT NULL, description VARCHAR(500) NOT NULL );
ALTER TABLE changelog ADD CONSTRAINT Pkchangelog PRIMARY KEY (change_number, delta_set);
Automated Deployment With Phing - Daniel Cousineau
49
Basic Delta: 01_init.sql
ZendCon '09
--//
CREATE TABLE IF NOT EXISTS `users` ( `id` int(10) unsigned NOT NULL auto_increment, `handle` varchar(25) NOT NULL default '', PRIMARY KEY (`id`), UNIQUE KEY `users_handle_index` (`handle`) ) ENGINE=InnoDB DEFAULT;
--//@UNDO
DROP TABLE IF EXISTS `users`;
--//
Automated Deployment With Phing - Daniel Cousineau
50
Run Migration
• First add a task to your Phing build file
ZendCon '09
<target name="migrate">
</target>
Automated Deployment With Phing - Daniel Cousineau
51
Run Migration
• Generate the SQL migration file
ZendCon '09
<target name="migrate"> <dbdeploy url="mysql:host=${db.host};dbname=${db.name}" userid="${db.user}" password="${db.pass}" dir="${project.basedir}/db/deltas" outputfile="${project.basedir}/up.sql" undooutputfile="${project.basedir}/down.sql" /></target>
Automated Deployment With Phing - Daniel Cousineau
52
Run Migration
• Execute the SQL script
ZendCon '09
<target name="migrate"> <dbdeploy url="mysql:host=${db.host};dbname=${db.name}" userid="${db.user}" password="${db.pass}" dir="${project.basedir}/db/deltas" outputfile="${project.basedir}/up.sql" undooutputfile="${project.basedir}/down.sql" /> <exec command="/usr/bin/mysql -h${db.local.host} -u${db.local.user} -p ${db.local.pass} ${db.local.name} < ${project.basedir}/up.sql" dir="${project.basedir}" checkreturn="true"> </target>
Automated Deployment With Phing - Daniel Cousineau
53
Pitfalls
• Make sure you have a good CLI app for loading a SQL file– /usr/bin/mysql– mysql.exe
• Build script travelling across operating systems?– Write your own Phing task to execute the migration
output?• Was the changelog table created in the first
place?ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
54
Doctrine Database Migrations
• Integrated into the Doctrine CLI tool– ./doctrine migrate
• Same philosophies– BUT, deltas are PHP objects that contain an UP
and a DOWN method• Run using the Phing exec task
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
55
Other Database Migration Tools
• http://www.liquibase.org/
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
56
CLOSING THOUGHTS
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
57
Further Resources
• The people around you– More people than you know automate their
deployment– There are many, many different techniques– My techniques may suck for your particular setup
• #phpc on freenode– Many of the speakers hang out there– Many smart people hang out there– I HANG OUT THERE!– Therefore I am smart
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
58
Further Resources
• Blogs– http://phpdeveloper.org
• Documentation– http://phing.info– http://dbdeploy.com
• Google• Experimentation– Pushing to a local directory– Pushing to a virtual machine
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
59
Questions?
ZendCon '09
http://joind.in/937