north east user group tour

©All rights reserved. Zend Technologies, Inc.

Northeast User Group Tour – April 2011

By Kevin Schroeder

Technology Evangelist

Zend Technologies


About Kevin

Past: Programming/Sys Admin

Current: Technology Evangelist/Author/Composer

@kpschrade


What do we do?


Linux(rpm/web repositories)

IBM i(PTF)

Windows(MSI)

ApplicationPerformance

Acceleration

Optimization

Caching

Reliability &Management

Monitoring

Root-Cause

Configuration

Scale-Out

Clustering

Job Queue

Downloads

Enterprise Enablement

Hot Fixes

Support

Java Bridge

Zend Framework

PHP

Zend Server

Zen

d S

tudio

(Ecl

ipse

-base

d)

Colla

bora

teD

ebug

Pro

file

Test


A quick word on something new…


Wanna try it?1. Go to

http://prerelease.adobe.com/callout/?callid=6C1235C63BDC414896951DDCEC01D5E8

… or go to http://www.eschrade.com/fbphp :-)

2. Email me at [email protected] (this will help expedite your approval)



http://www.eschrade.com/fbphp

mailto:[email protected]


Things we’re giving away


• 1 Zend Certification Test (you still have to pass it)

• 1 copy of Zend Studio

• ElePHPants

• Books

• Dart Boards

• Little Note Books (the paper kind)


Queuing


Could your PHP apps benefit from being able to process data or execute

asynchronously?

Twtpoll results


• Performance Execute logic apart from the main request (asynchronicity)

• Scalability The ability to handle non-immediate logic as resources are

available

Why would you want to queue?


• People often say that performance and scalability are separate things.

This is inaccurate (consider Google)

• Performance is the speed by which a request is executed

• Scalability is the ability of that request to maintain its performance as load/infrastructure increases

Performance & Scalability


• DON’T!!

• You are not Facebook

• You probably won’t be

• Don’t overcomplicate your problems by trying to be

So how do you scale to Facebook size?


Typical anatomy of a PHP Application

| 16

Presentation

Application Control

Database Access

Business Logic

Presentation

Application Control

Business Logic

Presentation

Bad for scala-bility!

©All rights reserved. Zend Technologies, Inc. | Apr 12, 2023

| 17

Presentation

Database Access

Business Logic

Application Control

Something.phtml

Something.phtml

Something.phtml

Something.phtml

Something.phtml

Something.phtml

Something.phtml

Something.phtml

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Something.p

hp

Good for Scalabilit

y


Defined tasks

Loose coupling

Resource discovery

What helps make software scalable?


The Golden Rule of Scalability

“It can probably wait”


• Pre-caching data

• Data analysis

• Pre-analysis (predicting where your users will go)

• Data processing

• Pre-calculating (preparing data for the next request)

Data is “out of date” once it leaves the web server

Immediacy is seldom necessary

Asynchronous execution uses


• A sledgehammer can hit a machine Scalability and High Availability are yin and yang

• A site that can’t keep running is not scalable

• A site that can’t scale will fail (if it gets really popular)

• Machines can be added and removed at will Not “cloudy” necessarily

• No single point of failure Data exists in at least two, preferably at least three, places

Characteristics


• Waste disk space

• Control usage (don’t let users do anything they want)

• Pre-calculate as much as possible Calculate and cache/store

• Don’t scan large amounts of data

• Keep data processing off the front end servers

• Don’t just cache Don’t let it substitute for thought

Cache hit rates can be meaningless if you have hundreds of cache hits for a request

Considerations


• Build a deployment mechanism with NO hardcoded values like directory or resource locations• Make as much as possible configurable/discoverable

• Decouple/Partition Don’t tie everything (relationships and such) into the

database

• Use queues/messaging Stomp interfaces are really good for PHP – Can also use

Java Bridge

Zend_Queue has several interfaces

• Try to use stateless interfaces polling is more scalable than idle connections; introduces

lag

Considerations


• Use Cron /w PHP CLI (probably shouldn’t do this)

• Use Gearman

• Use home-grown (don’t do this)

• Use pcntl_fork() (NEVER do this)

• Use Zend Server Job Queue

Options


Gearman* Zend Server Job Queue

FreeLightweightOpen Source(mostly) language agnosticDistributed queuing

Ready to goIntegrates with Event MonitoringIntegrates with Code TracingRuns over a widely known protocolLoad distribution can be accomplished outside of the queue

Your only real options

* I am not an expert on Gearman. Corrections will be taken in the spirit that they are given.

Very cloud friendly

For obvious reasons, I will focus on Zend Server


• Schedule jobs in the future

• Set recurring jobs

• Execute immediately, as resources are available (my fav)

• Utilize ZendJobQueue()

Using the Zend Server Job Queue


Job Queue Architecture – Elastic Backend

Users!

Web Server /w

JQ

Web Server /w

JQ

Web Server /w

JQ

Web Server

Web Server

Web Server

Load B

ala

nce

r

• Pros Scale the backend as necessary

Default (easy) mechanism

• Cons Getting the job status requires using a DB


Job Queue Architecture – Elastic Frontend

Users!

Web Server

Web Server

Web Server

Web Server /w

JQ

Web Server /w

JQ

Web Server /w

JQ

Load B

ala

nce

r

• Pros• Easy to communicate with the Job Queue server handling the job

Cons• Requires you to build your own interface


• Create a task-handling controller

• Create an abstract task class Understands the Job Queue

Self contained• If Elastic Backend: connects to localhost

• If Elastic Frontend: connects to load balancer (my preferred), load balanced JQ servers manage themselves

• Execute the task, have it serialize itself and send it to send to the task handler

Kevin’s favorite way to implement it


com\zend\jobqueue\Manager

Handles connecting to the queue and passing results back and forth

com\zend\jobqueue\JobAbstract

Abstract class that a job would be based off of

com\zend\jobqueue\Response

The response from the manager when a job is queued. Contains the server name and job number

org\eschrade\jobs\Scandir

The actual job that scans the directory

org\eschrade\jobs\ScandirResult

An object that represents the data found

Classes involved in the demo


Create job and set data

Execute job• Job passes itself to the queue manager

• Manager serializes job

• Manager uses HTTP call through a load balancer to queue the job

• The queue on the other end returns the job id and server name

• Job ID and server name is passed to the client

Client polls the manager to get a completed job• When the job is returned pass the serialized version of the

executed job

Execution Flow


Let’s write some code

(no I’m not copping out with slides. We’re all told to show our work in grade skool)


Deployment


Agenda

• What we will be talking about Development->Testing->Staging->Production

Rsync/SVN/PEAR/yum mechanisms

• What we will not be talking about Build tools (Phing, Maven, etc.)

Database versioning/deployment

Continuous Integration

MSI installs (Sorry MS folks, I really wanted to but, y’know, time)

Test-Driven Development


Before we get started

• Who here is a developer?

• Who here is a system administrator?

• Who here has an existing deployment strategy?

• Who here has edited a file in production?

• Who here has wished they didn’t?


Application Stages

| 36

Increasing Maturity of PHP Applications

Development Testing/QA Staging Production


Maturity

| 37

Increasing Maturity of PHP Applications

Testing & Validating

Application

Testing & Validating

Environment

Benefit

Structure

Distinct Dev Environments

Uptime!!

Where are you?


Development

• Purpose To provide developers an environment to write their code

• Characteristics Should be similar to production, though it usually isn’t

Often security is more open

Usually local to the developer


Consider Virtualization


Testing

• Purpose To provide a non-programming environment to test

functionality

• Characteristics Continuous Integration could be included

Generally no development occurs, only testing

Developers should not do the testing, if possible

Restricted outbound networking– Use Zend Server Monitoring and Code Tracing to help reproduce

errors


Prepared for and handed off to System Administrators

(the stuff we’re here to talk about)


Staging

• Purpose To test your deployment process/scripting (not your code)

• Characteristics Developers generally do not have access, unless they are

also the sysadmin

Very restricted outbound networking

Mirrors production as best as possible


Change Control

• A formal process used to ensure that changes are introduced in a controlled and coordinated manner

• Protect against unnecessary changes introduced to a system without forethought

• Requires documenting the release process Software version number

Plans for rollback (tested? y/n)

Expected outage time

Customer impact

• Done prior to doing anything in production


Pre-Production (Optional)

• Purpose Test the code in the production environment without

impacting customers

• Characteristics Not likely to have use in the cloud or large scale

deployments

Deployed in production immediately prior to making it live

Test the application with production settings without customer interaction


Production

• Purpose• Do whatever it is that it’s supposed to be doing

• Characteristics• Developers do not have access (as they might be tempted

to fix something)

• Deployment should be done without requiring developer input

• Very limited inbound traffic – generally only the service that is being served. i.e. HTTP


Application considerations

• Build the application to be aware of different environments Zend_Config_* is good at this

• Do not set the environment type in your code Environment variables can be placed in server config

• It means that you won’t accidentally keep your development settings turned on in production

• Make production your default environment Keeps accidental data leakage to a minimum


Deployment Option 1 – rsync

• Benefits Easy

Already installed on your system

• Drawbacks No deployment script can be run

Not integrated with either PHP or the OS

Rollbacks require rolling back the entire source server• Does not understand versioning

– One option is to move the source tree prior to the rsync

• Uses xinetd


Deployment Option 2 – Source Control

• Benefits Easy

You should already be using source control

Has versioning

Can have pre/post install scripts

Easy to do automatic deployments

• Drawbacks Do you want to put details on how to access your source

code on your production box?

Requires that your source control is accessible from production

Deny access to .svn directories


Deployment Option 3 – PEAR• Benefits

Designed for PHP

Very scriptable• Natural for doing things like clearing a cache

Has a better understanding of PHP

Cross platform compatible

• Drawbacks Requires admins to be familiar with PHP code

Limited to PHP deployment

Available tooling is restrictive/build your own package.xml

Define standardized locations (www, library, etc.)


Deployment Option 3 – PEAR

Creating your PEAR Channel

pear channel-discover pear.pirum-project.org

pear install pirum/Pirum-beta

vi /var/www/pear/pirum.xml

<server> <name>localhost/pear</name> <summary>OurOrg PEAR channel</summary> <alias>helloworld</alias> <url>http://localhost/pear/</url></server>

php ./pirum build /var/www/pear/


Deployment Option 3 - PEAR

Creating your PEAR package

php ./build-pear.php

pear package

pirum add /var/www/pear/ HelloWorld-0.1.3.tgz


Deploying your app

pear channel-discover localhost/pear

pear config-set www_dir /var/www

pear install helloworld/HelloWorld

pear clear-cache

pear upgrade helloworld/HelloWorld

Roll Back

pear uninstall helloworld/HelloWorld

pear install helloworld/HelloWorld-0.1.3


Deployment Option 4 - OS-based

• Benefits Easily added as part of your server deployment

• Very cloud friendly

Admins already know how to use it

Can describe OS-requirements/dependencies

Downtime limited to actual install, not network transfer time

• Drawbacks Depends on environment

Any PHP deployment scripting needs to be deployed with the application and executed in %post hook


Which should you use?

• Need something simple? rsync

• Need to deploy to multiple disparate platforms? PEAR

• Need to deploy internally/with minimal developer input?OS (yum/apt) If using the OS consider maintaining an internal “blessed”

repository instead of depending on the distribution


Continuous Deployment

• Using automated deployment

• Very tight end-to-end integration

• Requires a lot of trust that the developers have a fully tested application w/ backend and frontend code

• Probably more work to manage if you can count the servers in your production environment with your fingers

• Could be beneficial but make sure you know what you’re doing You will need to have several testing experts


Takeaways

• No need for copy-and-paste deployment

• No need for (S)FTP (Kinda not good for deployment)

• Have an easy rollback mechanism

• Prepare for errors in deployment

• Try to minimize the amount of scripting needed for deployment

• Consider using VMs on anything important to keep downtime to zero

bit.ly/phpDepAll


Follow us! Zend Technologies

http://twitter.com/zend

http://twitter.com/kpschrade (me!)


Get this information and all the examples at eschrade.com…

north east user group tour

Technology

job queue server

job status

cron w php cli

zend certification test

php applicationbad

thisuse pcntl

web serverimmediacy

localhostif elastic