handling long-running tasks in rails

Long-Running Tasks In RailsWithout Much Effort

Andy Stewart

April 2008

Not me

Still not me

Why?

User startssomething

lengthy

Delayedtask

Regulartask

Example

Synchronous link to Campaign Monitor

10

BackgroundJob (BJ)

Delayed Job (DJ)

BackgrounDRbBeanstalkd &

async-observer

AM4R

Workling

Spawn

BackgroundFu

Sparrow

Conveyor

Factors

DurabilityProgress reporting

SchedulingSerial vs. Parallel

Process managementExecution environment

Error handlingLearning curve

Installation burdenConstraints on task code

Durability

Durable Hopeful

Background JobBackgroundFuDelayed JobWorkling

BackgrounDRbBeanstalk

SpawnSparrowWorkling

50%0% 100%

Tool Method

Background Job Process’s stats

BackgroundFu Incremental

BackgrounDRb Ask workers

Workling DIY “return store”

Scheduling

BackgrounDRb

cron + {rake, script/runner, whatever}

Delaying(on purpose)

Delayed Job

Beanstalk

ProcessManagement

No Problem Hassle

Background JobSpawn

Everything else

Examples

BackgrounDRb

# Lots of configuration...

# Your worker codeclass BillingWorker < BackgrounDRb::MetaWorker set_worker_name :billing_worker

# Called when worker is loaded for the first time def create(args = nil) end

# The lengthy task. def charge_customer(customer_id = nil) # ... do stuff ... endend

# Your invocation codeclass CustomersController < ApplicationController def upgrade_account # ... MiddleMan.worker(:billing_worker).charge_customer(@customer.id) endend

Beanstalk# Some configuration...

# Comments controllerdef create @comment = Comment.new(params[:comment]) if @comment.save BEANSTALK.yput({:type => "comment", :id => @comment.id}) rescue nil # Then redirect and return endend

# Worker - Rake taskloop do job = BEANSTALK.reserve job_hash = job.ybody # ybody deserializes the job case job_hash[:type] when "comment" if Comment.check_for_spam(job_hash[:id]) job.delete else job.bury end else puts "Don't know what type of job this is: #{job_hash.inspect}" endend

Source: nubyonrails.com

Async Observer

# Some configuration...

# Start some workers$ ./vendor/plugins/async_observer/bin/worker

# Your codeclass Person < ActiveRecord::Base async_after_create do |person| SiteStats.increment_members() end

def befriend(other_person) Friendship.async_send(:create, self, other_person) endend

Delayed Job

# Small config - db migration

# A job is a Ruby object with #performclass NewsletterJob < Struct.new(:text, :emails) def perform emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) } endend

# Stick in queueDelayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))

# Delayed jobBatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv)

# Running tasks$ rake jobs:work <CTRL-C to cancel>

Background Job

# Command line

$ Bj.submit "./jobs/background_job_to_run"

$ Bj.submit "./script/runner ./jobs/background_job_to_run"

# Within Rails

def upload_to_s3 Bj.submit "./script/runner ./jobs/s3_uploader.rb #{self.id}" end

Source: slackworks.com

1

Background Job

Two-line installationZero configuration

No job-code constraintsAutomatic* process managementDurable, prioritised, taggable jobs

Comprehensive job resultsClustering

* Manual if you want

Feed my spaniel

Buy my PeepCode PDF

Text’s Licence:Creative Commons Attribution-Share Alike 2.0 UK: England & Wales Licence

My Photos’ Licence:Creative Commons Attribution-Noncommercial-No Derivative Works 2.0

UK: England & Wales Licence

Photo Credits:1. BBC2. Forever in Song (CD cover)3. Me4. Me