how to make ajax work for you

How to make

Simon Willison - http://simonwillison.net/Web 2.0 Expo Berlin

5th November 2007

work for you

http://simonwillison.net

http://simonwillison.net

In this talk

• Where Ajax came from

• When you should (and shouldn’t) use it

• Ajax fundamentals (including a DOM and JavaScript refresher)

• Advanced Ajax techniques

• Picking and using a JavaScript library

Please askquestions

at any time

February 2005

AJAX v.s. Ajax

“Asynchronous JavaScript + XML”

AJAX v.s. Ajax

“Asynchronous JavaScript + XML”

“Any technique that allows the client to retrieve more data

from the server without reloading the

whole page”

Why did Ajax happen?

• Two reasons:

• The browsers that mattered all added support for IE’s XMLHttpRequest object (which IE has had since 1998)

• Jesse James Garrett came up with a name that sucked a lot less than “XMLHttpRequest” did

Pre-Ajax...

• “Remote Scripting” (not as catchy)

• Various unpleasant but ingenious hacks:

• Scriptable Java applets

• Passing data through changes to a cookie

• Passing data through a hidden iframe

• (That one’s actually still in use)

Ajax scenarios

• Drag and drop shopping carts

• Massive page updates e.g. pagination

• Unlinkable content (see also bookmarking)

• Breaking browser expectations

• Forgetting the loading icon!

Some anti-patterns

Fundamentals

The three legged stool

(X)HTMLCSS JavaScript

The three legged stool

(X)HTMLCSS JavaScript

The DOM

The DOM

The DOM

<html>

<h1>

<div id="head">

<body>

<title>

<head>

<div id="main">

<p> <p><p>

Key DOM methods• document.getElementsByTagName,

document.getElementById

• el.childNodes, el.parentNode, el.firstChild, el.nextSibling, el.getAttribute

• el.appendChild, el.insertBefore, el.setAttribute, el.removeChild

• document.createElement, document.createTextNode

http://www.howtocreate.co.uk/tutorials/javascript/domstructure



JavaScript doesn’t suck!

• Just remember; your; semicolons;

• Avoid accidental globals: always declare your variables with “var”

• Take the time to figure out higher order programming, where you treat functions as objects

CSS class switching

• Often if you are dynamically making large-scale changes to the layout of a page...

• ... you can do most of the work by defining an alternative CSS class ...

• ... and then switching a container element’s className property in your JavaScript

Firebug!

Firebug

• Inspect HTML elements to see what CSS rules currently apply to them

• Interactive console for experimenting with JavaScript against the loaded page

• Full JavaScript debugger, including breakpoints

• Profile your code and your site’s assets and Ajax requests

• Logging with console.log() and friends

XMLHttpRequest

• The object that lets you make HTTP requests from within JavaScript

• IE had it first, using an ActiveX Object

• Almost nothing to do with XML

• Asynchronous (so you use callbacks)

var xhr = createXHR();xhr.onreadystatechange = onComplete;xhr.open('POST', '/helloworld.php', true);xhr.setRequestHeader( 'Content-Type', 'application/x-www-form-urlencoded');xhr.send('name=Simon&age=26');

xhr.open('POST', '/helloworld.php', true);

The third argument specifies that we want to make an asynchronous request (“tell me when you’re done by calling this function”). If you set this to false you’ll get a synchronous request, which will completely hang the browser until the request returns.

Don’t do that.

• That’s quite a lot to remember

• Best option: don’t remember any of it!

• Use a good library instead

• Lots more on those later on

Too much boilerplate?

doRequestdoRequest("GET", "/blah.txt", null, function(xhr) { doSomethingWith(xhr.responseText);});

doRequest("GET", "/getinfo.php", { "name": "Simon" }, function(xhr) { doSomethingWith(xhr.responseText);});

doRequest("POST", "/getinfo.php", { "name": "Simon" }, function(xhr) { doSomethingWith(xhr.responseText);});

Data transport options

• Requests can have anything you like in the body:

• HTML, XML, JSON...

• So if you really want to, you can access SOAP or XML-RPC services from JavaScript

• For almost all purposes, regular form arguments suffice

• REST APIs are a good fit as well

Client data submission

• Plain text

• HTML fragments

• CSV or similar

• XML

• Executable JavaScript

• JSON

Server responses

• Just sending back a simple value

• xhr.responseText is all you need

• Pros:

• Absurdly simple

• Cons:

• Only supports a single value or string

Plain text

• Return an HTML fragment, and inject it using innerHTML

function onSuccess(xhr) { var div = document.getElementById('response'); div.innerHTML = xhr.responseText;}

<div id="response"></div>

HTML fragments

• Pros:

• Again, really simple

• You can leverage existing server-side templates and techniques

• Cons:

• You have to change your server-side code to change client-side behaviour

• You can't reuse your endpoint

Fragment pros and cons

• We can pass JavaScript code back from the server and execute it directly

document.getElementById( 'response').innerHTML = 'Done!';

function onSuccess(o) { eval(o.responseText);}

Executable JavaScript

• Pros:

• Incredibly simple client-side code

• The server can do abolutely anything, without needing to modify the client-side code

• Cons:

• Weird factoring of code - dynamically generating code to run on the client makes it hard to keep track of the big picture

• How about just sending structured data?

• We want structured data!

Simon Willison,26,[email protected]

• Pros:

• Easy to parse: xhr.responseText.split(',');

• Cons:

• How do we pass values containing commas?

• Index based, so brittle and hard to maintain

CSV or similar

mailto:[email protected]

mailto:[email protected]

• Surely XML will save us!

<person> <name>Simon</name> <age>26</age></person>

function onSuccess(xhr) { var dom = xhr.responseXML; showPerson( dom.getElementsByTagName('name')[0].data, parseInt(dom.getElementsByTagName('age')[0].data, 10) });}

XML

• Pros:

• We can re-use our DOM knowledge

• xhr.responseXML parses it for us

• We can call existing Web Services directly

• Cons:

• Verbose handling code

• Is this the simplest thing that could possibly work?

XML pros and cons

• A standard for representing common data structures, "discovered" by Douglas Crockford

• Common data structures?

• Associative arrays (dictionaries / hashes)

• Arrays (lists)

• Numbers, strings, booleans

• A subset of JavaScript syntax

JSON

{"name": "Simon", "age": 26}

function onSuccess(xhr) { var person = eval('(' + xhr.responseText + ')') ...)

JSON example

{"people": [{ "name": "Simon", "email": "[email protected]" }, { "name": "Natalie", "email": "[email protected]" }]}

function onSuccess(xhr) { var people = eval('(' + xhr.responseText + ')'); ...}

More complex JSON

<people> <person> <name>Simon</name> <email>[email protected]</email> </person> <person> <name>Natalie</name> <email>[email protected]</email> </person></people>

To compare...

JSON XML

Bytes (excluding whitespace) 95 156

Lines of code to process 1 7+

JSON v.s. XML

• Libraries for consuming and generating JSON are available for many languages, including:

• Python

• Ruby

• PHP

• Java ...

• JSON works really well as a general purpose tool for exchanging data structures

It ain’t just for JavaScript

• Modern browsers support XSLT transforms - albeit with differing APIs

• These can be significantly faster than regular DOM manipulation

• If you're having performance problems, this may be worth looking in to

Aside: XSLT

• For quick and simple apps, HTML fragments offer the path of least resistance

• For most purposes, JSON makes the most sense

• Use XML if you already have an XML Web Service that you want to consume (or you need an XSLT speed boost)

Recommendations

Let’s see somerunning code

Advanced Ajax topics

Cross-domain Ajax

The same-origin restriction

• XMLHttpRequest is only allowed to communicate with the domain from which the page was loaded

• The same is true of JavaScript calls between windows, frames and iframes

• This is a critical browser security feature

The intranet attack

• http://wiki.private.corp/ contains private data

• User is tricked in to visiting evilexample.com

• evilexample.com uses XMLHttpRequest (or a hidden iframe child) to load and steal data from http://wiki.private.corp/

http://wiki.private.corp




• JSON-P offers a (hackish) solution to the cross-domain XMLHttpRequest restriction

http://example.com/query?name=Simon

callback({"name": "Simon", "age": 26});

function loadJSON(url) { var script = document.createElement('script'); script.type = 'text/javascript'; script.src = url; document.getElementsByTagName('head')[0].appendChild(script);}

function callback(person) { ...}

loadJSON('http://example.com/query?name=' + name);



http://example.com/query?name='

http://example.com/query?name='

• Pros:

• Lets you call services hosted on other domains, provided they opt-in

• del.icio.us, Flickr, other sites offer this

• Cons:

• Detecting errors is harder than with XMLHttpRequest

• You have to trust the site who’s code you are executing

Pros and cons

• XSS attacks occur when a malicious third party injects untrusted code (HTML, CSS or JavaScript) in to your page

• It’s generally a server-side consideration, but you need to take it in to consideration when writing JavaScript as well

• Ajax can compound the problem by allowing users to create self-propagating “worms”

<img src="http://example.com/admin/delete.php?id=5">

Trick someone who is signed in to example.com/admin/ in to visiting a page hosting that image and you’ll force them to delete something.

CSRF for GET

http://example.com/admin/delete.php?id=5

http://example.com/admin/delete.php?id=5

<form id="evil" action="http://example.com/admin/delete.php" method="POST"> <input type="hidden" name="id" value="5"></form><script>document.getElementById('evil').submit()</script>

CSRF for POST

• For regular forms, add a hidden field with a one-time secret token and check for that token on every submission

• For Ajax, either do the above or use:

CSRF protection

xhr.setRequestHeader( "X-Requested-With", "XMLHttpRequest");

Unobtrusive JavaScript

• JavaScript isn't always available

• Security conscious organisations (and users) sometimes disable it

• Some devices may not support it (mobile phones for example)

• Assistive technologies (screen readers) may not play well with it

• Search engine crawlers won't execute it

• Unobtrusive: stuff still works without it!

Unobtrusive JavaScript

• Start with solid markup

• Use CSS to make it look good

• Use JavaScript to enhance the usability of the page

• The content remains accessible no matter what

Progressive enhancement

• Unobtrusive JavaScript relies on adding event handlers after the fact

• The naive way of doing this:

window.onload = function() { for (var i = 0, link; link = document.links[i]; i++) { if (link.className == 'external') { link.onclick = function() { window.open(this.href); return false; } } }};

Adding events

• If you later assign something else to window.onload (or one of the link.onclicks) you will clobber your behaviour

• You should add events "properly"...

• Different in different browsers

• attachEvent v.s. addEventListener

• My advice: use a library!

Problem

Unobtrusive examples

• One of the earliest examples of this technique, created by Aaron Boodman (now of Greasemonkey and Google Gears fame)

labels.js

• Once the page has loaded, the JavaScript:

• Finds any label elements linked to a text field

• Moves their text in to the associated text field

• Removes them from the DOM

• Sets up the event handlers to remove the descriptive text when the field is focused

• Clean, simple, reusable

<label for="search">Search</label><input type="text" id="search" name="q">

How it works

• An unobtrusive technique for revealing panels when links are clicked

<ul> <li><a href="#panel1" class="toggle">Panel 1</a></li> <li><a href="#panel2" class="toggle">Panel 2</a></li> <li><a href="#panel3" class="toggle">Panel 3</a></li></ul>

<div id="panel1">...</div><div id="panel2">...</div><div id="panel3">...</div>

easytoggle.js

• When the page has loaded...

• Find all links with class="toggle" that reference an internal anchor

• Collect the elements that are referenced by those anchors

• Hide all but the first

• Set up event handlers to reveal different panels when a link is clicked

• Without JavaScript, links still jump to the right point

How it works

• Large multi-select boxes aren't much fun

• Painful to scroll through

• Easy to lose track of what you have selected

• Django's admin interface uses unobtrusive JavaScript to improve the usability here

Django filter lists

• Ajax is often used to avoid page refreshes

• So...

• Write an app that uses full page refreshes

• Use unobtrusive JS to "hijack" links and form buttons and use Ajax instead

• Jeremy Keith coined the term "Hijax" to describe this

• Live search / filtering

• Adding comments / tags

• Smart form validation

• Checking if usernames are already taken

• You could even make a progressively enhanced chat room

More progressive Ajax suggestions

• All of these examples use code that runs when the window "load" event is fired

• Wait until page is loaded, then manipulate the DOM

• Problem: If the page takes a while to load (large inline images) there will be a Flash Of Unstyled Content (FOUC)

• Also, if the user clicks things before the setup code has fired they won't get the expected behaviour.

The onload problem

• If the effect requires hiding some otherwise visible elements, you can document.write a stylesheet

<script type="text/javascript">document.write('<style type="text/css">';document.write('.hideme { display: none }');document.write('</style>');</script>

document.write(css)

• If your effect is triggered by the user clicking on something, attach code to the document.body "click" event right at the start of your code (no need to wait for the document "load" event)

YAHOO.util.Event.on(document.body, 'click' function(e) { var clicked_el = YAHOO.util.Event.getTarget(e); // ...}

body.onclick

• A number of attempts have been made to create an onDOMReady event that fires when the DOM has been constructed but before the page has loaded

• dean.edwards.name/weblog/2006/06/again/

• Modern libraries all support this in some form

• Problem: CSS may not have loaded, so calculated dimensions may be incorrect

onDOMReady

• Unobtrusive JavaScript is the Right Way to go about things, but runs in to browser differences even faster than regular JavaScript

• Which leads us neatly on to...

Frameworks and Libraries

JavaScript libraries

• ajaxpatterns.org lists over 40 general purpose JavaScript libraries

• ... and that’s not including the many libraries tied to a specific server-side language

• Why are there so many of them?

“The bad news: JavaScript is broken.

The good news:It can be fixed with more JavaScript!”

Geek folk saying

• Inconsistent event model (thanks, IE)

• Positioning and co-ordinates

• Memory management (thanks, IE)

• The DOM is a horrible API!

• JavaScript-the-language has quite a few warts

• But it’s powerful enough to let you fix them

• Classes and inheritance can be confusing

• Many useful JS utility functions are missing

• Drag and drop and Animation are really hard

• Prototype (and Scriptaculous)

• The Yahoo! User Interface Library - YUI

• jQuery

• The Dojo Toolkit

Narrowing them down...

Honourable mentions

• MochiKit

• No updates since early 2006

• The Google Web Toolkit

• I don’t do Java

• mooTools

• Lots of buzz, but I haven’t figured out why yet

Download API Docs Tips and Tutorials Blog Discuss Contribute

Prototype is a JavaScript Framework that aims toease development of dynamic web applications.

Featuring a unique, easy-to-use toolkit for class-driven

development and the nicest Ajax library around, Prototype

is quickly becoming the codebase of choice for web

application developers everywhere.

Prototype and script.aculo.us: The "Bungee

book" has landed!

Core team member Christophe

Porteneuve has been hard at work

for the past few months tracking

and documenting Prototype for his

new book Prototype and

script.aculo.us, which is now

available as a Beta Book from the

Pragmatic Programmers (and is

scheduled to ship later this year).

Read more !

DownloadGet the latest version—1.5.1

LearnOnline documentation and resources.

DiscussMailing list and IRC

ContributeSubmit patches and report bugs.

Who's using Prototype?

Meet the developers

© 2006-2007 Prototype Core Team | Licenses: MIT (source code) and CC BY-SA (documentation).

Prototype and Scriptaculous

• Prototype focuses on basic browser compatibility and JavaScript language enhancement

• Tries to make JavaScript more like Ruby

• Extends most of JavaScript’s built-in objects with new functionality

• Scriptaculous adds fancy effects, basic widgets and drag and drop

$$('#bmarks li').each(function(li){ Event.observe(li, 'click', function(e) { this.style.backgroundColor = 'yellow'; }.bindAsEventListener(li));});

The Yahoo UI Library

• Created at Yahoo!, BSD licensed

• Designed for both creating new applications and integration with legacy code

• Focused on browser issues; almost no functionality relating to JS language itself

• Extensively tested and documented

dom event connection

animation dragdrop

utilities

controls

autocomplete calendar

menu slider treeview

container

YAHOO.util.Event.on(window, 'load', function() { var div = YAHOO.util.Dom.get('messages'); setTimeout(function() { var anim = new YAHOO.util.Anim(div, { height: {to: 0}, opacity: {to: 0} }, 0.4); anim.animate(); anim.onComplete.subscribe(function() { div.parentNode.removeChild(div); }); }, 2000);});

$E = YAHOO.util.Event;$D = YAHOO.util.Dom;

$E.on(window, 'load', function() { var div = $D.get('messages'); ...});

Common YUI idiom

jQuery

jQuery(function($) { $("a.hideme"). css('color', 'red'). click(function() { $(this).hide("slow"); return false; });});

• Simple philosophy: find some nodes, then do something to them

• Minimal impact on your global namespace - it adds two global symbols: jQuery and $, and $ can be easily reverted

• API designed around “chaining” - other libraries are now emulating this

• Outstanding node selection, based on CSS 3 and custom extensions

• Small core library with an intelligent plugin mechanism

• The oldest of the current popular libraries, pre-dating even Ajax

• Incredible amounts of functionality

• Used to suffer from a tough learning curve, although the 0.9 release simplifies things greatly

• dojo

• Core library, similar to jQuery etc

• Smart package managment with dynamic code loading

• dijit

• Advanced widget system

• dojox

• Dojo eXperimental - crazy voodoo magic

Dojo components

<div dojoType="dijit.layout.TabContainer" sizeShare="40"><div id="tab1" dojoType="dijit.layout.ContentPane" title="Form Feel"> <h2>Various Form Elements:</h2> <form name="dijitFormTest"><p><input type="checkBox" dojoType="dijit.form.CheckBox" checked="checked">Standard Dijit CheckBox<br><input type="checkBox" dojoType="dijit.form.CheckBox" disabled="disabled">Disabled Dijit<br><input type="checkBox" dojoType="dijit.form.CheckBox" disabled="disabled" checked="checked">Checked and Disabled Dijit</p>...

• Graphics (cross-browser drawing API)

• Offline storage

• Cryptography

• Templating

• Data grids and more

• “The future of the browser today”

My library selection criteria

• Enables unobtrusive JavaScript

• Plays well with other code

• Smart use of namespacing

• Global variable impact kept to a minimum

• Tie breaker: the less code I have to write the better!

• I’m currently using and recommending jQuery for most situations

• But... there’s cut-throat competition between the various libraries at the moment

• This is one of the reasons I care about interoperability - commit to a single library and you might lose out when one of the others jumps ahead of it

The law of leaky abstractions

The more you rely on abstractions, the worse off you’ll

be when one of them leaks

http://www.joelonsoftware.com/articles/LeakyAbstractions.html

My interpretation:



Thank you!

how to make ajax work for you

Technology

var xhr

createxhr xhr

oncomplete xhr

true xhr

formurlencoded xhr

ajax asynchronous javascript

javascript library

javascript debugger