django Forms in an API World

http://www.northwesternflipside.com/2011/01/10/2011-where-are-the-flying-cars/

XHR

Lets talk about me

Tareque HossainLead Software Engineer

Lets talk about forms

django Forms

And how we can use them in this day & age of APIs!

What can you expect…

• What’s wrong with forms as it is• How we use forms• Issues using form in an API

world• Approaches for tackling the

issues• The solution

The good old days..

• Write up some HTML• Throw some fancy template tags in

there{{ my_awesome_form.as_p }}

• WIN

Fast forward a few years..

You really dig APIs

Your setup looks like this:

I give you APINo $#!7.

Me likey!

Nuevo mundo..

• Django forms live on API server–Validates/ saves API requests–Doesn’t get rendered via template

• You’ve been writing forms in the frontend–Hardcoded HTML–Trying to match up data that API

expects

API Clients

• Your website no longer lives on the same application space as the API

• Common API clients–A JavaScript MVC powered website–An android app–An iOS app

Forms on API Clients

The Issue

• You can serve most platforms with an HTML app–Write form in HTML on your

webapp

• If you write native application for mobile–You recreate forms using the

interfaces available

The Issue

• These interfaces you write–Don’t have any idea about the django

forms residing on the API server–Only know what data to collect when

you explicitly code them on each device

• There’s a disconnect

Houston we have a problem..

http://epicdemotivational.com/tag/beer/page/2/

Lets take a step back

What is a form?

Lets take a step back

a printed or typed document with blank spaces for insertion of required or requested information

ˈform (noun)

Entry #4 at http://www.merriam-webster.com/dictionary/form

In the world of HTML

Part of an HTML document with input interfaces for inserting required or requested information

In the world of web apps

• A form is the interface we provide the application user to collect information

• It’s essential to any application where we collect data

In the world of django

django Forms

• A construct that:–Binds data (request.POST)–Validates data (clean)–Processes data (save)–Renders interface (as_p)

django Forms

• ModelForm–Turns your model into a form–Easiest way to get an interface for

your data

• Widgets–Define specific properties for interface

element–Separates presentation from data

types

Why not just render via template?

You can’t if:–You only use django to power your

API and the consumers are arbitrary–You run several django API servers

each dealing with different data space

Think about this architecture

Content API

Profile API

Analytics API

User AppAdmin App

Your services are distributed

• Web applications we design are increasingly becoming:–Separated between presentation

and data layer via API–Dependent on multiple API

endpoints–Rich and complex in interface

Your services are distributed

• Your site content is retrieved using the Content API–You collect user feedback on

content using forms–You provide admin interface to

manage content using forms

Your services are distributed

• Information for users are stored and retrieved using Profile API–You allow log in, creation and

update of profiles using forms–You provide admin interface to

manage profiles using forms

Your services are distributed

• Site performance and user traffic is recorded to Analytics API–You provide admin interface to

access and create custom reports using forms

Think again.

Content API

Profile API

Analytics API

User AppAdmin App

The Issue (contd.)

• At WiserTogether we love APIs & have a similar distributed setup

• We’ve been hardcoding forms in the frontend, collecting data and sending to API

The Issue (contd.)

• Whenever a data point in the backend changed, we had to update the form

• We have multiple clients who require different set of fields present on registration forms–Again, hardcoding in frontend

It was a mess$#^*!

What to do..

• django forms is used to validate and process our API data

• We wanted django forms to determine our frontend interface–But it was completely agnostic

about backend forms!

What to do..

• Deliver form definition over API• Render interface in the frontend

from the retrieved definition–No more hardcoding–Forms in the user facing

application changes as soon as backend form changes

What to do..

• Adjust form in whatever way necessary–Add/ remove fields from

registration form–Frontend renders form exactly the

way you want–No code change necessary in

frontend

What to do..

• Contain form definition in one spot• Allow a single central point to

control interface on all applications• Allow different API consumers to

retrieve form definition–And render interface appropriate for

the platform or device

3 step solution

Step 1

Serialize form with all information necessary for reproduction at frontend

Step 2

• Devise methods to handle the following over API:–Deliver form definition–Receive form data–Validate form and deliver errors– If valid save the form

Step 3

• Handle forms in the frontend using API data–Render form–Submit data– If not valid, then display errors– If valid, then display success

message, reload page or redirect as necessary

Step 1

Serialize form with all information necessary for reproduction at frontend

django Remote Forms

• Extracts all information from a given django form or model form instance

• Goes through each field & widget to extract all necessary information

• Presents the information as dictionary for further manipulation & easy serialization into JSON

As easy as π

A JSON form

Step 2

• Devise methods to handle the following over API:–Deliver form definition–Receive form data–Validate form and deliver errors– If valid save the form

Points to Ponder

• Handle CSRF yourself of using X-CSRFToken–django CSRF middleware is not

JSON friendly

• Encapsulate form processing in save method, similar to Model Form

Step 3

• Handle forms in the frontend using API data–Render form–Submit data– If not valid, then display errors– If valid, then display success

message, reload page or redirect as necessary

HTML/JS/CSS Implementation

• We created a set of rendering and data handling tools for the frontend using:

• In future, we’ll be working towards iOS implementations as well

Backbone Form Handler

• Renders forms based on definition received over API

• Uses Handlebars template for rendering:– Core form structure (form tag, fields

container, global errors)–Field & widget structure (help text,

errors)

• Error rendering

Backbone Form Handler• Allows instant validation–Similar to autocomplete–Field can be validated as soon as you

move to next one

• Allows preloading of data• Disallow editing of fields– Including selects, radio and checkboxes

• Provide submit buttons (if not supplied)

Handlebars Templates

Handlebars Widgets

Sample Backbone View

Instantiate form model

Instantiate form view

Initiate rendering by fetching the form definition

django Remote Admin

• A reviewer expressed interest–Use remote forms to expose django

admin interface over API

• So I implemented a set of API endpoints– Exposes django admin

app/model/instance data– Exposes admin forms

• And wrote a backbone app implementing django admin

Goals of django Remote Admin

• Allow administration of django projects over API

• No more ties to the same old interface!

• Use awesome Handlebars snippets of your own to spice up the interface

How does it work?

• Cycle through admin site registry– Extract app/model info and expose

over API

• Create ModelForm from the model– Expose over API using django remote

forms

• The backbone app calls the API– Allows browsing apps/ models– Allows creating/editing model

instances

Further Work

• django Remote Forms– Implement file/ image uploading

over API

• django Remote Admin–Load form/widget customizations

from Admin classes– Implement pagination for foreign

key loader

Demo

• Ask me about WiserTogether– github.com/WiserTogether/django-remote-

forms– github.com/tarequeh/django-remote-admin

• Follow my tweets @tarequeh• Shout out to Carlo Costino• ind this presentation–slideshare.net/tarequeh

Q/A

• Ask me about WiserTogether– github.com/WiserTogether/django-remote-

forms– github.com/tarequeh/django-remote-admin

• Follow my tweets @tarequeh• Shout out to Carlo Costino• ind this presentation–slideshare.net/tarequeh