a summer in the wild

8/4/2019 A Summer in the Wild

1/31

A Summer in the Wild

Alex Gaynor

DjangoCon.us 2011

6 September 2011

Alex Gaynor


2/31

A Summer in the Wild

Alex Gaynor


3/31

Me

Django/CPython/PyPy core developer

Mostly been building websites for the past 4years or so.

I was getting kind of sick of websites.

Alex Gaynor


4/31

PyPy

Implementation of Python, written in [a restricted

subset of] PythonAdvanced tracing just-in-time compiler.

Fast: http://speed.pypy.org/

Alex Gaynor
http://speed.pypy.org/http://speed.pypy.org/


5/31

Quora

Question and answer, knowledge database site

All Python

Wanted it to go faster.

Alex Gaynor


6/31

This Summer

Internship at Quora, to port the site to run on

PyPy.

Ultimately successful.

Alex Gaynor


7/31

This Talk

Everything I learned in 4 months... in 30

minutes.

Stuff I missed terribly when I wasnt using

Django

Stuff you need to be doing, on all your projects.

Alex Gaynor


8/31

First Mover

Quora was (to my knowledge) the largest

website to migrate to PyPy

Not all packages worked out of the box

Alex Gaynor


9/31

MySQLdb

Wrote a fresh implementation of MySQLdb using

ctypesKnew it was going to be needed going in.

Still a pain, next guy doesnt have to do that.

Alex Gaynor


10/31

MYSQL RAGE

32-bit vs. 64-bit

Client has been blocked from this server

Alex Gaynor

P


11/31

Paster

Under CPython used pasters HTTP server

It had obscure bugs under PyPy with chunkedencoding

Switched to gunicorn: Just Worked (tm)

Alex Gaynor

C h


12/31

Cython

Division between .py and .pxd files

Better support hopefully coming soon.

Alex Gaynor

O b i fi t


13/31

On being first

Any major infrastructure change carries inherent

risk.Know what that risk is going in.

Tests are an absolute must (more on this later).

Alex Gaynor

P fili


14/31

Profiling

It sucks.

siege for very basic comparison.Editing gunicorn itself for actual profiling with

cProfile (wtf?)

Alex Gaynor

Thi Dj i D i V V Ri ht


15/31

Things Django is Doing Very Very Right

URLs

The ORMEscaping

Conventions

Alex Gaynor

URLs


16/31

URLs

With 100% ease, I can start from the root URL

conf, and track down what view serves a given

request.

DO NOT VIOLATE THIS WITH MIDDLEWARE

HACKS.

Alex Gaynor

The ORM


17/31

The ORM

You forget how awesome it is, until you dont

have it.

80-20 solution, more like 95-5 solution.

Alex Gaynor

Escaping


18/31

Escaping

Security as a top priority.

You have to actively work at it to give yourself an

XSS hole.

Alex Gaynor

Conventions


19/31

Conventions

You always know where your models, views,urls, etc. are within an app.

Doesnt require Django itself to do anything.

We need to keep going with this, organize

projects in the same way.

Conventions for using WSGI middleware with

Django (coming soon)

Alex Gaynor

Seperation of concerns


20/31

Seperation of concerns

Practically un-workaround-able

Fundamental syntactic difference means younever forget.

Alex Gaynor

Tools everyone must use


21/31

Tools everyone must use

virtualenv, pip

No discussions.

Alex Gaynor

Project startup


22/31

Project startup

cvs checkoutmkvirtualenv

pip install -r

Edit some settings.py

We need better conventions for settings.

python manage.py syncdb

python manage.py runserverAnd if those last two arent enough, your

README damn well better say so.

Alex Gaynor

We do not


23/31

We do not

Check python binaries into our VCS

Check compiled modules into our VCS

Vendor our dependencies.

Run your own PyPI server if you feel the need,

dont check crap into the repository.

Have a process that isnt a goodprocess. Just

a process isnt enough.

Alex Gaynor

Automate Everything


24/31

Automate Everything

Content on Quora automatically updates if its

changed by anyone on the site.

General framework for doing this, not done

manually. (Cool idea, someone should write it for

Django)

To a first approximation anything done manually

will break.

Alex Gaynor

Automate EVERYTHING


25/31

Automate EVERYTHING

DRY applies to the human side of things as well.

Write tests so you dont need to test everything

by hand when you make a change

Have a buildserver to run tests

One-click deploys

If you run the same 3 commands for occasional

maintenance, PUT THEM IN A SHELL SCRIPT

Alex Gaynor

Caching


26/31

Caching

At a minimum organized.

Better is automated (easier for simpleobject-caches).

No caching in your views.

Alex Gaynor

Tests


27/31

Tests

How do you know what youre writing works?

How do you know you didnt just break it?How do you know your big refactor still does the

right thing?

Alex Gaynor

Build server


28/31

Build server

Because I dont trust myself, much less

everyone else, to run all the tests every time

Alex Gaynor

One click deploy


29/31

One click deploy

{fabric, gondor, ep.io, chef, puppet} deploy

git push deploy

WHATEVER

Deployment is hard enough, without having to

re-invent it each time.

Alex Gaynor

In Summary


30/31

In Summary

Infrastructure changes can be dangerous,

mitigate risk by doing research up front

Django is lovely, take advantage.

Automate everything, or you and your

colleagues will slowly go mad.

Alex Gaynor

Questions?


31/31

Q

@alex_gaynor

http://alexgaynor.net/

Alex Gaynor
http://alexgaynor.net/http://alexgaynor.net/

a summer in the wild

Documents