Redis And Python

PyCon India, 2011

(Work in Progress)Sunil Arora

Raising Hands...

How many of you have used Redis before ?

How many of you have got a laptop here ?

Who

Sunil Arora / @_sunil_ Work for ShopSocially (http://shopsocially.com) Interests:

Backend, Frontend, scaling, internet technologies in general, startups ... all kind of shit

Who

Sunil Arora / @_sunil_ Work for ShopSocially (http://shopsocially.com) Interests:

Backend, Frontend, scaling, internet technologies in general, startups...

Today's talk

What is Redis

How it works and what you can do with it

Real life use-cases

What is Redis ?

Its between lot of stuff, so difficult to categorize it precisely

What is Redis ?

I see Redis definitely more as a flexible tool that as a

solution specialized to solve a specific problem: his mixed soul of cache,

store, and messaging server shows this very

well

-Salvatore Sanfilippo

Picture by herzogbrhttp://www.flickr.com/photos/herzogbr/2274372747/sizes/z/in/photostream/

Brief History of Redis

Released in March 2009 by Salvator Sanfilippo (@antirez)

Open source, BSD licensed

VMWare sponsored its development in March, 2010

A few fundamentals

Written in C (25K LOC) Uses memory as main storage Single Threaded Uses disk for persistence Screamingly fast performance 50K read/write operations per seconds 200K read/write ops per second on a regular

EC2 instance

Installation

$ git clone http://github.com/antirez/redis

$ cd redis

$ make

$ ./src/redis-server

..........

..........

redis-py

The most popular python client library Andy McCurdy (sedrik@gmail.com) Github: http://github.com/andymccurdy/redis-py easy_install redis OR pip install redis Optional: easy_install hiredis or pip install

hiredis

Lets get started...

$ cd redis

$ ./src/redis-server

.........

>>> from redis import Redis

>>> redis_client = Redis()

>>> redis_client.keys()

>>> help(redis_client)

Redis Keys

Not binary safe. Should not contain space or newline character A few rules about keys:

Too long keys are not a good idea Too short keys is also not a good idea “object-type:id:field” can be a nice idea, i.e.

“user:1001:name”

Operations on Keys

KEYS EXISTS DEL EXPIRE OBJECT PERSIST

RANDOMKEY RENAME TYPE TTL EXPIREAT MOVE

Lets play with keys

>>>redis_client.keys()

>>>redis_client.exists('key')

>>>redis_client.delete('key')

>>>redis_client.type('key')

>>>......

>>>......

Data Structures

Strings Lists Sets Sorted Sets Hashes

Strings

SET GET MSET MGET SETEX

INCR INCRBY DECR DECRBY

Strings – with redis client

>>> redis_client.set('key', 'value')

>>> redis_client.get('key')

>>> redis_client.delete('key')

Fetch multiple keys at once

mget/mset redis_client.mset({'key1': 'val1', 'key2': 'val2'}) redis_client.mget('key1', 'key2',......)

Expiration

Set a value with expire

>>>redis_client.setex('key', 'value', 2) #key to expire in 2 secs

>>>redis_client.expire('key', 2)

>>>redis_client.get('key')

>>>None

Uses

To store transient states in your web application

Uses

Who is online?

Uses

Redis as LRU cache (http://antirez.com/post/redis-as-LRU-cache.html)

Atomic Increments

>>>help(redis_client.incr)

>>>help(redis_client.decr)

>>>

>>> redis_client.incr('counter', 1)

>>>1

>>> redis_client.incr('counter')

>>>2

>>> redis_client.incr('counter')

>>>3

Uses

High Speed counters (views/clicks/votes/likes..)

Uses

API Rate Limiting

Uses

Generating unique IDs

Lists

Ordered list of binarysafe strings Doubly linked list Memory footprint optimized for smaller list O(1) insertion/deletion at both ends

Lists - operations

LPUSH RPUSH LSET LRANGE LPOP BLPOP BRPOP BRPOPLPUSH

LINSERT RPOP RPOPLPUSH LPUSHX RPUSHX

Uses

Web apps are full of lists :)

Uses

Capped List

Uses

Normal QueueReal time message Queue

Uses

Social Activity Streams

Sets

An unordered collection of distinct byte strings

Nothing different from the data type in python

Sets - Operations

SADD SCARD SDIFF SDIFFSTORE SINTER SINTERSTORE SISMEMBER SMEMBERS

SMOVE SPOP SRANDMEMBER SREM SUNION SUNIONSTORE

Uses

Picking random items from a set using SRANDMEMBER

Uses

Primitive to construct filtering mechanism on items

Which of my friend's are online right now?

Uses

Sorted Sets

ZADD ZCARD ZCOUNT ZINCRBY ZINTERSTORE ZRANGE ZRANGEBYSCORE ZRANK

ZREM ZREMRANGEBYRA

NK ZREMRANGEBYSC

ORE ZREVRANGE ZREVRANGEBYSCO

RE ZSCORE ZUNIONSTORE

SORT

SORT KEY

SORT key BY pattern (e.g. object-type:*:age)

SORT key LIMIT 0 10

SORT key GET user:*:username

SORT key BY pattern STORE dstkey

Uses

Generating sorted transient views

Publish/Subscribe

A simple and efficient implementation of publish/subscribe messaging paradigm

Client can subscribe/psubscribe to receive messages on channels (key patterns)

Publish/Subscribe

PSUBSCRIBE PUBLISH PUNSUBSCRIBE SUBSCRIBE UNSUBSCRIBE

Uses

Many to Many message passing

Uses

Web Chat

Questions