redis - your advanced in-memory key-value store

Jordi Boggiano@seldaekhttp://nelm.io/

Redis - Your AdvancedIn-Memory Key-Value Store

About Me

Belgian living in Zürich, Switzerland

Weby stuff for 10 years http://seld.be

Symfony2, Composer and other OSS contributions http://github.com/Seldaek

Working at Nelmio http://nelm.io Symfony2 & frontend performance consulting

Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Agenda

IntroFeaturesUse CasesUsing it with PHP

You put data in it

Key-Value Store

Like NoSQL?

In-Memory Key-Value Store

Like Memcached?

In-Memory Key-Value Store

Memory is fast, but ephemeral

Persist to Disk

Fast and lasting

Created by @antirez

Sponsored by VMWare

So how does it work?

Simple Text Protocol

Human Readable!

Example - Client Library1 SET key value

Example - Client Library1 SET key value2 > OK

Example - Client Library1 SET key value2 > OK3 GET key

Example - Client Library1 SET key value2 > OK3 GET key4 > "value"

Example - Low Level1 *32 $33 SET4 $35 key6 $57 value

1 *22 $33 GET4 $35 key

1 $52 value

Example - Low Level1 *3\r\n2 $3\r\n3 SET\r\n4 $3\r\n5 key\r\n6 $5\r\n7 value\r\n

1 +OK\r\n

1 *2\r\n2 $3\r\n3 GET\r\n4 $3\r\n5 key\r\n

1 $5\r\n2 value\r\n

Data Types

Strings1 SET name Bob2 SET age 203 MGET name age4 > Bob5 > 20

1 GETSET name Alice2 > Bob

Strings1 SETEX age 3 202 GET age3 > 204 // .. 3 seconds later ..5 GET age6 > null

Integers1 INCR count2 > 13 INCR count4 > 25 INCRBY count 36 > 5

Hashes01 HMSET user name Alice email

alice@example.org02 HGET user email03 > alice@example.org04 HKEYS user05 > name06 > email07 HGETALL user08 > name09 > Alice10 > email11 > alice@example.org

Lists01 RPUSH admins Alice02 > 103 RPUSH admins Bob04 > 205 LINDEX admins 006 > Alice07 LLEN admins08 > 209 RPOP admins10 > Bob

Sets01 SADD page:3:visitors 13402 > 103 SADD page:3:visitors 25304 > 105 SADD page:3:visitors 25306 > 007 SCARD page:3:visitors08 > 209 SMEMBERS page:3:visitors10 > 13411 > 25312 SISMEMBER page:3:visitors 34913 > 0

Sets1 SADD page:6:visitors 2532 SADD page:6:visitors 9233 SADD page:6:visitors 134 SINTER page:3:visitors page:6:visitors5 > 253

Sorted Sets1 ZADD highscores 2930 Alice2 ZADD highscores 1546 Bob3 ZREVRANGE highscores 0 10 WITHSCORES4 > Alice5 > 29306 > Bob7 > 1546

Sort of Lists? Listed Sets?Lists

Sorted Sets

1 array('foo', 'bar')

1 shuffle(array('foo', 'bar'))

1 ksort(array(3 => 'foo', 1 => 'bar'))

The basic datatypes of everylanguage exist in Redis

Process data in Redisinstead of PHP

A Few Features

Atomic Operations1 SETNX name Alice2 GET name3 > Alice4 SETNX name Bob5 GET name6 > Alice

1 INCR foo2 GET foo3 > 14 INCRBY foo 35 GET foo6 > 4

EXPIRE / EXPIREAT / PERSIST

Pipelining

SORT - SQL in your NoSQL1 SORT key2 SORT key LIMIT 0 10 DESC3 SORT key ALPHA4 SORT page:6:visitors BY user_*->rank GET

user_* DESC

Append-Only File & Snapshots

Pub/Sub1 SUBSCRIBE foo12 PSUBSCRIBE foo*3 PUBLISH foo0 message

Master/Slave Replication

Transactions: MULTI / EXEC /DISCARD

Optimistic Locking with WATCH

Upcoming Features

Lua Scripting: EVAL / EVALSHA

This is great.

EVAL1 EVAL <body> <num_keys_in_args> [<arg1> <arg2> ... <arg_N>]

1 GET A2 EVAL "return redis.call('get', KEYS[1])" 1 A

Example: Atomic Conditional Decrement, Client-Side

This may return -ERR, or will block.

1 WATCH foo2 $val = GET foo3 $newVal = max(0, $val - 1); // decrement if foo > 0 client-side4 MULTI5 SET foo $newVal6 EXEC

Example: Atomic Conditional Decrement, Server-Side

This is instant and can not fail.

01 EVAL "local value = tonumber(redis.call('get', KEYS[1]))02 if value == nil03 then04 return {err="Value at key is not integer"}05 end06 if value > tonumber(ARGV[1])07 then08 value = value - 109 redis.call('set', KEYS[1], value)10 end11 return value" 1 foo 0

Redis Cluster

Almost as delayed asDuke Nukem Forever

Redis is FAST.CPU is unlikely to be the

bottleneck.

What do you do whenyou run out of RAM?

Wait for Redis Cluster?

If you can. Cluster looks great.

Use sharding, most clientlibraries can do it.

Use Edis, a protocol compatiblestorage-oriented server.http://inaka.github.com/edis/index.html

Use it without persistence, store to disk/DB

and load hot data in memory.A good example in these slides from Wooga

Ok, great. But..

What is it good for?

WEB SCALE

THE CLOUD

ELASTIC

HORIZONTAL SCALING

VERTICAL TOO

WEB SCALE

THE CLOUD

ELASTIC

HORIZONTAL SCALING

VERTICAL TOO

BULLSHIT

Get real"If you do actually have to scale,then your database isn't going to

magically do it for you."

Really.

What is it good for?

Storing Sessions

Caching Data

Other typical Memcached uses

Caching: Newspaper SiteExpensive readsFast-changing pagesDynamic tracking of what's read for users

Caching: Newspaper SiteCache generic results:

Store user read-state in sets

Combine and render for each user

1 SETEX <pageid>:content 600 <data>

1 SADD <pageid>:views <userid>

1 GET <pageid>:content2 SISMEMBER <pageid>:views <userid>

Logging, games and otherwrite-heavy usages

1 LPUSH logs "Log message"2 // keep the last 1000 entries3 LTRIM logs 0 999

Highscore tables

Note: players will only be listed once since it is a set.

1 ZADD scores 4290 <playerid>2 ZADD scores 390 <playerid2>3 // ...4 ZREVRANK scores <playerid> // 05 ZREVRANGE scores 0 10 WITHSCORES

Who's online now?1 ZADD visits <unix> <userid>2 ZADD visits 1327681399 523 ZADD visits 1327683245 184 ZRANGEBYSCORE visits <unix>-3600 <unix>5 ZREMRANGEBYSCORE visits 0 <unix>-3600

Job QueuesWorkers:

Application:

e.g. the Resque lib from GitHub

1 BRPOP queue

1 LPUSH queue "job description"

Behavior Tracking1 SETBIT click:<item>:<date> <userid> 12 GETBIT click:3:2012-01-28 55239

Last but not least, it puts thefun back in Datafunbases

Usage with PHP

Extension: phpredis

Comes with a session handler, sharding, up to date.

http://github.com/nicolasff/phpredis

1 $redis = new Redis();2 $redis->connect('127.0.0.1', 6379);3 $redis->watch('x');4 $val = $redis->get('x');5 $newVal = max(0, $val - 1);6 $result = $redis->multi()7 ->set('x', $newVal)8 ->exec();9 echo $result !== false ? 'Success' : 'Race lost, try again';

Extension: phpiredis

Basic hiredis bindings, protocol parsing, low level.

http://github.com/seppo0010/phpiredis

1 $redis = phpiredis_connect('127.0.0.1', 6379);2 phpiredis_command($redis, 'WATCH x');3 $val = phpiredis_command($redis, 'GET x');4 $newVal = max(0, $val - 1);5 phpiredis_command($redis, 'MULTI');6 phpiredis_command($redis, 'SET x '.$newVal);7 $result = phpiredis_command($redis, 'EXEC');8 echo $result !== false ? 'Success' : 'Race lost, try again';

Plain PHP: Predis

Very complete, sharding, master/slave auto-select,can use phpiredis for parsing.

http://github.com/nrk/predis

http://github.com/snc/SncRedisBundle (Symfony2integration)

01 $redis = new Predis\Client('tcp://10.0.0.1:6379');02 03 $options = array(04 'cas' => true, // enable Check-and-Set05 'watch' => 'x',06 'retry' => 10, // automatic retries07 );08 09 $result = $redis->multiExec($options, function($transaction) {10 $val = $transaction->get('x');11 $newVal = max(0, $val - 1);12 $transaction->multi();13 $transaction->set('x', $newVal);14 });15 16 echo $result !== false ? 'Success' : 'Race lost 10 times, giving up';

It's fast. It's fun. Try it.

try.redis-db.com

Thank you.

Slides: http://slides.seld.be

Find Out MoreRedis.ioTry RedisThe Little Redis BookRedis Cluster Specification

Questions?

jordi@nelm.io

@seldaek

Feedback:

http://joind.in/4764

redis - your advanced in-memory key-value store

Technology

deploying memcached and redis on oci - …€¦ · 0 |...

open source in-memory data structure...

memcached and redis - university of california, san...

redis document - wordpress.comredis document | ducdt - pmt 4...

deploying memcached and redis on oci | oracle white paper:...

repocket documentation · repocket is an active record that...

redis post-exploitation - 2018.zeronights.ru · intro redis...

jason brelloch and william...

redis 用 amazon elasticache - redis 用 elasticache...

redis : nosql key-data store

redis based in-memory data grid...

deploying a highly available distributed caching layer on...

welcome to nosql · redis supports in-memory operations and...

redis - a key value store

intel® memory drive technology redis performance ... ·...

intel® memory drive technology benefits · redis* +...

capi-flash wp 09apr2015 final · 2017-08-10 · 3.2 redis...

memory short-term memory & working memory. the multi-store...

cs 2 main memory and backing store. 2 0100 1000 memory...

cloud onload redis cookbook - xilinx.com2.1 redis overview...