ibm power systems solution for redis

IBM Power Systems solution for Redis

A solution guide to implement Redis on IBM Power Systems running Linux

featuring the new IBM POWER8 technology

Hari Reddy, Maya Pandya, Mark Nellen IBM Systems and Technology Group ISV Enablement

Sangamesh Mallayya, Luke Browning

IBM Systems and Technology Group, Enterprise Systems and Technology Development

Bill Phu

IBM Software Group, Power Systems Performance

July 2014

© Copyright IBM Corporation, 2014

https://twitter.com/IBMSystemsISVs�


Table of contents Abstract........................................................................................................................................1

Introduction .................................................................................................................................1

Prerequisites ...............................................................................................................................1

Advantages of Redis...................................................................................................................2

Data structures in Redis.............................................................................................................2

Redis architecture.......................................................................................................................3 Redis server............................................................................................................................................. 4 Redis replicas .......................................................................................................................................... 4 Redis clients............................................................................................................................................. 5

Implementation of Redis ............................................................................................................5 System configuration ............................................................................................................................... 5

Hardware configuration ..................................................................................................... 5 Software configuration....................................................................................................... 5

Build and install Redis.............................................................................................................................. 5 Redis server............................................................................................................................................. 6 Redis clients............................................................................................................................................. 8

Redis strings...................................................................................................................... 9 Redis hashes..................................................................................................................... 9 Redis lists ........................................................................................................................ 10 Redis sets........................................................................................................................ 11 Redis sorted sets............................................................................................................. 12

Power Systems built with POWER8 technology ....................................................................12 Redis process placement ...................................................................................................................... 14 Hardware prefetch ................................................................................................................................. 15

Summary....................................................................................................................................15

Resources..................................................................................................................................16

About the authors .....................................................................................................................17

Trademarks and special notices..............................................................................................18

1

Abstract This white paper describes Redis (Remote Dictionary Server), which is an open source NoSQL style, memory resident data structure server. The paper also describes the main features of the Redis solution, its architecture, and implementation on IBM Power Systems running Linux servers. The target audience is users and system integrators who implement Redis on IBM Power Systems running Linux servers. No familiarity with Redis or IBM Power Systems running Linux is required; however, some familiarity with the basics of Linux commands is required.

Introduction Relational databases such as IBM® DB2®, MySQL, and Oracle are in widespread use in applications dealing with large amounts of data that fit well into a tabular structure, and support a well-defined query

language to retrieve data. NoSQL type databases do not typically store the data into a well-defined table, but can be in any other format. Redis is one such open source NoSQL database. Redis is notable in that the data resides in memory and this contributes to extremely low transaction processing times because of

its high performance. Redis is typically used as a caching layer in front of other database products. Examples of other NoSQL database management tools include MongoDB, Cassandra, Riak and they each have their own non-standardized application programming interface (API) for accessing data, and

have unique benefits.

IBM’s latest entry (at the time of this publication) into its Power Systems™ portfolio is built for bigger demands of data stored and manipulated in these next-generation databases such as Redis. These are

systems built on first generation of IBM POWER8™ innovative design for very demanding, data-hungry applications.

Built with the first processor designed for such data workloads, IBM Power Systems design combines the

computing power, memory bandwidth, and I/O in ways that are easier to consume and manage, building on strong resiliency, availability and security, demonstrated by:

Computing power with 50% more cores and smart acceleration enabled by the Coherent

Accelerator Processor Interface (CAPI).

Massive memory with over twice the bandwidth of prior generation servers to process data faster and achieve greater speed and efficiency for transactional applications such as Redis.

Systems are easy to deploy and manage with open source technologies such as OpenStack, KVM, simplified virtualization management, and flexible capabilities to drive rapid adoption and dramatically simplify IT consumption.

Better cloud economics for scale out infrastructures, with price-performance advantages and security to confidently move data-centric applications to the cloud.

The subsequent sections provide an introduction to the advantages of Redis, the data structures and

associated commands supported in Redis, the architecture of Redis, the instructions to build, install, and run Redis on IBM Power Systems running Linux.

Prerequisites No prior knowledge of Redis is assumed. Basic familiarity with commands and tools used in Linux® and

Power Systems running Linux will be very helpful.


2

Advantages of Redis Key advantages of Redis are its speed, its support for rich data types, the atomicity of its operations, and its flexibility,

Performance Redis database resides in memory, and time complexity for most Redis commands is O(1). It is possible to reach over 1 million transactions per second for certain commands.

Data Structures

Redis provides a rich set of foundational data types familiar to most programmers and

primitives for manipulating these data types Atomicity

Redis is single threaded and this assures that each Redis command is not interfered with the execution of other Redis commands. In addition to the command-level atomicity, Redis has constructs that allow users to build atomicity into the execution of a grouping of Redis

commands so that entire sequence of commands can be run without interference.

Data structures in Redis The following data types are supported in Redis (refer to Figure 1).

Strings

Hashes Lists Sets

Sorted sets For more details about Redis data structures, refer to http://redis.io/topics/data-types-intro.


3

Figure 1: Examples of Redis data structures

Redis architecture Redis is based on the client/server architecture (refer to Figure 2) and it consists of the following components.

Redis server Redis replica servers (optional) Redis clients


4

Figure 2. Redis client/server architecture

Redis server

Redis server is written in American National Standards Institute for the C programming language (ANSI C). The main components of Redis server are:

In-memory Redis database Redis (AOF), disk-based append-only log Persistent Redis database (disk-based)

Redis works with in-memory database in order to process the queries from the clients. In addition, optionally, it can take snapshot of the in-memory database and can also append to a log file, AOF. These two options ensure the persistency of the database and can be reloaded in case the server process is

restarted. Because all the user queries are processed entirely from the in-memory database, the response times of the queries are in the order of a few microseconds resulting in extremely large rates of throughput.

Redis replicas

Redis replicas served by secondary servers are used to provide additional capacity to handle high customer loads. This is accomplished by sending the entire database initially to the replica server and subsequently sending incremental updates.


5

Redis clients

If both client and server reside on the same system, they can communicate using either TCP/IP or UNIX® domain socket protocols. UNIX domain sockets offer faster communication and hence higher transaction

rates between the client and the Redis server.

Although the Redis server is written in ANSI C language, clients can be written using a variety of language bindings. Redis supplies a client that is written in ANSI C. Clients written in Python, Ruby, Perl, and a

variety of other languages are available in the open source community.

Implementation of Redis In this section, a sequence of steps to build, install, and run Redis on Power Systems running Linux is

provided.

System configuration

The configuration that was used to install and run Redis is described in this section.

Hardware configuration

IBM Power Systems:

Model: IBM Power® S824 server Processor: Two IBM POWER8 4.15 GHz 8 core

Number of cores: 16 Memory: 128 GB

Software configuration

OS: Red Hat Enterprise Linux 6.5 Redis: 2.8 Redis configuration: Both Redis server and Redis client instances ran on the same

server

Build and install Redis

Prebuilt binaries for Redis on Power Systems running Linux are not available. The easiest way to install Redis is to download the source from the official repository website: http://redis.io/download, and then

compile and install the application (refer to Listing 1). The destination directory, specified by PREFIX, can be adjusted to point to your site specific location to contain the binaries. The following prerequisites are required to build Redis:

wget – GNU wget is a software package for retrieving files libevent – The libevent API provides a mechanism to run a callback function The GNU Compiler Collection (GCC) 4.4.7 or later


6

# yum install libevent # yum install wget # cd /data/bench/paper # wget http://download.redis.io/releases/redis-2.8.8.tar.gz # ls redis-2.8.8.tar.gz # tar -xvf redis-2.8.8.tar.gz # ls redis-2.8.8 redis-2.8.8.tar.gz # cd redis-2.8.8 # make install PREFIX=/data/bench/paper # cd /data/bench/paper # ls bin redis-2.8.8 redis-2.8.8.tar.gz # ls bin redis-benchmark redis-cli redis-server

User input is black System response is in orange

Listing 1. Building and installing Redis

Three binaries redis-cli, redis-server, and redis-benchmark are created. These are Redis

client (ANSI C based), Redis server, and Redis benchmark respectively.

Redis server

Redis server communicates with Redis clients through a socket interface. If both Redis client and Redis server reside on the same server, there is a choice of using either UNIX domain or TCP/IP socket interface. UNIX domain sockets are more efficient and are recommended. The operation of the Redis

server is controlled by the several operational parameters that are specified in a configuration file. A default configuration file, redis.conf, is supplied with the installation. Create a working directory and copy the redis.conf file from the installation directory to the working directory, as shown in Listing 2.

# cd /data/bench/paper # mkdir bm # cd bm # cp /data/bench/paper/redis-2.8.8/redis.conf . #

Listing 2. The redis.conf file supplied by the installation package

Some of the parameters that are used in the illustration presented in this paper are:

Socket communication (port for TCP/IP and UNIX domain socket file) The frequency of saving Redis database to disk

In the example that follows, both Redis client and server are started on the same server and the UNIX

domain socket is used for communication between the client and the server. The saving of the Redis database (DB) to disk is tuned off. The changes that are entered in the redis.conf file are shown in Listing 3 . To follow the illustration, edit the redis.conf file using your favorite editor, make the changes


7

presented in Listing 3 and save the file. For more information about all the parameters that can be specified in the configuration file, refer to: http://redis.io/topics/config.

# Specify the path for the unix socket that will be used to listen for # incoming connections. There is no default, so Redis will not listen # on a unix socket when not specified. # unixsocket /data/bench/paper/redis-unix.sock ################################ SNAPSHOTTING ############################### # Save the DB on disk: # # save <seconds> <changes> # # Will save the DB if both the given number of seconds and the given # number of write operations against the DB occurred. # # In the example below the behaviour will be to save: # after 900 sec (15 min) if at least 1 key changed # after 300 sec (5 min) if at least 10 keys changed # after 60 sec if at least 10000 keys changed # # Note: you can disable saving at all commenting all the "save" lines. # # It is also possible to remove all the previously configured save # points by adding a save directive with a single empty string argument # like in the following example: # save "" # save 900 1 # save 300 10 # save 60 10000

Listing 3. Modifications to the redis.conf file

Follow the instructions in Listing 4 to start the Redis server as a background process. A log of a successful start of the server is shown in Listing 4.


8

# cd /data/bench/paper/bm # export PATH=/data/bench/paper/bin:$PATH # redis-server redis-conf > server.log 2>&1& # cat server.log _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 2.8.8 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in stand alone mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379 | `-._ `._ / _.-' | PID: 29851 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' [29851] 02 Apr 15:14:06.636 # Server started, Redis version 2.8.8 [29851] 02 Apr 15:14:06.636 * The server is now ready to accept connections on port 6379 [29851] 02 Apr 15:14:06.636 * The server is now ready to accept connections at /data/bench/paper/bm/redis.sock

Listing 4. Starting the Redis server

Redis clients

Redis installation supplies an ANSI C based Redis client. External clients using a variety of language bindings are available in the open source community (refer to http://redis.io/clients).

.

# cd /data/bench/paper/bm # export PATH=/data/bench/paper/bin:$PATH # redis-cli –s /tmp/redis.sock redis /tmp/redis.sock> #

Listing 5. Starting the Redis client

Now, you can experiment with the commands that work with any of the five Redis data structures. You can try some of the following examples. An interactive online tutorial is available at http://try.redis.io/.


9

Redis strings

A Redis string is the most basic data structure in Redis.

redis /tmp/redis.sock> set cust:tkt1 1005 OK redis /tmp/redis.sock> expire cust:tkt1 10 (integer) 1 redis /tmp/redis.sock> ttl cust:tkt1 (integer) 7 redis /tmp/redis.sock> get cust:tkt1 "1005" redis /tmp/redis.sock> get cust:tkt1 (nil)

Listing 6. Redis strings

Redis hashes

Hashes are used to save multiple attribute-value pairs associated with a key.

# redis-cli -s /tmp/redis.sock redis /tmp/redis.sock> hmset user:1000 name "Tom Jone" email [email protected] password s3cret OK redis /tmp/redis.sock> hgetall user:1000 1) "name" 2) "Tom Jone" 3) "email" 4) "[email protected]" 5) "password" 6) "s3cret" redis /tmp/redis.sock> hset user:1000 songs 1001 (integer) 1 redis /tmp/redis.sock> hincrby user:1000 songs 100 (integer) 1101 redis /tmp/redis.sock> hmget user:1000 name songs 1) "Tom Jone" 2) "1101"

Listing 7. Redis hashes


10

Redis lists

Lists are used to store an array of values for a given key.

redis /tmp/redis.sock> lpush Cities Walla-Walla (integer) 1 redis /tmp/redis.sock> lpush Cities Dallas (integer) 2 redis /tmp/redis.sock> lpush Cities Houston (integer) 3 redis /tmp/redis.sock> lrange Cities 0 2 1) "Houston" 2) "Dallas" 3) "Walla-Walla" redis /tmp/redis.sock> sort Cities alpha 1) "Dallas" 2) "Houston" 3) "Walla-Walla" redis /tmp/redis.sock> rpop Cities "Walla-Walla" redis /tmp/redis.sock> lpop Cities "Houston" redis /tmp/redis.sock> lrange Cities 0 2 1) "Dallas"

Listing 8. Redis lists


11

Redis sets

Sets are used to store unordered and unique values that can be manipulated by set operations.

redis /tmp/redis.sock> sadd Alex:Pets cats rabbits fish (integer) 3 redis /tmp/redis.sock> sadd Becky:Pets cats dogs (integer) 2 redis /tmp/redis.sock> sadd Cory:Pets cats birds fish turtles (integer) 4 redis /tmp/redis.sock> sunion Alex:Pets Becky:Pets Cory:Pets 1) "birds" 2) "dogs" 3) "turtles" 4) "fish" 5) "cats" 6) "rabbits" redis /tmp/redis.sock> sinter Alex:Pets Becky:Pets Cory:Pets 1) "cats" redis /tmp/redis.sock> sismember Alex:Pets rabbits (integer) 1 redis /tmp/redis.sock> smembers Alex:Pets 1) "fish" 2) "cats" 3) "rabbits"

Listing 9. Redis sets


12

Redis sorted sets

Sorted sets contain items each with a score, and the set is sorted based on the score.

# redis-cli -s /tmp/redis.sock redis /tmp/redis.sock> zadd Presidents 1921 "Warren Harding" (integer) 1 redis /tmp/redis.sock> zadd Presidents 1889 "Benjamin Harrison" (integer) 1 redis /tmp/redis.sock> zadd Presidents 1913 "Woodrow Wilson" (integer) 1 redis /tmp/redis.sock> zadd Presidents 1865 "Andrew Johnson" (integer) 1 redis /tmp/redis.sock> zadd Presidents 1945 "Harry Truman" (integer) 1 redis /tmp/redis.sock> zrange Presidents 2 4 1) "Woodrow Wilson" 2) "Warren Harding" 3) "Harry Truman"

Listing 10. Redis sorted sets

Power Systems built with POWER8 technology In this section, after presenting an overview of POWER8 technology, instructions to run Redis benchmark on Power Systems running Linux that is built with POWER8 technology are provided.

POWER8 is a multicore, multichip (node), and multisocket system. The number of chips and sockets available vary with the model purchased. A representative layout of the POWER8 processor is given in Figure 3. With double the memory bandwidth when compared to IBM POWER7+™, POWER8 is ideally

suited to run Redis operating on a database that resides in memory.


13

Figure 3. POWER8 processor

The following commands are used to get and set important system configuration parameters:

numactl

ppc64_cpu

numactl can be used to query core and memory configuration (refer to Listing 11). This information

taken in conjunction with the output from the ppc65_cpu command helps to identify how many sockets

and cores are available in the system and their identities. Depending on the operational requirements, this information can be useful in deciding whether to bind the Redis processes to specific cores or sockets.


14

# numactl –hardware numactl --hardware available: 4 nodes (0-3) node 0 cpus: 0 4 8 12 16 20 node 0 size: 63488 MB node 0 free: 43968 MB node 1 cpus: 24 28 32 36 40 44 node 1 size: 63488 MB node 1 free: 38798 MB node 2 cpus: 48 52 56 60 64 68 node 2 size: 64256 MB node 2 free: 59046 MB node 3 cpus: 72 76 80 84 88 92 node 3 size: 62208 MB node 3 free: 31554 MB

This configuration has:

Two sockets Each Socket has two chips (nodes) Each chip has

Six cores and 64GB memory

Listing 11. numactl output

Some of the system features that might have some effect on the operation of Redis are:

Redis process placement Hardware prefetch

Redis process placement

Each processor chip in a POWER8 processor-based server is independent with its own caches and

memory controllers. A process currently running within a chip trying to access memory on other chips incurs additional delay affecting application performance. Ideally, it would be good to bind the redis-server process to a core in a chip so that the data it needs can be stored or retrieved from memory

attached to the same chip. However, in case of the redis-server process, there are some optional I/O threads and these threads will be pinned to the same core as the main redis-server process thread.

This might result in poor performance. A better policy is to bind the Redis process to the chip and allow the OS to spread all the redis-server threads evenly across the cores within the chip. If the size of the

Redis database in memory grows beyond the size of the local memory, then remote memory accesses are unavoidable; however, pinning the process to the chip can still result in better performance.

The numactl command can be used to pin the redis-server process to a chip. The usage of this command is illustrated by running redis-server and the Redis benchmark suite called, redis-benchmark, supplied by the Redis package (refer to Listing 12). The illustration uses a subset of large

number of Redis operations that are supported by redis-benchmark. The other options used in this

example are pipelining and a request size of 512 bytes. Pipelining is used where there are several queries to be processed and can be submitted as a batch. For more details about the Redis benchmark, refer to

http://redis.io/topics/benchmarks.


15

# cd /data/bench/paper/bm # export PATH=/data/bench/paper/bin:$PATH # numactl --cpunodebind=1 ./redis-server redis.conf > server.log 2>&1&# numactl --cpunodebind=1 ./redis-benchmark -s /data/bench/hari/redis.sock -d 512 -r 1000000 -n 2000000 -t get,set, mset -q -P 100 SET: 382409.19 requests per second GET: 474608.44 requests per second MSET (10 keys): 67136.62 requests per second #

Note: cpunode is the term: node used in Figure 1.

Listing 12. Binding of processes to POWER8 processors

Hardware prefetch

When the data access patterns are predictable, such information can be used by the processor to prefetch

the data from memory in order to hide the latency delay in getting the data to faster processors. In the case of Redis, the memory accesses are very random and therefore, might not result in any improvement in performance and might even hinder performance. It is recommended that the hardware prefetch is

turned off as indicated in Listing 13.

# ppc64_cpu --dscr dscr is 1 # ppc64_cpu –dscr=1

dscr=0 means HW prefetch is ON ddsc=1 means HW prefetch is OFF

Listing 13.Hardware prefetch flag

Summary Popular relational databases that are available in the industry today use predefined schema to manipulate data. Redis belongs to a new class of databases where a predefined schema does not exist. Thus, Redis is very flexible and adaptive to rapidly changing requirements. In addition, Redis database is memory

resident and hence is very fast, taking just a few microseconds on the average to respond to a query. This makes it ideal as a caching front-end for more robust and mature back-end database applications. In this solution guide, the concepts behind Redis and how to install Redis and run some sample Redis

commands on IBM Power Systems running Linux were introduced. Then, an overview of IBM POWER8 technology was given. POWER8 is IBM’s latest offering (at the time of this publication) to tackle the ever-increasing, data-demanding applications. Finally, a few recommendations to run Redis applications

efficiently on IBM Power Systems running Linux built with POWER8 technology were given.


16

Resources The following websites provide useful references to supplement the information contained in this paper:

IBM Systems on PartnerWorld

ibm.com/partnerworld/systems

IBM Power Systems

ibm.com/systems/in/power/?lnk=mhpr IBM Power Systems Hardware Documentation

http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp

IBM Power Systems running Linux – resources

ibm.com/systems/power/software/linux/resources.html

Redis official website

http://redis.io

Interactive Redis tutorial

http://try.redis.io/

Redis clients

http://redis.io/clients

IBM Power Systems running Linux Information Center

http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/index.jsp

IBM Publications Center

www.elink.ibmlink.ibm.com/public/applications/publications/cgibin/pbi.cgi?CTY=US


17

About the authors Hari Reddy is a consultant in IBM Systems and Technology Group ISV Enablement organization. You can reach Hari at [email protected].

Sangamesh Mallayya is a software Engineer in IBM Systems and Technology Group, Enterprise Systems and Technology Development organization. You can reach Sangamesh at [email protected].

Luke Browning is a Senior Technical Staff member in IBM Systems and Technology Group, Enterprise Systems and Technology Development organization. You can reach Luke at [email protected].

Maya Pandya is a Technology Manager in IBM Systems and Technology Group, ISV Enablement

organization. You can reach Maya at [email protected].

Bill Phu is a Software Engineer in IBM Software Group, Power Systems Performance organization. You can reach Bill at [email protected].

Mark Nellen is a Program Manager in IBM Systems and Technology Group, ISV Enablement organization. You can reach mark at [email protected].


https://twitter.com/IBMSystemsISVs�

mailto:[email protected]




18

Trademarks and special notices © Copyright IBM Corporation 2014.

References in this document to IBM products or services do not imply that IBM intends to make them

available in every country.

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked

terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A

current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

Information is provided "AS IS" without warranty of any kind.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly

available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the

supplier of those products.

All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the

full text of the specific Statement of Direction.

Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to

any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the

storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here.



19

Photographs shown are of engineering prototypes. Changes may be incorporated in production models.

Any references in this information to non-IBM websites are provided for convenience only and do not in

any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk.