Hardware Acceleration of Key-Value StoresSagar Karandikar, Howard Mao, Albert Ou, Yunsup Lee Advisor: Krste Asanovic
Motivation
I In datacenter applications, path through CPU/kernel/application accounts for
86% of total request latency
I Goal: Serve popular requests without CPU interruption
I Solution: Hardware key-value store attached to the network interface controller
I Many workloads have an access pattern suitable for a small dedicated cache
I Per a Facebook study, 10% of keys represent 90% of requestsI Most values are relatively small in size (1 kB)
Related Work
I A 2013 paper by Lim et al. proposed a system dubbed “Thin Servers with
Smart Pipes”, which served memcached GET requests from FPGA hardware.
I However, the FPGA hardware handled GET requests by accessing DRAM, not
a local SRAM cache.
Infrastructure
SFP Cage(1 GbE SFP,not pictured)
DDR3 SDRAM
XilinxXC7Z045FPGA: Rocket Core Accelerator Traffic Manager DMA Engine
Xilinx ZC706 Evaluation Platform
I ZYNQ-7000 SoC
I Brocade 1GbE Copper SFP Transceiver
I Xilinx Tri-Mode Ethernet MACI Xilinx 1000Base-X PCS/PMA
I 64-bit RISC-V Rocket Core (50 MHz)
I Single-issue, in-order, 6-stage pipelineI ASIC version most nearly comparable
with ARM Cortex-A5
Int.RF
InstructionDecode
ITLB
I$ Access
PCGeneration
Int.EX
DTLB
D$ Access Commit
FP.RF FP.EX1 FP.EX2 FP.EX3
I No pre-existing I/O peripherals for the Rocket core
I Built first RISC-V hardware device: register-mapped NIC
I Programmed I/O with custom Linux kernel driverI First telnet/ssh session into a physical RISC-V machine
I Evolved to DMA-based NIC for performance
Software
I Manages what keys and values are stored on the accelerator
I Controls the accelerator through the RoCC co-processor interface, which
provides custom instructions for setting keys and values
I Responsible for implementing cache replacement policies
I Identification of the most popular keys as candidates for offloadingI Invalidation of stale entries
System Architecture
Baseline
Rocketscalar core
16 KiB I$ 32 KiB D$
2:1 arbiter
HTIF
DMA
L2 Coherence Agent
TileLink
512 MiB SDRAM
Tile
Uncore
AXI
NIC
Enhanced
Rocketscalar core
Key-Value Storeaccelerator
RoCC
16 KiB I$ 32 KiB D$
2:1 arbiter
2:1 arbiter
HTIF
DMA
L2 Coherence Agent
TileLink
512 MiB SDRAM
Tile
Uncore
AXI
TrafficManager NIC
Accelerator
Accelerator
Hasher
Hasher
WriterCurrent
Key CacheAll Keys
KeyCompare
ValueMemory
Key In
Value Out
MemoryHandler
Controller
RoCC Memory RoCC Command
Mux
Traffic Manager
MainWriter
MainBuffer
DeferBuffer
Splitter
Arbiter
ArbiterResponder
Controller
MAC Rx
DMA Rx
DMA Tx
Mac Tx
Key Out Result In
I The accelerator accepts a key and computes primary and secondary hash
values, which it uses to retrieve the value from its local block RAM.
I The traffic manager, interposed between the NIC and the DMA engine,
implements the specialized Memcached logic.
I For intercepted Memcached GET requests, the traffic manager queries the
accelerator and constructs the response packet if the key is found.
I Unhandled frames are forwarded to the DMA engine for processing by the
operating system.
DMA Engine
I Performs uncached memory accesses via the TileLink protocol
I Transfers a 512 bit cache block per request
I Front-end/back-end decoupling allows load prefetching to hide memory latency
I Buffer descriptor rings exposed as queues through processor control registers
I Provides 250 times the query throughput as compared to programmed I/O
TxFront-end
acquire
grant
finish
TxBack-end
data
last?
addrcount
descriptor
rptr
wptr
CSRs
IRQ
RxFront-end
data
last?
RxBack-end
acquire
grant
finish
cptr cptr
addrcountCSRs
descriptor
wptr rptr
IRQ
Floorplan
Utilization
Resource w/o A+TM w/A+TM
Slice LUTs 17.09% 21.79%
Slice Registers 6.18% 8.01%
Memory 21.65% 63.85%
Latency Evaluation
0 10 20 30 40 50 60 70 80 90Percentiles
1600
1700
1800
1900
2000
2100
2200
2300
2400
Late
ncy
(us)
Uniform Distribution Latencies
Requests served by memcached on RocketRequests served by accelerator
0 10 20 30 40 50 60 70 80 90Percentiles
1500
1600
1700
1800
1900
2000
2100
2200
2300
2400
Late
ncy
(us)
Normal Distribution Latencies
Requests served by memcached on RocketRequests served by accelerator
0 10 20 30 40 50 60 70 80 90Percentiles
0
500
1000
1500
2000
2500
Late
ncy
(us)
Pareto Distribution Latencies
Requests served by memcached on RocketRequests served by accelerator
0 10 20 30 40 50 60 70 80 90Percentiles
0
500
1000
1500
2000
2500
Late
ncy
(us)
Facebook ETC Distribution Latencies
Requests served by memcached on RocketRequests served by accelerator
Conclusion
I By moving some of the keys to the accelerator and serving directly to the NIC
from hardware, we gained an order of magnitude speed-up over memcached
software running on the Rocket core (1700µs vs 150µs response latency).
I The accelerator serves 40% of keys at this reduced latency for Facebook ETC.
I However, we still have a long way to go before reaching production quality.
Future Work
I Place the DMA engine, traffic manager, and accelerator in faster clock domains
with asynchronous FIFOs, rather than be constrained by the core frequency
I Widen I/O interfaces for greater throughput
I Investigate replacing the fixed-function traffic manager with a programmable
co-processor (reminiscent of mainframe channel I/O)
I Conduct torture testing for reliability
I Explore opportunities for measuring and improving energy efficiency
University of California, Berkeley ASPIRE Winter 2015 Retreat Department of Electrical Engineering & Computer Sciences
