eliminating the hardware/software divide satnam singh, microsoft research cambridge, uk
TRANSCRIPT
Eliminating the Hardware/Software Divide
Satnam Singh, Microsoft Research Cambridge, UK
!IRQ, NMI
t
locksmonitorscondition variablesspin lockspriority inversion
multipleindependentmulti-ported
memories
fine-grainparallelism
andpipelining
hard and softembeddedprocessors
LUTs are just higher order functions
i o
lut1
oi1
i0
lut2 lut3
i0
i1i2
o
lut4
i0
i1i2i3
o
inv = lut1 not
and2 = lut2 (&&)
mux = lut3 (l s d0 d1 . if s then d1 else d0)
XC6VLX760 758,784 logic cells, 864 DSP blocks, 1,440 dual ported 18Kb RAMs
32-bitintegerAdder(32/474,240)>700MHz
332x1440
14820 sim-adds1,037,400,000,000additions/second
XD2000i FPGA in-socketaccelerator for Intel FSB
XD2000F FPGA in-socketaccelerator for AMD socket F
XD1000 FPGA co-processormodule for socket 940
Case Study – Spam Filtering (Alessandro Forin, MSR Redmond)
• Benchmark– ~50,000 regular expressions from
Forefront Team (snapshot fromtheir Exchange server in Aug ‘09)
• Performance– Up to 6000x faster than standard Intel
processors– Capable of processing at line rate of gigabit
Ethernet
• Power Requirement– 7 – 10 watts rather than 200++ watts
27
Software Version FPGA Version
~6000 Messages/Sec~1 Message/Sec
<10 Watts200++ Watts
“E-mail Server”
Reg Ex Processing
“E-mail Server”
Reg Ex Processing
René Müller (ETH)
FPGAs + SQL [VLDB]
CPU FPGA
Speed Grade -1 -2 -3
With Layout 270MHz 320MHz 362MHz
Without Layout 210MHz 260MHz 280MHz
541 seconds
1896 seconds
opportunity
scientific computingdata miningsearchimage processingfinancial analytics
challenge
The Accidental Semi-colon
;
public static int[] SequentialFIRFunction(int[] weights, int[] input) { int[] window = new int[size]; int[] result = new int[input.Length]; // Clear to window of x values to all zero. for (int w = 0; w < size; w++) window[w] = 0; // For each sample... for (int i = 0; i < input.Length; i++) { // Shift in the new x value for (int j = size - 1; j > 0; j--) window[j] = window[j - 1]; window[0] = input[i]; // Compute the result value int sum = 0; for (int z = 0; z < size; z++) sum += weights[z] * window[z]; result[i] = sum; } return result; }
PLDI 1998
PLDI 2003
12345
PLDI 2010
1 2 3 4 50
10
20
30
40
50
60
70
80
Series1Series2
POPL 1998
POPL 2002
POPL 2010
ray of light
Signal
Liquid Metal
PRET-C
Bluespec
Feldspar
AcceleratorRapidMind /Ct
Streams-C
EsterelSHIM
universallanguage?
embeddedhigh levelsoftware
FPGA
GPU
DSP
machinelearning
grand unificationtheory polygots
Gannet
DSLs
Our High Level Synthesis Projects
Kiwi: concurrent C# programs for control-oriented
applications[David Greaves,
Univ. Cambridge]
shape analysis: synthesis of
dynamic data structures (C)
[MPI and CMU]
Accelerator/FPGA:synthesis of data
parallel programs in C++/C#/F#
[MSR Redmond]
HLINQeDSLs
[Gavin Bierman]
+ compilation of self-recursive Haskell functions to FPGA circuits!
Redmond Accelerator TeamBarry Bond
Kerry HammilLubomir Litchev
<anonymous other person>
Effort vs. Reward
loweffort
lowreward
higheffort
highreward
mediumeffort
mediumreward
CUDAOpenCLHLSLDirectComputeAccelerator
Accelerator
Application.EXE Accelerator.DLL
Windows on Intel/AMDprocessor
DX9
GPU(ATI, Nvidia, …)
open Systemopen Microsoft.ParallelArrayslet main(args) = let x = new FloatParallelArray (Array.map float32 [|1; 2; 3; 4; 5 |]) let y = new FloatParallelArray (Array.map float32 [|6; 7; 8; 9; 10 |]) let z = x + y use dx9Target = new DX9Target() let zv = dx9Target.ToArray1D(z) printf "%A\n" zv 0
open Systemopen Microsoft.ParallelArrayslet main(args) = let x = new FloatParallelArray (Array.map float32 [|1; 2; 3; 4; 5 |]) let y = new FloatParallelArray (Array.map float32 [|6; 7; 8; 9; 10 |]) let z = x + y use sse3Target = new X64MulticoreTarget() let zv = sse3Target.ToArray1D(z) printf "%A\n" zv 0
open Systemopen Microsoft.ParallelArrayslet main(args) = let x = new FloatParallelArray (Array.map float32 [|1; 2; 3; 4; 5 |]) let y = new FloatParallelArray (Array.map float32 [|6; 7; 8; 9; 10 |]) let z = x + y use fpgaTarget = new FPGAMulticoreTarget() fpgaTarget.ToArray1D(z) 0
[1; 2; 3; 4; 5]
FloatParallelArray
CPU Address Space
F# Array
GPU Address Space
EncapsulatedData-parallelarray
x
[6; 7; 8; 9; 10]
FloatParallelArray
y
100010101101011010
x+yGPU code
GPU memory
GPU code
y
[7; 9; 11; 13; 15]F# Array
using System;using Microsoft.ParallelArrays;
namespace AddArraysPointwise{ class AddArraysPointwiseDX9 { static void Main(string[] args) { var x = new FloatParallelArray (new[] {1.0F, 2, 3, 4, 5}); var y = new FloatParallelArray (new[] {6.0F, 7, 8, 9, 10}); var dx9Target = new DX9Target(); var z = x + y; foreach (var i in dx9Target.ToArray1D (z)) Console.Write(i + " "); Console.WriteLine(); } }}
module Mainwhere
import Accelerator
x = fpa [1.0, 2.0, 3.0, 4.0, 5.0]y = fpa [6.0, 7.0, 8.0, 9.0, 10.0]
z = x + y
main = do dx9Target <- c_DX9Target_Create r <- acceleratorCompute dx9Target z putStrLn (show r)
rX *
pa
Shift (0,0) k[0]
+
+
*
Shift (0,1) k[1]
+
…
let rec convolve (shifts : int -> int []) (kernel : float32 []) i (a : FloatParallelArray) = let e = kernel.[i] * ParallelArrays.Shift(a, shifts i) if i = 0 then e else e + convolve shifts kernel (i-1) a
public static int[] SequentialFIRFunction(int[] weights, int[] input) { int[] window = new int[size]; int[] result = new int[input.Length]; // Clear to window of x values to all zero. for (int w = 0; w < size; w++) window[w] = 0; // For each sample... for (int i = 0; i < input.Length; i++) { // Shift in the new x value for (int j = size - 1; j > 0; j--) window[j] = window[j - 1]; window[0] = input[i]; // Compute the result value int sum = 0; for (int z = 0; z < size; z++) sum += weights[z] * window[z]; result[i] = sum; } return result; }
shift (x, 0) = [7, 2, 5, 9, 3, 8, 6, 4] = xshift (x, -1) = [7, 7, 2, 5, 9, 3, 8, 6]shift (x, -2) = [7, 7, 7, 2, 5, 9, 3, 8]
y = [y[0], y[1], y[2], y[3], y[4], y[5], y[6], y[7]]= a[0] * [x[0], x[1], x[2], x[3], x[4], x[5], x[6], x[7]] + a[1] * [x[-1], x[0], x[1], x[2], x[3], x[4], x[5], x[6]] +
a[2] * [x[-2], x[-1], x[0], x[1], x[2], x[3], x[4], x[5]] + a[3] * [x[-3], x[-2], x[-1], x[0], x[1], x[2], x[3], x[4]] + a[4] * [x[-4], x[-3], x[-2], x[-1], x[0], x[1], x[2], x[3]]
y = a[0] * shift (x, 0) + a[1] * shift (x, -1) + a[2] * shift (x, -2) + a[3] * shift (x, -3) + a[4] * shift (x, -4)
using Microsoft.ParallelArrays;using A = Microsoft.ParallelArrays.ParallelArrays;namespace AcceleratorSamples{ public class Convolver { public static float[] Convolver1D(Target computeTarget, float[] a, float[] x) { var xpar = new FloatParallelArray(x); var n = x.Length; var ypar = new FloatParallelArray(0.0f, new [] { n }); for (int i = 0; i < a.Length; i++) ypar += a[i] * A.Shift(xpar, -i); float[] result = computeTarget.ToArray1D(ypar); return result; } }}
for (int i = 0; i < a.Length; i++) ypar += a[i] * A.Shift(xpar, -i);
using System;using Microsoft.ParallelArrays;namespace AcceleratorSamples{ public class Convolver2D { static FloatParallelArray convolve(Func<int, int[]> shifts, float[] kernel, int i, FloatParallelArray a) { FloatParallelArray e = kernel[i] * ParallelArrays.Shift(a, shifts(i)); if (i == 0) return e; else return e + convolve(shifts, kernel, i - 1, a); } static FloatParallelArray convolveXY(float[] kernel, FloatParallelArray input) { FloatParallelArray convolveX = convolve(i => new [] { -i, 0 }, kernel, kernel.Length - 1, input); return convolve(i => new [] { 0, -i }, kernel, kernel.Length - 1, convolveX); } static void Main(string[] args) { const int inputSize = 10; var random = new Random(42); var inputData = new float[inputSize, inputSize]; for (int row = 0; row < inputSize; row++) for (int col = 0; col < inputSize; col++) inputData[row, col] = (float)random.NextDouble() * random.Next(1, 100); var testKernel = new float[]{2, 5, 7, 4, 3} ; var dx9Target = new DX9Target(); var inputArray = new FloatParallelArray(inputData); var result = dx9Target.ToArray2D(convolveXY (testKernel, inputArray)); for (var row = 0; row < inputSize; row++) { for (var col = 0; col < inputSize; col++) Console.Write("{0} ", result[row, col]); Console.WriteLine(); } } }}
static FloatParallelArray convolve(Func<int, int[]> shifts, float[] kernel, int i, FloatParallelArray a){ FloatParallelArray e = kernel[i] * ParallelArrays.Shift(a, shifts(i)); if (i == 0) return e; else return e + convolve(shifts, kernel, i - 1, a);}
static FloatParallelArray convolveXY(float[] kernel, FloatParallelArray input) { FloatParallelArray convolveX = convolve(i => new [] { -i, 0 }, kernel, kernel.Length - 1, input); return convolve(i => new [] { 0, -i }, kernel, kernel.Length - 1, convolveX); }
using System;using System.Linq;using Microsoft.ParallelArrays;namespace AcceleratorSamples{ static class Convolver2D { static FloatParallelArray convolve(this FloatParallelArray a, Func<int, int[]> shifts, float[] kernel) { return kernel .Select((k, i) => k * ParallelArrays.Shift(a, shifts(i))) .Aggregate((a1, a2) => a1 + a2); } static FloatParallelArray convolveXY(this FloatParallelArray input, float[] kernel) { return input .convolve(i => new[] { -i, 0 }, kernel) .convolve(i => new[] { 0, -i }, kernel); } static void Main(string[] args) { const int inputSize = 10; var random = new Random(42); var inputData = new float[inputSize, inputSize]; for (int row = 0; row < inputSize; row++) for (int col = 0; col < inputSize; col++) inputData[row, col] = (float)random.NextDouble() * random.Next(1, 100); var testKernel = new[] { 2F, 5, 7, 4, 3 }; var dx9Target = new DX9Target(); var inputArray = new FloatParallelArray(inputData); var result = dx9Target.ToArray2D(inputArray.convolveXY(testKernel)); for (var row = 0; row < inputSize; row++) { for (int col = 0; col < inputSize; col++) Console.Write("{0} ", result[row, col]); Console.WriteLine(); } } }}
static FloatParallelArray convolve(this FloatParallelArray a, Func<int, int[]> shifts, float[] kernel) { return kernel .Select((k, i) => k * ParallelArrays.Shift(a, shifts(i))) .Aggregate((a1, a2) => a1 + a2); } static FloatParallelArray convolveXY(this FloatParallelArray input, float[] kernel) { return input .convolve(i => new[] { -i, 0 }, kernel) .convolve(i => new[] { 0, -i }, kernel); }
open Systemopen Microsoft.ParallelArrays[<EntryPoint>]let main(args) = // Declare a filter kernel for the convolution let testKernel = Array.map float32 [| 2; 5; 7; 4; 3 |] // Specify the size of each dimension of the input array let inputSize = 10 // Create a pseudo-random number generator let random = Random (42) // Declare a psueduo-input data array let testData = Array2D.init inputSize inputSize (fun i j -> float32 (random.NextDouble() * float (random.Next(1, 100)))) // Create an Accelerator float parallel array for the F# input array use testArray = new FloatParallelArray(testData) // Declare a function to convolve in the X or Y direction let rec convolve (shifts : int -> int []) (kernel : float32 []) i (a : FloatParallelArray) = let e = kernel.[i] * ParallelArrays.Shift(a, shifts i) if i = 0 then e else e + convolve shifts kernel (i-1) a // Declare a 2D convolver let convolveXY kernel input = // First convolve in the X direction and then in the Y direction let convolveX = convolve (fun i -> [| -i; 0 |]) kernel (kernel.Length - 1) input let convolveY = convolve (fun i -> [| 0; -i |]) kernel (kernel.Length - 1) convolveX convolveY // Create a DX9 target and use it to convolve the test input use dx9Target = new DX9Target() let convolveDX9 = dx9Target.ToArray2D (convolveXY testKernel testArray) printfn "DX9: -> \r\n%A" convolveDX9 0
let convolveXY kernel input = // First convolve in the X direction and then in Y let convolveX = convolve (fun i -> [| -i; 0 |]) kernel (kernel.Length - 1) input let convolveY = convolve (fun i -> [| 0; -i |]) kernel (kernel.Length - 1) convolveX convolveY
0 5 10 15 20 25 30 35 40 450
5
10
15
20
25
x64 multicore target benchmark for 2D convolver (24 core server Xeon E7540)
6 core speedup12 core speedup18 core speedup24 core speedup
kernel size
spee
dup
over
one
core
Convolver
8.249ns max delay3 x DSP48Es63 slice registers24 slice LUTs
DRAM
technology node 130nm CMOS(2006)
45nm CMOS(2008)
transfer 32bacross-chip
20 computations 57 computations
transfer 32boff-chip
260 computations 1300 computations
Power of Computation vs. Communication
numbers derived from work by W. Dally, Stanford
Virtex-6 XC6VLX240T-1317MHzArea 6%108 DSP48E1 (out of a768)2.7W (at 25C)1,110mega-samples per secondcf. CUDA version on 470GTX at 552 mega-samples per second(single precision)
Kiwi Thesis
thread 2
thread 3
thread 1
»
Kiwi
structural imperative (C)parallelimperative
gate-level VHDL/Verilog Kiwi C-to-
gates
&0
0
0
Q
QSET
CLR
S
R
;
;
;
jpeg.cthread 2
thread 3
thread 1
KiwiLibrary
Kiwi.cs
circuitmodel
JPEG.cs
Visual Studio
multi-thread simulationdebuggingverification
Kiwi Synthesis
circuitimplementation
JPEG.v
parallelprogram
C#
Thread 1
Thread 2
Thread 3
Thread 3
C togates
C togates
C togates
C togates
circuit
circuit
circuit
circuitVerilog
for system
Ports and Clockspublic static class I2C { [OutputBitPort("scl")] static bool scl;
[InputBitPort("sda_in")] static bool sda_in;
[OutputBitPort("sda_out")] static bool sda_out;
[OutputBitPort("rw")] static bool rw;
circuit ports identified by
custom attribute
public static int max2(int a, int b){ int result; if (a > b) result = a; else result = b; return result;}
.method public hidebysig static int32 max2(int32 a, int32 b) cil managed{ // Code size 12 (0xc) .maxstack 2 .locals init ([0] int32 result) IL_0000: ldarg.0 IL_0001: ldarg.1 IL_0002: ble.s IL_0008
IL_0004: ldarg.0 IL_0005: stloc.0 IL_0006: br.s IL_000a
IL_0008: ldarg.1 IL_0009: stloc.0 IL_000a: ldloc.0 IL_000b: ret}
max2(3, 7)
stack
local memory
0
377
7
Writing to a Channelpublic class Channel<T>{ T datum; bool empty = true; public void Write(T v) { lock (this) { while (!empty) Monitor.Wait(this); datum = v; empty = false; Monitor.PulseAll(this); } }
Reading from a Channel
public T Read(){ T r; lock (this) { while (empty) Monitor.Wait(this); empty = true; r = datum; Monitor.PulseAll(this); } return r;}
systems level concurrency constructsthreads, events, monitors, condition variables
rendezvous join patterns transactionalmemory
dataparallelism
user applications
domain specificlanguages
Filter Example
thread one-placechannel
Transposed Filter
static void Tap(int i, byte w, Kiwi.Channel<byte> xIn, Kiwi.Channel<int> yIn, Kiwi.Channel<int> yout){ byte x; int y; while(true) { y = yIn.Read(); x = xIn.Read(); yout.Write(x * w + y); }}
Inter-thread Communication and Synchronization
// Create the channels to link together the tapsfor (int c = 0; c < size; c++){ Xchannels[c] = new Kiwi.Channel<byte>(); Ychannels[c] = new Kiwi.Channel<int>(); Ychannels[c].Write(0); // Pre-populate y-channel registers with zeros}
// Connect up the taps for a transposed filterfor (int i = 0; i < size; i++){ int j = i; // Quiz: why do we need the local j? Thread tapThread = new Thread(delegate() { Tap(j, weights[j], Xchannels[j], Ychannels[j], Ychannels[j+1]); }); tapThread.Start();}
static public void echo() { tx_sof_n = !false; // We are not at the start of a frame tx_src_rdy_n = !false; tx_eof_n = !false; // We are not at the end of a frame bool start = !rx_sof_n && !rx_src_rdy_n; // The start condition int i, j; bool doneReading;
while (true) // Process packets indefinately { // Wait for SOF and SRC_RDY while (!start) { Kiwi.Pause(); // Wait for a clock tick start = !rx_sof_n && !rx_src_rdy_n; // Check for start of frame } // Read in the entire frame i = 0; doneReading = false;
// Read the remaining bytes while (!doneReading) { if (!rx_src_rdy_n) { buffer[i] = rx_data; i++; } doneReading = !rx_eof_n; Kiwi.Pause(); }
C#
softprocessor
fib :: Int -> Intfib 0 = 0fib 1 = 1fib n = n1 + n2 where n1 = fib (n – 1) n2 = fib (n - 2)
STATE 1 FREE PRECASE ds1 := ds CASE ds1 WHEN 1 => RETURN 1 WHEN 0 => RETURN 0 WHEN others => v0 := ds1 - 2 RECURSE [v0] 2 [ds1] END CASESTATE 3 FREE n2 n1 := resultInt v2 := n1 + n2 RETURN v2STATE 2 FREE ds1 n2 := resultInt v1 := ds1 - 1 RECURSE [v1] 3 [n2]
gcd_dijkstra :: Int -> Int -> Intgcd_dijkstra m n = if m == n then m else if m > n then gcd_dijkstra (m - n) n else gcd_dijkstra m (n - m)
relocation viavirtualization???
+ encryption + virtualization + data-processing
no standard ABIno FPGA-kernel-userspace modelThe cloud is just an extension of existing OS paradigms… FPGAs get left behind… they lack abstraction boundaries
Split Trust
managing physical devicevs.
using a physical devicemanagement
domain?
FPGAs Improve Cloud Security
Barrelfish Heterogeneous Operating System
IT’S The End of the World as we Know It