capabilities, costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท...
TRANSCRIPT
Capabilities, Costs โ๐ท๐๐๐๐๐๐๐๐๐๐/๐พ๐๐๐
$
ASICsFPGAs
Source: Bob Broderson, Berkeley Wireless group
Xeon CPU NIC
Xeon CPU NICSearch Acc. (FPGA)
Search Acc. (ASIC)
Wasted Power,
Holds back SW
Xeon CPU NICSearch Acc. v2 (FPGA)
NICXeon CPU Math Accelerator
Wasted Power,One more thing that
can break
http://www.globalfoundationservices.com/posts/2014/january/27/microsoft-contributes-cloud-server-specification-to-open-compute-project.aspx
โข Two 8-core Xeon 2.1 GHz CPUs
โข 64 GB DRAM
โข 4 HDDs, 2 SSDs
โข No cable attachments to server
Stratix V
8GB DDR3
PCIe Gen3 x8
Data Center Server (1U, ยฝ width)
FPGA FPGA FPGA FPGAWeb Search Pipeline
FPGA FPGA FPGA FPGA
Math Acceleration
Service Comp.
Vision
Service
Physics
Engine
Web Search Pipeline
West SLIII
East SLIII
South SLIII
North SLIII
x8 PCIe Core
DMA Engine
ConfigFlash (RSU)
DDR3 Core 1DDR3 Core 0
JTAG
LEDs
Temp Sensors
Application
Shell
I2C
xcvrreconfig
2 2 2 2
4256 Mb
QSPI ConfigFlash
4 GB DDR3-1333 ECC SO-DIMM
4 GB DDR3-1333 ECC SO-DIMM
Host CPU
72 72
Role
8
Inter-FPGA RouterSEU
IFM 1
IFM 2
IFM 44
IFM 3
IFM 1
IFM 2
IFM 44
IFM 3
IFM 1
IFM 2
IFM 44
IFM 3
SaaS 1
SaaS 2
SaaS
48
SaaS 3
Ranking-as-a-Service (RaaS)
- Compute scores for how relevant each selected
document is for the search query
- Sort the scores and return the results
Selection-as-a-Service (SaaS)
- Find all docs that contain query terms,
- Filter and select candidate documents for
ranking
Selection as a Service (SaaS)
IFM 1
IFM 2
IFM 44
IFM 3
IFM 1
IFM 2
IFM 44
IFM 3
IFM 1
IFM 2
IFM 44
IFM 3
RaaS 1
RaaS 2
RaaS
48
RaaS 3
Ranking as a Service (RaaS)
Query
Selected
Documents10 blue links
Query: โFPGA Configurationโ
NumberOfOccurrences_0 = 7 NumberOfOccurrences_1 = 4 NumberOfTuples_0_1 = 1
{Query, Document}
~4K Dynamic
Features
~2K Synthetic
Features
L2 Score
Document
Score
FFE #1 =(2*NumberOfOccurrences_0 + NumberOfOccurrences_1)
(2 * NumberOfTuples_0_1)
NumberOfTuples_0_1 = 1NumberOfOccurrences_1 = 4NumberOfOccurrences_0 = 7
FFE #1 = 9
{Query, Document}
~4K Dynamic
Features
~2K Synthetic
Features
L2 Score
Document
Score
PCIe
Distribution latchesControl/Data
Tokens
Compressed
Document
Feature
Gathering
Network
Free Form
Expression
(FFE)
Stream
Preprocessing
FSM
โข 196 feature families
โข 54 state machines
โข 2.6K dynamic features extracted in
less than 4us (~600us in SW)
Core 0 Core 1 Core 2
Core 3 Core 4 Core 5
ComplexFST
Ou
tpu
t
Cluster
0
FFE: Free-Form
Expressions
FE: Feature Extraction FPGA 0
FPGA 1
FPGA 2
FPGA 3
FPGA 4
FPGA 5
FPGA 6
FPGA 7
Server
Server
Server
Server
Server
Server
Server
Server
Document
Scoring
Request
8-Stage Pipeline
Compute
Score
Route to
Head
Return
Score
RaaS ServersDocument
Score
Document
Scoring
Request
Compute
Score
Route to
Head
Return
Score
FPGA 0
FPGA 1
FPGA 2
FPGA 3
FPGA 4
FPGA 5
FPGA 6
FPGA 7
8-Stage Pipeline
FPGA 5
FPGA 6
FPGA 0
FPGA 1
FPGA 2
FPGA 3
FPGA 4
8-Stage Pipeline
FPGA 2
1,632 Servers with FPGAs Running Bing Page Ranking Service (~30,000 lines of C++)
But when will an FPGA handle my Bing Search?
โข Bing is going into production with FPGAs
Huge thanks to our partners at
Top Row: Eric Peterson, Scott Hauck,
Aaron Smith, Jan Gray, Adrian M.
Caulfield, Phillip Yi Xiao, Michael
Haselman, Doug Burger
Bottom Row: Joo-Young Kim, Stephen
Heil, Derek Chiou, Sitaram Lanka,
Andrew Putnam, Eric S. Chung,
Not Pictured: Kypros Constantinides,
John Demme, Hadi Esmaeilzadeh,
Jeremy Fowers, Gopi Prashanth Gopal,
Amir Hormati, James Larus, Simon
Pope, Jason Thong
Microsoft Privacy Policy statement applies to all information collected. Read at research.microsoft.com
Save the planet and return your name badge before you
leave (on Tuesday)
