qpace qcd parallel computing on the cell broadband engine™ (cell/b.e.)

© Copyright IBM Corporation 2009 IBM Deutschland Research & Development GmbH 10/15/09

QPACE QCD Parallel Computing on the Cell Broadband Engine™ (Cell/B.E.)

Heiko Joerg Schick Firmware Project and Bring-up Lead Böblingen, 2009-06-29

© Copyright IBM Corporation 2009 IBM Deutschland Research & Development GmbH 10/15/09 2

Agenda

Overview

QPACE Architecture

QPACE Node Card


Overview

The QPACE project is a research collaboration of IBM Development and European universities and research institutes with the goal to build a prototype of a cell processor-based supercomputer.

The major part of this project will be funded by the German Research Foundation (DFG – Deutsche Forschungsgemeinschaft) as part of a Collaborative Research Center (SFB – Sonderforschungsbereich [TR55]). Additional funds come the PRACE project.

QPACE = QCD Parallel Computing on the Cell Broadband Engine™ (Cell/B.E.)


Architecture

System: Node card with PowerXCell 8i processor and network processor (NWP) Commodity processor interconnected by a custom network 256 node-cards per rack = 26 Tflops peak performance

QPACE Architecture

Network: 3-dimensional Torus Network: nearest-neighbour communication, 3-dimensional torus topology Tree Network: evaluation of global conditions and synchronization Gigabit Ethernet: One link per node, rack-level switches

Applications: Optimized for calculations in theoretical particle physics: Simulation of Quantum Chromodynamics Target sustained performance of 20-30%

Liquid cooling system: Closed node card housing acts as heat conductor Housing is connected to liquid-cooled “cold plate” Cold Plate is placed between two rows of node cards


QPACE Architecture

Backplane (8 per rack)

Power Supply and Power Adapter Card (24 per rack)

Rack

Root Card (16 per rack)

Node Card (256 per rack)


Features

Components: PowerXCell 8i processor 3.2 GHZ 4 Gigabyte DDR2 memory 800 MHZ with ECC Network processor (NWP) Xilinx FPGA LX110T FPGA Ethernet PHY 6 x 1GB/s external links using PCI Express physical layer Service Processor (SP) Freescale 52211 FLASH (firmware and FPGA configuration) Power subsystem Clocking

QPACE Node Card

Network Processor: FLEXIO interface to PowerXCell 8i processor, 2 bytes with 3 GHZ bit rate Gigabit Ethernet UART FW Linux console UART SP communication SPI Master (boot flash) SPI Slave for training and configuration GPIO


QPACE Node Card

Memory PowerXCell 8i Processor

Network Processor (FPGA)

Network PHYs


QPACE Node Card

PowerXCell 8i

FPGA Virtex-5

800MHz

Power Subsystem

FLEXIO 6GB/s

SPI

Compute Network

SP Freescale MCF52211

RS232

384 IO@250MHZ 4*8*2*6 = 384 IO

680 available (LX110T)

FLEXIO 6GB/s Clocking

UART

Flash

6x 1GB/s PHY

SPI

I2C

SPI I2C

GigE

RW (Debug)

DDR2 DDR2 DDR2 DDR2

PHY

Block Diagram


QPACE Node Card

FELX iO

Rocket IO

GBIF

IOC (IOIF)

Master(BE) Slave(BE)

Switch / Address Decode / FIFOs Bus Controller

6 x 1GB/S

Make Requests Receive Requests

IOC (IOIF)

FLEXIO

IBM:

• RocketIO Logic

• IOC Logic

• GBIF Logic

Academic Partners:

• Network Processor Logic

FPGA Architecture


FlexIO Processor Interface

Interface: High bandwidth interface between IBM PowerXCell 8i processor and Xilinx Viretx-5 FPGA via an

interface implementation from Rambus Inc. Optimized for intra-board environments.

QPACE Node Card

Challenges: QPACE FlexIO connection is very challenging:

– Speed, Latency, Bandwidth and Timing (Clock) – 3 Gbyte/sec communication channel – 2 Byte link wide

Requirements: FlexIO requires link training after power-on:

– Phase calibration (aligns the data for optimal sampling point) – Parallel calibration (synchronizes the receive deserializer with the transmit serializer) – Levelization calibration (aligns all data lanes)


Thank you very much for your attention.


Disclaimer

IBM®, DB2®, MVS/ESA, AIX®, S/390®, AS/400®, OS/390®, OS/400®, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere®, Netfinity®, Tivoli®, Informix und Informix® Dynamic ServerTM, IBM, BladeCenter and POWER and others are trademarks of the IBM Corporation in US and/or other countries.

Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license there from. Linux is a trademark of Linus Torvalds in the United States, other countries or both.

Other company, product, or service names may be trademarks or service marks of others. The information and materials are provided on an "as is" basis and are subject to change.

Weitere Themen

qpace qcd parallel computing on the cell broadband engine™ (cell/b.e.)

Sports

copyright ibm corporation

ibm powerxcell

research institutes

rack node card

fpga memory processor

qpace project

network processor logic6

qpace architecture root