gbt project: present & future

[email protected] 1

GBT Project: Present & Future

Paulo MoreiraOn Behalf of the GBT Project Collaboration

18 March 2014CERN, Switzerland

http://cern.ch/proj-gbt

[email protected] 2

Outline

• Radiation Hard Optical Link Architecture• The GBT System• GBT Chipset:

– Status– Schedule

• GBT-FPGA Status• GBT Building Blocks Status• LpGBTX:

– Architecture– SerDes (Resources optimization)– Power Consumption– Footprint– Project effort


[email protected] 3

Radiation Hard Optical Link Architecture


On-DetectorRadiation Hard Electronics

Off-DetectorCommercial Off-The-Shelf (COTS)

GBTX

GBTIA

GBLD

PD

LD

Custom ASICs

Timing & Trigger

DAQ

Slow Control

Timing & Trigger

DAQ

Slow Control

FPGA

GBT GBT

Versatile Link

[email protected] 4

The GBT System


FEModule

FEModule

Phase – Aligners + Ser/Des for E – Ports

FEModule

E – PortE – Port

E – Port

GBT – SCA

E – Port

Phase - Shifter



CDR

DEC/D

SCR

SER

SCR/ENC

I2C MasterI2C Slave

Control Logic Configuration(e-Fuses + reg-Bank)

Clock[7:0]

CLK Manager

CLK Reference/xPLL

External clock reference

controldata

One 80 Mb/s port

I2CPort

I2C (light)

JTAG

JTAGPort

80, 160 and 320 Mb/s ports

GBTIA

GBLD

GBTXe-Link

clock

data-up

data-down

ePLLTxePLLRx

clocks

[email protected] 5

GBLD StatusGBLD V4.1• Main Specs

– Bit rate 5 Gb/s (min)– Modulation:

• Current sink• Single-ended/differential

– Laser modulation current: 2 to 24 mA– Laser bias: 2 to 43 mA– Equalization:

• Pre-emphasis/de-emphasis• Independently programmable for rising/falling

edges– Supply voltage: 2.5 V– Die size: 2 mm × 2 mm– I2C programming interface

• Status– Available in small quantities

• Integrated in the VTRx and VTTx– Fully functional– Excellent performance– Radiation hardness proved (total dose) – Heavy Ion and Neutron SEU tests:

• Revealed “some sensitivity” of the configuration registers

• Proton tests shown no upsets• Majority of the errors related with reset function

(not triplicated) can be easily improved.– Technology: 130 nm DM metal stack– Device is production ready


4.8 Gb/s, pre-emphasis on

Total jitter: ≈ 25 ps

[email protected] 6

GBTIA StatusGBTIA V2.0 / V2.1• Main specs:

– Bit rate 5 Gb/s (min)– Sensitivity: 20 μA P-P (10-12 BER)– Total jitter: < 40 ps P-P– Input overload: 1.6 mA (max)– Dark current: 0 to 1 mA– Supply voltage: 2.5 V– Power consumption: 250 mW– Die size: 0.75 mm × 1.25 mm

• Status:– Fully functional– Integrated in the VTRx– Excellent performance– Radiation hardness proved

• Tested up to 200 Mrad (SiO2)

– Device is production ready• LM metal stack


[email protected] 7

GBTX Status• Main specs:

– 4.8 Gb/s transceiver– User bandwidth:

• 3.28 Gb/s up/down-links, GBT mode• 3.52 Gb/s up-link, 8B/10B mode• 4.48 Gb/s up-link, wide-bus mode

• Status:– Chip is fully functional– A few modifications will be

introduced in the production version:• XPLL will be adapted for a higher

value of the motional resistance of the quartz crystal

• Tolerance to SEUs of the e-link programing register will be improved

• Startup state machine expanded– Prototypes availability:

• 22 + 32 (drilled) GBTX available• Additional 240 will be available in

August


17 m

m

17 mm

Total height including solder balls: ~3 mm

[email protected] 8

GBT – SCAGBT Slow Control Adapter:• ASIC dedicated to slow control functions.• System Upgrades for SLHC detectors.• “Replacement” for the CCU & DCU ASICs • Flexible enough to match the needs of different

Front-End systems.Key Features:• Dual redundant e-Ports for GBTX e-links.• 16 I2C master controllers @ 100k/s or

1 Mb/s.• 1 SPI master controller• 1 JTAG master controller• 32 multiplexed ADC channels:

– 12-bit dual slope integrating ADC @ 3.5KHz• 4 DAC channels• 32 Digital I/O lines individually programmable.• 8-bit memory bus• 4 Interrupt inputs• Technology: CMOS 130nm using radiation

tolerant techniques.Status:• Chip ready for prototyping:

– Tapout May 2014http://cern.ch/proj-gbt 11.6 mm

Total height including solder balls: ~3 mm

BOTTOM WIEW

ADC_in_pad<0>

ADC_in_pad<2>

ADC_in_pad<5>

ADC_in_pad<8>

ADC_in_pad

<11>

ADC_in_pad

<14>

ADC_in_pad

<17>

ADC_in_pad

<20>

ADC_in_pad

<23>

ADC_in_pad

<26>Pad_

resOutDAC_ou

t_pad<2>

DAC_out_pad<3>

JTAG_reset_pad

ADC_in_pad<1>

ADC_in_pad<1>

ADC_in_pad<4>

ADC_in_pad<7>

ADC_in_pad

<10>

ADC_in_pad

<13>

ADC_in_pad

<16>

ADC_in_pad

<19>

ADC_in_pad

<22>

ADC_in_pad

<25>

DAC_out_pad<28>

DAC_out_pad<0>

DAC_out_pad<1>

TDO_pad

RESET_B_pad

FuseProgram

Pulse_pad

ADC_in_pad<3>

ADC_in_pad<6>

ADC_in_pad<9>

ADC_in_pad

<12>

ADC_in_pad

<15>

ADC_in_pad

<18>

ADC_in_pad

<21>

ADC_in_pad

<24>

ADC_in_pad

<27>

ADC_in_pad

<29>

ADC_in_pad

<30>TDI

_pad

rx_sdRx_sd_nAGNDAGNDAGNDAGNDAVDDAVDDAVDDAVDD

SCL_pad<14>

SCL_pad<15>

TCK_pad

link_clklink_clk_n

SPI_ss_pad<0>

AGNDAGNDAGNDAGNDAVDDAVDDAVDDAVDDSDA_pad<14>

SDA_pad<15>

TMS_pad

tx_sdtx_sd_n

SPI_ss_pad<1>

DVSSDVSSAGNDAGNDAVDDAVDDDVDDSCL

_pad<13>

SCL_pad<12>

SCL_pad<11>

SCL_pad<10>

rx_sd_aux

Rx_sdaux_n

SPI_ss_pad<2>

DVSSDVSSDVSSDVSSDVDDDVDDDVDDSDA_pad<13>

SDA_pad<12>

SDA_pad<1>

SDA_pad<10>

link_clk_aux

link_clk_aux_n

SPI_ss_pad<3>

DVSSDVSSDVSSDVSSDVDDDVDDDVDDSCL

_pad<9>

SCL_pad<8>

SCL_pad<7>

SCL_pad<6>

tx_sd_aux

tx_sdaux_n

SPI_ss_pad<4>

GNDGNDGNDGNDVDDVDDVDDSDA_pad<9>

SDA_pad<8>

SDA_pad<7>

SDA_pad<6>

link_aux_disable

_pad

SPI_miso_pad

SPI_ss_pad<5>

GNDGNDGNDGNDVDDVDDVDDSCL

_pad<5>

SCL_pad<4>

SCL_pad<3>

SCL_pad<2>

auxPortTestEn_pad

SPI_mosi_pad

SPI_ss_pad<6>

GNDGNDGNDGNDVDDVDDVDDSDA_pad<5>

SDA_pad<4>

SDA_pad<3>

SDA_pad<2>

auxPortSCL

_pad

SPI_clk

_pad

SPI_ss_pad<7>

GPIO_pad<4>

GPIO_pad<7>

GPIO_pad

<10>

GPIO_pad

<13>

GPIO_pad

<16>

GPIO_pad

<19>

GPIO_pad

<22>

GPIO_pad

<25>

GPIO_pad

<28>

SCL_pad<1>

SCL_pad<0>

auxPortSDA_pad

GPIO_pad<0>

GPIO_pad<2>

GPIO_pad<5>

GPIO_pad<8>

GPIO_pad

<11>

GPIO_pad

<14>

GPIO_pad

<17>

GPIO_pad

<20>

GPIO_pad

<23>

GPIO_pad

<26>

GPIO_pad

<29>

SDA_pad<1>

SDA_pad<0>

pad_GPIO_EXTCLK

GPIO_pad<1>

GPIO_pad<3>

GPIO_pad<6>

GPIO_pad<9>

GPIO_pad

<12>

GPIO_pad

<15>

GPIO_pad

<18>

GPIO_pad

<21>

GPIO_pad

<24>

GPIO_pad

<27>

GPIO_pad

<30>

GPIO_pad

<31>

N

M

L

K

J

I

H

G

F

E

D

C

B

A

N

M

L

K

J

I

H

G

F

E

D

C

B

A

1413121110987654321

1413121110987654321

pwr3_3pad

11.6

mm

[email protected] 9

GBT Chipset Schedule“Scenario 1” – MPW followed by production• 19 May 2014 – MOSIS MPW

• GBTX• GBT - SCA• GBLD

• Q3 – 2014– Chips available from the foundry– ASIC Packaging

• Q4 – 2014– Prototypes medium quantities: ~200– Samples distributed to the users

• Q1 – 2015:– Launch the production

• Q3 – 2015:– Production quantities available

“Scenario 2” – Production only• Q2 – 2014 (May)

– Split Engineering Run to produce in quantities:

• GBTX• GBTIA• GBLD V4• GBT - SCA

• Q3 – 2014– Chips available from the foundry– ASIC Packaging

• Q4 – 2014– ASIC production testing– First production ASICs distributed to the

users


[email protected] 10

GBT – FPGA Status


• Aim:– Implement the GBT serial link in all its

flavours as an IP core for most of the current FPGAs used on Back-End boards for upgrades

• Firmware versions– Serial link encoding schemes

• Reed-Solomon (aka “GBT frame” ),• 8b/10b (to be done)• Wide-bus

– Standard and Fixed latency versions• Targeted FPGA and reference design

– Altera:• Cyclone V GT

– Eval kit & GBTx-SAT board• Stratix V

– AMC 40

– Xilinx:• Virtex 6

– ML605 & GLIB• Kintex 7

– KC705• Virtex 7

– VC705

• Project Resources– 50% of one Fellow since last summer– More than 85 users registered– A sharepoint site: https://

espace.cern.ch/GBT-Project/GBT-FPGA/default.aspx

• A SVN repository:– https://

svnweb.cern.ch/cern/wsvn/ph-ese/be/gbt_fpga

• Contact us:– [email protected]– [email protected]

See poster by Manoel Barros forfurther information on the GBT – FPGA

MAJOR RELEASE 11 APRIL 2014

https://espace.cern.ch/GBT-Project/GBT-FPGA/default.aspx



https://svnweb.cern.ch/cern/wsvn/ph-ese/be/gbt_fpga



mailto:[email protected]

mailto:[email protected]


GBT Building Blocks (IP) StatusAvailable “IP” to facilitate the implementation ofe-Link transceivers in the frontend ASICs:• SLVS Receiver

– Wire-bond, DM metal stack– C4, LM metal stack

• SLVS Driver– Wire-bond, DM metal stack– C4, LM metal stack

• SLVS Bi-directional– C4, LM metal stack

• HDLC transceiver– Synthesizable Verilog

• 7B/8B CODEC– Synthesizable Verilog

• ePLL-FM– Frequency Multiplier PLL– Radiation Hard– 130 nm CMOS technology with the DM metal stack

(3-2-3).– Input frequencies: 40/80/160 MHz– Output frequencies: 160/320 MHz regardless the

input frequency– Programmable phase of the output clocks with a

resolution of 11.25° for the 160 MHz clock and 22.5° for the 320 MHz clock

– Programmable charge pump current, loop filter resistance and capacitance to optimize the loop dynamics

– Supply voltage: 1.2 V - 1.5 V– Nominal power consumption: 20 mW @ 1.2 V - 30

mW @1.5 V– Operating temperature range: -30°C to 100°C

• ePLL-CDR (currently under testing)– Data rate: 40/80/160/320 Mbit/s– Output clocks: data clock + 40/80/160/320 MHz

with programmable phase– Internal or external calibration of the VCO frequency– Possibility to use it as a frequency multiplier PLL

without applying input data– Programmable charge pump current, loop filter

resistance and capacitance to optimize the loop dynamics

– Supply voltage: 1.2 V - 1.5 V– Operating temperature range: -30°C to 100°C– Prototype fabrication: May 2013


ePLL- FM


The LpGBTX

• Low Power Dissipation and Small Footprint:– Critical for pixel detectors– Critical for tracker/triggering detectors

• Bandwidth:– Low-Power mode

• 4.8 Gb/s for Up and Down links– High-Speed mode:

• 9.8 Gb/s for the Up-link• 4.8 Gb/s for the Down-link

• e-Links:– 80, 160 and 320 Mb/s for down-links– Low-Power Mode:

• 80, 160 and 320 Mb/s for up-links– High-Speed Mode:

• 160, 320 and 640 Mb/s for up-links– Programmable: Single-ended / differential

• Functionality:– “Replica” of the GBTX– Small subset of the GBT-SAC functionality will be included



LpGBTX Architecture


SerDes

4.8 Gb/s

4.8 or 9.6 Gb/s

refClk40MHz

cdrOut[119:0]

serIn[239:0]

DEC&

DSCR

rxData[79:0]

SCR&

ENC

rxEcData[1:0]ePortTx

eLinkOut[39:0]

ecOut

ePortRx

eLinkIn[39:0]

eLinkIn[57:40]

ecIn[1]

40/80/160/320/640 MHz 40 MHz

40 MHz

160/320 MHz

160/320/640 MHz

• Downlink: 4.8 Gb/s– GBT mode

• Uplink: 4.8/9.6 Gb/s– GBT / Wide-Bus / 8B/10B

• EC link: 80 Mb/s• SCA functionality added• Output e-Links:

– 80/160/320 Mb/s 40 e-Links (max)

• Phase programmable clocks:– 40/80/160/320/640 MHz 8 Clocks (max)

• Output e-Clocks:– 40/80/160/320 Mb/s 40 e-Clocks (max)

• Input-Links:– 80/160/320 Mb/s @ 4.8 Gb/s– 160/320/640 Mb/s @ 9.6 Gb/s

– GBT mode 40 e-Links (max)– Wide-bus mode 56 e-Links (max)

160/320/640 MHz

“Idle & data headers”could be used to distinguishbetween the “odd and even frames”when operating at 9.6 Gb/s!

SCA(Reduced set)

txData[159:0]

txEcData[3:0]

txData[223:160]

PhaseShifter

eClock39:0]

Newcomer!


SerDes Optimization

• 1 high frequency PLL drives the CDR and SER:– Half Rate CDR (same architecture as GBTX)– Serializer:

• Full rate @ 4.8 Gb/s• Half rate @ 9.6 Gb/s

• PLL can work as:– CDR– Frequency Multiplier– This enables: TX, RX and TRANS

Clock Divider• For the e-Links the clock frequencies are:

– 40/80/160/320/640 MHz

• Half rate CDR requires:– 2.4 GHz– I and Q phases– Differential for CML

• Serializer: Half rate 9.8 Gbps / Full rate 4.8 Gbps– 4.8 GHz


CDR/FMPLL

De-Serializer4.8 Gb/s

4.8 or 9.6 Gb/s

refClk40MHz

Oscillator

1/N

Serializer

cdrOut[119:0]

serIn[239:0]

40/80/160/320/640 MHz

Aided Lock Control

For efficiency the SER and DES have to

be co-designed


LpGBTX: Power (1/2)

LpGBTX power estimate

• Assumes:– 65 nm CMOS technology– Supply voltage: 1.2V– Merged SerDes architecture– 4.8 Gb/s on both up and down

links– e-Links with 200 mV signaling– “SCA” functionality not taken

into account in this preliminary study


I/OSERDESCOREPhase-ShifterClock Manager

Power [mW] FractionI/O 548 54%SERDES 157 15%CORE 121 12%Phase-Shifter 120 12%Clock Manager 73 7%Total (max) 1019

Preliminary&

Non-Binding!


LpGBTX: Power (2/2)Case study:• Hypothetical configuration:

– E-Links:• 20 × Data-In @ 160 Mb/s• 1 × Clock @ 160 MHz• 2 × Data-Out @ 160 Mb/s

– Phase-Adjustable clocks:• 2 × Clock @ 40 MHz• 2 × Clock @ 160 MHz


Preliminary&

Non-Binding!

Mode: TRANSCEIVERE-Links: Data rate [Mb/s]: 160 # Data outputs: 2 # Clock outputs: 1 # Data inputs: 20EC-Channel: State: DisabledPhase-Shifter: State: Enabled # Channels 4Circuit: Power [mW] SERDES: 157 Clock Manager E-PLL: 33 XPLL: 40 Total: 73 Phase-Shifter: PLL: 17 Channel: 26 SLVS-TX: 26 Total: 69 I/O: Data SLVS-Tx: 13 CLK SLVS-Tx: 7 Data SLVS-Rx: 5 Total: 24 CORE: Standard Cells: 110 Phase-Aligners: 6 Total: 115 Total Power [mW]: 439


LpGBTX: Footprint


General PurposeLpGBTX with programmable I/O

(Single-ended /Differential)

Tracker Specific Development(Single-ended I/O only)

Two ASICs:• LpGBT-SerDes (tracker specific)!• LpGBTX (general purpose)Although many functions are the same in thetwo ASICs, these are effectively two projects!• Almost doubles the budget and the

manpower needs!

I/O set to differentialI/O se

t to sin

gle-ended

• Pin-count: ≈ 500• BGA pitch: 0.65 mm• Package Size: ≈ 16 mm × 16 mm

• Pin-count: ≈ 150 (to be confirmed)• BGA pitch: 0.65 mm• Package Size: ≈ 10 mm × 10 mm

Preliminary&

Non-Binding!


LpGBTX: 2014 – and beyond, Activities / Manpower / Budget2014• Q3: First draft of LpGBTX specifications• Q3: Discussions with the Experiments• Q4: LpGBTX final specifications2015 and beyond• LpGBTIA

– (The GBTIA is already relatively low power. Perhaps the LpGBTIA is not a strictly necessary development…)– Technology: 65 nm CMOS– Manpower: 2 MY (design and testing)

• LpGBLD10– (Some work already going on)– Technology: 130 nm CMOS– Manpower: 2 MY (design , packaging and testing)

• LpGBTX– Technology: 65 nm CMOS– Two packaging flavours:

• “Tracker” & “General Purpose”– Manpower:

• Design: 8 MY• Packaging: 1 MY• Testing: 2 MY

– We have to seriously consider “building” a stable LpGBT team if a LpGBT chipset is to be a reality in useful time!

– The move to 65 nm and Low Power is not just a “copy-paste exercise”!!!• The LpGBT has not yet been defined as an approved “project” or “common project”:

– Effort going on to secure budget and manpower for the project!http://cern.ch/proj-gbt

Total manpower: 15 MY

[email protected] 19http://cern.ch/proj-gbt

Extra Slides

M. Barros Marin, P. Leitao, S. Baron 20

E-LINK SERDES

GBT-FPGA TX MGT TX

GBT-FPGA RX

MGT RX

SFP

GBTxGBT-FPGA

SFP

GBTx SAT board

GLIB

GBT SERDESE-LINK SERDES

CDR

CLOCK

CDR

BACK END FRONT END

FE EMULATOR

Cyclone V

PLL

DOWNLINK

UPLINK

EportRx: phase-aligner (set to maximum and minimum; FE

position dependant)EportTx: no phase align necessary

on GBTx

Flags are latched with e-link’s clock and triggered by

the e-links dataGBT-FPGA uplink and downlink flag latched with

40 MHz clock

18/03/2014

21

E-LINK SERDES

GBT-FPGA TX MGT TX

GBT-FPGA RX

MGT RX

SFP

GBTxGBT-FPGA

SFP

GBTx SAT board

GLIB

GBT SERDESE-LINK SERDES

CDR

CLOCK

CDR

BACK END FRONT END

FE EMULATOR

Cyclone V

PLL

4 ns5 ns1 ns

DOWNLINK

UPLINK

M. Barros Marin, P. Leitao, S. Baron

51.2 ns 30.6 ns

29 ns 21 ns

18/03/2014

22

UPSTREAM LATENCYUP

Encoding

E-link speed All All - passive All - passive - GBT_FPGA*

GBT Frame

80MHz224 ns 214 ns

~9 BC163 ns

~7 BC237 ns 227 ns 176 ns

160MHz190 ns 180 ns

~8 BC129 ns

~5 BC202 ns 192 ns 141 ns

Wide Bus

Frame

80MHz224 ns 214 ns

~9 BC163 ns

~7 BC237 ns 227 ns 176 ns

160MHz190 ns 180 ns

~8 BC129 ns

~5 BC202 ns 192 ns 141 ns

M. Barros Marin, P. Leitao, S. Baron18/03/2014

Remaining: MGT, SFPs, GBTx

23

DOWNSTREAM LATENCYDOWN

Encoding

E-link speed All All - passive All – passive - GBT_FPGA*

GBT Frame

80MHz 174 ns 164 ns ~7 BC 135 ns ~5 BC

160MHz 173 ns 163 ns ~7 BC 134 ns ~5 BC

M. Barros Marin, P. Leitao, S. Baron18/03/2014

Remaining: MGT, SFPs, GBTx

gbt project: present & future

Documents