gfex - cern...a single board system. both efex and jfex are multi-board systems. 6 see reiner...

gFEXthe ATLAS Calorimeter Trigger Global Feature

Extractor

Helio Takai, Michael Begel and Hucheng Chen Brookhaven National Laboratory

21st International Conference on Computing in High Energy Physics Okinawa, Japan - April 13-17,2015

[email protected]

1

mailto:[email protected]

The ATLAS Experiment

2

Phase I UpgradeLuminosity ~ 2 x1034 cm-2s-1

25 ns bunch crossing Pile up of <𝜇>~60

Level 1 rate =100 kHz HLT rate = 1 kHz

Motivation

3

• High pT bosons and fermions are a key component of ATLAS physics. • W,Z and H bosons, top quarks and exotic particles. • Many analyses with boosted objects.

• Analyses that addresses this kind of physics use large R jets with R > 1.

• The ATLAS Level 1 trigger is designed for narrow jets, with limited acceptance for large objects.

y-1 -0.5 0 0.5 1 1.5 2 2.5

φ

0

0.5

1

1.5

2

2.5 Preliminary Simulation ATLAS = 1.75 TeVZ’m event, tt → Z’

= 186.5 GeVWbm = 77.3 GeV, Wm

= 1.0R tkAnti-Calorimeter clusters

= 0.2RSubjets, C/A bosonW

jetbTop radiationISR

L1 Jet

Mass [GeV]0 50 100 150 200 250 300 350 400

Entri

es /

10 G

eV

0

50

100

150

200

250

300

1 subjet2 subjets3 subjets4+ subjets

Simulation PreliminaryATLAS,-1 L dt = 20.3 fb∫ = 8 TeVs

|<1.2η>200 GeV, |T

Trimmed jet p

narr

ow je

ts

W/Z

Top

Increase trigger efficiency for Fat-jets in ATLAS

Mass [GeV]0 50 100 150 200 250 300 350 400

Entri

es /

10 G

eV

0

50

100

150

200

250

300



|<1.2η>200 GeV, |T

Trimmed jet p

Mass [GeV]0 50 100 150 200 250 300 350 400

Entri

es /

10 G

eV

0

50

100

150

200

250

300



|<1.2η>200 GeV, |T

Trimmed jet p

No

trev

iew

ed

,fo

rin

tern

al

circu

la

tio

no

nly

[GeV]jet

Tpuncalibrated

100 200 300 400 500E

vent E

ffic

iency

0.0

0.2

0.4

0.6

0.8

1.0

=80⟩µ⟨=14 TeV s tt R=1.0 (5% trimmed)Tkanti-

>20 GeVT

p D=0.3 subjet with Tk1 ≥

|<2.5jet

η| [GeV]<220

jet100<m

L1_J100 (Run 1 L1Calo sim.)L1_HT200 (Run 1 L1Calo sim.)L1_G140 (seed>15 GeV)

ATLAS PreliminarySimulation

[GeV]jet

Tpuncalibrated

100 200 300 400 500

Eve

nt E

ffic

iency

0.0

0.2

0.4

0.6

0.8

1.0

=80⟩µ⟨=14 TeV sbb ν l→WH R=1.0 (5% trimmed)Tkanti-

>20 GeVT


|<2.5jet

η| [GeV]<150

jet100<m

L1_J100 (Run 1 L1Calo sim.)L1_HT200 (Run 1 L1Calo sim.)L1_G140 (seed>15 GeV)


Figure 2: Event-level Level-1 trigger e�ciency turn-on curves as a function of the o✏ine trimmed jetpT for (left) tt̄ events and (right) WH ! `⌫bb̄ events. Both processes are simulated with a pile-up levelequivalent to an average number of interactions per bunch-crossing, or hµi, of 80. The e�ciency curvesare shown as a function of the o✏ine trimmed anti-kt R = 1.0 jet pT , where the o✏ine trimmed jet isrequired to have a mass between 100 < mjet < 220 GeV (left, top) or 100 < mjet < 150 GeV (right,Higgs). The trimming parameters specify that any subjets with a pT fraction of the original jet lessthan 5% are to be discarded. The subjets are defined using the kt clustering algorithm with a nominalradius parameter of D = 0.3. In each case, three trigger selections are shown: (blue open circles) fullsimulation of the existing Run I Level-1 calorimeter jet trigger with a 100 GeV threshold, (black opensquares) full simulation of a sum jet transverse energy (ET ) trigger, or HT trigger, with a 200 GeVthreshold, formed using Run 1 Level-1 calorimeter jets, (red closed circles) and a Phase I gFEX-basedreconstruction algorithm with a 140 GeV threshold [1]. The gFEX reconstruction implements a simpleseeded cone algorithm with a nominal radius of R = 1.0 and with a seed selection of 15 GeV applied tocalorimeter towers with area 0.2 ⇥ 0.2 in ⌘ ⇥ �. The 140 GeV gFEX trigger threshold is chosen to matchthe L1 J100 single subjet turn-on curve.

2

No

trev

iew

ed

,fo

rin

tern

al

circu

la

tio

no

nly

1 Level 1 jet trigger e�ciency turn-on curves

[GeV]jet

Tpuncalibrated

100 200 300 400 500

per-

Jet E

ffic

iency

0.0

0.2

0.4

0.6

0.8

1.0


>20 GeVT


R>2∆|<2.5; isolated by jet

η|

L1_J100 (Run 1 L1Calo sim.)

1 subjet

2 subjets

3 subjets≥

L1_G140

1 subjet

2 subjets

3 subjets≥


[GeV]jet

Tpuncalibrated

100 200 300 400 500

per-

Jet E

ffic

iency

0.0

0.2

0.4

0.6

0.8

1.0


>20 GeVT


R>2∆|<2.5; isolated by jet

η|

L1_J100 (Run 1 L1Calo sim.) 1 subjet 2 subjets

3 subjets≥


Figure 1: Per-jet e�ciency turn-on curves in Monte Carlo (MC) simulation for multiple Phase I upgradeLevel-1 jet trigger options. A global feature extraction (gFEX) reconstruction algorithm (closed redmarkers, left) from the TDAQ Phase I Upgrade Technical Design Report (TDR) [1] with a 140 GeVthreshold is compared to full simulation of the Run I Level-1 calorimeter jet trigger (open blue markers,left and right) with a 100 GeV threshold. The gFEX reconstruction implements a simple seeded conealgorithm with a nominal radius of R = 1.0 and with a seed selection of 15 GeV applied to calorimetertowers with area 0.2⇥ 0.2 in ⌘⇥�. The 140 GeV gFEX trigger threshold is chosen to match the L1 J100single subjet turn-on curve. Pair-produced top quark MC simulation samples are simulated with a pile-up level equivalent to an average number of interactions per bunch-crossing, or hµi, of 80. For eachalgorithm, the e�ciency curves are shown as a function of the o✏ine trimmed anti-kt R = 1.0 jet pT

with di↵erent o✏ine subjet multiplicities. The trimming parameters specify that any subjets with a pT

fraction of the original jet less than 5% are to be discarded. The subjets are defined using the kt clusteringalgorithm with a nominal radius parameter of D = 0.3. For subjet counting, the subjets are required tohave a subjet pT > 20 GeV. The o✏ine trimmed jets are required to be isolated from any other o✏inejet by at least a radial distance of �R > 2.0 radians and to be within the pseudorapidity range |⌘| < 2.5.The turn-on curves measure per-jet e�ciencies after requiring a that the the Level-1 gFEX jet be within�R < 1.0 of the o✏ine trimmed jet.

1

boosted top

Simulation Studies

H bb -

Simulations performed for 14 TeV, and for a <𝜇> = 80.

Larger trigger acceptance

4

Simulation Studies

Correlation between the event energy density (x) and estimated by gFEX (y). The correlation is better than 90%.

Jet energy pile up subtraction

offline calculation

gFEX

cal

cula

tion

offline calculationgF

EX c

alcu

latio

n

5

MuCTPi

Barrel

Endcap

Muon

LiquidArgon

TileCal.

LTDBDPS

TBB

JetFeatureExtractor

ElectronFeatureExtractor

GlobalFeatureExtractor

OpticalPlant

nMCM

pre-Processor

Hub

ROD

Hub

ROD

Hub

ROD

CMX

CMX

Jet EnergyProcessor

ClusterProcessor

L1Topo

L1CTP

L1A

toDAQ

ROI

Hit C

ount

s

TOBs

supercells

η x φ0.2x0.2

η x φ0.1x0.1

e/γ, τ

Jets, τ, ΣE ,E T Tmiss

Large Jets,E Tmiss

L1Calo

gFEX in ATLAS L1Calo

gFEX is a component of the ATLAS L1 Calo system in Phase I upgrade. It complements the electron and jet feature extractors. It is a single board system. Both eFEX and jFEX are multi-board systems.

6

See Reiner Hauser’s talk on ATLAS data flow in Track 1, tomorrow.

See Eduard Simioni’s presentation in Track 1

tomorrow.

See Julian Glatzer’s presentation in this

session

October 16, 2014 – 13 : 53 DRAFT 5

#3 #1 #2 #4 #3 #1 #2 #4

Figure 4 Left: Seeding step for identifying large-R jets by selecting towers over a threshold ET value.Right: Summing the energy around the seeds within ∆R ! 1.0.

#3 #1 #2 #4

Figure 5 The final large-R jets. Each jet is stored on the Processor FPGA that produced the seed.

techniques. For this study, the seeding step required ET > 15 GeV and a noise threshold of ET > 3 GeV

was applied to all gTowers entering the sum. The results are shown in Fig. 6. The open symbols represent

the efficiencies as would be measured by the Run 1 L1Calo for a threshold of 100 GeV (L1 J100). The

red closed symbols correspond to a 140 GeV gFEX-jet threshold (L1 G140), which was matched to the130

L1 J100 plateau for QCD-like jets with only one subjet. The turn-on curves in Fig. 6 (left) measure

per-jet efficiencies as a function of the offline trimmed R = 1.0 jets, isolated (∆R > 2.0) and matched to

the online objects within ∆R < 1.0. There are three curves for each of the J100 and the G140 selections,

corresponding to the number of subjets within the offline trimmed jets (1 through 3). These subjets are

[GeV]jetTpuncalibrated

100 200 300 400 500

per-J

et E

fficie

ncy

0.0

0.2

0.4

0.6

0.8

1.0


>20 GeVTp D=0.3 subjet with Tk1 ≥

R>2∆|<2.5; isolated by jetη|


3 subjets≥

L1_G140 1 subjet 2 subjets

3 subjets≥



100 200 300 400 500

Even

t Effi

cienc

y

0.0

0.2

0.4

0.6

0.8

1.0



|<2.5jetη|

[GeV]<220jet100<m L1_J100 (Run 1 L1Calo sim.)L1_HT200 (Run 1 L1Calo sim.)L1_G140 (seed>15 GeV)



100 200 300 400 500

Even

t Effi

cienc

y

0.0

0.2

0.4

0.6

0.8

1.0



|<2.5jetη|



Figure 6 Trigger efficiency turn-on curves comparing the gFEX R = 1.0 jet trigger to the Run 1 styleL1 J100 and L1 HT200 jet triggers (both expected to be unprescaled in Run 2). All samplesuse the ⟨µ⟩ = 80 Upgrade Monte Carlo simulation. The left two plots are for tt̄ while theright plot is WH → ℓνbb̄. The left plot shows the efficiency per “isolated” jet binned in thenumber of subjets identified offline, while the right two plots display the event-level triggerefficiency. The first 12 bunches from each bunch train were removed prior to analysis incorrespondence with the TDAQ TDR [2].

October 16, 2014 – 13 : 53 DRAFT 5

#3 #1 #2 #4 #3 #1 #2 #4


#3 #1 #2 #4











100 200 300 400 500

per-J

et E

fficie

ncy

0.0

0.2

0.4

0.6

0.8

1.0





3 subjets≥


3 subjets≥



100 200 300 400 500

Even

t Effi

cienc

y

0.0

0.2

0.4

0.6

0.8

1.0



|<2.5jetη|




100 200 300 400 500

Even

t Effi

cienc

y

0.0

0.2

0.4

0.6

0.8

1.0



|<2.5jetη|




Finding Large Radius Jets

325 ns latency. 7

October 16, 2014 – 13 : 53 DRAFT 5

#3 #1 #2 #4 #3 #1 #2 #4


#3 #1 #2 #4











100 200 300 400 500

per-J

et E

fficie

ncy

0.0

0.2

0.4

0.6

0.8

1.0





3 subjets≥


3 subjets≥



100 200 300 400 500

Even

t Effi

cienc

y

0.0

0.2

0.4

0.6

0.8

1.0



|<2.5jetη|




100 200 300 400 500

Even

t Effi

cienc

y

0.0

0.2

0.4

0.6

0.8

1.0



|<2.5jetη|




STEP 1 - Find seeds. Seeds are towers with energy above a set threshold.

STEP 2 - Sum energy from neighboring towers. Concurrently, estimate pileup energy.

STEP 3 - Subtract pileup energy and “join” results. The final result is stored on processing FPGAs.

October 16, 2014 – 13 : 53 DRAFT 22

Hybrid FPGA

FPGA–4 FPGA–3

FPGA–2FPGA–1

/2

\2

/2

\2

/112/112 /112/112

/112

/112

/112

/112

/

20

\

20

\

20

/

20

/

20

\

20

\

20

/

20

Monitoring & TTC communications (MGT)Algorithm connections (GPIO)(50 DDR pairs data + 6 pair clock)

TTC transmission (GPIO)(8 DDR pairs data + 2 pair clock)

IPbus communications (GPIO)(8 Tx & 8 Rx + 4 handshake)

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

o

Figure 13 Interconnections used for information transfers between the Processor FPGAs and betweenthe Processor FPGAs and the Hybrid FPGA. The numbers indicate the sum of pins associ-ated with each GPIO connection or the number of MGT.

configuration of gTowers that could be sent to the Hybrid FPGA is detailed in Table 9 (only half the bits

are needed as EM and HAD layers will be pre-summed). Sending these gTowers to the Hybrid FPGA

at the 1 MHz L0A rate in Run 4 requires that each Processor FPGA transmit up to 5.13 Gb/s (including

8b/10b encoding). Each Processor FPGA must also send the TOBs requiring an additional bandwidth of510

0.615 Gb/s. It is also useful, for monitoring purposes, to histogram TOBs in random events at a prescaled

rate independent of the L0A.

Output from the Hybrid FPGA is via an Ethernet connection to an online monitoring system including

a web server running on the Hybrid FPGA. Ethernet connectivity will be implemented via an external

PHY chip (such as the Marvell R⃝ 88E1116R [31]) with an interface to IPcores “Tri mode eth MAC”515

and an “AXI based FIFO” between the IPbus core and the Ethernet UDP layer in the Hybrid FPGA.

The Hybrid FPGA GPIOs in the FPGA fabric can be used for parallel data/control/timing lines with this

design.

gFEX Concept

• Coarse granularity data (𝛥𝜂x𝛥𝜑=0.2x0.2) from calorimeters are received by high speed optical links and processed by four large FPGAs.

• The processing FPGAs are monitored and programmed by a Hybrid FPGA (SoC).

• Results are transferred to the next level in L1 trigger.

8

October 16, 2014 – 13 : 53 DRAFT 3

4×Processor FPGA

rear front

Real-Time Data Path

Readout Data PathAT

CA

Sh

elfM

an

ager

Hu

bT

TC

RO

D

IPMC Card

IPMC

Hybrid FPGA

PHY IPbus Int.

TTC Interface

GBT Interface

Monitoring

opticalfi

ber

plant

Op

tical

Con

n.

/4×72

Op

tical

Rx

/24×12

Deseria

lizer

/264

gT

ow

erB

uild

er

Jet Finder

Pileup Density

Event Moments

TO

BB

uild

er

Seria

lizer

Op

tical

Tx

/4×12

Op

tical

Con

n.

/4×12

L1T

op

o

/1×48

Readout Buffer

Serializer

/4×2

/2×4×1

Op

tical

Con

n.

FE

LIX

TT

C

Op

tical

Rx

Op

tical

Tx/

4×2

2

//3

/7

/ 4 × 1

/1×12

/1×12

/1×24

Ethernet

RJ45PHY

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

o

L1Cal

oL1C

alo

L1Cal

oL1C

alo

L1Cal

o

L1Cal

o

Figure 3 A block diagram of the gFEX module. Shown are the real-time (to L1Topo) and readout datapaths (to Hub/ROD and FELIX).

The gFEX is an ATCA module conforming to the PICMG R⃝3.0 Revision 3.0 specification [14]. The

gFEX module will likely be placed in a sparsely populated ATCA shelf so that it can occupy two slots if

needed: one for the board and one for cooling (e.g., large heat sinks), fiber routing, etc.

2.2 Input Data

The gFEX receives data from the electromagnetic and hadronic calorimeters on optical fibers. For most75

of the detector, the towers — called gTowers — correspond to an area of ∆η×∆φ = 0.2×0.2. Calorimeter

locations with towers different from this standard size are detailed in Sec. 3.3.

Calorimeter data are transmitted as continuous serial streams. The gFEX logic must be aligned with

the word boundaries in this serial data for conversion into parallel data. The scheme for achieving this is

not yet specified, however, possible mechanisms are being explored.80

The calorimeter data are accompanied by a CRC code. This is checked in the Processor FPGA

immediately following conversion from serial streams into parallel data. All data with a detected error

are sent to the readout path without modification but are zeroed on the real-time trigger path. In addition,

information about errors is collected as follows:

Input Error Count Incremented for any clock cycle with at least one error in any input channel.85

Input Error Latch A bit is set for any channel with at least one error. These bits remain set until

cleared via an external signal.

Error Check Result Formed from an OR of all error checks for the current BC.

Global Input Error Formed from the OR of all bits in the Input Error Latch.

The Input Error Count, Input Error Latch, Error Check Result, and Input Error bits can be read via an90

IPbus command [15] and will be reported via the Hybrid FPGA. An IPbus command will be able to clear

any or all of these registers. The Error Check Result and Input Error Count are included in the readout

data for the current BC. Additionally, the quality of the input data is assessed:

gFEX: Processing Chain and Interfaces

9

See Jorn Schumacher’s presentation in Track 1

tomorrow.

Floor Plan of Prototype gFEX � Input/Output

o 4 72-fold MPO connectors (back) o 2 48-fold MPO connectors (front) o 24 miniPOD Rx ←− gFOX o 4 miniPOD Tx −→ L1Topo o 1 miniPOD Rx ←− FELIX o 1 miniPOD Tx −→ FELIX o JTAG, UART, RJ45 (front) o ATCA Zone 1 & J23 ADF Plus (back)

� FPGA o 4 Virtex-7 XC7VX690T-FFG1927

(speed grade -3) o ZYNQ XC7Z045-FFG900

� Notes: o neighboring slot empty ⇒ more space

for fiber routing and better cooling o switched to eFEX ATCA form factor

proposal o identifying vendors for 72-fold

MPO;1.1 dB light loss expected (purchase for gFEX, jFEX, FOX?)

2/24/2015 2

gFEX Floor Plan

1. The baseline FPGA is the XC7VX690T-FFG1927 speed grade -3 @ 320 MHz.

2. The SoC is the ZYNQ XC7Z045-FFG900, with dual ARM core and Linux OS (PetaLinux).

3. Input/Output • 4x 72-fold MPO* connectors (IN) • 2x 48 MPO* connectors (OUT) • miniPODs for Rx and Tx • JTAG, UART, RJ45 (Front) • ATCA Zone 1 & J23 ADF+ (Back)

4. The number of input fibers is 264 (at 6.4 Gb/s) or 232 (at 11.2 Gb/s).

10(*)MPO - Multiple-Fiber Push-On/Pull-off

Floor Plan of Pre-prototype gFEX � Input/Output

o 4 72-fold MPO connectors (back) o 1 48-fold MPO connectors (front) o 6 miniPOD Rx ←− gFOX o 1 miniPOD Tx −→ L1Topo o 1 miniPOD Rx ←− FELIX o 1 miniPOD Tx −→ FELIX o JTAG, UART, RJ45 (front) o ATCA Zone 1 & J23 ADF Plus (back)

� FPGA o 1 Virtex-7 XC7VX550T-FFG1927

(speed grade -3) o ZYNQ XC7Z045-FFG900

� Note: o Verify critical technical problems,

such as PCB design, ZYNQ SOC, High speed routing and Vertex 7 FPGA GTH link with high density MiniPods.

o Functional Board. Almost all the important gFEX functions can be tested on this board.

2/24/2015 3

Development Plans

1. gFEX is being developed in stages. The first phase (ongoing) is to prototype a board to assure full integration with L1 Calo. In a second stage a board with four FPGAs will be produced.

2. High speed optical links will be tested. Supported I/O speeds are 6.4, 9.6, 11.2 and 12.8 Gb/s.

3. Integration and link speed tests will take place at the end of 2015.

4. Firmware for data processing is being developed in parallel using commercial boards.

Prototype

11

Pre-prototype Layout

� Schematic Entry : complete � Layout & Routing: 80% � On Schedule for March

2/24/2015 4

Pre-prototype Layout

� Schematic Entry : complete � Layout & Routing: 80% � On Schedule for March

2/24/2015 4

The Prototype Board

Prototype Schematic and Layout are complete! It is implemented in an ATCA board form factor.

12

PCB Stackup � Months of simulation effort to develop

stack up for prototype gFEX module � 26-layer PCB

o high-speed signals in layers 5/7 and 20/22 o back-drilled vias o Megtron 6 PCB material o several vendors make similar boards

� Thickness is ∼ 2.6 mm o mill edges to ATCA standard

Design File: gth_v7_fr4_simulation.ffsHyperLynx LineSim v8.2.1

U1

xilinx_7gth_txxilinx_7gth_tx

1n

1p

U2

xilinx_v7_gth_ami_...rx_p

2n

2pV1TOP

inn...

TL1

61.1 ohms7.435 ps0.050 inCoupled Stackup

TL2


V3TOP

inn...

TL3

65.9 ohms2.108 ns12.000 inCoupled Stackup

TL4

65.9 ohms2.108 ns12.000 inCoupled Stackup

TL5


TL6


V4TOP

inn...

TL7


TL8


V5TOP

inn...

TL9


TL10


TL11


TL12


J1

Xilinx_Virtex-7_...

Port1

Port2

Port3

Port4

J2

Xilinx_Virtex-7_...

Port1

Port2

Port3

Port4

C1

100.0 nF

C2

100.0 nF

GTH-to-GTH Link simulation

14 inch/FR4/with stub/9.6 Gbps 14 inch/FR4/without stub/12.8 Gbps 14 inch/M6/without stub/12.8 Gbps 14 inch/M6/with stub/12.8 Gbps

2/24/2015 5

Challenges

1. gFEX will be a 26 layer board, with a total thickness of ~2.6 mm.

2. The choice of material is critical for the high speed traces within the board. Selected Megtron 6.

13

Summary

The global feature extractor, gFEX, will add to ATLAS the capability to trigger on large radius jets at Level 1.

It is based on FPGA processing. Four large FPGAs will receive coarse data (0.2x0.2) from the EM and HAD calorimeters and select events of interest.

Prototype is now being built. Initial prototype addresses the interfaces with ATLAS L1. After this phase a full processor will be built.

gFEX is processor board that has a large number of high speed I/O lines and could be used in other applications.

14

Extra Slides

October 16, 2014 – 13 : 53 DRAFT 17

Table 6 Count of input fibers by section of the calorimeter for several bandwidth assumptions. Thenumber of bits per gTower is the maximum possible for a given detector configuration andbandwidth.

Link Speed

6.4 Gb/s 9.6 Gb/s 11.2 Gb/s 12.8 Gb/s

Partition Coverage gT

ower

s/F

iber

bit

s/gT

ower

Fib

ers

gT

ower

s/F

iber

bit

s/gT

ower

Fib

ers

gT

ower

s/F

iber

bit

s/gT

ower

Fib

ers

gT

ower

s/F

iber

bit

s/gT

ower

Fib

ers

Barrel EM |η| < 1.6 8 15 64 8 22 64 8 26 64 8 30 64

Tile (Phase I opt.2) |η| < 1.6 16 ? 32 16 ? 32 16 ? 32 16 ? 32

Tile (Phase I opt.3) |η| < 1.6 12 10 48 12 ? 48 12 ? 48 12 ? 48

Tile (Phase II) |η| < 1.6 8 15 64 8 22 64 8 26 64 8 30 64

Standard EMEC 1.6 < |η| < 2.4 7 17 32 7 26 32 7 30 32 7 35 32

Special EMEC 2.4 < |η| < 3.2 10 12 32 14 12 24 14 12 24 20 12 16

HEC 1.5 < |η| < 3.2 12 10 48 18 10 32 18 11 32 18 11 32

FCAL 1 3.1 < |η| < 4.9 12 11 12 16 11 8 16 11 8 22 11 6

FCAL 2&3 3.2 < |η| < 5.0 12 11 12 16 11 8 16 11 8 22 11 6

Total (Phase II) 264 232 232 220

Table 7 Coverage and number of fibers incident on the Processor FPGA.

η φ Fibers at Link Speed (Gb/s)FPGA coverage coverage 6.4 9.6 11.2 12.8

1 −1.6 < η < 0.0 0.0 < φ < 2π 64 64 64 64

2 0.0 < η < 1.6 0.0 < φ < 2π 64 64 64 64

3 1.6 < η < 5.0 0.0 < φ < 2π 68 52 52 46

4 −5.0 < η < −1.6 0.0 < φ < 2π 68 52 52 46

be made following additional performance studies and in consultation with the Calorimeter and L1Calo

communities.

Four configurations of fibers and FPGA as a function of link speed are presented in Table 7. It

is interesting to note that by changing from the 80 MGT Virtex R⃝-7 [27] to the 120 MGT Virtex R⃝UltraScaleTM [28], it is possible to reduce the number of Processor FPGA from four to two at 9.6 Gb/s390

or higher link speed while retaining spare capacity for future calorimeter upgrades.

3.4 Real-Time Output Data

The real-time output from the gFEX are TOBs which contain information from the feature identification

algorithms running on the Processor FPGA (Sec. 2.4). There are two category of TOBs: global and

jet. The global TOBs encode data associated with global event observables such as the pileup density ρ,395

EmissT

, HmissT

, etc. The jet TOBs contain information about the identified jets and any applicable moments.

The TOBs are transmitted to L1Topo on optical fibers assuming a baseline bandwidth of 6.4 Gb/s [29].

It is foreseen that the bandwidth could be increased up to 12.8 Gb/s for the Phase-I upgrade. It is as-

sumed that each Processor FPGA will have one or more independent fibers to each L1Topo FPGA. The

protocol and payload for transmission to L1Topo is the same as the receipt of data from the calorimeters400

as described in Sec. 3.1.

gFEX: number of optical fiber connections

gFEX References

https://twiki.cern.ch/twiki/bin/view/AtlasPublic/JetTriggerPublicResults#Global_Feature_Extraction_gFEX_P

Performance Plots (Simulation)

https://edms.cern.ch/file/1425502/1/gFEX.pdf

gFEX Prototype Technical Specification

https://twiki.cern.ch/twiki/bin/view/AtlasPublic/JetTriggerPublicResults#Global_Feature_Extraction_gFEX_P

https://edms.cern.ch/file/1425502/1/gFEX.pdf

gfex - cern...a single board system. both efex and jfex are multi-board systems. 6 see reiner...

Documents