ftk: a fast track trigger for atlas

FTK: a Fast Track Trigger for ATLAS

Anton Kapliy

Anton Kapliy - Sugarman Award Presentation

2

Overview

6/4/2012

FTK is a hardware system for fast online track finding in the ATLAS detector that will be installed in 2014

Chicago plays a central role in FTK collaboration: 3 professors, 4 postdocs, 5 graduate students, 2

engineers Principal Investigator: Mel Shochet

I’ve been involved in FTK since 2006 and participated in design, simulation, hardware, and initial commissioning. Motivation for having FTK Conceptual description of the system Simulation and expected performance Electronics development and commissioning


3

An LHC collision

6/4/2012

LHC collides bunches of protons, not individual protons Currently: ~25 interactions per collision. Future: >70 Most of these interactions are “boring”

Our job is to find interesting physics among this mess

April 15th 2012: a Z➛ 𝝻𝝻 candidate event


4

Interesting physics

6/4/2012

LHC goals: Higgs, SUSY, other BSM This stuff is rare!

Tracking system is critical for selecting such events from vast backgrounds: Isolated electrons and muons

Produced in decays of W and Z bosons 3rd generation fermions: taus, b-quarks

Expect a lot of them because whatever participates in EWK symmetry breaking couples to particle mass

Rates of production:

new physics

Example I:Three collimated tracks in a triple-prong tau decay:

Example II:Displaced secondary vertex in B-hadron decay


5

ATLAS Tracker

6/4/2012

Pixel and SCT tracking detectors are relatively small But due to high granularity, they contain a whopping

86M readout channels (>98% of the ATLAS total!)

2T solenoid B-field

A typical charged track bends its way out through 3 Pixel and 8 SCT layers:


6

ATLAS Trigger System

6/4/2012

Trigger necessary to reduce event rate in real-time from 40 MHz to ~200 Hz Of course, we would like to keep all “interesting physics” and reject boring stuff

Using tracking detectors for this rejection is very challenging due to high occupancy Impossible in LVL1 hardware. In LVL2, it is only possible within narrow cones (even that is slow!)

Available data:• 40 million events/sec• Each event is O(1

MB)→ 1 Library of Congress / sec

Able to save on disk:~100 MB/s→ about 200 events/sec


7

FTK comes to the rescue

6/4/2012

FTK receives a complete copy of tracking data at LVL1 rate (75-100 KHz) Quickly reconstructs all tracks with pT>1GeV and makes them available for LVL2 Tracks are ready by the time LVL2 starts its selection of interesting physics!

FTK

FTK Buffer

FTKA dedicated hardware system for track-finding that sits between LVL1 and LVL2 trigger stages


8

FTK approach: pattern recognition

Superstrips: ~1mm

1. Given a collection of hits in an event, we gang them together to form coarse “superstrips” – about 1 mm wide.

Original hits: ~50 microns

2. Build a pre-calculated lookup table of all coarse paths (“patterns”) that a charged track might take through the tracking layers. For full ATLAS detector, we expect about 1 billion such patterns

Monte-Carlo tracks

3. Load these patterns into specialized hardware – Associative Memories – that can simultaneously compare the event with ALL stored patterns and quickly return only those that match.MATCH!


9

FTK approach: track fitting4. Restore full-resolution hits inside each matched pattern.

Ultimately, we need to create a list of tracks with:• χ2 (track quality)• Curvature (~ 1/pT)• Phi• Impact parameter,

etc

5. Because our patterns are narrow, we have very few hits inside of them.The remaining combinatorial problem can be solved via the brute-force method – i.e., trying out every combination.

6. For each combination, perform a linearized fit to arrive at final track parameters.Since these fits involve only scalar products, they can be performed VERY quickly in modern FPGA chips: 1 fit / ns

Matched pattern from previous stage along with all of its hits

Pre-computed constants

hit coordinatestrack parametersand χ2 componentsThis pattern has two combinations:

Finally, we apply a χ2 cut to remove bad tracks, perform duplicate removal, and send all final tracks to LVL2 trigger.


10

Simulation – challenges

6/4/2012

If we could simulate FTK very quickly, we wouldn’t need to build it – just stick the software algorithm into LVL2 trigger!

In reality, FTK simulation is very challenging: Associative Memories need a lot of RAM to store 1 billion patterns CPUs are not well-suited to perform millions of track fits → each event must be simulated in ~100 separate jobs

Requires unprecedented parallelism not seen anywhere else in ATLAS! To simulate 1 million FTK events for 2016-2017 luminosity

profile, we needed about 500k PC-hours This would take ~50 years to simulate on a single PC

We wrote an advanced simulation framework that fully exploits the parallelism afforded by the HEP grid. Simulation time reduced to about 1 week


11

Simulated FTK performance

6/4/2012

FTK performance is comparable to offline reconstruction However, FTK hardware is 3 orders of magnitude faster!

Per-event time: ~25 µs for FTK, 10-100 ms for offline

Reconstruction of track impact parameter (D0) is particularly important to identify jets with a B-hadron.

D0

12

Example: FTK selects W boson events

6/4/2012Anton Kapliy - Sugarman Award Presentation

Identification of W→µν decays becomes challenging as the number of interactions per collision increases: Electromagnetic calorimeter isolation used in the trigger

becomes powerless due to other “stuff” contaminating the event Track isolation using FTK tracks restores good efficiency

Add FTK tracks


13

FTK architecture

6/4/2012

FTK operates in 64 θxϕ towers divided between 8 core crates.Each crate consists of a number of boards, each with a dedicated function.

HOLAs

Finished and commissioned by ChicagoInstalled @ CERN

Design started @ Fermilab

Prototypes @ Italy

Design started @ Argonne

Design started@ Illinois

Design + prototypes @ Chicago

14

First FTK installation @ ATLAS

6/4/2012Anton Kapliy - Sugarman Award Presentation

to FTK

HOLA: a high-speed (2 GBps) link that provides a copy of all tracker hits to FTK

to ATLAS

Development & testing@ Chicago

FPGA

Highlights:• Data transmission protocol similar to IEEE 802.3

(Ethernet)• Transmission medium: optical fiber• Flow control from downstream cards on both

channels• If this card fails, ATLAS loses the tracker

• → extensive stress-testing to ensure reliable operation

• Produced 270 cards (total system bandwidth ~50 GB/sec):• 268 passed our stress tests; 2 had a soldering

problem• 31 installed at CERN and are being used now

Installation & commissioning@ CERN


15

Conclusions

6/4/2012

As LHC luminosity increases, it will be essential to efficiently select interesting physics from the vast background of boring events

Tracking plays a key role in identifying isolated leptons, b-quarks, and taus

FTK brings the power of tracking-based event selection to LVL2 trigger

Schedule: Tested board prototypes & TDR – by Summer 2013 Full system ready after the 2013-2014 LHC shutdown

FTK is a challenging, exciting and very educational project: Development and simulation of a complex trigger system State-of-the-art electronics development, testing, and

commissioningTHANK YOU!

ftk: a fast track trigger for atlas

Documents

tracking system

lvl1 hardware

tracking layers

atlas detector

fast track trigger

sct tracking detectors

ftk collaboration

event rate