routing and streaming in the hlt tdaq week sander klous wednesday, may 17, 2006

Routing and Streaming in the HLT

TDAQ Week

Sander Klous

Wednesday, May 17, 2006

LVL2RoiB

L2sv

L2pu

Ros/Robin pRos

DFM

SFI (partially) build

Input

ExtPT

ExtPT

Trash

Output Output

EFD

B

CD

Ath / CalStr

Ath / CALIB

PT

SFOStream

selection

StreamTag scenarios (UC01) Stream nStream 1 Stream 2

From LVL1

Ath / CALid

Ath / PESA

PT

Event

LVL1 Info

• Use Case 01– Events triggered by LVL1– Bypass LVL2 (force accepted by L2SV)

• Based on LVL1 info• Based on statistics• Based on L2pu timeout

– RoutingTag build by L2SV– Event and StreamTag build by SFI– Bypass EF

CreateRoutingTag

CreateStreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

• Use Case 02/03/04– Events triggered by LVL1– RoutingTag created by L2SV– RoutingTag info added by L2PU

• For UC02: a force accept by L2PU– Accepted by LVL2– Event (partially-UC04) and StreamTag build by SFI– Bypass EF

Input

ExtPT

ExtPT

Trash

Output Output

EFD

B

CD

LVL2RoiB

L2sv

L2pu

Ros/Robin pRos

DFM


SFOStream

selection

Ath / CalStr

Ath / CALIB

PT

Ath / CALid

Ath / PESA

PT

From LVL1

Event

LVL1 Info

CreateRoutingTag

Add toRoutingTag

StreamTag scenarios (UC02/UC03/04) Stream nStream 1 Stream 2

CreateStreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

Input

ExtPT

ExtPT

Trash

Output Output

EFD

B

CD

LVL2RoiB

L2sv

L2pu

Ros/Robin pRos

DFM

SFI builds event

SFOStream

selection

StreamTag scenarios (UC05-B) Stream nStream 1 Stream 2

Ath / CalStr

Ath / CALIB

PT

Ath / CALid

Ath / PESA

PT

• Use Case 05-B– Events triggered by LVL1– Accepted by LVL2– RoutingTag info added by L2PU– Event and StreamTag build by SFI– Accepted by EF– StreamTag info added by EF– If necessary, stripping done by outputTask

From LVL1

Event

LVL1 Info

CreateStreamTag

Stripping

Add toRoutingTag

CreateRoutingTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

Add toStreamTag

Input

ExtPT

ExtPT

Trash

Output Output

EFD

B

CD

LVL2RoiB

L2sv

L2pu

Ros/Robin pRos

DFM


SFOStream

selection

StreamTag scenarios (UC05-C) Stream nStream 1 Stream 2

Ath / CALid

Ath / PESA

PT

• Use Case 05-C– Events triggered by LVL1– Accepted by LVL2– LVL2 RoutingTag info added by L2PU– Event and StreamTag build by SFI– Accepted by EF– StreamTag info added by EF– If necessary, stripping done by outputTask

From LVL1

Event

LVL1 Info

Stripping

Ath / CalStr

Ath / CALIB

PT

Add toRoutingTag

CreateStreamTag

CreateRoutingTag

Add toStreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

Input

ExtPT

ExtPT

Trash

Output Output

EFD

B

CD

LVL2RoiB

L2sv

L2pu

Ros/Robin pRos

DFM


SFOStream

selection

StreamTag scenarios (UC06/07) Stream nStream 1 Stream 2

Ath / CALid

Ath / PESA

PT

• Use Case 06/07– Events triggered by LVL1– Accepted by LVL2– LVL2 RoutingTag info added by L2PU– Event and StreamTag build by SFI– Accepted by EF– StreamTag info added twice by EF– If necessary, stripping done by outputTask

From LVL1

Event

LVL1 Info

Add toStreamTag1

Stripping

Ath / CalStr

Ath / CALIB

PT

Add toStreamTag

CreateStreamTag

Routing

Note for UC05/06/07 If PT times out, ExtPTs can force accept the event. This event does not contain EF StreamTag information, nor did the LVL2 RoutingTag mark it to “Bypass EF”.

Duplicating

Routing

(Partial) Event

LVL1 Info

RoutingTag

StreamTag 1

Event

LVL1 Info

RoutingTag

StreamTag 2

Event

LVL1 Info

RoutingTag

StreamTag 2

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag 1

Add toStreamTag2

3/15/2006 StreamTags in the HLT 7

Use Case analysis

• The definition of the words “TriggerResult”, “RoutingTag” and “StreamTag”.– RoutingTag and StreamTag are not the same as TriggerResult. – The TriggerResult contains information about which triggers fired.– The “RoutingTag” describes the Route of an event in the HLT.– The “StreamTag” describes the destination(s) stream of an event.– Of course, all three are highly correlated.

• Requirements– An event should never end up twice in the same stream.

• Hence, the StreamTags of two instances of the same event should be mutually exclusive.

– The TriggerResults of two instances of the same event should be identical.

• Design implications– Each level guides the next active level in routing/streaming.– Reserved header bits for routing and streaming information.– Duplication of events is only possible in the last stage of the EF.


SFO

A generic StreamTag generation scheme

LVL2SV

EFD

PT2

UC01 UC02/03/04

UC02/03/04 andUC05/06/07

All UCs

LVL2PU

UC05 andUC06/07

UC06/07

UC05 B/C UC06/07

Event Header8 bit LVL1 Trigger Type

32 bit LVL2 Trigger Info

4 x 32 bit EF Info

From ATL-D-ES-0019, raw eformat V3

LVL1

PT1 or PT2

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

(Partial) Event

LVL1 Info

RoutingTag

StreamTag

Event

LVL1 Info

RoutingTag

StreamTag 2

(Partial) Event

LVL1 Info

RoutingTag

StreamTag 1


Current specificationsTrigger classes

• Calibration (missing)• Bypass LVL2 (missing)• Bypass LVL2PU• Calo / muon• e/, jet, E• Low pT muons• CTP priority level

Table 11-19, page 392

Missing functionality:• Possible duplication warning (1 bit).• Bypass EF (1 bit).• LVL 2 StreamTag version number (4 bits).• Trigger class (16 bits).

– Table 11-20, page 393, physics TDR.• Might choose similar construction as LVL1, where few

bits control the meaning of the other bits.

Bit Use

0-7 LVL1 Trigger Type (copy)

8-15 Debug type (to be defined)

16-23 Partial EB (Subdetector ID)

24-26 Not used (0)

27 Debug event (Yes/No)

28 Calibration event (Yes/No)

29 LVL1 Calibration (Inverse of MSB)

30 Partial event building (Yes/No)

31 Physics event (Yes/No)

Bit Use

0-6 Trigger class (to be defined)

7 Physics or Calibration



4 x 32 bit EF Info8 bit LVL1 Trigger Type (from draft proposal N. Ellis)

Physics

Other (i.e. Calibration or Test)

32 bit LVL2 Trigger Info(from ATL-DQ-ES-0076)

Bit Use

0-2 Trigger class (to be defined)

3-6 Sub-detector identifier

7 Physics or Calibration

Specification will change to 64 bits.Needed for partial event building.


“128 EF BitPattern” (EF status bits) and internal EF routing

64 bits filled by PESA Athena algorithm:

bits of “fired” (0 or 1) TriggerElementsaccording to TriggerMenu in TrigConfDB

AND some PESA Steering-specific errors(not “visible” outside PESA)

e20i mu60i j250

32 bits filled byCALIB Athena algorithm:

“fired” bits (0 or 1) for typesof calibration according to DB

AND errors specific to theCalibration Athena algorithm

32 bits filled byPSC/DF (outside Athena)

various PSC/DF errors, etc.according OKS ConfDB

muon Calibration LAr Calibration Null pointer

e20i

mu60i

j250

muon Calibration

LAr Calibration

Null L2Result

Null EFResult

Error A Error B

Error A

Error C

a FEW DF/Routing Answersas summary of all given set of StatusBits,For example:“Good physics”“Calibration”“Various COMBINATIONS” (e.g. good physics and calibration)“VARIOUS kinds of Error/Debug/ForcedAccept” (MANY !!)

GENERAL IDEA:Mapping from StatusBits to Answers

… and yet ANOTHER mapping from Answers to Routes !

Not distinguishTRIGGER MENUdetails to avoidconnect to TrigConfDB

BUT be able to usePESA error code bitswhich CAN be in OKS


Possible Use Cases/Combinations of StatusBits/Answers/ROUTES:P = “PhysicsAccept” (at least one TE bit “fired”, no Err) ROUTE “Physics” (no 2nd CalPT)C = “Calibration Accept” (at least one Calibration bit “fired”, no Err) ROUTE “Calibration” (2nd CalPT)

B = EF Bypassed (Bypass from L2, no any processing in EF) corresponds to ROUTE “P” with

flag Bypassed in StatusBitsF = Various Types of Forced Accept (from PESA/PSC/OKS) corresponds to ROUTE “P” with

additional details in StatusBits patternE = ANY Error (in PESA, in Calibration Algorithm or in PSC/PT) corresponds to ROUTE “P” with

flag “DEBUG” in StatusBits pattern(F & E = different Use Cases/streams !!)

all possible useful combinations: P & !C & !E & !F = “pure” P!P & C & !E & !F = “pure” C

2=“both” P & C & !E & !F = ROUTE “P+C” (send both P&C events to 2nd ExtPT task)

N !P & !C & !E & !F = nothing needed in this event TrashTask

F & !E & any P & any C = corresponds to ROUTE “P” with details of Forced Accept in StatusBits

E & any F & any P & any C = corresponds to ROUTE “P” with flag “DEBUG” in StatusBits

So, (algorithm) ANSWERS can be: P, C, 2, E, F, N, B

For ROUTING we do NOT need details(i.e. routes “E” = “F” = “P*” with details in StatusBits),

so, the possible ROUTES are: P, C, 2, N, B


PESA Algorithm PT

EF

Bit

Pat

tern

(st

atu

s b

its)

PE

SA

CA

LIB

RP

SC

/ D

F

Fil

l bit

s in

PE

SA

or C

AL

IBR

par

t

EF Answer RoutingTag(from Athena/PSC) (routing/DF path in EFD)

0/1 P=Physics Accept (!Calibration ) P = “pure” Physics (NO 2nd CalPT)0/1 C=Calibration (! Physics Accept) C = 2nd CalPT (“pure Calibration”)0/1 2=Calibration AND Physics accept 2 = P + C0/1 E=Error/Debug any Algorithm or PSC0/1 F=Any ForcedAccept0/1 N=NOTHING needed N = Trash0/1 B=EF bypassed

P.S.C.

Fil

l PS

C/D

F e

rror

bit

s

PLANS: 1. EF Routing: mapping from StatusBits to Answer

SECOND configurable mappingin OKS: Answers Routing

CONFIGURABLE MAPPINGIN OKS: StatusBits Answers

In principle the simplest would be 1-to-1 correspondencefrom Answers to Routes, but we better “move” it from codeto OKS for even more flexibility (see next page for ex.) !!


“128 EF BitPattern” (EF status bits)

Calibration bits from TrigDB (not avail. In DF apps)

PSC/DF bits – all in OKS

e20i

mu60i

j250

muon Calibration

LAr Calibration

Null L2Result

Null EFResult

Status Bits ANSWER ( Use Case ) ROUTE

EF_BYPASS B B (no PT processing)

ANY bit reserved for physics P ( “pure” Physics ) P

PESA Error A F ( Forced Accept type 2 ) P *

Any Calibration bits fired C ( “pure” Calibration ) CCalibration and Physics 2 ( Phys + Calibration ) 2“NONE” (Physics nor Calibration) N P * (Forced Accept type 3) Calibration Error B E ( wrong detector ) N (delete: can not be)

L2Result absent E P * (debug 1)EFResult PESA lost / created PSC E P * (debug 2)EFResult ANY absent E ( not expected ) N (delete: program crash)

Error A

Mapping table in OKS configuration file (no TrigDB access):

Bits for Physics Trigger Menu defined in TrigConfDB and NOT available in DF

Calibration error bits CAN be in OKS (available for DF)

PESA error bits CAN be in OKS and available for DF

e20i mu60i j250 muon Calibration LAr Calibration Null pointerError A Error B Error C


PLANS: universality in the code level

1. Data / PSC Operation results fill corresponding status bits and do not take any action on it (do not touch “answer” or “route”)

2. Configurable mappings “StatusBits Answer” & “Answers Routes” should allow flexibility and avoiding any hard coded routing


2. SFO Streams: mapping from Answer & S.B. to streams

e20i

mu60i

j250

muon Calibration

LAr Calibration

Null L2Result

Null EFResult

Error A

P=Physics Accept (!Calibration)C=Calibration (! Physics Accept)2=Calibration and Physics AcceptE=Error/Debug Any algorithm or PSCF=Any ForcedAcceptN=NOTHING needed (N/A)B=EF bypassed

ROUTING is needed only “internally” inside EF (or L2), so, no need to use it here

1. The set of all defined TriggerElements is mapped to “Logical Streams” in TrigConfDB (HERE we DO access it !!). The particular FIRED (for this event) TriggerElements in Algorithm BitPattern are translated (using map of TrigDB) to “Logical Streams”.

2. The SFO configuration in OKS contains file names of SFO streamsWith additional mapping (in OKS ? In TrigDB ?) we can translate “logical Streams stream file names”

“file” p1

“file” p2

“file” p9

“file” c1

“file” c4

“file” D1

“file” D2

“file” Mon

DAQConfDB(OKS)

TrigConfDBE gamma

muons

jets

Muon Cal.

LAr Cal.

Debug

Express

Monitor

2

1this mappingis also possible- do we need it?


EF Trigger Info

EF Info

(4 x 32 bit words)

1 word overhead

1 word physics streams

1 word other streams

1 word spare

OverheadBit Use

0-7 EF StreamTag version

8 Duplication warning

9 Debug event (yes/no)

10 Calibration event (yes/no)

11 Partial event (yes/no)

12 Physics event (yes/no)

13 ROI/Subdetector stripping



4 x 32 bit EF Info

Physics streams

Bit Use Bit Use

0 Express 16 3j165

1 Electrons 17 4j110

2 Muons 18 jET1000

3 Jets 19 g60i

4 Photons 20 2g20i

5 Missing ET and taus

21 t60i

6 B physics 22 t35i+xE45

7 Luminosity and 0 bias

23 xE200

8 e25i 24 ET1000

9 2e15i 25 j70+xE70

10 e15imu10 26 2mu6+mB

11 e25 27 Other B

12 mu20i 28 Min. bias

13 2mu10 29 Zero bias

14 j400 30 Rom. Pots

15 2j350 31 Other

Other streams

Bit Use Bit Use

0 High pT e/ 16 Sub ID

1 Gen. high pT 17 Sub ID

2 Z di-lepton 18 Sub ID

3 Iso. hadron 19 Sub ID

4 LAr5sample 20 Sub ID

5 Overlap 21 Sub ID

6 Min. bias 22 Sub ID

7 23 Sub ID

8 Monitoring 1 24 ROI ID

9 Monitoring 2 25 ROI ID

10 ? 26 ROI ID

11 ? 27 ROI ID

12 ? 28 ROI ID

13 ? 29 ROI ID

14 ? 30 ROI ID

15 ? 31 ROI ID

•Overhead is from various emails/meetings•Physics is from data stream meeting:http://agenda.cern.ch/fullAgenda.php?ida=a06860•Other is from Hawkings/Gianotti document


Summary

• Specification of Routing and Streaming in the HLT is completed (after a lot of long discussions).

• DataFlow applications do not need access to TrigDB.• OKS does not need knowledge about Trigger Scheme• No hard coded relations, all routing and streaming is

done through configuration schemes• Implementation details under discussion

– E.g. bit patterns

• Write up to be completed (ATL-DQ-ES-0076)• Implementation needs quite some work

routing and streaming in the hlt tdaq week sander klous wednesday, may 17, 2006

Documents