debugging of m2-m5 g. passaleva. 11/02/2009 muon system mini review g. passaleva 2 outline short...

Debugging of M2-M5

G. Passaleva

11/02/2009 Muon System mini review G. Passaleva 2

outline

Short summary of Muon readout channel organization

Connectivity tests, timing tests Status of debugging since Oct. 2008 Plans for the coming weeks Summary


Muon channel organization Physical channels (PC) are grouped in logical OR

to form Logical Channels (LC). Time of LC is measured by the SYNC chip inside the ODE

Logical channels are those stored on disk and sent to L0

They are crossed at software (L0) level to get the Logical Pads (LP) i.e. the muon hits

In some regions LCs are directly grouped into LPs

Logical channel

Logical channel

Logical pad Physical channels


Muon channel organization

Sync channel

Sync channel

Delays can be adjusted:1. at physical channel level inside the DIALOG

chips

2. at sync channel level

3. at ODE level (sync to sync synchronization)

1)

2)

2)

3)


The Muon Detector readout

Data chain& Tell1 boards

C. D

epla

no


Principle of the testsAll the physical channels are masked and then unmasked and pulsed one after the other. The corresponding logical channels are read out

Sync channel

Sync channel


Connectivity: FE to ODE Connection Complexity

12 different type of electronic chain connections (up to ~1k Front-End CH, ~200 Read-Out CH and 48 muon chambers)

Test Connectivity using noise: On Front End

set low thresholds: noisy signals On Read Out

Look if the noise histogram is filled in the right read-out channel, and only there!

About 99% of the channel are well connected and correctly reachable via PVSS control software

C. Deplano 23 Sept 2008 8

C. D

epla

no


Time Alignment using the FE pulse system (FE to ODE)

DIALOG

ASD

MWPC

FEE

ASD

to ODEfrom PDM

PDM: pulses to FEE synchronous to a given BX number • Use LHC Clock• Pulse reproduced by

DIALOG and sent to CARIOCA

• Chambers are not involved

• Pulse distribution cables length compensated

C. Deplano 23 Sept 2008

Time Measurements done on the ODE board (SYNCs).The delay parameters are calculated and

saved intoPVSS recipes

C. D

epla

no


Status of debugging since Oct. 2008 After the cancelation of LHC run we starteda systematic campaign of

debugging to fix the problems left open

October/November C. Deplano + GP; December F. Dettori + A. Sciubba and M. Palutan; January: GP

30/10/2008 20/12/2008 Today

Tot log. channels

Bad log. channels

fraction Bad log. channels

fraction Bad log. channels

fraction

Side A 8352 390 4.7% 322 3.9% 277 3.3%

Side C 8352 257 3.1% 208 2.5% 128 1.5%

Total 16704 647 3.9% 530 3.2% 405 2.4%

Fixed 117 18% 242 37%


The actual status The remaining problems need careful interpretation of

the tests and intervention on the hardware

Typically the problems are in the chain dialog (IB) TB

ODE

Debugging time is hard to predict… but is never short !

Access and modify hardware

Test the single part

If successful re-test the whole hardware unit

…hope that no new problem show up…


connectivity timing bad Sync ChannelsQ1 M2 R1 HU1 noisy channel 1Q1 M2 R2 HU1 Chamber Chamber 48Q1 M2 R2 HU2 warn Verificare L0 2Q1 M2 R3 HU2 noise noisy channel 3Q1 M2 R3 HU1 chaos ! to be verified 7Q1 M2 R4 HU2 Chamber Chamber 1Q1 M3 R2 HU2 conn missing channel! Expected in timing 1Q1 M3 R3 HU2 timing change ODE ? 56Q1 M4 R3 HU1 connectivity (expected) conn 3Q1 M4 R4 HU1 connectivity (expected) conn 8Q1 M5 R1 HU1 noisy channel noise connectivity (expected+unexpected) 11Q1 M5 R4 HU1 timing change sync 8Q2 M2 R2 HU2 noise VERIFY NOISE 11Q2 M2 R3 HU2 Chamber Chamber 35Q2 M2 R3 HU1 conn expected in timing 2Q2 M2 R4 HU2 CONNECTIVITY conn CONNECTIVITY 1Q2 M3 R2 HU2 conn CONNECTIVITY 48Q2 M3 R2 HU1 noise confirmed noisy FE channels in 16A3 3Q2 M4 R2 HU1 timing problems on CMB 13A1 16Q2 M4 R4 HU1 timing was OK before PATCH ! 2Q2 M5 R3 HU1 timing problems were different before PATCH 10Q3 M3 R3 HU2 noisy channel noise 1Q3 M3 R4 HU2 noisy channel noise CANALE RUMOROSO ACCERTATO 3Q3 M4 R2 HU1 conn CONNECTIVITY 1Q3 M4 R3 HU1 conn expected;additional timing problem after PATCH 3Q3 M4 R4 HU1 conn CONNECTIVITY 7Q3 M5 R1 HU1 Chamber Chamber 96Q3 M5 R3 HU1 conn was OK before PATCH ! 5Q4 M4 R3 HU1 connectivity (unexpected) noise VERIFY NOISE 7Q4 M5 R1 HU1 conn expected in CT 2Q4 M5 R3 HU1 expected in timing timing problems were different before PATCH 3

405


M2

Fixed

problem

chamber

Channel m

aps Sep. 2008


M3

Fixed

problem

chamber

Channel m

aps Sep. 2008


M4

Fixed

problem

chamber

Channel m

aps Sep. 2008


M5

Fixed

problem

chamber

Channel m

aps Sep. 2008


Debugging databaseThanks to G. Graziani we have set up a database to save the history of all the tests we are doing, including diagnostic reports, timing tables and TDC histograms produced by the ECS tests. This helps our experts to spot the problems and get indications on where to intervene on the hardware


Plans for the debugging All the problems have been categorized

Will attack first the cases were a big number of channels are

lost.

Tests are very time consuming: connectivity tests require

several hours per ODE; timing tests about 1 hour

Very complex system: to spot a problem we have to

investigate many boards, cables etc.

Four problematic chambers. Need to access them. However:

problematic chambers in M2 front(2) can be accessed without

opening the walls. The other two need opening


Plans for the debuggingIn the next months a task force will be dedicated full time to this task:

Francesco Dettori (Cagliari) full time

GP > 50% (10 weeks from now to june); Matteo Palutan (from March, t.b.q)

Remote assistance by the experts (Sandro, Adalberto)

Actively looking for another full time person

Contribution from other people coming for shorter time

Next week Adalberto will be at CERN for a full week of hardware debugging.

Support from a STRONG group of people very expert on the hardware and on the ECS system:V. Bocci, S. Cadeddu, M. Carletti, C. Deplano, M. Lenzi, R. Nobrega, D. Pinci, A. Sciubba

In particular the ECS group is actively working to make our software more robust, faster and well organized


Plans for the debugging

In principle, (some) residual dead regions can be “faked” at L0 level. Since the spatial correlation is small, ghosts should not be a big problem

HLT: with some care (performance, ghosts, efficiency !) dead channels can be faked also there

Software studies in progress. Muon ID: a lot of work already done to study a looser

muon selection How to deal with dead regions (Condition DB etc) Online monitoring of efficiency


Summary

Debugging started Nov 2008: 37% bad channels recovered

Still missing 2.4% of the channels mainly concentrated in a few large groups

Four chambers with problems: 2 of them can be accessed without opening the walls. The other 2 need opening. To be decided

debugging of m2-m5 g. passaleva. 11/02/2009 muon system mini review g. passaleva 2 outline short...

Documents

physical channel level

noisy fe channels

logical channels lc

right readout channel

corresponding logical

odelogical channels

16a33q2m4r2hu1timing

2q2m5r3hu1timing problems