cms/hcal/tridas. sept, 2003 hcal tridas 1 tridas status drew baden university of maryland september...
Post on 19-Dec-2015
217 views
TRANSCRIPT
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 1
Tridas Status
Drew BadenUniversity of Maryland
September 2003
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 2
HTR Status
Dual-LC O-to-E
VME
Deserializers
Xilinx XC2V3000
-4
Sti
ffen
er
s
SLBs (6)
TTC mezzanine
• Rev 3 – 30 boards made in March 2003–Board production changes:
• New assembler, in-house X-ray, DFM review, QC
• Gold plated (Rev 1 was white-tin) for better QC
–Changes to HTR:• Change from FBGA (1.00mm)
to BGA (1.27mm)• Added stiffeners• Moved all SLB/TPG output to
front-panel daughterboards• Modified Rx refclk scheme (the
usual TTC/refclk clocking concerns)
–Full 48 channel capability • Rev 1 in 2002 was “half HTR”
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 3
TTC receiver - TTCumd
• General purpose TTC receiver board (TTCumd)
– TTCrx ASIC and associated PMC connectors
• Will be used to receive TTC signal by HTR, DCC, and clock fanout boards• No signal receivers.
– Copper/fiber receivers must be on the motherboard
– Signal driven through TTC connectors
• Tested successfully by Maryland, Princeton, BU groups
Princeton Fanout Card
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 4
Clocks and Synchronization
• Clocking considerations can be divided into 2 parts:– Deserializers REFCLK: stability critical (80MHz frame rate)
• Stability: must have a very low jitter – 40ps pkpk spec• Frequency: TI TLK2501 spec is 100ppm (8kHz) to lock
– Measured ~350ppm (30kHz) needed to establish link» LHC variation expected to be few kHz
– Once link is established, just needs to be stable (it’s a REFCLK!!!)
• Phase: relationship to LHC clock totally irrelevant
– Phase critical clock for pipeline synchronization• Must be in phase with LHC clock• Jitter spec is less stringent
– For FPGA sequential logic
– For SLB transmission 80ps pkpk
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 5
HCAL Clock Fanout
• HTR clocks provided by a single 9U VME board– Chris Tully/Jeremy Mans from Princeton
– Has fiber TTC input
• Signals fanned out over Cat6 twisted pair:– TTC stream
• To be used by each HTR and by DCC to decode commands & L1A
– BC0• To be used by SLBs to synchronize TPGs
– “40MHz” clock• To be used by FPGA and SLBs to maintain pipeline
– Comes from QPLL
– “80MHz” clean clock• To be used for deserializer REFCLK
– Comes from QPLL
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 6
Clock Distribution
HTR
TTC fiber
TTC
CLK80
BC0
CLK40
distributionto 6 SLBs
and to 2 Xilinx
Brdcst<7:0>,BrcstStr, L1A
O/E
BC0
TTCTTC
TTC
FPGA
..
..
TestPoints forRxCLKand RxBC0
..
..
..
..
80.18 MHz
..
..
..
..
TTCrx
to Ref_CLK of SERDES(TLK2501)
CLK40
CLK80
Princeton FanoutBoard
TTCrx
QPLL
HTRHTR
Cat6Eor Cat7Cable
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 7
May Testbeam Setup
• Lack of a QPLL – we had to improvise:– Front-end used commercial Cypress PLL
• Beats GOL 100ps pkpk jitter spec in the lab
• Terry Shaw measured sufficiently small jitter on backplane
– Fanout card • No clean 80MHz REFCLK, so provided 2 alternatives:
– 2xLHC clock from crystal oscillator– High quality clock from HP signal and pulse generators– Jumper selectable on mezzanine cards
• No clean 40MHz system clock– Just used 40MHz output from TTCrx chip anyway– Worked well enough
» The system clock doesn’t need a low jitter spec, and Xilinx has a DLL to clean it up
– Much struggling with the fiber links, synchronization issues, etc.• Almost all issues
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 8
May 2003 Testbeam
• What we struggled with and learned about– Fiber links
• FE Tx clock critical. No QPLL so we used backup plan.• Great deal of experience gained here.
– TTC• Had a few minor issues, discover how to best synch L1A with pipeline
– DAQ• Production DCC logic board
– Able to operate SLINK64 after BU group discovered bug in documentation– Logic in place for inline error checking, data integrity, etc
• XDAQ implementation experience
– Calorimeter• Took useful calorimeter data!
– HB, HE, HO, HF
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 9
May Testbeam Experience• Fiber synchronization
– FE• With crystal oscillator on FE, virtually no troubles• TTC 40MHz clock cleaned up by Cypress “roboclock” chip (Cy7B993)
– Many synch errors, varied from fiber-to-fiber, hard to maintain link on all fibers over ~few hours time period
– During synchronized beam running, sent reset between spills to ensure link. This seemed to work ok. Exploring using rescynch in abort gap…
– Fanout card• Fanout from onboard ~80MHz crystal oscillator for REFCLK• Fanout TTC 40MHz clock for system clock
– HTR• 40MHz system clock scheme worked
– Some firmware changes necessary to synchronize with L1A
• TLK2501 link circuitry always enabled.
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 10
Fiber Links (cont)• Tests at Maryland using TI eval board and Stratos transmitter
– Link exceedingly stable (see slides below)• Current plan
– Study FE → HTR link at FNAL• FNAL test stand is setup, link tests underway, more to come this fall
– Investigate noise characteristics of H2 environment• H2 is clearly different than FNAL, Maryland (and BU) experience
– Review of HTR and Fanout card• Will learn what we need to do from the above
• Best guess– All tests in US indicate solid link, but experience in H2 disagree– Probably a linear combination of:
• TTC clock jitter (most likely)• Some kind of new noise component (less likely – FE works with crystal oscillators)
• Bottom line:– We have a system that works in our labs (BER<10-14) but…
• H2 environment? Need more studies during the next 6-12 months• Lack of working QPLL. Looking forward to getting correct QPLL/crytals
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 11
Optical Attenuation and BER• “Typical setup”
– VECSEL transmitter, coupled to fiber via LC connector• Not locked, but fixed in place
– Fiber to LC to 8-way MTP male on HTR front panel– Single fiber to LC connector for connection to STRATOS receiver
• Output power:– VECSEL advertised to put out 500W (-3dBm)
• Terry Shaw measured 570W for a particular VECSEL– UMD uses STRATOS LC transmitter
• Advertised output 100-400W (-4 to -10dBM)• Measured to be 90W for a particular STRATOS• About 6dB below what we will use in CMS• Working on FE emulator now using GOL+VECSEL…
• Attenuations measured:– At each LC connector, 10 – 50% (0.5 to 1.5 dBm)– At MTP connector, same thing (.75dB advertised)– Fibers are about ¼ dBM per 100m
FE
VECSEL
LC
MTP (8-way)
LCLCStratos
HTR
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 12
Optical Power (cont)
• What do we need at the receiver to maintain link?– Did a series of measurements with
known attenuator– Varied attenuation, looked at:
• BER• TTL “signal detect” (SD) signal provided
by Stratos part
– Found:• SD signal goes away when power is
below about 2W– Measured 1.5W but accuracy of meter
is probably ±.2W
• BER climbs very fast right at this shoulder
• NB: achieved BER<10-15 with multiple fibers in parallel with crystals
Measured ~5k errors in 10sec
Points with error bars are worst case BER: <1 error
See next slide
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 13
Link Loss Observation
• Ran a test at 7.4W (-21dBm), found 44 errors in 18 hours• Simultaneously triggered scope on VCC to transmitter and
crate• Found that the scope had triggered on transmitter voltage
but not crate– Voltage spike - 1-2V oscillation with ~50ns rise time– Crate was isolated through UPS
• These errors were due to a spike on the A/C line from equipment being power cycled, noise in building, etc.
• Moral: noise in the power/grounds will be our nightmare
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 14
Optical Attenuation (cont)
• Input power required to maintain link:– Measured failure for power < ~2W (-
33dBm)
• Power output by VECSEL:– 500W output
• Divide by 2 for digital averaging• Gives 250W (-6dBm) output at source
• Expected Attenuations– Maximum of 8 couplings until the signal
gets to the Stratos receiver on the HTR• 8x(0.5 - 1.5)dBm = (4 – 12)dBm
– Add another ~1dBM due to fibers
• Total power at inputs to HTRs:– -6dBm – (4-13)dBm = -10 to -19 dBm– FNAL measured/calculated 7.3dB
• Operating would be -13dBm
• We should have at least 10dB margin– Probably more like 15dB
VECSEL
Operating
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 15
Longitudinal Separation Attenuation
• MTP connector ends are spring loaded into adapter
• Measured attenuation as a function of the separation– Separation should be ~0 if keys
and adapters are working well– This should not be an issue for us
(famous last words….)
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 16
Changes to HTR for Rev4
• Moved 2 LC’s down to giver more clearance for fibers– Upper rear of card
• Spread out routing of differential pairs for 6 SLB and 2 FPGA system clocks
• Removed hot swapping circuits– Worry about noise, decided not to
require HTR to be hot swappable• Front-panel changes
– Rotary switch, LEDs, eliminate REFCLK mux via jumper, etc
• Miscellaneous changes– Fixed what was found to be wrong with
Rev3 board, add test points, other minor stuff
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 17
Project Timeline
2003 2004
Firmware TB 03 firmware should be adequateBoards Can use current crop of Rev3
To do: Commission QPLL, global clocking. Level 1 Trigger: SLB, TPG, latency…
“Vertical Slice” (~March)
2005
HB, HE, and ME plus Level 1/TPGFiber synchronization Check in alcove…compare environmentsFirmware TPG firmware has to be ready. Fall 03 task.Boards Must have Rev4. How many TBD.
FE Commissioning (~Feb)
“Magnet” Tests (no HCAL)
CMS Magnet Integration System TestBoards Will have Rev4.
H2 Testbeam
Fiber synchronization Check in H2…compare environmentsBoards Will have Rev4. How many TBD.
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 18
HTR Firmware
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 19
TPG Path
• Still under development– The following is already coded/simulated but not tested in HTR
• Things to do (not necessarily in order)– Linearize QIE data to 10 bits
• With .5GeV resolution gives 512 GeV max
– Apply BCID filter• Probably will sum over 2 buckets and assign based on high/low patter
around the bucket that has the max energy
– Sum or divide depending on HB, HE, or HF– Extract a muon window for the “feature bit”– Apply logic to eliminate false muons from shower leakage, etc.– Compress and send to SLB
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 20
TPG Path Schematic
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 21
TODO - TPG
• HTR production can begin after:– SLB/HTR connectivity check
• Logic analyzer card on SLB site already shows connectivity
• Need to plug in real SLBs to check that they can coincide
– SLB/HTR/Wisconsin check• Check that we can maintain link• Will bring the setup from Princeton to
Maryland this month and do the tests. Measure BER, etc.
• Data validation– We propose to build a 6U VME board
with sites for the Wisconsin Vitesse receiver boards
– Will fifo data and read out over VME– Use this to check data validity, try
different TPG tests, etc.– Plan to have this board ready in a few
months. Will use this for the early slice tests to test TPG output.
FPGA
Clock Input
CMS/HCAL/TriDas. Sept, 2003 HCAL TriDAS 22
TODO (cont)
• Firmware– Focus on L1 latency optimization
• Measure full latency, scheme for random TLK latency, how to meet L1 latency budget, etc.
• This will be the main activity from now until we go into production.
– Bells and whistles for error reporting/recover• Meeting next week in Princeton to finalize what we learned in 2003• More to be learned in 2004
• HTR boards– Make Rev4 this fall, go into production early 04
• ASAP given schedule• Depends on results from above