active buffer status in cbm daq w. gao, a. kugel, r. männer, g. marcus, m. stapelberg, a. wurz 06...
Post on 29-Jan-2016
218 Views
Preview:
TRANSCRIPT
Active Buffer Status in CBM DAQ
W. Gao, A. Kugel, R. Männer, G. Marcus, M. Stapelberg, A. Wurz
06 Oct 200914th CBM Collaboration Meeting
Split
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 2
Agenda
• Test set-up and traffic classes• Hardware preparation
– ABB DMA logic improvement– Larger Event Buffer: DDR FIFO– Optical Link (Frank, CA Group, ZITI)
• Software preparation– ABB Daemon– Misc. test programs
• Performance tests
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 3
Test Set-up
ABB2
ROCs
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 4
Traffic Classes in Link Protocol
• DAQ– Hit data packet– Uni-directional: downstream– Major data flow
• CTL– Commands and responses– Bi-directional
• DLM (Deterministic Latency Message)– Timing purpose– Synchronization
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 5
ABB2 Block Diagram
DDR2SFP-0
SFP-1
ABB2
4x PCIe
V5LX110T
LP
FIFOwrapper
DMAFibreBRAMFIFO
BRAM
DGen IGen
xbar
(CAG)
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 6
DMA Upgrade
• Interface upgrade– 32-bit to 64-bit– Back-to-back transactions
• BAR[2] access bugs• Time-out’s
– DMA time-out: does nothing– Event Buffer time-out: padding with -1– Tx arbitration time-out: padding with -1
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 7
DDR FIFO
• Reason– 128 KB BRAM FIFO is a little too small to avoid data loss
• Size– 256 MB = 64 bits × 32M
• Peak bandwidth– 8B × 2 × 125 MHz = 2 GB/sec
• Behaves as standard FIFO (not FWFT yet)– Reset takes longer time (3.6 ms) than normal FIFO
• Data count issue– Disagree between count and empty flag crashed the host
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 8
Optical Link Module(CAG, ZITI)
• 2.5 Gbps / link• Both for ABB2 and ROC• Protocol
– packet-based
• Combining 2 links outputs in ABB2– Additional XBAR module
for ABB2
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 9
ABB Daemon
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 10
ABB Daemon (cont’d)
• Uses UNIX IPC mechanism• A transparent replacement for direct driver usage • Control requests have a higher priority than data transfer• Takes care of the concurrent access of the device by
multiple software pieces • Provides a high-level interface of network messages for
the ROC– PUT, GET, NPUT, NGET, …
• Provides a DMA interface for the FIFO read• Minimal overhead
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 11
and, with all these …
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 12
Commissioning Tests• (nXyter –) ROC – DCB – ABB2 – ABB Daemon – upper-level test programs• Data generators available at two positions
– ABB2: without optical links– ROC: with optical links
• Loop-back– inside ABB2 before asymmetric DAQ channel– inside DCB: CTL verification in this way
• Traffic class tests status– CTL: no error– DAQ: no error. Much more effort than CTL– DLM: primarily tested
• 2 ROCs are also tested– Connected to ABB2
• Software support– PCI Driver + MPRACE Library + ABB Daemon + testNXyter
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 13
CTL Message Latency Tests
Write-Read Read
Latency (mSec) 0.138 0.053
• Software layer• 2 tests are made
– Read latency– Read-after-write latency
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 14
DAQ Performance
DMARead Perf.
DDR FIFO
(large)
BRAM FIFO
(small)1-ROC 272.7 MB/s 199.2 MB/s
2-ROC 308.1 MB/s 305.8 MB/s
• ROC DGen data source– Instead of nXyter format
• Software layer– Instead of data-link layer
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 15
Status Summary
• Hardware upgrade– DMA transportation– 256 MB DDR FIFO– CAG.ZITI: Optical link + XBar
• Software– ABB Daemon– nXyter-format data source DMA read test
• “United” tests in stability– DCB-ABB2 for CTL robustness test– ROC-ABB2– 2 ROCs - ABB2
06.10.2009 CBM[14] Split
Wenxue Gao ZITI.Uni-Heidelberg 16
End of Talk
top related