a readout system-on-chip for a cubic kilometer submarine neutrino telescope
DESCRIPTION
S.Anvar , V.Gautard , H.Le Provost , F.Louis , K.Menager , Y.Moudden , B.Vallage , E.Zonca , on behalf of the KM3NeT consortium 1 IRFU/SEDI-CEA Saclay F-91191 Gif-sur-Yvette, France 1 Supported by the European Commission through FP6 and FP7. - PowerPoint PPT PresentationTRANSCRIPT
A READOUT SYSTEM-ON-CHIP FOR A CUBIC KILOMETER SUBMARINE
NEUTRINO TELESCOPE
S.Anvar, V.Gautard, H.Le Provost, F.Louis, K.Menager, Y.Moudden, B.Vallage, E.Zonca, on behalf of the KM3NeT consortium1
IRFU/SEDI-CEA SaclayF-91191 Gif-sur-Yvette, France
1 Supported by the European Commission through FP6 and FP7
TWEPP-11, Vienna, Austria, 26-30 September 2011
The KM3Net Detector
Digital Optical Module (DOM)
31 PMTs/DOM
12800 DOMs
320 lines
http://www.km3net.org
Technical Design Report (ISBN 978-90-6488-033-9) Conceptual Design Report (ISBN 978-90-6488-031-5)
The DOM processor board
HV PMT base
Signal collection board
Processor board & Softwaredesigned at CEA-IRFU
Power conversion boardHeat conductor
PMT
DOM designed at NIKHEF institute (Amsterdam, Netherlands)
RTOS
DDR2Memory(64 MB)
FlashMemory(32 MB)
FPGABitstream
/Processor
Boot
The KM3Net prototype Offshore Processor
Board
PPC440 processor
SlowControl
SlowControl
Task
Slow-Controlfor the Storey (I2C, SPI)
1Gb/s Ethernet LinkTo shore station
DataTask
Data
ReadoutLogic
Readout System On Chip (RSOC) / VIRTEX-5
SC ProtocolLogic
Clock/CommandExtraction
PMTs
x 31 small PMTs
31x
1 GHzTDC
31 TOT Data
Firmware TDC Development
1 GHzSampling
PLL500 MHz
/500 MHz
125 MHz
RecoveredETHERNET
Clock@125MHz
XILINX VIRTEX-5 ISERDES
Readout Logic@125 MHz
‘1’ Detection&
Coarse Time Stamping
1 1
1
0 0
0
0 0
0 0 1 1 1 0 0
TOT LVDS PMTasynchronous input
8-bit
0
Packetized and Sent over ETHERNET
TDC #16-bitCoarse Time Stamp #24-bitEvent type/TDC number
[Originally designed by A.Zwart (NIKHEF) / small PMTs test bench for ALTERA]
Clock distribution
Transmit Reference Clock @62.5 MHz
STANDARD1000BASE-XETHERNET
Phase and Latency
Measurements
Command (CMD) insertion
Custom Logic
On Shore VIRTEX-6 Board
IDLE CMD DATA
Command(CMD) extraction
Custom Logic
Off Shore VIRTEX-5 RSOC
1.25 Gbps
PPC440 Bus
Recovered Clock @62.5 MHz / Bit slice analysis (known latency)
CMD
IDLE
DATA
Event Time Stamping
Embedded62.5 MHz
Transmit/Receive clock skew
0 2000 4000 6000 8000 10000 12000 14000 160004.4
4.5
4.6
4.7
4.8
4.9
5
5.1
5.2
5.3
Time since cold start in seconds
Tx/Rx clock skew measured on shore with DMTD and oscilloscope (ns)
DMTD (Dual Mixer Time Difference)
Scope
Store & Forward Acquisition model
31 TDC
&
RoutingMatrix
Dynamic Memory
AcousticEvent
Format(128 bytes)
TDC event
format(6 bytes)
CircularBuffer
FIFO 0
PMT 0-31
PPC440 processor@ 300 MHz
1 Gb/s Ethernet Port
134 ms Data
Time Slice To Computer 1
Time Slice To computer 2
Interrupt@134 ms(Typical)
Slow control (Configuration, Readout on requests)
134 ms Data
Detector Clock(ETHERNET)
Slow ControlObjects
FIFO 5
DAT
A RO
UTE
R
ReadoutLogic
Acoustic Data
FIFO 6
FIFO 0
CircularBuffer FIFO 6
134 ms
134 ms
Time Slice BuildingIntrinsic Parallelism
Performance Parallelism
OffshoreNodes
OnshoreFarm
Multi-gigabit Standard Ethernet Switching
FarmComputing
Clock Distribution/Commands Insertion Detector Clock(Start) Commands
• Track reconstruction• Data routing
EmbeddedComputing
Time Slice 2
Time Slice 1Time Slice 0
Start
NetworkEmbedded
TargetSC Server
RunControl
Data routers / writer
ElectronicsControl & Config (SC & Elec. RC)
EmbeddedTarget
SC ServerCommunication
middleware (ICE)
Communicationmiddleware (ICE)
Electronics control & config
Data acquisition setup
Reference Clock @62.5 MHz
STANDARD1000BASE-XETHERNET
Start Command
On Shore VIRTEX-6
Board• Run Control• Target Configuration• Data acquisition• vxWorks RTOS boot
server
Host PC
Synchronous1000BASE-XETHERNET
RSOC VIRTEX5FX-70• vxWorks RTOS• 1 TDC Channel• 1 dedicated TDC ASIC
channel• PPC440@300 MHz
Bus@75 MHz
Pulsegenerator
Acquisition results
8 ns2 ns 2 ns
Standalone measured TCP/IP throughput PPC440@400 MHz / Bus@100 MHz /WindRiver Zero Copy buffer/Jumbo frames : 988 Mb/s
Current acquisition setup (Source limitation): 60 Mb/s
Pulse @ f=10 kHz
ns ns
Scope measurementMean : 100,10667 usRMS : 1,75 ns
Power : 7 W
CONCLUSION
Common Readout system functions integrated in a single component (RSOC):Event Time stamping @ 1 GHzClock and command distributionSlow-control and data acquisition performed in a RTOS
multi-tasking embedded system
RSOC is a node designed to be plugged in a complete Data acquisition SystemServer/Client topology (ICE)Scalable system