the application of daq-middleware to the j-parc e16 experiment e hamada 1, m ikeno 1, d kawama 2, y...
Post on 28-Dec-2015
219 Views
Preview:
TRANSCRIPT
The Application of DAQ-Middleware
to the J-PARC E16 Experiment
E Hamada1, M Ikeno1, D Kawama2, Y Morino1, W Nakai3, 2, Y Obara3,
K Ozawa1, H Sendai1, T N Takahashi4, M M Tanaka1, S Yokkaichi2
1High Energy Accelerator Research Organization (KEK)2RIKEN Nishina Center
3The University of Tokyo4Research Center for Nuclear Physics, Osaka University
2015/4/14 CHEP2015 1
Track 1 Session: #3(Data acquisition and electronics)April 14, 2015 18:00 – Village Center
Outline
1. DAQ-Middlewareoverview and features
2. DAQ System at E16 Experimentrequirements and architecture
3. DAQ System Performancemethod and result
4. Summary and Future Plan
2015/4/14 CHEP2015 2
1. DAQ-Middleware
2015/4/14 CHEP2015 3
What DAQ-Middleware Is
• A framework for network based DAQ software– Easy to use, configure and develop
• Target– Medium-scale experiments– Test benches (sensors, electronics)
2015/4/14 CHEP2015 4
DAQ-Middleware Architecture
• Develop DAQ system by configuring DAQ components
2015/4/14 CHEP2015 5
DispatcherGatherer
・・・
Read-out-modules
PC
PC
XML
System Configuration
DaqOperator
Logger
Monitor
Command / Status
software unit that achieve DAQ functions
DAQ Component
2015/4/14 CHEP2015 6
InPort OutPort
Service Port
(command/status)
Logics
(for data handling)
Data
InPort OutPort
Service Port
(command/status)
(for data handling)
Data+ =DAQ-Middleware
provides users writes
Logics
DAQ Component Features
2015/4/14 CHEP2015 7
READOUTREADOUT
READOUTREADOUT
READOUTREADOUT
READOUTREADOUT
Flexibility
Reusability
Scalability
logic
Ring Buffer
・・・
Users can flexibly change DAQ component combination.
Users can improve performance by adding new PCs and deploying DAQ components.
Users can use a DAQ component in various DAQ system.
DAQ component has a ring buffer. Users do not need to implement a buffer.
List of DAQ-Middleware Users
2015/4/14 CHEP2015 8
DAQ-Middleware Working
J-PARC MLFNeutron beam lines
• Experiments [19 experiments]- Material and Life Science
- J-PARC MLF Neutron/Muon
- DAQ system of Depth-resolved XMCD (KEK PF)
- Elementary Particle/Nuclear Physics
- J-PARCE16 Experiment
- CANDLES
- SuperNEMO (planning)
• Test benches for sensors and electronics [9 test benches]–Liquid Argon TPC
–SOI Pixel Detector
–ILC CCD Vertex
–ADC-SiTCP
・ 2D gaseous detector with readout ASIC using printing technologies
and so on….
2. DAQ System
at E16 Experiment
2015/4/14 CHEP2015 9
Requirements
2015/4/14 CHEP2015 10
660MBdata
receive
660MBdata
receive
notreceive
1spill
notreceive
Trigger rate fluctuates due to beam rate variation.Event size per one event is almost constant.
Estimation of data transfer to DAQ PCs
Data rate per spill 660MB/spill
Trigger rate (average) 1kHz
(max) 2kHz
Instantaneous data rate
(average) 330MB/s
(max) 660MB/s
System Architecture
2015/4/14 CHEP2015 11
Function ・ Event build・ Analyze data & show the result
Read-out-module Network switch Storage PC
Network switch Monitor PCHDD
HDD
HDD
Function ・ Read data・ Save data on each HDD・ Send data of a part of event to Monitor PC
・・・・・・
DAQ Component Configuration
2015/4/14 CHEP2015 12
Storage PC Monitor PC
・・・
Gatherer
Merger Dispatcher
Logger
Filter
Merger Eventbuilder Monitor
Component Function
Gatherer Read data
Merger Receive data from multiple Gatherers
Dispatcher Send data to Logger and Filter
Logger Save data on each HDD
Filter Send data of a part of event to Merger of Monitor PC
Component Function
Merger Receive data from multiple Filters
Eventbuilder Event build
Monitor Analyze data & show the result
DAQ Component Features
2015/4/14 CHEP2015 13
READOUTREADOUT
READOUTREADOUT
READOUTREADOUT
READOUTREADOUT
Flexibility
Reusability
Scalability
logic
Ring Buffer
・・・
Users can flexibly change DAQ component combination.
Users can improve performance by adding new PCs and deploying DAQ components.
Users can use a DAQ component in various DAQ system.
DAQ component has a ring buffer. Users do not need to implement a buffer.
If data volume increase by detector upgrade, we add Storage PC.
Advantage of DAQ Component Features
2015/4/14 CHEP2015 14
Flexibility & Scalability
Reusability
Ring BufferUsing DAQ component which is prepared, we saved time and effort for development.
We can use no event time effectively.
Sample Dispatcher, LoggerGatherer, Monitor
DAQ system
receive
process
receive
process
3. DAQ System Performance
2015/4/14 CHEP2015 15
Purpose & Method
2015/4/14 CHEP2015 16
Emulator Storage PCNetwork switch Monitor
PCHDD
HDD
Emulator
Emulator
EmulatorEmulator
Emulator
EmulatorEmulator
Emulator
test data
send data once every 10 events
Purpose
Evaluation of a total throughput using a DAQ-Middleware for the J-PARC E16 experiment
• Emulators run with 1 cycle per 6 seconds. During one cycle, emulators send test data for 2 seconds and do not send test data for 4 seconds.
• Monitor PC shows data value of a part of data regularly.
• We changed the number of emulators to change transfer data volume and measured processing data speed.
Method
Result
The result met the requirements of the E16 experiment!
2015/4/14 CHEP2015 17
requirement throughput(Trigger rate is always max [2kHz]. )
HDD write limit
requirement throughput(Trigger rate is always average [1kHz]. )
During sending test data,one event data size per one emulator is 14kB.
2kHz
1kHz
4. Summary and Future Plan
2015/4/14 CHEP2015 18
Summary and Future Plan
• DAQ-Middleware is a framework for network based DAQ software.
• DAQ component has following features.– Flexibility– Scalability– Reusability– Ring Buffer
• We have developed DAQ system for E16 experiment by using DAQ-Middleware.• The requirements from E16 experiment have been
met.• In the future, we are going to connect DAQ system to
read out module, and evaluate the system.
2015/4/14 CHEP2015 19
Backup
2015/4/14 CHEP2015 20
J-PARC E16 Experiment DAQ system overview
2015/4/14 CHEP2015 21
Evaluation Environment
2015/4/14 CHEP2015 22
HDD
HDD
PC1
PC2PC3
CPU Intel(R) Xeon(R) X5650 @ 2.67GHz 6Cores
Memory 24GB
Network 1Gbps x 5
OS Scientific Linux 6.4
HDD Hitachi HDS724040ALE64TB
PC1CPU Intel(R) Xeon(R)
CPU E5-2640 0 @ 2.50GHz 6Cores
Memory 32GB
Network 1Gbps x 5
OS Scientific Linux 6.6
HDD Hitachi HDS724040ALE64TB
PC2CPU Intel(R) Xeon(R)
CPU E3-1220 v3 @ 3.10GHz
Memory 8GB
Network 2Gbps
OS Scientific Linux 6.6
PC3
CiscoCatalyst 2960G
HDD write speed check
2015/4/14 CHEP2015 23
SRS (Scalable Readout System)
2015/4/14 CHEP2015 24
SRS is general purpose multi-channel readout system.User can choose variety of frontend chips.
FECADC
FEC
Front-End Card
FEC
Front-End Card
FEC
Front-End Card
Eurocrate
Front-End ASIC
Front-End ASIC
Front-End ASIC
SRU
Evaluation by using SRS
2015/4/14 CHEP2015 25
SRSADC/EFC
ADC/FEC send test data.Maximum transfer speed of test data is 1Gbps + 1Gbps.We configured that DAQ PC could process all of data.
Storage PCNetwork switch Monitor PC
HDD
HDD
test data
send data once every 10 events
Storage PC Performance
2015/4/14 CHEP2015 26
Emulator Storage PC
Test PC
emulatoremulator
emulatoremulator
CPU Intel(R) Xeon(R) X5650 @ 2.67GHz 6CoresMemory 24GBNetwork 1Gbps x 10OS Scientific Linux 6.4SSD Intel SSD520Series 240GB
only receive data
Throughput of 1 Storage PC is 1000MB/spill.
write data on SSD
test datatrigger rate = 2kHz1 event size = 14kB
DAQ Component & Configuration Example
2015/4/14 CHEP2015 27
InPort OutPort
Service Port(command/status)
Logics (for data handling)
Data
InPort OutPort
Service Port(command/status)
Logics (for data handling)
Data+ =DAQ-Middleware provides user writes
Monitor
DaqOperator
Gatherer
Online Monitor only
Examples of DAQ component combination
Logger
DaqOperator
Gatherer
Logging only
Dispatcher
Logger
Monitor
DaqOperator
Gatherer
Online Monitor & Logging
top related