lhcb computing status report meeting with lhcc referees october 19th, 1999 john harvey cern/ ep-alc
Post on 12-Jan-2016
218 Views
Preview:
TRANSCRIPT
LHCb Computing Status Report
Meeting with LHCC RefereesOctober 19th, 1999
John HarveyCERN/ EP-ALC
Report to LHCC Referees October 1999 Slide 2
Outline
Frontend / DAQ workshop DAQ - TFC, Readout Unit, Readout Network ECS - Fieldbus, OPC, SCADA Software - Migration to GAUDI Computing Model Studies Status and plans for next production of simulated events
Report to LHCC Referees October 1999 Slide 3
Frontend DAQ Workshop An LHCb workshop was held 12-14 October at CERN to
discuss issues of Front-End Electronics and DAQ. 35 participants from all the various subsystems and institutes A great deal of discussion and a positive outcome :
establishing a common language between all participants underlining advantages of taking a uniform approach and adopting
uniform solutions whenever possible agreement on the need to review specifications before building
hardware (qualification) global decisions (choice of parameters) to be easily/widely visible areas where clarification of ideas are needed were identified
Agenda and slides :http://lhcb.cern.ch/electronics/meetings/FE-DAQ_workshop_october99/agenda.htm
Report to LHCC Referees October 1999 Slide 4
Trigger/DAQ ArchitectureTrigger/DAQ Architecture
Read-out Network (RN)
RU RU
Control &
Monitoring
RU
4 GB/s
20 MB/s
Variable latencyL2 ~10 ms
L3 ~200 ms
LA
N
Read-out units (RU)
Timing&
FastControl
Front-End Electronics
VDET TRACK ECAL HCAL MUON RICH
LHCb Detector
L0
L1
Level 0Trigger
Level 1Trigger
40 MHz
1 MHz
40 kHz
Fixed latency 4.0 s
Variable latency <2 ms
Datarates
40 TB/s
1 TB/s
Front-End Multiplexers (FEM)1 MHz
Front End Links
Trigger Level 2 & 3Event Filter
SFC SFC
CPU
CPU
CPU
CPU
Sub-Farm Controllers (SFC)
Storage
Th
rott
le
Report to LHCC Referees October 1999 Slide 5
Timing and Fast Control Responsibilities
transmission of LHC clock, L0 and L1 triggers to front-end electronics monitor state of buffers at all stages of readout - throttle trigger to avoid overflows react to control commands such as ‘resets” support partitioning
The Readout Supervisor (RS) is the heart of the TFC system. It must model state of frontend electronics and throttle trigger according to rules It must react to control commands, DAQ throttle, resets etc. It must encode channel A (L0) and channel B (L1, resets) for TTC
Effort has been started to make a detailed specification of RS functions next step is to define use cases to be able to specify format of channel B types, frequency and timing of resets, commands to pulse detectors etc.
Warsaw group starting to work on prototype for RS switch Effort will shortly be put on making a detailed specification of RS itself
gL0gL1
ReadoutSupervisor
L0 L1
gL0gL1
ReadoutSupervisor
Localtrig.
gL0gL1
LHCClock
Clk Fanout
SD1 TTCtx SD2 TTCtx SDn TTCtx
Readout SupervisorSwitch
L1 TTCtx
ReadoutSupervisor
L-0
Localtrig.
L0 L1
L0 L1
L0 TTCtx
L-1
LHCTurn Signal
LHCTurn Signal
LHCTurn Signal
TFC : Signal Generation and Treatment
RD12
Report to LHCC Referees October 1999 Slide 7
Trigger/DAQ Architecture
Read-out Network (RN)
RU RU
Control &
Monitoring
RU
4 GB/s
20 MB/s
Variable latencyL2 ~10 ms
L3 ~200 ms
LA
N
Read-out units (RU)
Timing&
FastControl
Front-End Electronics
VDET TRACK ECAL HCAL MUON RICH
LHCb Detector
L0
L1
Level 0Trigger
Level 1Trigger
40 MHz
1 MHz
40 kHz
Fixed latency 4.0 s
Variable latency <2 ms
Datarates
40 TB/s
1 TB/s
Front-End Multiplexers (FEM)1 MHz
Front End Links
Trigger Level 2 & 3Event Filter
SFC SFC
CPU
CPU
CPU
CPU
Sub-Farm Controllers (SFC)
Storage
Th
rott
le
Report to LHCC Referees October 1999 Slide 8
Front-End Multiplexing/Readout Unit
There are ~2000 sources of data from L1 electronics and these have to be combined onto ~200 links and fed to the Readout Units (RU)
There is agreement to aim to have a common module for this based on RU The RU project is well underway. A number of issues were identified and
some resolved single fragment per event (simple protocol, performance) maximum size of fragments to be determined - issue is cost of fast memory adopted a single data frame format for transport through various readout stages data payload to be self describing project starting to investigate use of fieldbus for remote control/monitoring
A first prototype of RU should be ready by the beginning of 2000 Auxiliary equipment and software to allow testing is also being built
Report to LHCC Referees October 1999 Slide 9
Trigger/DAQ ArchitectureTrigger/DAQ Architecture
Read-out Network (RN)
RU RU
Control &
Monitoring
RU
4 GB/s
20 MB/s
Variable latencyL2 ~10 ms
L3 ~200 ms
LA
N
Read-out units (RU)
Timing&
FastControl
Front-End Electronics
VDET TRACK ECAL HCAL MUON RICH
LHCb Detector
L0
L1
Level 0Trigger
Level 1Trigger
40 MHz
1 MHz
40 kHz
Fixed latency 4.0 s
Variable latency <2 ms
Datarates
40 TB/s
1 TB/s
Front-End Multiplexers (FEM)1 MHz
Front End Links
Trigger Level 2 & 3Event Filter
SFC SFC
CPU
CPU
CPU
CPU
Sub-Farm Controllers (SFC)
Storage
Th
rott
le
Report to LHCC Referees October 1999 Slide 10
Event Building
After discussion with representatives from industry we are convinced that a switching network supporting 4 GB/s sustained rate will not be a problem to acquire at affordable cost on the timescale of the LHCb DAQ.
Several technologies are in principle available Gb Ethernet or 10Gb Ethernet Myrinet latest estimate of cost per port for 1 Gbps technology is now 500$ - 1000$
In this light we see no reason to deviate from our ‘full readout protocol’ as described in the Technical Proposal.
We are currently concentrating on studying ‘intelligent Network Interfaces’ that would allow to perform the assembly of the incoming fragments locally and only send complete events to the host CPU. A project based on an ATM card from IBM has been started beginning of October
Report to LHCC Referees October 1999 Slide 11
Experiment Control System
Read-out Network (RN)
RU RU
Control &
Monitoring
RU
4 GB/s
20 MB/s
Variable latencyL2 ~10 ms
L3 ~200 ms
LA
N
Read-out units (RU)
Timing&
FastControl
Front-End Electronics
VDET TRACK ECAL HCAL MUON RICH
LHC-B Detector
L0
L1
Level 0Trigger
Level 1Trigger
40 MHz
1 MHz
40 kHz
Fixed latency 4.0 s
Variable latency <2 ms
Datarates
40 TB/s
1 TB/s
Front-End Multiplexers (FEM)1 MHz
Front End Links
Trigger Level 2 & 3Event Filter
SFC SFC
CPU
CPU
CPU
CPU
Sub-Farm Controllers (SFC)
Storage
Th
rott
le
Report to LHCC Referees October 1999 Slide 12
Control/Monitoring structure
Experimental equipment
. . .
LAN
WAN
Storage
Oth
er
syst
em
s(L
HC
, S
afe
ty,
...)
Configuration DB, Archives,Logfiles, etc.
Controller/PLC VME
FieldBus
LAN
Supervision
ProcessManagement
FieldManagement
Sensors/devices
Field buses
PLC/ PC
OPC
CommunicationProtocols
SCADA
Technologies
SCADA = supervisory control and data acquisitionOPC = OLE for Process ControlPLC = Programmable Logic ControllerField buses = CAN, ProfiBus, WorldFip, ...
Report to LHCC Referees October 1999 Slide 13
LHCb Priorities Follow standards at all levels
LHCb, CERN, HEP, Industry
First priority is to adopt guidelines for choice of fieldbus and integration with hardware Milestone : end of this year
Use PLCs for specialised domains (gas, magnet…), where safety is issue
Use PCs for other domains cheaper, programming flexibility
Adopt OPC JCOP recommendation (Sept ‘99)
Adopt SCADA JCOP tender under discussion timescale target is ~ end 2000
Hardware resources
Field Bus controller
Field bus
I/O Interfaces
Software Interface
Device behavior
GUI/MMI
Network comm.
Sub-system supervision
High level services
Sensors/devices
Field buses
PLC / PC
OPC
CommunicationProtocols
SCADA
TechnologiesLayers
Process control Pri
ori
ty
high
low
Report to LHCC Referees October 1999 Slide 14
Fieldbus
Collect requirements in terms of #boards, #bytes/board, frequency and speed types of boards and interface to electronics technical constraints (radiation environment, power consumption, space) issue questionnaire (next week)
Match requirements to fieldbus capabilities and produce LHCb guidelines Concrete projects to get hands-on experience : fieldbus controller for RU
Profibus CANbus Worldfip Ethernet Firewire
SpeedMbps
12 1 2.5/5 100 400
MessagesizeBytes
256 8 128
#nodes 127 127 256
Distancemetres
100 40 1000
Report to LHCC Referees October 1999 Slide 15
Supervisory Software (SCADA) LHCb requirements :
The number of different device types is of the order of a 100 The number of devices is of the order of 17’000 The number of parameters is of the order of n•107
The number of monitored quantities is of the order of n•105
Implications for choice of SCADA system : A tag-oriented system is unrealistic if each parameter is an entry. We need a namespace hierarchy (Device->Parameters). For highly repetitive items (e.g. individual detector channels in an electronics board)
arrays are needed (don’t want to name each of them individually).
Short term SCADA - Bridgeview Longer term - adopt scalable device-oriented commercial product (JCOP tender) Invest in OPC to ease transition
Report to LHCC Referees October 1999 Slide 16
Migration to GAUDI
Final objective: Produce a complete set of fully functional data processing applications
using exclusively OO technology
Sub-objectives: Provide a fully functional framework (GAUDI) Assemble new and old algorithms into a single and complete suite of data
processing applications. Be able to run productions. Convert all the existing FORTRAN code to C++
Report to LHCC Referees October 1999 Slide 17
Possible strategiesFortran
C++
SICb
Gaudi?1
SICb
Gaudi2
Fast translation of Fortran into C++
SICb
Gaudi3
Wrapping Fortran
Framework development phase
Transition phase
Hybrid phase
Consolidation phase
Report to LHCC Referees October 1999 Slide 18
Framework development phase
At the end of this phase the GAUDI framework should be functionally complete Data access services Generic event model Generic detector description model Data visualization Basic set of services
Develop some physics algorithms to prove architecture concept We started this phase one year ago We expect to be completed by middle of November 1999 with the
release of v3 of GAUDI
Report to LHCC Referees October 1999 Slide 19
Transition phase
At the end of this phase we should be able to reconstruct and analyze simulated data within the GAUDI framework. The Monte Carlo data production will still be done using SICb.
Incorporate reconstruction and analysis parts of SICb in the GAUDI framework - wrap FORTRAN code Analyse SICb to identify all modules, their inputs and outputs Develop a complete OO event data model Write converters to allow access to data in both formats
Development of new algorithms can proceed within GAUDI Caveats
lot of work to make converters in both directions we could discover technical difficulties (size, commons, initialization,…)
Report to LHCC Referees October 1999 Slide 20
GeneratorDet. Simul.
MCHits
DST
HistoNtuples
Recon-struction +
Analysis
Transition phase -2
SICbGEANT 3
FrameworkGAUDI
Framework
Report to LHCC Referees October 1999 Slide 21
Transition Phase - 3
GeneratorDet. Simul.
(Sicb)MCHits
FA
Bank 1 ZEBRA
GAUDI store
FB FC
C++A C++B
Cnv
Commons
Bank 2 Bank4
Obj 2 Obj 3 Obj 4
Cnv
DST
HistoNtuples
Reconstruction + Analysis
Report to LHCC Referees October 1999 Slide 22
Hybrid Phase
One single program with FORTRAN and C++ cooperating to produce physics results.
Replace wrapped FORTRAN code incrementally. At the end of this phase we should be able to retire the FORTRAN
compiler and libraries Already known problems:
Two different detector descriptions. Difficult to maintain. Output file format. Which one to choose? Different set of input “card files”. Large memory needs for data and code.
The hybrid phase should be as short as possible to minimize pain.
Report to LHCC Referees October 1999 Slide 23
Consolidation phase
Complete the new detector description Re-iterate with the O-O Event Model Re-engineer some of algorithms Incorporate Geant4 with GAUDI to make framework for new
simulation program …..
Report to LHCC Referees October 1999 Slide 24
Planning
Qtr 1Qtr 3 Qtr 2 Qtr 3 Qtr 41998 1999
Architecture Design
Gaudi Development v1
Gaudi Development v2
Gaudi Development v3
Qtr 1 Qtr 2 Qtr 3 Qtr 42000
Qtr 4
Framework Functional
Analysis Sicb
Transition phase
Production program
Hybrid phase
Report to LHCC Referees October 1999 Slide 25
Computing Model Studies We are revising our computing requirements
event sizes and storage requirements triggering cpu requirements (optimisation strategies) availability of calibration/alignment constants simulation requirements
We are looking at 'use cases’for the analysis of selected physics channels (+-, J/ Data retained at each stage Processing required at each stage Frequency of processing cycles
Generate a baseline model for distribution and role of experiment-specific, CERN and regional computing facilities
Timescales fixed by impending Hoffmann Review
Report to LHCC Referees October 1999 Slide 26
SICb Status
Generators: Pythia 6.125 + QQ (CLEO) for decays Full GEANT simulation Realistic detector desciption:
WARM magnet design realistic beampipe model calorimeters,muons and RICHes follow current optimisation
Reconstruction: no pattern recognition in tracking full pattern recognition in RICHes
Report to LHCC Referees October 1999 Slide 27
PLANs up to end 1999
Trigger optimisation CERN (PCSF) 200k mbias - 4 different settings(luminosity, generator) 20k b -> mX 20k b -> eX 20k B0
d -> +-
20k B0d -> J/()K0
s
20k B0d -> J/(ee) K0
s
Background studies with GCALOR at Lyon 50k mbias
Physics production 500k b inclusive at Rutherford lab (PCSF) 100k other physics channels at CERN (PCSF)
Report to LHCC Referees October 1999 Slide 28
Training Formal training through CERN Technical Training services
Hands-on Object-Oriented Analysis, Design and Programming with C++ C++ for Particle Physicists (Kunz) at least 40 people followed courses
Follow-up training books - Design Patterns (Gamma), Software Design (Lakos),..
Learning on the job - we already do following : design discussions where developers present their designs for review tutorials on new GAUDI releases during LHCb software weeks
Learning on the job - ideas for future include : code walkthroughs document patterns (those well-tried and successful and those that failed)
Consultancy and mentoring have GAUDI team spend 50% of their time helping algorithm builders
top related