steps towards c++/oo offline software in star valery fine, yuri fisyak, pavel nevski, victor...
TRANSCRIPT
Steps Towards C++/OO Offline Software in STAR
Valery Fine, Yuri Fisyak, Pavel Nevski,
Victor Perevoztchikov, Torre Wenaus
Brookhaven National LaboratoryDoug Olson, R. Jefferson Porter, Craig Tull,
David Zimmerman
Lawrence Berkeley National Laboratory Herbert Ward
University of Texas, Austin
(STAR software infrastructure group)
STARSolenoidal Tracker At RHIC
An international collaboration of over 400 physicists located at 34 institutes
STAR will start taking data next summer (1999) Expect to process over 300 TBytes of real and simulated
data per year STAR faces, on a short time, scale many of the same
software and computing challenges that the next generation of high-energy physics experiments will be facing at the LHC .
STAR detector:
- SVT
- TPC
- FTPC
- EMC
- TOF
- Solenoid
magnet
-...
Present STAR software framework: StAF The STAR offline software has been developed within the
Standard Analysis Framework , StAF, which provides: IDL-based CORBA-compliant encapsulation of data analysis
algorithms , modules --- user-written function in C, C++, or FORTRAN
these components are controlled at run time by a high-level scripting language (KUIP) and/or by Graphical User Interfaces
tools for the manipulation of the basic components of the data model --- StAF tables
organization of these tables in Data Set containers; I/O for these containers via XDR library with support of schema
evolution built on top of PAW/CERNLIB dynamical loading of modules
Present STAR software (cont.) The system currently has 307 tables and 148 modules and ~14klocs
of kumacs to control them.
• AGI stands for Advanced GEANT Interface
Reconstruction TotalStAF AGI generators
klocs (%) klocs (%)FORTRAN 18.3 23.9 104.2 66.0 42 212.4 53MORTRAN 21.3 7.3 5 28.4 7C 42.1 4.8 14.0 9 60.9 15C++ 17.8 0.6 44.7 29 63.1 16KUIP 7.0 0.9 0.1 14.2 9 22.2 6IDL 1.0 9.2 6 10.2 3Total 87.2 51.5 104.3 155.4 398.4
Present STAR software (cont.) After several years of successful development we feel that the
present framework has some problems: Extremely rapidly growing number of kumacs to control growing
number of objects to handle --- tables and modules. This makes the offline system fragile and unstable.Thus the present STAR framework is facing a real problem of scalability.
The framework supports very important but only one kind of objects --- tables. It is not convenient to have only one kind of furniture in a house. In event reconstruction we lack simple ways of pointer handling.
The framework is heavily based on CERNLIB/PAW. These tools are no longer supported by CERN, do not scale to our data volumes, and are poorly matched to the C++/OO software environment we are moving towards.
To C++/OO Offline software STAR software infrastructure group has been looking into
how to replace CERNLIB and related software: Possibility to replace KUIP with Tcl/Tk in StAF was considered, but
poorly received because PAW histogramming could not be carried over into a Tcl/Tk environment
Commercial software was found to have promising presentation tools but did not offer the data analysis and fitting capabilities of PAW
We have followed the development of CERN's official PAW replacement in the LHC++ project but it was the common consensus that this software is not yet mature enough for our use, and the considerable expense its use would entail could NOT be justified.
After looking at the alternatives, and considering the limited manpower we have, which precludes significant in-house development, ROOT has emerged as the only comprehensive PAW replacement that might be usable on our time scale.
To C++/OO Offline software (cont.) A potential problem with ROOT is the lack of official support. But
ROOT appears to be emerging as a de facto standard, in which case support within the community will inevitably follow.
Currently, with the high quality of support offered by ROOT's developers today, this is not a big problem.
The principal objective of the new effort: is to achieve a full integration which would provide a ROOT command
line and analysis environment with full access to STAR offline codes in the same way that the current StAF provides a KUIP command line and PAW environment.
An essential requirement is that code (modules) and data structures (tables) be supported without change in the ROOT environment, consistent with the STAR policy of migration to preserve existing investment rather than `revolution'.
Kumacs, of course, as KUIP-based scripts, are not supported in ROOT, and must be converted to C++ with essential semantic change.
StAF ROOT The architecture of StAF makes the effort to build a
ROOT-capable infrastructure practical. The IDL-based definitions of modules and tables and the in-
house IDL compiler, stic, made possible the automatic generation of ROOT C++ wrappers for modules and tables through the extension of stic.
The modularity and component-based design of modules make their invocation within ROOT reasonably straightforward.
The modularity and design for 'graceful retirement' of components of the StAF system code itself makes it possible to use components of it as needed together with ROOT.
Goals: Automatically generated interface code for all modules and tables
that makes it possible to access tables and invoke modules from within ROOT
Use of StAF system code to provide standard XDR I/O of data set hierarchies and tables into standard StAF data structures. ROOT access to these data structures is direct; data is not copied to ROOT objects
The same dynamic loading capability for modules in ROOT as in StAF
makefiles supporting both ROOT and StAF based use of the offline software
Direct histogramming access to tables from ROOT; the entire data model becomes effectively an Ntuple, rather than converting tables to Ntuples as in StAF
Automatically generated ROOT-style documentation for tables, their wrappers, module wrappers, and ROOT code implementing the StAF interface
Implementation:
To meet the above requirements the following C++ class library was developed which includes 10 base classes at four levels:
"wrapper" level objects to wrap and describe StAF tables ( St_Table) objects to wrap and call modules ( St_Module) objects to wrap and execute kuip action ( St_Kuip) for GEANT3
"container" level objects to create the hierarchical structure of datasets( St_DataSet)
objects to navigate the hierarchical structure of datasets ( St_DataSetIter) objects to create the hierarchical structure of datasets from the "native" file system structure ( St_FileSet)
``converter level'' table to/from ( St_DataSet) converter: reads and writes table with XDR format compatible with the current StAF framework ( St_XDFFile) GEANT3 geometry banks to ROOT geometry objects ( St_Geometry)
C++ object level objects to describe a whole production chain (to replace the existing kumacs) ( StChain) object to describe a single step of the whole chain ( StMaker)
Implementation (cont.)
Wrapper, container and converter levels provide us the StAF functionality at least. They allow migration from the "flat table" data model to a tree-like representation with "structural" and "reference" links.
The classes of C++ object level open the window of a "real" object oriented approach (here we borrowed the idea of Chain/Maker classes developed for ATLFast project).
During the transition period this approach offer dual options of working in the ROOT environment or working in the StAF environment
Browser of
GEANT3
geometry
in ROOT
Forward TPC
in ROOT
ROOT browser for data
produced with Big Full Chain
Further examples can be seen at our URL: http://www.rhic.bnl.gov/STAR/html/comp_l/root/index.html
Implementation (cont.) Using Chain/Maker schema:
TPC Cosmic Ray and Laser Calibration analysis chain has been implemented in ROOT.
Big Full Chain --- data processing starting from simulation of detector response to reconstruction and DST production --- has been implemented in ROOT and is tested right now.
During coming Mock Data Challenge 1 (Sep. - Oct. 1998) we are planning to process about 2 Tbytes of data.
BFC based on both StAF and ROOT will be tested in parallel.
Work to do: It is needed to understand at what scale we will use
ROOT: as a framework for everything or as an analysis tool
What interface with Objectivity data base will be used ? To develop a new STAR data model more suitable for
new framework. The old data model has to be redesigned based on experience of MDC1 during coming fall, and tested during the MDC2 (Feb. 1999) and fixed
Education to use new framework
Conclusions Thanks to the framework which we have (StAF) the approach has been
developed in STAR to allow graceful transition from FORTRAN/C environment to C+
+/OO world without breaking the system and preserving all reliable FORTRAN/C software developed so far.
Our plan of migration from StAF to ROOT has been endorsed by the STAR collaboration.
The scale of ROOT usage in STAR software infrastructure depends on lessons learned during the coming Mock Data Challenge 1.
There is a chance to have a homogeneous software infrastructure
environment for all components of the system: starting from Data Acquisition Online monitoring Data production, and analysis
and we would like to take it.