alberto colla - cern alice off-line week 1 alberto colla alice off-line week cern, may 31, 2005...
DESCRIPTION
Alberto Colla - CERN ALICE off-line week 3 “History” of DB access classes Original idea and first implementation of DB access classes by T. Kuhr (late 2004) Since February 2005: work on the framework implementation is carried out in the core offline group we are receiving a lot of comments and suggestions from detector experts!TRANSCRIPT
Alberto Colla - CERNALICE off-line week 1
Alberto Colla ALICE off-line weekCern, May 31, 2005
Table of contents:
● Summary of requirements
● Description of the DB access classes
● Some working examples
● Comments/Remarks
● Work in progress
AliRoot DataBase access classes
Alberto Colla - CERNALICE off-line week 2
• simple user interface (data accessible by strings & run numbers)
• Distributed file system as the final storage/retrieval environment
• Possibility to dump data locally• possibility to retrieve data in parallel from different sources (local files + Grid)
• technique used to store/retrieve object should be transparent to user
• automatic versioning (objects will be read-only)
• run-dependent objects ( retrieval handled by a RunLoader)
• Underlying framework: ROOT/AliRoot + UNIX/Grid
Summary of requirements
Alberto Colla - CERNALICE off-line week 3
“History” of DB access classes
• Original idea and first implementation of DB access classes by T. Kuhr (late 2004)
• Since February 2005: work on the framework implementation is carried out in the core offline group
we are receiving a lot of comments and suggestions from detector experts!
Alberto Colla - CERNALICE off-line week 4
DB access classes schema
AliRunData
DB object
AliObjectMetaData
AliRunDataStorage
AliRunDataFile
AliRunDataOganizedFile
AliRunDataAlien
Local file
Loacl file in a defined folder structure
AliEn file
AliRunLoader
(AliSelectionMetaData)
User code
GetRunObject
Get/PutGetEntry/PutEntry
Note: classes names are under discussion and are subject to change!
Alberto Colla - CERNALICE off-line week 5
AliRunData
• Container for a basic databse object
• Contains:● The object● the object’s MetaData: a set of information fully describing the object
• Something on the database objects:
• It is detector specific● can be any object inheriting from TObject● each AliRunData object contains a single object (which can be a container of more elementary objects)● the validity of the object is specified by a RUN range set in the object’s metadata
Alberto Colla - CERNALICE off-line week 6
AliMetaData and derived classes
AliMetaData
AliOBjectMetaData AliSelectionMetaData
Base class, contains methods to compare different AliMetaData objects, etc...
Contains a set of information which fully describes the object
Contains the subset of information used to retrieve the object
Alberto Colla - CERNALICE off-line week 7
AliObjectMetaData
• Each object is associated to its “AliObjectMetaData” which fully
describes it
• pieces of information (AliObjectMetaData data members) constituting
the object’s MetaData:- const char *fName: see later
- int fFirstRun, int fLastRun: validity range of the object
- int fPeriod: LHC fill Period
- const char *fFormat: object`s format (TTree structure, histo structure, etc...)
- const char *fResponsible: name of the person `responsible´ of the object
- const char *fExtraInfo: free field (anything else you may want to know)
- int version: automatically set during storage
• ideas and suggestions on the metadata descriptors are being collected and th efinal
list will change
Alberto Colla - CERNALICE off-line week 8
AliSelectionMetaData
• Only a subsample of all the ObjectMetaData entries are used for data retrieval:
- name- Run validity range
(firstRun - lastRun)- version
• AliSelectionMetaData is used uniquely during data retrieval, to specify a series of selection criteria:
`For the object called “name” and for all runs from firstRun to lastRun, I want this version´
AliSelectionMetaData class
Alberto Colla - CERNALICE off-line week 9
MetaData name
• The metadata name must have a simple (and fixed) structure in
order to ease the retrieval:
Detector/DBType/DetSpecType
Detector´s name(ZDC, TPC...)
DataBase type(Calib, Align...)
Detector specific type of the object (Pedestals, Gains, DeadChanMaps ...)
• Example: “ZDC/Calib/Pedestals”
• In the current implementation this structure is fixed (AliMetaData complains if it is different)
Alberto Colla - CERNALICE off-line week 10
AliRunDataStorage class
• Base class, provides a virtual interface to the database
• Singleton (only one instance of AliRunDataStorage is allowed at any time)Many ideas have been gathered on the “singleton issue” and on the possibility
to retrieve objects from different sources (see Magali’s talk). We are discussing about it…
• Storage and retrieval of objects will be performed through AliRunDataStorage
public methods:
• Bool_t Put(const TObject* object, const AliObjectMetaData& objMetaData)
• const TObject* Get(const char* name, Int_t runNumber)
• The Get function is also called by AliRunLoader::GetRunObject(const char *name)
method (the RunLoader knows the run number)
• Implementation of storage/retrieval of object is performed by the active instance of AliRunDataStorage.
Alberto Colla - CERNALICE off-line week 11
AliRunDataStorage class (2)
• Two other public methods of AliRunDataStorage:
• Bool_t RecordToFile(const char *filename=“DB.root”)Prepares for recording into a local file the entries (AliRunData objects) as they are retrieved from the database
• void Select(const AliSelectionMetaData& selMetaData)specifies one or more selection criteria used for data retrieval
• Examples of the functionality of these two methods will be given later
Alberto Colla - CERNALICE off-line week 12
AliRunDataFile
• Access class for a locally stored object• inherits from AliRunDataStorage• allows one or more objects to be stored in a single Root file• Structure of a stored object:
DBFile.root:
TKey: AliRunData “name1;1”TKey: AliRunData “name1;2”TKey: AliRunData “name2;1”...
Key cycle (beware, this is not the object‘s version!)
“name” is defined by the object‘s MetaData; if name contains slashes (´/´) a TDirectory structure is created
• constructor: AliRunDataFile(const char *filename=“DB.root”, Bool_t readOnly=kTRUE)
• probably not the most suitable way to store large amounts of data; this class is
useful for “local” work on the objects by sub-detector experts
Alberto Colla - CERNALICE off-line week 13
AliRunDataOrganizedFile
• Access class for a object stored into a local file inside a well defined directory structure• inherits from AliRunDataStorage• each root file contains one single object • Structure of a stored object:
DBFolder
Object‘s name1
Object‘s name 2
...
Run1-10_v0.rootRun11-20_v0.rootRun21_v0.rootRun22-30_v0.rootRun22-40_v1.root...
Single AliRunDataentry
Alberto Colla - CERNALICE off-line week 14
AliRunDataOrganizedFile (2)
• The only data member of AliRunDataOrganizedFile class is the folder name (default: “$ALICE_ROOT/DB” )
• When an object is stored with AliRunDataOrganizedFile:
– Its location is determined by MetaData's name (if a folder does not exist itis created)
– The file name is determined by the run validity set in the MetaData;version number is automatically set
– Note: Version is incremented also in case the run validity range of the object “overlaps“ the one of an already existing object!
– File name sintax: Run#firstRun-#lastRun_v#Vers.root orRun#Run_v#Vers.root
– The run validity and version of the stored object's MetaData areautomatically set to -1, to avoid inconsistencies if the file name ischanged by hand– When retrieved, the run validity and version of the object's MetaData arereset according to the file name
Alberto Colla - CERNALICE off-line week 15
AliRunDataAlien
• access classes to a object stored into a Grid file
• inherits from AliRunDataStorage
• not yet implemented – will be based on the ROOT TGrid/TAlien plugin
• the file structure will be probably similar to that built with the
AliRunDataOrganizedFile class (one object per file)
Alberto Colla - CERNALICE off-line week 16
Two examples of calibration objects
1. $ALICE_ROOT/ZDC/AliZDCCalibData class:
• container for ZDC calibration data
• Data members: an array of pedestal values (floats)
an array of energy calibration coefficients (floats)
2. Calibration object for SPD (ITS):
• Under development, see Paul Nilsson´s talk
• Preview: map(s) of SPD dead and noisy channels• object: (array of) TClonesArray(s)
These are two “simple” calibration objects:
• object size will be small (few Kbytes)
• update frequency will be low (< 1/day, say) constant within a run
Alberto Colla - CERNALICE off-line week 17
Usage of the AliRoot DB access classes
1. Storing data
First: specify the object's MetaData
Suppose we want to store a set of ZDC pedestals (set in the object AliZDCCalibData* zdcdata) which we'll use to calibrate runs 10 to 100
AliObjectMetaData md(“ZDC/Calib/Pedestals”,10,100,...);
Run validityName
Second: Create an AliRunDataStorage instance (e.g. an AliRunDataOrganizedFile)
AliRunDataOrganizedFile *dof = new AliRunDataOrganizedFile(“DBFolder”);
Third: Store the object with AliRunDataStorage::Put method
dof->Put(zdcdata, md);
The object is written in file: DBFolder/ZDC/Calib/Pedestals/Run10-100_v0.root
Alberto Colla - CERNALICE off-line week 18
Usage of the AliRoot DB access classes (2)
2. Retrieving data
Now you want to retrieve the ZDC calibration object to calibrate, say, run 50 from our local DB storage
First: Create the AliRunDataStorage instance (AliRunDataOrganizedFile)
AliRunDataOrganizedFile *dof = new AliRunDataOrganizedFile(“DBFolder”);
Second: if necessary, specify some selection criteria using an AliSelectionMetaData object and the AliRunDataStorage::Select method
AliSelectionMetaData smd(“ZDC/Calib/*”,1,100,2);dof->Select(smd);
Continue…
This means: For all objects named“ZDC/Calib“, and for all runs between 1
And 100, I want version 2
Alberto Colla - CERNALICE off-line week 19
Usage of the AliRoot DB access classes (3)
2. Retrieving data (continued)
Fourth: get the AliZDCCalibData object valid for run 50 with AliRunDataStorage::Get method:
Third: if you want to dump the retrieved object(s) into a local (AliRunData) file, call the AliRunDataStorage::RecordToFile option before getting the objects:
dof->RecordToFile(“$HOME/MyDataBase.root”);
AliZDCCalibData *zdcdata = AliRunDataStorage::Instance()->Get(“ZDC/calib/Pedestals”,50);
Most general way to call the active AliRunDataStorage instance
Alberto Colla - CERNALICE off-line week 20
Usage of the AliRoot DB access classes (4)
For further examples:
• See the note “AliRoot Framework for Access to the Offline Calibration and Alignment Database” distributed by email
• See the tutorial macro $ALICE_ROOT/macros/DBAccessTutorial.C
• Follow the session “Hands on the DB access classes” next Friday morning
Alberto Colla - CERNALICE off-line week 21
Further comments
• About the AliRunDataStorage::Select method:
More than one selection criterion can be specified in cascade, e.g.:
// Select version 2 for all TPC objectsAliSelectionMetaData smd(“TPC/*”);smd.SetVersion(2);AliRunDataStorage::Instance()->Select(smd); // but version 3 for “TPC/Calib/Gain”smd.SetName(“TPC/Calib/Gain”);smd.SetVersion(3);AliRunDataStorage::Instance()->Select(smd); // and version 0 for “TPC/Calib/DriftVelocity” //for runs 10 to 20AliSelectionMetaData smd2(“TPC/Calib/DriftVelocity”,
10,20,0);AliRunDataStorage::Instance()->Select(smd2);
Note: if during retrieval a selection criterion is specified, but that particular version of the object is not found, then the closest preceding version is returned. Suggested behaviour: is that what we want?
Alberto Colla - CERNALICE off-line week 22
Further comments (2)
• About the automatic versioning of the stored objects (with
AliRunDataOrganizedFile class):
I stress that the version of the stored object is incremented if its run range “overlaps“ the one of an already existing object.
Example: suppose I want to update the calibration object stored in DBFolder/.../Run1-10_v0.root.
After the update, I realize that the object is valid up to run 15.
Then I just need to set the new run range in the object‘s metadata and the object will be stored in the file DBFolder/.../Run1-15_v1.root.
Alberto Colla - CERNALICE off-line week 23
Further comments (3)
• Finally, remember that the objects can be retrieved via the
AliRunLoader class, through the function GetObject(“objectName”).Example (user macro):
// create an AliRunLoader objectAliRunLoader *runLoader = new AliRunLoader()…// open some AliRunDataStorage instance, specify selection // criteria, etc etc……// get the objectTObject *obj = runLoader->GetObject(“objectName”);
Alberto Colla - CERNALICE off-line week 24
Work in progress
• First implementation of AliRunDataStorage::TagForProduction method:
• With the AliRunDataOrganizedFile (and probably with AliRunDataAlien) access
classes the simplest solution may be to create symbolic links of the files and label them with a “production version”.
void TagForProduction(AliSelectionMetaData &smd, Int_t prodVersion)
The AliSelectionMetaData contain all the
information needed to identify the object(name, run range, version)
• We have been suggested to provide a means to “tag” a specific version of every objects, in order to ease their retrieval in case they should be used for global data reconstruction
“production” version that will label the
symbolic link to the DB object
Alberto Colla - CERNALICE off-line week 25
Work in progress (2)
• Example of usage of TagForProduction: I want to tag the object in the
“organized” file DBFolder/ZDC/Calib/Pedestals/Run1-10_v3.root for production 0.
// open the AliRunDataOrganizedFile instanceAliRunDataOrganizedFile *dof = new AliRunDataOrganizedFile(“DBFolder”);// Create the SelectionMetaData object with the info needed // to identify the objectAliSelectionMetaData smd(“ZDC/Calib/Pedestals”,1,10,3)// Tag the file:dof->TagForProduction(smd,0)
• The following symbolic link is created in the same directory as the object:
DBFolder/…/Run1-10_Prod0.root → Run1-10_v3.root
Alberto Colla - CERNALICE off-line week 26
Work in progress (3)
• I propose a method to prepare AliRunDataStorage to retrieve the data valid for production:
void AliRunDataStorage::SetStorageMode(int storageMode)
• If storageMode=1 (“Production mode”) then the retrieval is done in the standard
way for all files tagged for production.
Note: Discussion on this issue is under way within the core offline group (see Magali‘s talk).