DB-based DAQ monitoring and Physics analysis tools

Emiliano BarbutoEuropean Emulsion Group

(LNGS May 2003)

The Scanning Cluster

Mic PC #1

Mic PC #2

Mic PC #3

Batch Manager

File Server

Web Interface

Local DB(ORACLE)

Building blocksBuilding blocks

Worldwide access through standard ways (HTML, ODBC)Worldwide access through standard ways (HTML, ODBC)

Scanning cluster operators onlyScanning cluster operators only

Batch Manager

What is the BM and what is it for?

BM is a service application (“daemon” in UNIX language)

At startup the BM explores the scanning cluster configuration using Local DB info: centralized cluster administration through DB.

Scanning tasks are grouped in batches and sent to the BM; BM registers batch progress into the DB in order to recover procedures in case of HW failure.

All operations are logged: easy to discover malfunction cases.

Batch Manager

The flow diagram that schematically illustrates the Vertex location procedure in the ECC brick, as presented in the proposal.

Extra-DAQ tasks for the BM:

• data acquisition monitoring;

• systematic errors correction;

• fragment linking;

• next sheet predictions;

• vertex confirmation (through simple volume tracks reconstruction);

All this operations will be logged in the DB: every correction and operations performed on scanning files (both automatic and custom) will be recorded

The BM is also supposed to perform other important tasks, related to the vertex location procedure (red box in the diagram);

DAQ MonitoringDAQ Monitoring can be performed in two different ways:

• web interface (application running on the web server interacting with remote users);

• usual desktop application (applications running locally on the machine);

Web Interface

Local DB(ORACLE)

Web interface for DAQ monitoring running under KDE (Netscape browser), it also accesses both files and DB

Desktop application: access both files and DB

Track Post-Processing

Easy Link (relying on track linking library) : microtrack linking through the base

Easy Correct (relying on track correction library): field to field shifts and conversion factor correction.

These post-processing applications (not libraries!) will be integrated into the BM.

Parameters and configurations will be stored in the DB, so they are forever documented.

Coherent Shifts

Field 1 Field 2Tk 1 Tk 1’

Tk 2’Tk 2

Shift

Shift

Batch Manager tasks

All tasks present some standard features, concerning the following items:

• I/O operations (scanning files & DB);

• Parameter settings for data processing algorithms come from the DB;

• separation between algorithms and I/O operations + user interface (software structure).

This structure allows to easily integrate them into the BM

We will see just an example in detail: the sheet alignment and vertex detection (confirmation) task.

Batch Manager

File Server

Local DB(ORACLE)

Sheet Alignment & Vertex Detection

Running in Chorus/Opera setup

Total Scan Reconstruction

Alpha Omega Reconstruction

Optimized for Opera setup

Sheet Alignment & Vertex Detection

Alpha Omega Reconstruction Library

AReconstruction steps:

1. an affine transformation, sheet focusing and correction of slope systematic errors, for each pair of sheets;

2. then volume-tracks reconstruction is performed using a tracking-like algorithm that relies on base-tracks (optional);

3. low momentum tracks reconstruction using the risk-factor/background density algorithm (optional – step 2 necessary);

4. vertex and decays detection (optional – step 2 necessary).

It performs all steps typical of a volume scanning, from rough sheet alignment (alpha) up to vertex and decay detection (omega)

I/O

Alpha Omega Reconstruction Program

Results of scanning procedure on several emulsion sheets: microtracks/linked base tracks.

Tracks linked throughout the emulsion sheets volume with topological information

• “tlg” files (conventional output files of the scanning system);

• Root files (“tlg” files converted into root format);

• Oracle DB (scanning sheet selected from DB by means of sql instructions);

• tsr files (conventional output files of the reconstruction already used in Chorus);

• Oracle DB (reconstruction inserted into DB by means of sql instructions);

• Root files (converted from Oracle DB);

Storage in DB of algorithm parameters could represent a problem because algorithms may be discarded or modified during the experiment analysis

changing DB schema!!!

Indeed, this won’t be a problem if our solution for parameter storage in the DB is a versatile one.

Parameters SettingParameters of the reconstruction

Several track linking parameters allow to tune the alignment procedure.

It is possible to tune also parameters of vertex and decay detection (E. Barbuto – Strasbourg).

Parameters SettingParameters Input/Output: XML format• XML files are similar to html files and can be accessed by a simple browser (remote access is also possible);

• They are ASCII files and can be modified using every text editor

• a simple XML string can be dumped into the DB collecting all the parameters into a single field rather than a record (several fields).

A xml parameter file as it appears in MSIE browser.

Changing algorithm, deleting or adding parameters is not a problem because DB tables and fields will not be modified.

We plan to store the algorithm id (or the module itself) in the DB along with its settings. Using XML, the application intrinsically knows how to handle its settings.

Software StructureHow this software is structured.

Library

Appl#1

Appl#2

Appl#3

...

All algorithms are coded in the library.It runs under several OS: Windows/Linux/FreeBSD (talk by CB).

They care about input/output operations, user interfacing and parameters setting.They can run with the usual windows aspect or under command lines (under Win/Linux/ FreeBSD, depending on the Application)

An ExampleFeatures of the Sheet Alignment & Vertex Detection: Vertices/Tracks/Segments Display

Sheet per Sheet analysis manager

Debug and TestAdditional features of the application: data generation.

The volume generation library allows to generate tracks and segments across a scanning volume.

• segments can be smeared according to usual measurements accuracy;

• multiple scattering and energy loss are taken into account;

• segments can be “moved” and “modified” according to typical affine transformation, sheet focusing and systematic errors computed during emulsion data processing.

The last feature turned out to be important to debug and test reconstruction ability

Computing library is a independent module Easy to include in custom programs (e.g. efficiency studies with “official” MC-generated data) .

Every data format can be used .

Conclusions and Perspectives

1. BM and Database features have been defined according to the various steps, trying to be as flexible as possible.

2. This will be important for the goal of next test beams: vertex location.

3. We started to include I/O for Database in all our applications.

4. Our policy is to separate as much as possible I/O operations, user interface and parameter setting from algorithm itself in order to write “computation code” that is OS-independent.