vincenzo innocente, cern/ep central software 19 september 2002, cern internal review of cms software...

Vincenzo Innocente, CERN/EPCentral Software

19 September 2002, CERNInternal Review of CMS Software and Computing

A Baseline forCMS Central Software

Vincenzo InnocenteCERN/EP/CMC

Software architecture, framework and servicesin simulation, high level trigger, reconstruction

and analysis


19 September2002, CERNInternal Review of CMS Software and Computing

Architecture Framework Toolkit TasksArchitectureGeneric FrameworkFramework specializations

Fast Simulation Detector Simulation Detector Description Reconstruction Analysis Visualization

Utility ToolkitsGraphics toolsTechnology tracking and evaluation



This TalkWhat happened since last internal review

Exit Objectivity New LHC schedule LCG

CMS Software Architecture Overview LCG “Blueprint” Baseline

Highlight on recent developments Prototype for a new Persistency Solution Simulation Framework Detector Description Data-Base Visualization Technology Tracking



2002: A transition yearCMS Software development plans affected by

CERN budget crisis LHC delay New “LHC Computing Grid” project Choices and decisions of CERN IT Person-power shortage

The only possible plan was planning for changes

New Strategy Terminate autonomous large scale R&D

Contribute to LCG Concentrate on CMS specific issues

Consolidate current software to finalize DAQ-TDR Initiate prototyping for a new persistency solution Continue review of architecture and implementation

Plan for software baseline end 2003



Baselining Architecture Framework Toolkit

Define and prototype the initial production software for LHC operation

Review Architecture Choose products Prototype and implement middleware Implement framework and toolkit

Needs collaboration (or at least synchronization) with LCG. CMS Strategy:

Proactive collaboration within LCG projects Define priorities, inject CMS person-power in critical areas

Early evaluation and integration Get a new working software already in the next months

Mitigate risks in areas where convergence seems slow using a more flexible architecture making our own choices and implementing ad-interim CMS-specific

interfaces and abstractions



CCS role in LCG eraBasic Software components will be provided by LCG as

Supported External software (mainly public domain) In house developed components

Part of a larger effort (HEP or wider) Specific to LHC

The main role of CMS CCS Directly contribute to LCG projects Develop specific CMS software to integrate, interface, extend the

components provided by LCG Provide a coherent and consistent framework Hide “native” API and UI if conflicting or not conforming Ensure transparency to Physics and Reconstruction Software

New Plan 9/02 Prototype (confirmed for october) 6/03 Match current functionality and performance 9/03 Baseline CMS CCS Software



CMS Contribution to LCG Project

Participations in RTAGs at the highest level of responsibility and technical competence First experience very positive

Collaborative, friendly atmosphere Real effort to define common products

Direct involvement in the project CMS “Chief Architect” in PEB & Architect Forum Already one person working full time in persistency project Ready to contribute with > 50% of AFT person-power

Full commitment to use products of common LCG projects



Architecture (Framework) ReviewCurrent implementation, design and foundations hit by:

Objectivity fate in HEP GRID claims for responsibility Unclear future in CMS of Python, Lizard, SWIG, ROOT, CINT Obsolescence of many basic components (some chosen before 1998)

Many fundamental issues reopened: Loose or tight integration with LCG persistency Sharing of responsibility with GRID MetaData: how, where, who Which UI, which GUI Scripting: yes, no, at which level C++ interpreter OR compiled and dynamically-loaded code Integration with build system and running environment

This very issues are addressed by the Blueprint RTAG whose report is due in few days. We can anticipate that CMS endorses it.



Build on Current SuccessSuccess of 2002 Data Challenge (Spring production)

Distributed production Simple installation Smooth running Easy crash-recovery Easy problem-detection Flexible “data product” selection

Distributed analysis Fast data validation and publishing Data and software distribution working from Tier1 down to Tier5 (portable PC) Efficient selection and cloning mechanism at sub-event level Direct contribution of many physicists to ORCA software

Thanks (in spite of) COBRA , Objectivity/DB, ORCA and CMS software-release and data-production toolsNew software should provide at least the same functionality and performance to start with



Federation

wizards

Detector/EventDisplay

Data Browser

Analysis jobwizards

Generic analysis Tools

ORCAORCA

FAMOSFAMOS

ObjyObjytoolstools

GRIDGRID

OSCAROSCARCOBRACOBRADistributedData Store

& ComputingInfrastructure

CMSCMStoolstools

Architecture Overview

ConsistentUser Interface

Coherent set of basic tools and mechanisms

Software development and installation



CompSci Analogy

DistributedData Store

& ComputingInfrastructure

TCP/IPTCP/IPprotocolprotocol

FileFileSystemSystem

SocketSocketAPIAPI

PosixPosixAPIAPI

FTPFTP

TELNETTELNET

SSHSSH



(O/R)DBMS

GEANT 3 / 4 CLHEP PAW

ReplacementC++ Standard Library+ Extension Toolkits

Component ArchitectureFramework Layering

CalibrationObjects Generic Generic

Application Application FrameworkFramework

Physics modulesPhysics modulesGrid-UploadableGrid-Uploadable

BasicBasicServicesServices

Adapters and ExtensionsAdapters and Extensions

ConfigurationObjects Event

Objects

(Grid-aware) Data-Products

SpecificSpecificFrameworksFrameworks

EventEventFilterFilter

ReconstructionReconstructionAlgorithmsAlgorithms

PhysicsPhysicsAnalysisAnalysis

DataDataMonitoringMonitoring

LCG



Blueprint Architectural ElementsTypes usable in public interfacesObject dictionaryObject bus (whiteboard)Component bus (Plug-in manager)Scripting language (ROOTCINT and Python both available)Component configurationBasic framework services

Framework infrastructures: creation of objects (factories), lifetime, multiplicity and scope (singleton, multiton, smart pointers), communication & discovery (eg. registries), …

Core services: Component management, incident management, monitoring & reporting, GUI manager, exception handling, …

System servicesFoundation and utility librariesUse of external software



Blueprint Domain Decomposition

Object dictionary and object model

Persistency and data management

Event processing frameworkEvent modelEvent generationDetector simulationDetector geometry and materialsTrigger/DAQEvent reconstructionDetector calibration

Scripting, interpreterGUI toolkitsGraphicsAnalysis toolsMath libraries and statisticsJob managementCore servicesFoundation and utility librariesGrid middleware interfaces

Red: Agreed to be common: project startedBrown: at least partly foreseen as common

projectGrey: not foreseen as common project



Blueprint architecture design precepts

Software structure: foundation; basic framework; specialized frameworksComponent model: APIs, collaboration (‘master/slave’, ‘peer-to-peer’), physical/logical module granularity, plug-ins, abstract interfaces, composition vs. inheritance, …Service model: Uniform, flexible access to functionalityObject models: dumb vs. smart, enforced policies with run-time checking, clear and bullet-proof ownership modelDistributed operationGlobal objectsDependenciesInterface to external components: generic adaptersException handling…



Do you remember ZEBRA?ZEBRA was essentially obsolete already at the start-up of LEPIt has been absolutely impossible to replace it (even reengineer it) with a modern dynamic memory manager because of

the way it was designed and implemented internally the pervasive way it was used down into the heart of physics and

reconstruction software.We should avoid that this situation will reproduce itself at LHC

Components should be independent and isolated The use of external products should be limited to their own domain Avoid gratuitous and exaggerated re-use

One hammer does not fit all screws The core of the physics and reconstruction software should not depend

from the details of the event-processing framework

The cost of code duplication and adapter-layers will pay as soon as a component will need to be replaced

Happening already: exit objectspace enter boost. exit Objy, enter Root.



Baseline: not a mere shopping list

Objective is a manageable component architecture made of units realistically replaceable

Define the global software model Granularity, role and nature of “Modules”

Physical vs logical modules(At last CMS plenary M.Livny concluded asking for statically linked, check-

pointable executables…) Reuse model of sub-components Which integration technologies (“glues”) have to be used, where and

howDefine THE set of basic components

Their functionalities The domain of their applicability Identify independent “vertical slices” and model their integration

Agree on Metrics to measure modularity Not only Frameworks, also applications based on them



Metrics: NCCD vs Cycles

0

2

4

6

8

10

12

0% 10% 20% 30% 40% 50% 60% 70%

Fraction of Packages in Cycles

NC

CD

Toolkits &Frameworks

ATLAS

ORCA

AnapheIGUANA

COBRAG4

ROOT



User Entry-Point



User Entry-Point

Persistent Classes



Prototyping new Persistency SolutionJune-December 2001 Oracle Evaluation

Oracle 9i judged not mature enough to be used as Event StoreNovember 2001-September 2002 Root Evaluation

Vanilla ROOT: use ROOT-Trees to model CMS event COBRA-ROOT: use COBRA event model replacing ooRef with TRef

April 2002- LCG Hybrid Store Remove or isolate explicit Objectivity dependencies in COBRA

PRS does not have any already now Produce a description of the ROOT file format Model MetaData using a RDBMS Identify all missing features required by a “production-grade” software

January-December 2002 Evaluation of RDBMS for Analysis Use RDBMS to model user data (event tags, n-tuple) Oracle 9i, Java, distributed WEB services PostgreSQL, C++, Python, Lizard



COBRA-ROOTMinimal perturbation of current framework, PRS software and production environmentUsage of minimal ROOT specific functionalities

Based on new ROOT functionalities such as TRefs and support for std containers

Full port of all COBRA persistency to Root (event and metadata) Integration with a RDBMS or flat-file MetaData catalog should be easy

Easy to port to LCG-POOL implementationMay suffer performance penalties w.r.t. a full “Rootification”

Prototype performance already similar to current Objy implementation Compatibility with Root-data-analysis unclear

Current prototype able to run DAQ-TDR jobs sequentially More work required for parallel population of a dataset



“Vanilla” ROOTFull exploitation of all ROOT capabilities as event modeler and manager

Use of Trees, Folder, CloneArrays etc Optimization of data structure to match ROOT requirements Use foreign classes for “data” objects (mitigate impact on PRS)

Heavy impact on framework and partially on PRS code Only the event has been re-modeled, MetaData require a new model based on a RDBMS

Should give maximal ROOT performanceFull compatibility with ROOT as analysis environmentPorting to final LCG-POOL will require major changes

LCG-POOL is supposed to hide the use of implementation-specific data-structure such as TTrees



A temporary persistent solution During next CMS week final reports on the two prototypes will be presentedWe propose to base the new version of COBRA (COBRA 7) on the COBRA-Root prototype as it offers a seamless transition from the current architectureWe suggest to maintain the Vanilla-Root prototype as a test-bed for new ROOT features and to study integration issues between POOL and ROOT in a controlled environmentThe expertise developed in the Vanilla-Root prototype should also be used to implement analysis objects based on Root-trees in the framework of COBRA 7.

Working products (COBRA ORCA OSCAR IGUANA) based on the new persistency solution, with all the current basic functionalities, should be ready for January 2003.The ability to test and integrate LCG-POOL components, as they are released, should be an integral part of the new product.



SimulationFull reengineering of all framework aspects of CMS simulation software (OSCAR and FAMOS)

Inline with reconstruction framework Full re-use of common interfaces, mechanisms and components Integration of DDD

Cleaner division between CCS and PRS responsibilities Generic Mechanisms in COBRA (Mantis sub-system) CMS specific software in OSCAR Specific packages to host detector code

This work incorporates and expands experiences from pre-existing software from CMS and ATLASWe consider it as an important contribution toward a future LCG simulation framework



Mantis Packages



Mantis application



Detector DescriptionDDD marked a major milestone with a full round-trip within Geant4

Full persistency mechanism for G4 geometry

Integration tests with ORCA to replace G3 geometry model are in progress (Tracker ready)CMS Prototype essentially finished:

Not CMS specific Supports several geometry models Not bound to XML Rely on external geometry modelers (currently G3 or G4)

An RTAG on the subject is in progress and we hope it will recommend a common project



Detector Description DataBase



VisualizationSpecialized Framework (IGUANA)

Working on full integration with COBRA

Interactive ORCA Visualisation Reconstruction Geometry Reconstructed Objects MonteCarlo truth

Interactive Geant4 Visualisation Explore and visualise

the physical volume tree, with all the usual IGUANA 3D features: view, picking, slices, …

Trajectories (soon hits) Magnetic field

Integrated with DDD overlap detection Find overlaps, show result details in a list and highlight overlaps in 3D

Geant4 command line A wizard to guide through OSCAR settings G3 functionalities (DCUT, DTREE) Volume tree selectors (using attributes: by material, sensitive, etc



Technology tracking and evaluationTracking the evolution of technology is essential to anticipate the need for changes.Recent activities:

Python XML RDBMS Template Meta Programming (boost, Loki) Math libraries (gsl, TNT, Blitz) New versions of Qt, OIV gcc 3.1, icc, Solaris CC 5.4 Non intrusive performance profilers (valgrind, oprofile)

CMS software R&D activities have been always tightly coupled with LCB RD-projects first, and IT/DB and IT/API later.Today they are fully integrated into the activities of LCG.



ScheduleOct 02 COBRA 7 prototype

Jan 03 COBRA 7 first releaseJan 03 start LCG integration

Jun 03 release production sw

Sep 03 baseline DC04 software

Jan 04 release DC04 software

Nov 02 first Hybrid store release

Jun 03 first LCG-1 releaseJun 03 Hybrid event storeJul 03 Start Preparation DC04

Nov 03 LCG-1 fully operational

Feb 04 DC04Apr 04 distributed analysis



ConclusionsWe have to guarantee a working software system that satisfies CMS needs at any given timePast and recent experience shows that we should make sure that any of its component can be replaced in a timescale of one year with negligible negative impact on PRS.LCG software will provide the foundation of CMS baseline

Expect to integrate first LCG components early next year DC04 will make use of LCG-1 software DC04 preparation may still require ad-hoc solutions

DC04 experience will tell us where to go next

vincenzo innocente, cern/ep central software 19 september 2002, cern internal review of cms software...

Documents