IEEE TRANSACTIONS ON NUCLEAR SCIENCE VOL 46 NO 3 JUNE 1999 392

The Use of Production Management Techniques in the Construction of Large Scale Physics Detectors

A Bazan G Chevenie F Estrell Z Kovac T Le Flour J-M Le Goffl S Lieunard R McClatchey S Murray L Varga J-PVialle M Zsenei6

LAPP IN2P3 Annecy-le-Vieux France Centre for Complex Cooperative Systems Univ of the West of England Frenchay Bristol BS16 1QY UK

31nstitut fur Teilchenphysik Eidgenossische Tech Hochschule Zurich SwitzerIand EP Division CERN121l-Geneva 23 Switzerland

SzTAKI Kende U 13-17 Budapest H-1111 Hungary KFKI-RMKI Budapest 114 H-1525 Hungary

Abstract The construction process of detectors for the Large

Hadron Collider (LHC) experiments is large scale heavily constrained by resource availability and evolves with time As a consequence changes in detector component design need to be tracked and quickly reflected in the construction process With similar problems in industry engineers employ so-called Product Data Management (PDM) systems to control access to documented versions of designs and managers employ so-called Workflow Management software (WfMS) to coordinate production work processes However PDM and WfMS software are not generally integrated in industry The scale of LHC experiments like CMS demands that industrial production techniques be applied in detector construction This paper outlines the major functions and applications of the CRISTAL system (Cooperating Repositories and an Information System for Tracking Assembly Lifecycles) in use in CMS which successfully integrates PDM and WfMS techniques in managing large scale physics detector construction This is the first time industrial production techniques have been deployed to this extent in detector construction

I INTRODUCTION Each CMS [l] detector will be constructed from a large

number of precision parts and will be produced and assembled during the next few years by centres distributed worldwide Each constituent part of each detector will be measured and tested locally prior to its assembly and integration in the experimental area at CERN Much of the information collected during this phase will be needed not only to construct the detector but for its calibration to facilitate accurate simulation of its performance and to assist in its lifetime maintenance The construction process is heavily dependent on many areas of research (materials science electronics computing) so that systematic tracking of the evolution of individual detector parts and quality control of the assembly process are essential for the subsequent calibration and maintenance of the detector Furthermore coordinating operations required for construction can be complex given the number of parts

requiring characterisation particularly when the operations are distributed over geographically separated centres

Large-scale industrial production systems (such as airplane manufacture) have similar requirements and often employ product data management (PDM [2]) tools to manage the data and documents accumulated in the design of product These systems are normally based on commercially available PDM products (eg IBMs PM Adras Matrix or Sherpa) and successfully support the creative and collaborative stages of product design where access to documents needs to be controlled between groups of designers However their use in supporting the unstructured processes inherent in product research and development is somewhat limited [3] - they provide limited facilities for activity definition and no facilities for the enactment of production activities Workflow management systems (WfMS [4]) on the other hand do provide utilities which facilitate the coordination and scheduling of the activities of organisations to optimise the flow of information between resources and there are many such tools becoming available commercially However WfMSs alone are weak at handling the dynamic evolution of process and activity definitions which occurs during design and during the enactment of workflow processes such as is characteristic of LHC detector development

No commercial products provide the workflow and product data management capabilities required by CMS A research project entitled CRISTAL SI [6] [7] has therefore been initiated using object oriented computing technologies where possible to facilitate the management of the engineering data collected at each stage of production of CMS CRISTAL uses the so-called as designed view of the detector (potentially stored in a Product Data Management system) to build a distributed production scheme which spans construction centres As the detector construction evolves during assembly and also during the lifetime of the experiment versions of the production scheme are dispatched from CERN to the centres where the so-called as built view of the detector is gathered as a consequence of following the execution of pre-defined activities On execution of these activities CRISTAL captures all the physical characteristics of detector components which are later required for detector construction calibration and maintenance This paper

0018-949999$1000 0 1999 IEEE

393

outlines the major functionalities of CRISTAL and how it is being used to manage CMS construction For clarity a glossary of acronyms used in this paper is presented in section VIII

11 CRISTAL REQUIREMENTS AND OBJECTIVES In the first instance CRISTAL is being used to monitor

and control the production and assembly process of the lead tungstate mono-crystals and their associated electronics to be installed in the CMS Electromagnetic Calorimeter (ECAL) [ 11 The software employs workflow and task management techniques and is generic in design and will therefore be reusable for other CMS detector groups such as the MSGC Tracker and the Hadronic calorimeter (HCAL) or indeed by groups with similar requirements outside of CMS

Following the growth and characterisation of the ECAL Barrel crystals in Production Centres in China and Russia assembly of the crystals with their Avalanche Photo-Diodes (APDs produced in France) and associated support structures (alveoli) will take place in Local Centres located in Italy UK and CERN Each of the component parts (crystals electronics alveoli) will have their physical characteristics individually measured and recorded to facilitate calibration and to ensure consistency of the production process (see figure 1)

SIC Bogorodisk IPN Lyon LPNHE (China) (Russia) (France) (France) Crystals Crystals Electronic Alveoli

v Parts

CERN EN EN1 NFN (Switzerland) + (Italy) Modules amp Modules I SuperModules

I DAPNIA I (France)

(Switzerland) (Switzerland) ECAL in CMS SM in Testbeam

I I

Figure 1 The distributed construction of CMS ECAL Barrel

Since the overall costs and timescales of ECAL crystal production must be strictly controlled the efficiency of the production process will be paramount Quality control must be enforced at each step in the fabrication process The CRISTAL system must support the testing of detector parts the archival of accumulated information controlled access to data and the on-line control and monitoring of all production and assembly centres

A legacy PDM system (called EDMS [8]) is currently being established at CERN to manage store and control all the documents relevant for the conception construction and exploitation of the LHC accelerator and experiments

throughout their life cycbe EDMS is defining these systems in terms of product breakdown structures and work breakdown structures (the as-designed view) All the engineering drawings blueprints construction procedures part definitions and part nominal values will be stored in EDMS The CRISTAL system could use EDMS to store the workflow definitions required to control the detector construction process CRISTAL needs to be both a production management and workflow management facility that tracks parts through the manufacturing assembly and maintenance life cycle Ulitimately it will provide the as-built view to information stored during ECAL construction which is required by physicists as the basis for all detector-related analyses and is capable of providing support for other views of the construction data

These views include the following as examples e A Calibration View Where physicists will want to view

and access part characteristic data for experiment calibration and event reconstruction purposes A Maintenance View Where engineers will refer to the production processes for assembly and disassembly procedures update information collected through maintenance operations and design modifications throughout the experiiments lifetime

An Experiment Systems Management View Where the so-called slow control system can view the part production history for configuration and fault management purposes

The Calibration view is investigated later as an example of how construction data can be accessed for physics analysis

In summary the CRISTAL project aims to implement a prototype distributed engineering information management system which will control the construction process of CMS ECAL Its specific objectives are to - 1 provide an information system to control quality in the

construction of detectors 2 monitor the global production process across distributed

centres 3 capture and store crystal data during detector

construction 4 integrate instruments used to characterise parts 5 provide controlled multi-user access to the production

management system and 6 provide access to engineering and calibration data for

CMS users

e

111 ARCHITECTURE AND DESIGN A CRISTAL system comprises one or more data

gathering centres (fig l) each of which is federated into the system These centres are a single Central System (for CMS this will be at CERN) (2nd Local Centres in which the CRISTAL software will run The CRISTAL system uses a set of roles (eg Coordinator Centre Supervisor Operator Physicist) to define user access to its software and data (see

394

[5] ) Each Local Centre will also have a set of Instruments defined in the database in terms of the commands that each instrument uses and the data formats expected as outcomes from the execution of activities by instruments [9]

Figure 2 shows the software architecture of a Local Centre The software comprises a set of Instrument Agents a set of Product Managers for handling all part data tofrom the database a Local Centre Manager which supervises the data gathering in a centre a set of Digital Control Panels (DCPs) which handle user interaction with CRISTAL and a Data Duplication Manager (DDM) which handles all duplication of data between the Local Centre and the Central System (for secure back-up) The Desktop Control Panels (DCP) are interpreted Java code which together provides the user interface to CRISTAL By swiping a barcode reader on a part the Operators are guided via the DCP through the possible construction task sequences that the part can follow and are instructed to carry out activities on the swiped part

Each instrument in the Centre (for example ACCOS [ 10)) has an instrument agent through which it can receive commands from the operator via the associated DCP and Product Manager The Instrument Agent communication protocol is ASCII based and is implemented either using CORBA or using a sockets interface driver depending on the instrumentrsquos computing capabilities The measurements from the instrument are converted by the Instrument Agent from ASCII into an object-based format that the Product Manager can use to store in the database

User Code

Object Request Broker I

1

Figure 2 Local Centre software architecture

CMS assembly will be carried out by groups with responsibilities for sub-detectors Each group needs to be only loosely coupled to others for final detector integration and must preserve their autonomy during the assembly process Each CRISTAL system is set up to manage the accumulation of physical characteristic data during the construction of a particular CMS sub-detector The construction follows a specific production plan and each detector is assembled and tested in a step-wise fashion At any point in the overall assembly of CMS there could be several CRISTAL systems active one for each logically

separated assembly process In the case of ECAL the assembly production plan is defined in steps from individual crystals APDs and capsules through sub-units and modules to supermodules Each step comprises a set of assembly sequences

To enable management of this construction procedure it is necessary to have knowledge of the detailed structure of the detector This structure is akin to the lsquoBill of Materialsrsquo mechanisms used in industry and it can be captured in a PDM system (such as the Cadim [ 111 system in use at CERN or the Matrix [ 121 system at Fermilab) Without this detector breakdown structure it would be impossible to keep track of the processes of detector component production and assembly Part quantities need to be attached to the part type information in the detector breakdown structure (see figure 3) and are used to allow visualisation of the complete Barrel structure and to provide mechanisms for the expansion of the type structure into an individual parts structure [13] It is this expanded ldquoas-builtrsquo parts structure which is ultimately required for gathering detector physics characteristics A subset of the overall ECAL Barrel breakdown structure revealing detail of a generic sub-module (of a specified type) is shown in figure 3 Each sub-module type comprises an alveolar structure type a tablet type five sub-units of type Left and five of type Right A Sub-unit type consists of a Capsule type and a Crystal type

I Figure 3 An ECAL Barrel Submodule and associated assembly sequences

In order to provide the mechanisms for controlling detector construction it is essential to specify an assembly sequence for each type of each element in the detector breakdown structure of figure 3 One example of an assembly sequence is shown in figure 4 that of gluing an APD onto a crystal Each assembly sequence is an ordered sequence of (potentially nested) activities each of which may require resources takes place at a certain location and may or may not be carried out in parallel with other activities Resources

395

that may be allocated are termed Agents and are of the type Instrument Human or User-provided code Agents execute activities and are the initiators of the data gathering process The assembly sequence may itself be composite in nature ie one assembly sequence may be composed of other assembly sequences

Figure 4 shows that activities (in the case shown lsquoglue crystal with APDrsquo) act on products (crystal capsule) if certain start conditions are fulfilled The start conditions are the pre-requisites for the activity to take place and include (at least) the products involved in the activity and if relevant the resources required to carry out the activity The conditions are tested at the outset of the activity and the activity can only progress when all start conditions have been fulfilled Each activity has a description takes place at a specified Local Centre(s) (eg Rome or CERN) can be allocated resources (eg an Operator Agent) can have a specified minimum and maximum duration may optionally be repeatable and when completed produces an outcome which is tested against a set of so-called end conditions It is this test against end conditions which ensures quality control in the execution of activities If end conditions fail it will be possible for exceptional activities to be invoked All of this information is specified for a product type named by the Coordinator in the central ECAL construction database for a particular class of activities Each instance of the activity is actually run at the specified Local Centre and if successful physical characteristics are computed and stored against the tested product for future use in for example detector calibration and maintenance

StaftConditions Parts reauired for lis activiiy to start

Activitv Name Glue Crystal with APD Outcome amp

DescriDtion Place capsule on End glueing stand put glue on crystal Conditions and lower crystal on capsule IsActivityRepeatable No

Resources Operator glueing

1 crystal amp capsule Amlicable Centres Rome CERN

Duration tmin =22 h tmax = 24 h Results I toDB 1 Figure 4 An activity definition (including resources Centres start and end conditions and outcomes)

Management information regarding the definition configuration version performance and operational state of the distributed CRISTAL production line is stored at CERN in a central repository based on the Objectivity object- oriented database product [ 141 This object-based repository also stores the definitions of the parts that make up the detector together with the definitions of the instruments used to produce or take measurements of parts It stores descriptions of the life-cycle of each part (its so-called production scheme) and descriptions of the tasks and activities performed on the part This production scheme (or sequence of tasks) determines the order of tasks that can be applied to any part The task execution script will run and

will prompt the operator to perform manual operations or may automatically trigger networked instruments to take measurements of the chalracteristics of the part (eg the so- called ACCOS rig [IO] which is used to measure dimensions and optical properties of the ECAL crystals) All the details of the task operations and the associated measurements and instruments involved are eventually stored in the central repository

The data handling aspects of CRISTAL must be transparent to any users of the system irrespective of location throughout the system Part task production scheme and system configuration datal will be defined by the Coordinator at CERN thereby controlling the configuration and management of all the centres Once the production centres are configured parts will be produced and then shipped to further local centres for testing and assembly All users can browse part characteristics and navigate through production schemes centre coordinators can modify local production schemes and tasks In each centre Product Managers provide access to the database optimise part information retrieval handle operator and instrument agents update part histories and interact with the software component that duplicates all part data centrally to CERN (the DDM see figure 2)

Iv PRODUCT DATA MANAGEMENT The development lifecycle of a large high-energy physics

detector is much like my other large-scale construction activity in that it follows al design-prototype-implement cycle It does differ from industrial production however in that it is research-based and highly iterative and consequently dynamic in execution The overall performance of the detector is highly dependent on its design and therefore any changes in design need to be permeated through from conceptual design to physical construction as quickly as possible The importance of rapidly reflecting design changes in production activities is becoming increasingly typical of examples from manufacturing engineering

Conceptual design is a collaborative activity with designers checking-out and checking-in documents and diagrams of components under some policy of configuration management [15] The data vault aspects of a PDM lend themselves well to this creative design process Product breakdown structures (PBS) are often strictly hierarchical in form and attributes can be assigned to each (sub-)part The advantages of using a PDM are well-documented elsewhere [2 3 81 PDM system provide a document change management service which can be used by engineering applications to assess control and minimise the impact of product and process changes that occur in complex manufacturing lifecycles According to [ 161 any PDM system used for the engineering of large-scale one-of-a-kind facilities (such as CMS) should hold the descriptions of both the PBS and the work breakdown structure (WBS) The PBS saves information pertaining to projects sub-projects documents items etc The WBS holds information about the organisation of tasks (or activities) to be performed and the resources required for each task The WBS defines the activities which enable the engineers to build the production

396

line that can be viewed as a collection of (versioned) interface and an execution service provided by a so-called activities The WBS holds the definitions of the production workflow engine The engine is the component that executes line and can be mapped onto workflows (see following the static workflow (production) descriptions which were section)and instantiations of these definitions are used to defined through the programming interface Commercial form the PBS WfMS products normally subsume both the specification and

Essentially the purpose of building a PBS for industrial enactment Of workflOws applications in a PDM is to facilitate the capture of a design To achieve integration between the PDM and workflow hierarchy of parts As the product becomes more complex in components of CRISTAL the PDM can be used to store a set structure however a parts explosion can take place in the of definitions of both the parts and the tasks that need to be PDM and data management becomes an issue In CRISTAL executed on the parts The workflow definitions are mapped the CMS detector will be constructed from millions of onto the WBS and the workflow instances can therefore be individual parts sets of which will be identical in nature It is derived from their definitions residing in the PDM The PDM simply infeasible to enter and manage these parts individually acts as the reference database for the enactment services of in the PDM Instead a concept of meta-data management the production workflow system and manages (versions of) [13] can be followed where definitions of parts are captured the PBS and the WBS (see figure 6)

In practice the CRISTAL system sits alongside the PDM in the PDM and manages the execution of tasks upon parts CRISTAL is in effect the enactment service of instances of the workflows as defined in the WBS part of the PDM As a consequence of this CRISTAL accesses the PDM for part definitions and for the specification of assembly sequences CRISTAL consequently integrates a PDM and Wfh4 facility and tracks parts through the manufacturing life cycle from a PDM- resident design to the final construction and assembly of the detector

In the same way that the PBS has been modelled using meta-objects the workflow definitions (WBS) of CRISTAL can also be modeled Figure 5 shows the overall data model Each part definition has at most one activity definition assigned to it Activities can be reused for several part definitions A part definition is an item of the PBS the activity definition is the workflow meta-object (WBS) A

The left side of Figure 5 Shows a UML [I71 object m d e l workflow meta-object is an ordered collection of activity Of the Product bmddown structure which underlies definitions which will be executed on every part registered in CRISTAL Part definitions are specified as either elementary the CRISTAL system with its corresponding part definition Or composite in nature Composite Parts are made UP of Other Activity definitions can be either elementary or composite in Parts and the ComPositeMember object reflects membership nature composite activities can be composed of several other of a part in a composite In implementing this meta-datu activity definitions

The workflow definitions (WBS) of CRISTAL can be management the tree representation of assembly breakdown is transformed into a graph representation [ 131 Individual modeled using the same concept of meta-objects as was the

definitions thus easing data management The next section given activity definition and a named part definition The CRISTAL data model has been designed so that each investigates how these meta-objects also facilitate the assignment of a Part Definition to an Activity Definition is integration of PDMs with WfMSs

Figure 5 Simplified UML Object Model

parts (Of the same type) are then associated with Common PBS Figure 5 shows that there is an association between a

v INTEGRATING PRODUCT DATA AND WORKFLOW MANAGEMENT

A workflow management syatem (WFMS) [4 181 ia a system that completely defines manages and executes activities through the execution of software whose order of execution is driven by a computer representation of the workflow logic The main components of a workflow management system are a workflow application programming interface and a workflow enactment service The workflow application programming interface allows for the specification of workflows and workflow activities The workflow enactment service consists of an execution

decfared for a specific purpose In detector construction the assignment is made to indicate the activity to be instantiated for the assembly of a particular instance of a part of a given part definition Each assignment has associated with it some Conditions in detector construction the data model captures the definition of the construction conditions required for each assignment of an activity definition to a part definition

This technique can be generalised for other applications For example the association of a maintenance activity to a part will require quite different conditions to be captured than when the detector was constructed Also the association of a calibration activity to a part would require calibration- specific conditions to be captured In other words the identified association between the activity and part

397

Centralized Database for Models

description worlds carries rich semantics This method of integrating PDM and WfMS through the definition of meta- objects and their mutual assignment is very powerful It allows many other links to be made between aspects of the overall CRISTAL data model the same mechanism can be used to assign agents to activity definitions for the purposes of enactment or the assignment of agents to part definitions for the purposes of resource management

PDM CERN Engineering Data Management system for Design

Product Breakdown Structure (Part Specification) (Taskspecification)

Mapping from Design to production

(Workflow Models) (Workflow Enactment)

Distributed Database for Production Data

I I Figure 6 The relationship between CRISTAL and a PDM

VI CALIBRATION DATA AND CRISTAL The construction database was first released in the

summer of 1998 enabling ECAL physicists to collect the salient characteristics of each Barrel component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector construction phase must be arranged and processed to allow population of a calibration database This section details how the ECAL construction database can be used as the source of calibration data and by extension as a generalised physics data source

For the purposes of physics analysis CMS ECAL needs to produce a pre-calibration database which will hold all the physical parameters necessary for calculating the ECAL static calibration constants for a single ECAL supermodule However construction and calibration are two quite different views of ECAL detector data Physicists must be able to extract sets of data relevant for determining calibration coefficients from the ECAL construction database (figure 7) into a pre-calibration database The information required for calibration includes data for crystals capsules supermodules and electronics Crystal-specific information will include light yields attenuation lengths longitudinal and transverse transmission (as functions of wavelength) and any crystal non-uniformities Capsule information needed for calibration includes gain and dark current vs high voltage and nominal biases All of this information will be captured and recorded in the ECAL construction database as the detector is constructed step-wise from individual crystals through sub- units to super-modules

Production

Datasubset I Test Beam Data

Beam

subset I Data

Figure 7 The extraction of calibration data from the CRISTAL construction database

The step-wise construction procedure leads to a data organisation necessarily different to that required for calibration the structure of the construction database follows the assembly ordering of the calorimeter while the structure of the calibration database must follow the ECAL readout structure where the unit of detector that is normally considered is the readout channel Calibration constants need to be determined for each readout channel (eg a crystal plus its Capsule electronics (ADC etc) and fibres) so that ADC counts can be translated into energy deposited in a single readout channel Therefore the calibration system must be able to extract subsets of physics data from the ECAL construction database for the calibration of particular physics elements (or sets of detector components) even if these elements are specified in a manner orthogonal to that in which structures are defined in the ECAL construction database

All parameters needed for the calibration of a single supermodule will be extracted from the ECAL construction database and a matrix i j of readout channels (or so-called physics elements) will be Ixdt in the pre-calibration database To these data information from test beam slow controls data acquisition and monitoring is added to build a complete picture of the conditions under which calibration-data was taken and on completion of the calibration runs these data are copied back into the construction database (see figure 7) The construction database then accumulates this calibration data for each supermodule and acts as the source of calibration data for all of 1 he 36 supermodules comprising the ECAL Barrel The final static calibration constants for the complete ECAL are extracted from the construction database into a calibration database Fast and efficient access from physicist programmes is allowed both to the pre-calibration database for a single supermodule in the test beam and to the set of constants in the full calibration database (see figure 7)

398

As the CMS construction process evolves so more data and the relationships between different aspects of the data must be permanently recorded in the CRISTAL data database Different groups of users will require flexible ways to find access and share this construction data The actual information required will depend on the viewpoint and the role of the user in the organisation User groups may well require a maintenance a geometry an alignment or an experiment slow- controls viewpoint The physicist therefore needs to define a viewpoint in terms of lsquophysics elementsrsquo (or sets of detector components) which are derived from the tree of physical locations of detector components resulting from detector assembly Figure 8 shows how a matrix of physics elements can be extracted from the lsquoas-builtrsquo detector construction hierarchy by providing software which can traverse the construction tree and can query and extract the construction data for a selected set of detector components

Dataease for SuperModule SuperModule Precalition in testbeam

Subunit ampI) = A physics Element ex A Readout channel for ECAL

Capsule rystal

Treeregresentatiou established during assembly Matrix representation extracted fian with ~nsm~o Part physical

Figure 8 The derivation of physics elements from the construction lsquoas-builtrsquo tree

Meta-modeling will assist the extraction of data for physics elements provided an extraction mechanism is developed which can navigaqe the meta-model can interpret the structures in the database and can present the data in a form meaningful to the end-user The extraction facility comprises a set of software processes which can be invoked either by a viewpoint-specific application (eg calibration) or by a viewpoint non-specific application That is the processes either navigate a generalised meta-model to project out viewpoint- specific data (ie lsquolooking intorsquo the meta-model) or they navigate the meta-model to correlate effects between viewpoints (ie lsquolooking outrsquo from the data model) In the lsquolooking inrsquo (viewpoint-specific) case the processes perform traversal of the detector description and extract the relevant physics data for the application In the lsquolooking outrsquo case (viewpoint non-specific) the processes determine the effect of a system-wide change on individual viewpoints or sets of viewpoints for example a temperature variation on calibration and slow controls viewpoints An extraction facility is currently under development for CRISTAL [ 191

VII CONCLUSIONS The scale of LHC experiments requires that rigorous

production management techniques be employed for the first time in high energy physics detector construction The CRISTAL project of CMS has implemented a construction database and information system for managing the construction of the CMS ECAL detector The software is currently being evaluated for use in CMS HCAL and Tracker and there has been interest from other experiments and from the LHC magnet construction teams

The CRISTAL data model has been developed using UML [ 171 and has been designed with a view to capturing both PDM and WfMS data The PDM-component of the data model [20] is used to store the full specification for detector construction It will contain all the definitions of individual parts and will describe what is to be built and how it is to be built The specifications are defined and held centrally and releases of versions of the production scheme are issued from the central database The workflow specification is also centrally defined and it will define the process of part characterisation and the complete assembly process from single parts to full-scale detector

The first phase of CRISTAL prototyping and technology evaluation was initiated in the summer of 1996 and delivery of a working prototype is due in the summer of 1998 based on a distributed Objectivity database This prototype will enable ECAL physicists to collect all of the salient characteristics of each ECAL component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector assembly phase must be arranged and processed to allow population of for example a calibration database

The second phase of the project aims to adopt a strong meta-data approach as described in [19] in the expectation that it will produce a more adaptable system capable of interoperating with future systems This meta-modeling approach will facilitate further integration between product data management and workflow management thereby providing consistency between design and production and speeding up the process of implementing design changes in a production system

Since the CRISTAL projectrsquos inception a considerable amount of interest has been generated in meta-models and meta-object description languages Work is in progress within the OMG on the Meta Object Facility (MOF) [21] which is expected to manage all kinds of meta-models relevant to the OMG Architecture It is expected that future releases of CRISTAL will benefit from this standardisation effort The meta-modeling approach adopted in the design of CRISTAL could reasonably be applied to organisations developing technologies for lsquovirtual enterprisesrsquo (such as in [22] [23]) where collections of autonomous databases could be related via a central enterprise meta-model Furthermore inrsquothe area of standardisation the Meta Data Coalition is working towards a Metadata Interchange Specification [24] which aims to

399

tackle the exchange sharing and management of metadata in industry The experiences of CRISTAL in providing viewpoint access (with location and application transparency) to enterprise is clearly relevant to research in these areas

VIII GLOSSARY OF ACRONYMS To aid the readability of this paper the authors include here

a glossary of all the acronyms deployed throughout the body of this paperrsquos text ACCOS Automatic Crystal Control System ADC Analogue-to-Digital Converter APD Avalanche Photo-Diode CMS The Compact Muon Solenoid CRISTAL Cooperating Repositories and an Information System for Tracking Assembly Lifecycles DAcq Data Acquisition DCP Digital Control Panel DDM Data Duplication Manager ECAL Electromagnetic Calorimeter EDMS Engineering Document Management System HCAL Hadronic Calorimeter LHC Large Hadron Collider MOF Meta-Object Facility MSGC Micro-Strip Gas Chambers OMG The Object Management Group PBS Product Breakdown Structure PDM Product Data Management UML Unified Modeling Language WBS Work Breakdown Structure WfMS Workflows Management Systems

Ix ACKNOWLEDGEMENTS The authors take this opportunity to acknowledge the

support of their home institutes In particular the support of P Lecoq J-L Faure and M Pimia is greatly appreciated Richard McClatchey is assisted by CERN and by the Royal Academy of Engineering under their Technology Foresight Award scheme during 1998 and acknowledges their support Laszlo Varga and Marton Zsenei are supported by the OMFB Budapest for their activities in CRISTAL

X REFERENCES [ 11 CMS Technical Proposal The CMS Collaboration Jan

1995 ftpcmsdoccernchTprefmPhtml [2] M Philpotts ldquoAn Introduction to the Concepts Benefits

and Terminology of Product Data Managementldquo Industrial Management amp Data Systems 1996 4 ppll-17

[3] P Pikosz amp J Malmqvist ldquoPossibilities and Limitations when Using EDM Systems to Support the Product

Development Processrdquo Proceedings of NordDesignrsquo96 pp165-175 Espoo Fiinland 1996

[4] D Georgakopoulos M Hornick amp A Sheth ldquoAn Overview of Workdlow Management from Process Modeling to Infrastructure for Automationrdquo Journal of Distributed and Parallel Database Systems 3 (2) April

[5] J-M Le Goff et al ldquoDetector Construction Management and Quality Control Establishing and using a CRISTAL Systemrdquo CERN CMS NOTE 1998033 Available from ftpcrnsdoccerncNdocuments98note98~033 pdf

[6] R McClatchey et al ldquoA Distributed Workflow amp Product Data Management Application for the Construction of Large Scale Scientifiic Apparatusrdquo NATO ASI Series F Computer amp Systems Sciences Vol 164 ISBN 3-540- 64411-3 Springer Verlag 1998 Presented at the NATO AS1 workshop on lsquoAdvances in Workflow Management Systems amp Interoperabilityrsquo Istanbul Turkey 1997

[7] Z Kovacs J-M Le Goff amp R McClatchey ldquoSupport for Product data from Design to Operationrdquo Journal of Computer Integrated Manufacturing Vol 11 No 1-2 Elsevier Science Ltd in press

[8] A P Hameri ldquoEDN[S - Concepts Motivations and Basic requirementsrdquo Procleedings of the CERN School of Computing 96 CERN 96-08

[9] E Auffray et al ldquoCertifying Procedure for Lead Tungstate Crystal Parameters during the Mass Production for CMS ECALrdquo Poster N20-31 at IEEE Nuclear Science Symposium Toronto Canada November 1998

[lo] J-P Peigneux et al ldquoStudies and Proposals for an Automatic Crystal Control Systemrdquo CERN CMS NOTE 19971036

[ l l ] Cadim a PDM product developed by Eigner amp Partner see URL httplwwwep-kade

[12] Matrix a PDM product from Adra Systems see URL httpwww adracom

[13] J-M Le Goff et al ldquoFrom Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systemsrdquo CERN CMS NOTE 19971104

[ 141 Objectivity an ob-ject-oriented database management system produced by Objectivity Inc httpwwwobj y co r (

[ 151 P H Feiler ldquoConfiguration Management Models in Commercial Environmentsrdquo Carnegie-Mellon Technical Reports CMUSEI-91-TR-7 ESD-91-TR-7 March 1991

[16] G Bachi amp A P Hameri ldquoWhat Should be Implemented at the Early Stages of a Lerge-Scale Projectrdquo CERN MT95-02 (DI) LHC Note 315

[17] M Fowler 2 K Scott ldquoUML Distilled - Applying the Standard Object Modeling Languagerdquo Addison-Wesley Longman Inc 1997

1995 ppll9-153

393

outlines the major functionalities of CRISTAL and how it is being used to manage CMS construction For clarity a glossary of acronyms used in this paper is presented in section VIII

11 CRISTAL REQUIREMENTS AND OBJECTIVES In the first instance CRISTAL is being used to monitor

and control the production and assembly process of the lead tungstate mono-crystals and their associated electronics to be installed in the CMS Electromagnetic Calorimeter (ECAL) [ 11 The software employs workflow and task management techniques and is generic in design and will therefore be reusable for other CMS detector groups such as the MSGC Tracker and the Hadronic calorimeter (HCAL) or indeed by groups with similar requirements outside of CMS

Following the growth and characterisation of the ECAL Barrel crystals in Production Centres in China and Russia assembly of the crystals with their Avalanche Photo-Diodes (APDs produced in France) and associated support structures (alveoli) will take place in Local Centres located in Italy UK and CERN Each of the component parts (crystals electronics alveoli) will have their physical characteristics individually measured and recorded to facilitate calibration and to ensure consistency of the production process (see figure 1)

SIC Bogorodisk IPN Lyon LPNHE (China) (Russia) (France) (France) Crystals Crystals Electronic Alveoli

v Parts

CERN EN EN1 NFN (Switzerland) + (Italy) Modules amp Modules I SuperModules

I DAPNIA I (France)

(Switzerland) (Switzerland) ECAL in CMS SM in Testbeam

I I

Figure 1 The distributed construction of CMS ECAL Barrel

Since the overall costs and timescales of ECAL crystal production must be strictly controlled the efficiency of the production process will be paramount Quality control must be enforced at each step in the fabrication process The CRISTAL system must support the testing of detector parts the archival of accumulated information controlled access to data and the on-line control and monitoring of all production and assembly centres

A legacy PDM system (called EDMS [8]) is currently being established at CERN to manage store and control all the documents relevant for the conception construction and exploitation of the LHC accelerator and experiments

throughout their life cycbe EDMS is defining these systems in terms of product breakdown structures and work breakdown structures (the as-designed view) All the engineering drawings blueprints construction procedures part definitions and part nominal values will be stored in EDMS The CRISTAL system could use EDMS to store the workflow definitions required to control the detector construction process CRISTAL needs to be both a production management and workflow management facility that tracks parts through the manufacturing assembly and maintenance life cycle Ulitimately it will provide the as-built view to information stored during ECAL construction which is required by physicists as the basis for all detector-related analyses and is capable of providing support for other views of the construction data

These views include the following as examples e A Calibration View Where physicists will want to view

and access part characteristic data for experiment calibration and event reconstruction purposes A Maintenance View Where engineers will refer to the production processes for assembly and disassembly procedures update information collected through maintenance operations and design modifications throughout the experiiments lifetime

An Experiment Systems Management View Where the so-called slow control system can view the part production history for configuration and fault management purposes

The Calibration view is investigated later as an example of how construction data can be accessed for physics analysis

In summary the CRISTAL project aims to implement a prototype distributed engineering information management system which will control the construction process of CMS ECAL Its specific objectives are to - 1 provide an information system to control quality in the

construction of detectors 2 monitor the global production process across distributed

centres 3 capture and store crystal data during detector

construction 4 integrate instruments used to characterise parts 5 provide controlled multi-user access to the production

management system and 6 provide access to engineering and calibration data for

CMS users

e

111 ARCHITECTURE AND DESIGN A CRISTAL system comprises one or more data

gathering centres (fig l) each of which is federated into the system These centres are a single Central System (for CMS this will be at CERN) (2nd Local Centres in which the CRISTAL software will run The CRISTAL system uses a set of roles (eg Coordinator Centre Supervisor Operator Physicist) to define user access to its software and data (see

394

[5] ) Each Local Centre will also have a set of Instruments defined in the database in terms of the commands that each instrument uses and the data formats expected as outcomes from the execution of activities by instruments [9]

Figure 2 shows the software architecture of a Local Centre The software comprises a set of Instrument Agents a set of Product Managers for handling all part data tofrom the database a Local Centre Manager which supervises the data gathering in a centre a set of Digital Control Panels (DCPs) which handle user interaction with CRISTAL and a Data Duplication Manager (DDM) which handles all duplication of data between the Local Centre and the Central System (for secure back-up) The Desktop Control Panels (DCP) are interpreted Java code which together provides the user interface to CRISTAL By swiping a barcode reader on a part the Operators are guided via the DCP through the possible construction task sequences that the part can follow and are instructed to carry out activities on the swiped part

Each instrument in the Centre (for example ACCOS [ 10)) has an instrument agent through which it can receive commands from the operator via the associated DCP and Product Manager The Instrument Agent communication protocol is ASCII based and is implemented either using CORBA or using a sockets interface driver depending on the instrumentrsquos computing capabilities The measurements from the instrument are converted by the Instrument Agent from ASCII into an object-based format that the Product Manager can use to store in the database

User Code

Object Request Broker I

1

Figure 2 Local Centre software architecture

CMS assembly will be carried out by groups with responsibilities for sub-detectors Each group needs to be only loosely coupled to others for final detector integration and must preserve their autonomy during the assembly process Each CRISTAL system is set up to manage the accumulation of physical characteristic data during the construction of a particular CMS sub-detector The construction follows a specific production plan and each detector is assembled and tested in a step-wise fashion At any point in the overall assembly of CMS there could be several CRISTAL systems active one for each logically

separated assembly process In the case of ECAL the assembly production plan is defined in steps from individual crystals APDs and capsules through sub-units and modules to supermodules Each step comprises a set of assembly sequences

To enable management of this construction procedure it is necessary to have knowledge of the detailed structure of the detector This structure is akin to the lsquoBill of Materialsrsquo mechanisms used in industry and it can be captured in a PDM system (such as the Cadim [ 111 system in use at CERN or the Matrix [ 121 system at Fermilab) Without this detector breakdown structure it would be impossible to keep track of the processes of detector component production and assembly Part quantities need to be attached to the part type information in the detector breakdown structure (see figure 3) and are used to allow visualisation of the complete Barrel structure and to provide mechanisms for the expansion of the type structure into an individual parts structure [13] It is this expanded ldquoas-builtrsquo parts structure which is ultimately required for gathering detector physics characteristics A subset of the overall ECAL Barrel breakdown structure revealing detail of a generic sub-module (of a specified type) is shown in figure 3 Each sub-module type comprises an alveolar structure type a tablet type five sub-units of type Left and five of type Right A Sub-unit type consists of a Capsule type and a Crystal type

I Figure 3 An ECAL Barrel Submodule and associated assembly sequences

In order to provide the mechanisms for controlling detector construction it is essential to specify an assembly sequence for each type of each element in the detector breakdown structure of figure 3 One example of an assembly sequence is shown in figure 4 that of gluing an APD onto a crystal Each assembly sequence is an ordered sequence of (potentially nested) activities each of which may require resources takes place at a certain location and may or may not be carried out in parallel with other activities Resources

395

that may be allocated are termed Agents and are of the type Instrument Human or User-provided code Agents execute activities and are the initiators of the data gathering process The assembly sequence may itself be composite in nature ie one assembly sequence may be composed of other assembly sequences

Figure 4 shows that activities (in the case shown lsquoglue crystal with APDrsquo) act on products (crystal capsule) if certain start conditions are fulfilled The start conditions are the pre-requisites for the activity to take place and include (at least) the products involved in the activity and if relevant the resources required to carry out the activity The conditions are tested at the outset of the activity and the activity can only progress when all start conditions have been fulfilled Each activity has a description takes place at a specified Local Centre(s) (eg Rome or CERN) can be allocated resources (eg an Operator Agent) can have a specified minimum and maximum duration may optionally be repeatable and when completed produces an outcome which is tested against a set of so-called end conditions It is this test against end conditions which ensures quality control in the execution of activities If end conditions fail it will be possible for exceptional activities to be invoked All of this information is specified for a product type named by the Coordinator in the central ECAL construction database for a particular class of activities Each instance of the activity is actually run at the specified Local Centre and if successful physical characteristics are computed and stored against the tested product for future use in for example detector calibration and maintenance

StaftConditions Parts reauired for lis activiiy to start

Activitv Name Glue Crystal with APD Outcome amp

DescriDtion Place capsule on End glueing stand put glue on crystal Conditions and lower crystal on capsule IsActivityRepeatable No

Resources Operator glueing

1 crystal amp capsule Amlicable Centres Rome CERN

Duration tmin =22 h tmax = 24 h Results I toDB 1 Figure 4 An activity definition (including resources Centres start and end conditions and outcomes)

Management information regarding the definition configuration version performance and operational state of the distributed CRISTAL production line is stored at CERN in a central repository based on the Objectivity object- oriented database product [ 141 This object-based repository also stores the definitions of the parts that make up the detector together with the definitions of the instruments used to produce or take measurements of parts It stores descriptions of the life-cycle of each part (its so-called production scheme) and descriptions of the tasks and activities performed on the part This production scheme (or sequence of tasks) determines the order of tasks that can be applied to any part The task execution script will run and

will prompt the operator to perform manual operations or may automatically trigger networked instruments to take measurements of the chalracteristics of the part (eg the so- called ACCOS rig [IO] which is used to measure dimensions and optical properties of the ECAL crystals) All the details of the task operations and the associated measurements and instruments involved are eventually stored in the central repository

The data handling aspects of CRISTAL must be transparent to any users of the system irrespective of location throughout the system Part task production scheme and system configuration datal will be defined by the Coordinator at CERN thereby controlling the configuration and management of all the centres Once the production centres are configured parts will be produced and then shipped to further local centres for testing and assembly All users can browse part characteristics and navigate through production schemes centre coordinators can modify local production schemes and tasks In each centre Product Managers provide access to the database optimise part information retrieval handle operator and instrument agents update part histories and interact with the software component that duplicates all part data centrally to CERN (the DDM see figure 2)

Iv PRODUCT DATA MANAGEMENT The development lifecycle of a large high-energy physics

detector is much like my other large-scale construction activity in that it follows al design-prototype-implement cycle It does differ from industrial production however in that it is research-based and highly iterative and consequently dynamic in execution The overall performance of the detector is highly dependent on its design and therefore any changes in design need to be permeated through from conceptual design to physical construction as quickly as possible The importance of rapidly reflecting design changes in production activities is becoming increasingly typical of examples from manufacturing engineering

Conceptual design is a collaborative activity with designers checking-out and checking-in documents and diagrams of components under some policy of configuration management [15] The data vault aspects of a PDM lend themselves well to this creative design process Product breakdown structures (PBS) are often strictly hierarchical in form and attributes can be assigned to each (sub-)part The advantages of using a PDM are well-documented elsewhere [2 3 81 PDM system provide a document change management service which can be used by engineering applications to assess control and minimise the impact of product and process changes that occur in complex manufacturing lifecycles According to [ 161 any PDM system used for the engineering of large-scale one-of-a-kind facilities (such as CMS) should hold the descriptions of both the PBS and the work breakdown structure (WBS) The PBS saves information pertaining to projects sub-projects documents items etc The WBS holds information about the organisation of tasks (or activities) to be performed and the resources required for each task The WBS defines the activities which enable the engineers to build the production

396

line that can be viewed as a collection of (versioned) interface and an execution service provided by a so-called activities The WBS holds the definitions of the production workflow engine The engine is the component that executes line and can be mapped onto workflows (see following the static workflow (production) descriptions which were section)and instantiations of these definitions are used to defined through the programming interface Commercial form the PBS WfMS products normally subsume both the specification and

Essentially the purpose of building a PBS for industrial enactment Of workflOws applications in a PDM is to facilitate the capture of a design To achieve integration between the PDM and workflow hierarchy of parts As the product becomes more complex in components of CRISTAL the PDM can be used to store a set structure however a parts explosion can take place in the of definitions of both the parts and the tasks that need to be PDM and data management becomes an issue In CRISTAL executed on the parts The workflow definitions are mapped the CMS detector will be constructed from millions of onto the WBS and the workflow instances can therefore be individual parts sets of which will be identical in nature It is derived from their definitions residing in the PDM The PDM simply infeasible to enter and manage these parts individually acts as the reference database for the enactment services of in the PDM Instead a concept of meta-data management the production workflow system and manages (versions of) [13] can be followed where definitions of parts are captured the PBS and the WBS (see figure 6)

In practice the CRISTAL system sits alongside the PDM in the PDM and manages the execution of tasks upon parts CRISTAL is in effect the enactment service of instances of the workflows as defined in the WBS part of the PDM As a consequence of this CRISTAL accesses the PDM for part definitions and for the specification of assembly sequences CRISTAL consequently integrates a PDM and Wfh4 facility and tracks parts through the manufacturing life cycle from a PDM- resident design to the final construction and assembly of the detector

In the same way that the PBS has been modelled using meta-objects the workflow definitions (WBS) of CRISTAL can also be modeled Figure 5 shows the overall data model Each part definition has at most one activity definition assigned to it Activities can be reused for several part definitions A part definition is an item of the PBS the activity definition is the workflow meta-object (WBS) A

The left side of Figure 5 Shows a UML [I71 object m d e l workflow meta-object is an ordered collection of activity Of the Product bmddown structure which underlies definitions which will be executed on every part registered in CRISTAL Part definitions are specified as either elementary the CRISTAL system with its corresponding part definition Or composite in nature Composite Parts are made UP of Other Activity definitions can be either elementary or composite in Parts and the ComPositeMember object reflects membership nature composite activities can be composed of several other of a part in a composite In implementing this meta-datu activity definitions

The workflow definitions (WBS) of CRISTAL can be management the tree representation of assembly breakdown is transformed into a graph representation [ 131 Individual modeled using the same concept of meta-objects as was the

definitions thus easing data management The next section given activity definition and a named part definition The CRISTAL data model has been designed so that each investigates how these meta-objects also facilitate the assignment of a Part Definition to an Activity Definition is integration of PDMs with WfMSs

Figure 5 Simplified UML Object Model

parts (Of the same type) are then associated with Common PBS Figure 5 shows that there is an association between a

v INTEGRATING PRODUCT DATA AND WORKFLOW MANAGEMENT

A workflow management syatem (WFMS) [4 181 ia a system that completely defines manages and executes activities through the execution of software whose order of execution is driven by a computer representation of the workflow logic The main components of a workflow management system are a workflow application programming interface and a workflow enactment service The workflow application programming interface allows for the specification of workflows and workflow activities The workflow enactment service consists of an execution

decfared for a specific purpose In detector construction the assignment is made to indicate the activity to be instantiated for the assembly of a particular instance of a part of a given part definition Each assignment has associated with it some Conditions in detector construction the data model captures the definition of the construction conditions required for each assignment of an activity definition to a part definition

This technique can be generalised for other applications For example the association of a maintenance activity to a part will require quite different conditions to be captured than when the detector was constructed Also the association of a calibration activity to a part would require calibration- specific conditions to be captured In other words the identified association between the activity and part

397

Centralized Database for Models

description worlds carries rich semantics This method of integrating PDM and WfMS through the definition of meta- objects and their mutual assignment is very powerful It allows many other links to be made between aspects of the overall CRISTAL data model the same mechanism can be used to assign agents to activity definitions for the purposes of enactment or the assignment of agents to part definitions for the purposes of resource management

PDM CERN Engineering Data Management system for Design

Product Breakdown Structure (Part Specification) (Taskspecification)

Mapping from Design to production

(Workflow Models) (Workflow Enactment)

Distributed Database for Production Data

I I Figure 6 The relationship between CRISTAL and a PDM

VI CALIBRATION DATA AND CRISTAL The construction database was first released in the

summer of 1998 enabling ECAL physicists to collect the salient characteristics of each Barrel component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector construction phase must be arranged and processed to allow population of a calibration database This section details how the ECAL construction database can be used as the source of calibration data and by extension as a generalised physics data source

For the purposes of physics analysis CMS ECAL needs to produce a pre-calibration database which will hold all the physical parameters necessary for calculating the ECAL static calibration constants for a single ECAL supermodule However construction and calibration are two quite different views of ECAL detector data Physicists must be able to extract sets of data relevant for determining calibration coefficients from the ECAL construction database (figure 7) into a pre-calibration database The information required for calibration includes data for crystals capsules supermodules and electronics Crystal-specific information will include light yields attenuation lengths longitudinal and transverse transmission (as functions of wavelength) and any crystal non-uniformities Capsule information needed for calibration includes gain and dark current vs high voltage and nominal biases All of this information will be captured and recorded in the ECAL construction database as the detector is constructed step-wise from individual crystals through sub- units to super-modules

Production

Datasubset I Test Beam Data

Beam

subset I Data

Figure 7 The extraction of calibration data from the CRISTAL construction database

The step-wise construction procedure leads to a data organisation necessarily different to that required for calibration the structure of the construction database follows the assembly ordering of the calorimeter while the structure of the calibration database must follow the ECAL readout structure where the unit of detector that is normally considered is the readout channel Calibration constants need to be determined for each readout channel (eg a crystal plus its Capsule electronics (ADC etc) and fibres) so that ADC counts can be translated into energy deposited in a single readout channel Therefore the calibration system must be able to extract subsets of physics data from the ECAL construction database for the calibration of particular physics elements (or sets of detector components) even if these elements are specified in a manner orthogonal to that in which structures are defined in the ECAL construction database

All parameters needed for the calibration of a single supermodule will be extracted from the ECAL construction database and a matrix i j of readout channels (or so-called physics elements) will be Ixdt in the pre-calibration database To these data information from test beam slow controls data acquisition and monitoring is added to build a complete picture of the conditions under which calibration-data was taken and on completion of the calibration runs these data are copied back into the construction database (see figure 7) The construction database then accumulates this calibration data for each supermodule and acts as the source of calibration data for all of 1 he 36 supermodules comprising the ECAL Barrel The final static calibration constants for the complete ECAL are extracted from the construction database into a calibration database Fast and efficient access from physicist programmes is allowed both to the pre-calibration database for a single supermodule in the test beam and to the set of constants in the full calibration database (see figure 7)

398

As the CMS construction process evolves so more data and the relationships between different aspects of the data must be permanently recorded in the CRISTAL data database Different groups of users will require flexible ways to find access and share this construction data The actual information required will depend on the viewpoint and the role of the user in the organisation User groups may well require a maintenance a geometry an alignment or an experiment slow- controls viewpoint The physicist therefore needs to define a viewpoint in terms of lsquophysics elementsrsquo (or sets of detector components) which are derived from the tree of physical locations of detector components resulting from detector assembly Figure 8 shows how a matrix of physics elements can be extracted from the lsquoas-builtrsquo detector construction hierarchy by providing software which can traverse the construction tree and can query and extract the construction data for a selected set of detector components

Dataease for SuperModule SuperModule Precalition in testbeam

Subunit ampI) = A physics Element ex A Readout channel for ECAL

Capsule rystal

Treeregresentatiou established during assembly Matrix representation extracted fian with ~nsm~o Part physical

Figure 8 The derivation of physics elements from the construction lsquoas-builtrsquo tree

Meta-modeling will assist the extraction of data for physics elements provided an extraction mechanism is developed which can navigaqe the meta-model can interpret the structures in the database and can present the data in a form meaningful to the end-user The extraction facility comprises a set of software processes which can be invoked either by a viewpoint-specific application (eg calibration) or by a viewpoint non-specific application That is the processes either navigate a generalised meta-model to project out viewpoint- specific data (ie lsquolooking intorsquo the meta-model) or they navigate the meta-model to correlate effects between viewpoints (ie lsquolooking outrsquo from the data model) In the lsquolooking inrsquo (viewpoint-specific) case the processes perform traversal of the detector description and extract the relevant physics data for the application In the lsquolooking outrsquo case (viewpoint non-specific) the processes determine the effect of a system-wide change on individual viewpoints or sets of viewpoints for example a temperature variation on calibration and slow controls viewpoints An extraction facility is currently under development for CRISTAL [ 191

VII CONCLUSIONS The scale of LHC experiments requires that rigorous

production management techniques be employed for the first time in high energy physics detector construction The CRISTAL project of CMS has implemented a construction database and information system for managing the construction of the CMS ECAL detector The software is currently being evaluated for use in CMS HCAL and Tracker and there has been interest from other experiments and from the LHC magnet construction teams

The CRISTAL data model has been developed using UML [ 171 and has been designed with a view to capturing both PDM and WfMS data The PDM-component of the data model [20] is used to store the full specification for detector construction It will contain all the definitions of individual parts and will describe what is to be built and how it is to be built The specifications are defined and held centrally and releases of versions of the production scheme are issued from the central database The workflow specification is also centrally defined and it will define the process of part characterisation and the complete assembly process from single parts to full-scale detector

The first phase of CRISTAL prototyping and technology evaluation was initiated in the summer of 1996 and delivery of a working prototype is due in the summer of 1998 based on a distributed Objectivity database This prototype will enable ECAL physicists to collect all of the salient characteristics of each ECAL component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector assembly phase must be arranged and processed to allow population of for example a calibration database

The second phase of the project aims to adopt a strong meta-data approach as described in [19] in the expectation that it will produce a more adaptable system capable of interoperating with future systems This meta-modeling approach will facilitate further integration between product data management and workflow management thereby providing consistency between design and production and speeding up the process of implementing design changes in a production system

Since the CRISTAL projectrsquos inception a considerable amount of interest has been generated in meta-models and meta-object description languages Work is in progress within the OMG on the Meta Object Facility (MOF) [21] which is expected to manage all kinds of meta-models relevant to the OMG Architecture It is expected that future releases of CRISTAL will benefit from this standardisation effort The meta-modeling approach adopted in the design of CRISTAL could reasonably be applied to organisations developing technologies for lsquovirtual enterprisesrsquo (such as in [22] [23]) where collections of autonomous databases could be related via a central enterprise meta-model Furthermore inrsquothe area of standardisation the Meta Data Coalition is working towards a Metadata Interchange Specification [24] which aims to

399

tackle the exchange sharing and management of metadata in industry The experiences of CRISTAL in providing viewpoint access (with location and application transparency) to enterprise is clearly relevant to research in these areas

VIII GLOSSARY OF ACRONYMS To aid the readability of this paper the authors include here

a glossary of all the acronyms deployed throughout the body of this paperrsquos text ACCOS Automatic Crystal Control System ADC Analogue-to-Digital Converter APD Avalanche Photo-Diode CMS The Compact Muon Solenoid CRISTAL Cooperating Repositories and an Information System for Tracking Assembly Lifecycles DAcq Data Acquisition DCP Digital Control Panel DDM Data Duplication Manager ECAL Electromagnetic Calorimeter EDMS Engineering Document Management System HCAL Hadronic Calorimeter LHC Large Hadron Collider MOF Meta-Object Facility MSGC Micro-Strip Gas Chambers OMG The Object Management Group PBS Product Breakdown Structure PDM Product Data Management UML Unified Modeling Language WBS Work Breakdown Structure WfMS Workflows Management Systems

Ix ACKNOWLEDGEMENTS The authors take this opportunity to acknowledge the

support of their home institutes In particular the support of P Lecoq J-L Faure and M Pimia is greatly appreciated Richard McClatchey is assisted by CERN and by the Royal Academy of Engineering under their Technology Foresight Award scheme during 1998 and acknowledges their support Laszlo Varga and Marton Zsenei are supported by the OMFB Budapest for their activities in CRISTAL

X REFERENCES [ 11 CMS Technical Proposal The CMS Collaboration Jan

1995 ftpcmsdoccernchTprefmPhtml [2] M Philpotts ldquoAn Introduction to the Concepts Benefits

and Terminology of Product Data Managementldquo Industrial Management amp Data Systems 1996 4 ppll-17

[3] P Pikosz amp J Malmqvist ldquoPossibilities and Limitations when Using EDM Systems to Support the Product

Development Processrdquo Proceedings of NordDesignrsquo96 pp165-175 Espoo Fiinland 1996

[4] D Georgakopoulos M Hornick amp A Sheth ldquoAn Overview of Workdlow Management from Process Modeling to Infrastructure for Automationrdquo Journal of Distributed and Parallel Database Systems 3 (2) April

[5] J-M Le Goff et al ldquoDetector Construction Management and Quality Control Establishing and using a CRISTAL Systemrdquo CERN CMS NOTE 1998033 Available from ftpcrnsdoccerncNdocuments98note98~033 pdf

[6] R McClatchey et al ldquoA Distributed Workflow amp Product Data Management Application for the Construction of Large Scale Scientifiic Apparatusrdquo NATO ASI Series F Computer amp Systems Sciences Vol 164 ISBN 3-540- 64411-3 Springer Verlag 1998 Presented at the NATO AS1 workshop on lsquoAdvances in Workflow Management Systems amp Interoperabilityrsquo Istanbul Turkey 1997

[7] Z Kovacs J-M Le Goff amp R McClatchey ldquoSupport for Product data from Design to Operationrdquo Journal of Computer Integrated Manufacturing Vol 11 No 1-2 Elsevier Science Ltd in press

[8] A P Hameri ldquoEDN[S - Concepts Motivations and Basic requirementsrdquo Procleedings of the CERN School of Computing 96 CERN 96-08

[9] E Auffray et al ldquoCertifying Procedure for Lead Tungstate Crystal Parameters during the Mass Production for CMS ECALrdquo Poster N20-31 at IEEE Nuclear Science Symposium Toronto Canada November 1998

[lo] J-P Peigneux et al ldquoStudies and Proposals for an Automatic Crystal Control Systemrdquo CERN CMS NOTE 19971036

[ l l ] Cadim a PDM product developed by Eigner amp Partner see URL httplwwwep-kade

[12] Matrix a PDM product from Adra Systems see URL httpwww adracom

[13] J-M Le Goff et al ldquoFrom Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systemsrdquo CERN CMS NOTE 19971104

[ 141 Objectivity an ob-ject-oriented database management system produced by Objectivity Inc httpwwwobj y co r (

[ 151 P H Feiler ldquoConfiguration Management Models in Commercial Environmentsrdquo Carnegie-Mellon Technical Reports CMUSEI-91-TR-7 ESD-91-TR-7 March 1991

[16] G Bachi amp A P Hameri ldquoWhat Should be Implemented at the Early Stages of a Lerge-Scale Projectrdquo CERN MT95-02 (DI) LHC Note 315

[17] M Fowler 2 K Scott ldquoUML Distilled - Applying the Standard Object Modeling Languagerdquo Addison-Wesley Longman Inc 1997

1995 ppll9-153

394

[5] ) Each Local Centre will also have a set of Instruments defined in the database in terms of the commands that each instrument uses and the data formats expected as outcomes from the execution of activities by instruments [9]

Figure 2 shows the software architecture of a Local Centre The software comprises a set of Instrument Agents a set of Product Managers for handling all part data tofrom the database a Local Centre Manager which supervises the data gathering in a centre a set of Digital Control Panels (DCPs) which handle user interaction with CRISTAL and a Data Duplication Manager (DDM) which handles all duplication of data between the Local Centre and the Central System (for secure back-up) The Desktop Control Panels (DCP) are interpreted Java code which together provides the user interface to CRISTAL By swiping a barcode reader on a part the Operators are guided via the DCP through the possible construction task sequences that the part can follow and are instructed to carry out activities on the swiped part

Each instrument in the Centre (for example ACCOS [ 10)) has an instrument agent through which it can receive commands from the operator via the associated DCP and Product Manager The Instrument Agent communication protocol is ASCII based and is implemented either using CORBA or using a sockets interface driver depending on the instrumentrsquos computing capabilities The measurements from the instrument are converted by the Instrument Agent from ASCII into an object-based format that the Product Manager can use to store in the database

User Code

Object Request Broker I

1

Figure 2 Local Centre software architecture

CMS assembly will be carried out by groups with responsibilities for sub-detectors Each group needs to be only loosely coupled to others for final detector integration and must preserve their autonomy during the assembly process Each CRISTAL system is set up to manage the accumulation of physical characteristic data during the construction of a particular CMS sub-detector The construction follows a specific production plan and each detector is assembled and tested in a step-wise fashion At any point in the overall assembly of CMS there could be several CRISTAL systems active one for each logically

separated assembly process In the case of ECAL the assembly production plan is defined in steps from individual crystals APDs and capsules through sub-units and modules to supermodules Each step comprises a set of assembly sequences

To enable management of this construction procedure it is necessary to have knowledge of the detailed structure of the detector This structure is akin to the lsquoBill of Materialsrsquo mechanisms used in industry and it can be captured in a PDM system (such as the Cadim [ 111 system in use at CERN or the Matrix [ 121 system at Fermilab) Without this detector breakdown structure it would be impossible to keep track of the processes of detector component production and assembly Part quantities need to be attached to the part type information in the detector breakdown structure (see figure 3) and are used to allow visualisation of the complete Barrel structure and to provide mechanisms for the expansion of the type structure into an individual parts structure [13] It is this expanded ldquoas-builtrsquo parts structure which is ultimately required for gathering detector physics characteristics A subset of the overall ECAL Barrel breakdown structure revealing detail of a generic sub-module (of a specified type) is shown in figure 3 Each sub-module type comprises an alveolar structure type a tablet type five sub-units of type Left and five of type Right A Sub-unit type consists of a Capsule type and a Crystal type

I Figure 3 An ECAL Barrel Submodule and associated assembly sequences

In order to provide the mechanisms for controlling detector construction it is essential to specify an assembly sequence for each type of each element in the detector breakdown structure of figure 3 One example of an assembly sequence is shown in figure 4 that of gluing an APD onto a crystal Each assembly sequence is an ordered sequence of (potentially nested) activities each of which may require resources takes place at a certain location and may or may not be carried out in parallel with other activities Resources

395

that may be allocated are termed Agents and are of the type Instrument Human or User-provided code Agents execute activities and are the initiators of the data gathering process The assembly sequence may itself be composite in nature ie one assembly sequence may be composed of other assembly sequences

Figure 4 shows that activities (in the case shown lsquoglue crystal with APDrsquo) act on products (crystal capsule) if certain start conditions are fulfilled The start conditions are the pre-requisites for the activity to take place and include (at least) the products involved in the activity and if relevant the resources required to carry out the activity The conditions are tested at the outset of the activity and the activity can only progress when all start conditions have been fulfilled Each activity has a description takes place at a specified Local Centre(s) (eg Rome or CERN) can be allocated resources (eg an Operator Agent) can have a specified minimum and maximum duration may optionally be repeatable and when completed produces an outcome which is tested against a set of so-called end conditions It is this test against end conditions which ensures quality control in the execution of activities If end conditions fail it will be possible for exceptional activities to be invoked All of this information is specified for a product type named by the Coordinator in the central ECAL construction database for a particular class of activities Each instance of the activity is actually run at the specified Local Centre and if successful physical characteristics are computed and stored against the tested product for future use in for example detector calibration and maintenance

StaftConditions Parts reauired for lis activiiy to start

Activitv Name Glue Crystal with APD Outcome amp

DescriDtion Place capsule on End glueing stand put glue on crystal Conditions and lower crystal on capsule IsActivityRepeatable No

Resources Operator glueing

1 crystal amp capsule Amlicable Centres Rome CERN

Duration tmin =22 h tmax = 24 h Results I toDB 1 Figure 4 An activity definition (including resources Centres start and end conditions and outcomes)

Management information regarding the definition configuration version performance and operational state of the distributed CRISTAL production line is stored at CERN in a central repository based on the Objectivity object- oriented database product [ 141 This object-based repository also stores the definitions of the parts that make up the detector together with the definitions of the instruments used to produce or take measurements of parts It stores descriptions of the life-cycle of each part (its so-called production scheme) and descriptions of the tasks and activities performed on the part This production scheme (or sequence of tasks) determines the order of tasks that can be applied to any part The task execution script will run and

will prompt the operator to perform manual operations or may automatically trigger networked instruments to take measurements of the chalracteristics of the part (eg the so- called ACCOS rig [IO] which is used to measure dimensions and optical properties of the ECAL crystals) All the details of the task operations and the associated measurements and instruments involved are eventually stored in the central repository

The data handling aspects of CRISTAL must be transparent to any users of the system irrespective of location throughout the system Part task production scheme and system configuration datal will be defined by the Coordinator at CERN thereby controlling the configuration and management of all the centres Once the production centres are configured parts will be produced and then shipped to further local centres for testing and assembly All users can browse part characteristics and navigate through production schemes centre coordinators can modify local production schemes and tasks In each centre Product Managers provide access to the database optimise part information retrieval handle operator and instrument agents update part histories and interact with the software component that duplicates all part data centrally to CERN (the DDM see figure 2)

Iv PRODUCT DATA MANAGEMENT The development lifecycle of a large high-energy physics

detector is much like my other large-scale construction activity in that it follows al design-prototype-implement cycle It does differ from industrial production however in that it is research-based and highly iterative and consequently dynamic in execution The overall performance of the detector is highly dependent on its design and therefore any changes in design need to be permeated through from conceptual design to physical construction as quickly as possible The importance of rapidly reflecting design changes in production activities is becoming increasingly typical of examples from manufacturing engineering

Conceptual design is a collaborative activity with designers checking-out and checking-in documents and diagrams of components under some policy of configuration management [15] The data vault aspects of a PDM lend themselves well to this creative design process Product breakdown structures (PBS) are often strictly hierarchical in form and attributes can be assigned to each (sub-)part The advantages of using a PDM are well-documented elsewhere [2 3 81 PDM system provide a document change management service which can be used by engineering applications to assess control and minimise the impact of product and process changes that occur in complex manufacturing lifecycles According to [ 161 any PDM system used for the engineering of large-scale one-of-a-kind facilities (such as CMS) should hold the descriptions of both the PBS and the work breakdown structure (WBS) The PBS saves information pertaining to projects sub-projects documents items etc The WBS holds information about the organisation of tasks (or activities) to be performed and the resources required for each task The WBS defines the activities which enable the engineers to build the production

396

line that can be viewed as a collection of (versioned) interface and an execution service provided by a so-called activities The WBS holds the definitions of the production workflow engine The engine is the component that executes line and can be mapped onto workflows (see following the static workflow (production) descriptions which were section)and instantiations of these definitions are used to defined through the programming interface Commercial form the PBS WfMS products normally subsume both the specification and

Essentially the purpose of building a PBS for industrial enactment Of workflOws applications in a PDM is to facilitate the capture of a design To achieve integration between the PDM and workflow hierarchy of parts As the product becomes more complex in components of CRISTAL the PDM can be used to store a set structure however a parts explosion can take place in the of definitions of both the parts and the tasks that need to be PDM and data management becomes an issue In CRISTAL executed on the parts The workflow definitions are mapped the CMS detector will be constructed from millions of onto the WBS and the workflow instances can therefore be individual parts sets of which will be identical in nature It is derived from their definitions residing in the PDM The PDM simply infeasible to enter and manage these parts individually acts as the reference database for the enactment services of in the PDM Instead a concept of meta-data management the production workflow system and manages (versions of) [13] can be followed where definitions of parts are captured the PBS and the WBS (see figure 6)

In practice the CRISTAL system sits alongside the PDM in the PDM and manages the execution of tasks upon parts CRISTAL is in effect the enactment service of instances of the workflows as defined in the WBS part of the PDM As a consequence of this CRISTAL accesses the PDM for part definitions and for the specification of assembly sequences CRISTAL consequently integrates a PDM and Wfh4 facility and tracks parts through the manufacturing life cycle from a PDM- resident design to the final construction and assembly of the detector

In the same way that the PBS has been modelled using meta-objects the workflow definitions (WBS) of CRISTAL can also be modeled Figure 5 shows the overall data model Each part definition has at most one activity definition assigned to it Activities can be reused for several part definitions A part definition is an item of the PBS the activity definition is the workflow meta-object (WBS) A

The left side of Figure 5 Shows a UML [I71 object m d e l workflow meta-object is an ordered collection of activity Of the Product bmddown structure which underlies definitions which will be executed on every part registered in CRISTAL Part definitions are specified as either elementary the CRISTAL system with its corresponding part definition Or composite in nature Composite Parts are made UP of Other Activity definitions can be either elementary or composite in Parts and the ComPositeMember object reflects membership nature composite activities can be composed of several other of a part in a composite In implementing this meta-datu activity definitions

The workflow definitions (WBS) of CRISTAL can be management the tree representation of assembly breakdown is transformed into a graph representation [ 131 Individual modeled using the same concept of meta-objects as was the

definitions thus easing data management The next section given activity definition and a named part definition The CRISTAL data model has been designed so that each investigates how these meta-objects also facilitate the assignment of a Part Definition to an Activity Definition is integration of PDMs with WfMSs

Figure 5 Simplified UML Object Model

parts (Of the same type) are then associated with Common PBS Figure 5 shows that there is an association between a

v INTEGRATING PRODUCT DATA AND WORKFLOW MANAGEMENT

A workflow management syatem (WFMS) [4 181 ia a system that completely defines manages and executes activities through the execution of software whose order of execution is driven by a computer representation of the workflow logic The main components of a workflow management system are a workflow application programming interface and a workflow enactment service The workflow application programming interface allows for the specification of workflows and workflow activities The workflow enactment service consists of an execution

decfared for a specific purpose In detector construction the assignment is made to indicate the activity to be instantiated for the assembly of a particular instance of a part of a given part definition Each assignment has associated with it some Conditions in detector construction the data model captures the definition of the construction conditions required for each assignment of an activity definition to a part definition

This technique can be generalised for other applications For example the association of a maintenance activity to a part will require quite different conditions to be captured than when the detector was constructed Also the association of a calibration activity to a part would require calibration- specific conditions to be captured In other words the identified association between the activity and part

397

Centralized Database for Models

description worlds carries rich semantics This method of integrating PDM and WfMS through the definition of meta- objects and their mutual assignment is very powerful It allows many other links to be made between aspects of the overall CRISTAL data model the same mechanism can be used to assign agents to activity definitions for the purposes of enactment or the assignment of agents to part definitions for the purposes of resource management

PDM CERN Engineering Data Management system for Design

Product Breakdown Structure (Part Specification) (Taskspecification)

Mapping from Design to production

(Workflow Models) (Workflow Enactment)

Distributed Database for Production Data

I I Figure 6 The relationship between CRISTAL and a PDM

VI CALIBRATION DATA AND CRISTAL The construction database was first released in the

summer of 1998 enabling ECAL physicists to collect the salient characteristics of each Barrel component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector construction phase must be arranged and processed to allow population of a calibration database This section details how the ECAL construction database can be used as the source of calibration data and by extension as a generalised physics data source

For the purposes of physics analysis CMS ECAL needs to produce a pre-calibration database which will hold all the physical parameters necessary for calculating the ECAL static calibration constants for a single ECAL supermodule However construction and calibration are two quite different views of ECAL detector data Physicists must be able to extract sets of data relevant for determining calibration coefficients from the ECAL construction database (figure 7) into a pre-calibration database The information required for calibration includes data for crystals capsules supermodules and electronics Crystal-specific information will include light yields attenuation lengths longitudinal and transverse transmission (as functions of wavelength) and any crystal non-uniformities Capsule information needed for calibration includes gain and dark current vs high voltage and nominal biases All of this information will be captured and recorded in the ECAL construction database as the detector is constructed step-wise from individual crystals through sub- units to super-modules

Production

Datasubset I Test Beam Data

Beam

subset I Data

Figure 7 The extraction of calibration data from the CRISTAL construction database

The step-wise construction procedure leads to a data organisation necessarily different to that required for calibration the structure of the construction database follows the assembly ordering of the calorimeter while the structure of the calibration database must follow the ECAL readout structure where the unit of detector that is normally considered is the readout channel Calibration constants need to be determined for each readout channel (eg a crystal plus its Capsule electronics (ADC etc) and fibres) so that ADC counts can be translated into energy deposited in a single readout channel Therefore the calibration system must be able to extract subsets of physics data from the ECAL construction database for the calibration of particular physics elements (or sets of detector components) even if these elements are specified in a manner orthogonal to that in which structures are defined in the ECAL construction database

All parameters needed for the calibration of a single supermodule will be extracted from the ECAL construction database and a matrix i j of readout channels (or so-called physics elements) will be Ixdt in the pre-calibration database To these data information from test beam slow controls data acquisition and monitoring is added to build a complete picture of the conditions under which calibration-data was taken and on completion of the calibration runs these data are copied back into the construction database (see figure 7) The construction database then accumulates this calibration data for each supermodule and acts as the source of calibration data for all of 1 he 36 supermodules comprising the ECAL Barrel The final static calibration constants for the complete ECAL are extracted from the construction database into a calibration database Fast and efficient access from physicist programmes is allowed both to the pre-calibration database for a single supermodule in the test beam and to the set of constants in the full calibration database (see figure 7)

398

As the CMS construction process evolves so more data and the relationships between different aspects of the data must be permanently recorded in the CRISTAL data database Different groups of users will require flexible ways to find access and share this construction data The actual information required will depend on the viewpoint and the role of the user in the organisation User groups may well require a maintenance a geometry an alignment or an experiment slow- controls viewpoint The physicist therefore needs to define a viewpoint in terms of lsquophysics elementsrsquo (or sets of detector components) which are derived from the tree of physical locations of detector components resulting from detector assembly Figure 8 shows how a matrix of physics elements can be extracted from the lsquoas-builtrsquo detector construction hierarchy by providing software which can traverse the construction tree and can query and extract the construction data for a selected set of detector components

Dataease for SuperModule SuperModule Precalition in testbeam

Subunit ampI) = A physics Element ex A Readout channel for ECAL

Capsule rystal

Treeregresentatiou established during assembly Matrix representation extracted fian with ~nsm~o Part physical

Figure 8 The derivation of physics elements from the construction lsquoas-builtrsquo tree

Meta-modeling will assist the extraction of data for physics elements provided an extraction mechanism is developed which can navigaqe the meta-model can interpret the structures in the database and can present the data in a form meaningful to the end-user The extraction facility comprises a set of software processes which can be invoked either by a viewpoint-specific application (eg calibration) or by a viewpoint non-specific application That is the processes either navigate a generalised meta-model to project out viewpoint- specific data (ie lsquolooking intorsquo the meta-model) or they navigate the meta-model to correlate effects between viewpoints (ie lsquolooking outrsquo from the data model) In the lsquolooking inrsquo (viewpoint-specific) case the processes perform traversal of the detector description and extract the relevant physics data for the application In the lsquolooking outrsquo case (viewpoint non-specific) the processes determine the effect of a system-wide change on individual viewpoints or sets of viewpoints for example a temperature variation on calibration and slow controls viewpoints An extraction facility is currently under development for CRISTAL [ 191

VII CONCLUSIONS The scale of LHC experiments requires that rigorous

production management techniques be employed for the first time in high energy physics detector construction The CRISTAL project of CMS has implemented a construction database and information system for managing the construction of the CMS ECAL detector The software is currently being evaluated for use in CMS HCAL and Tracker and there has been interest from other experiments and from the LHC magnet construction teams

The CRISTAL data model has been developed using UML [ 171 and has been designed with a view to capturing both PDM and WfMS data The PDM-component of the data model [20] is used to store the full specification for detector construction It will contain all the definitions of individual parts and will describe what is to be built and how it is to be built The specifications are defined and held centrally and releases of versions of the production scheme are issued from the central database The workflow specification is also centrally defined and it will define the process of part characterisation and the complete assembly process from single parts to full-scale detector

The first phase of CRISTAL prototyping and technology evaluation was initiated in the summer of 1996 and delivery of a working prototype is due in the summer of 1998 based on a distributed Objectivity database This prototype will enable ECAL physicists to collect all of the salient characteristics of each ECAL component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector assembly phase must be arranged and processed to allow population of for example a calibration database

The second phase of the project aims to adopt a strong meta-data approach as described in [19] in the expectation that it will produce a more adaptable system capable of interoperating with future systems This meta-modeling approach will facilitate further integration between product data management and workflow management thereby providing consistency between design and production and speeding up the process of implementing design changes in a production system

Since the CRISTAL projectrsquos inception a considerable amount of interest has been generated in meta-models and meta-object description languages Work is in progress within the OMG on the Meta Object Facility (MOF) [21] which is expected to manage all kinds of meta-models relevant to the OMG Architecture It is expected that future releases of CRISTAL will benefit from this standardisation effort The meta-modeling approach adopted in the design of CRISTAL could reasonably be applied to organisations developing technologies for lsquovirtual enterprisesrsquo (such as in [22] [23]) where collections of autonomous databases could be related via a central enterprise meta-model Furthermore inrsquothe area of standardisation the Meta Data Coalition is working towards a Metadata Interchange Specification [24] which aims to

399

tackle the exchange sharing and management of metadata in industry The experiences of CRISTAL in providing viewpoint access (with location and application transparency) to enterprise is clearly relevant to research in these areas

VIII GLOSSARY OF ACRONYMS To aid the readability of this paper the authors include here

a glossary of all the acronyms deployed throughout the body of this paperrsquos text ACCOS Automatic Crystal Control System ADC Analogue-to-Digital Converter APD Avalanche Photo-Diode CMS The Compact Muon Solenoid CRISTAL Cooperating Repositories and an Information System for Tracking Assembly Lifecycles DAcq Data Acquisition DCP Digital Control Panel DDM Data Duplication Manager ECAL Electromagnetic Calorimeter EDMS Engineering Document Management System HCAL Hadronic Calorimeter LHC Large Hadron Collider MOF Meta-Object Facility MSGC Micro-Strip Gas Chambers OMG The Object Management Group PBS Product Breakdown Structure PDM Product Data Management UML Unified Modeling Language WBS Work Breakdown Structure WfMS Workflows Management Systems

Ix ACKNOWLEDGEMENTS The authors take this opportunity to acknowledge the

support of their home institutes In particular the support of P Lecoq J-L Faure and M Pimia is greatly appreciated Richard McClatchey is assisted by CERN and by the Royal Academy of Engineering under their Technology Foresight Award scheme during 1998 and acknowledges their support Laszlo Varga and Marton Zsenei are supported by the OMFB Budapest for their activities in CRISTAL

X REFERENCES [ 11 CMS Technical Proposal The CMS Collaboration Jan

1995 ftpcmsdoccernchTprefmPhtml [2] M Philpotts ldquoAn Introduction to the Concepts Benefits

and Terminology of Product Data Managementldquo Industrial Management amp Data Systems 1996 4 ppll-17

[3] P Pikosz amp J Malmqvist ldquoPossibilities and Limitations when Using EDM Systems to Support the Product

Development Processrdquo Proceedings of NordDesignrsquo96 pp165-175 Espoo Fiinland 1996

[4] D Georgakopoulos M Hornick amp A Sheth ldquoAn Overview of Workdlow Management from Process Modeling to Infrastructure for Automationrdquo Journal of Distributed and Parallel Database Systems 3 (2) April

[5] J-M Le Goff et al ldquoDetector Construction Management and Quality Control Establishing and using a CRISTAL Systemrdquo CERN CMS NOTE 1998033 Available from ftpcrnsdoccerncNdocuments98note98~033 pdf

[6] R McClatchey et al ldquoA Distributed Workflow amp Product Data Management Application for the Construction of Large Scale Scientifiic Apparatusrdquo NATO ASI Series F Computer amp Systems Sciences Vol 164 ISBN 3-540- 64411-3 Springer Verlag 1998 Presented at the NATO AS1 workshop on lsquoAdvances in Workflow Management Systems amp Interoperabilityrsquo Istanbul Turkey 1997

[7] Z Kovacs J-M Le Goff amp R McClatchey ldquoSupport for Product data from Design to Operationrdquo Journal of Computer Integrated Manufacturing Vol 11 No 1-2 Elsevier Science Ltd in press

[8] A P Hameri ldquoEDN[S - Concepts Motivations and Basic requirementsrdquo Procleedings of the CERN School of Computing 96 CERN 96-08

[9] E Auffray et al ldquoCertifying Procedure for Lead Tungstate Crystal Parameters during the Mass Production for CMS ECALrdquo Poster N20-31 at IEEE Nuclear Science Symposium Toronto Canada November 1998

[lo] J-P Peigneux et al ldquoStudies and Proposals for an Automatic Crystal Control Systemrdquo CERN CMS NOTE 19971036

[ l l ] Cadim a PDM product developed by Eigner amp Partner see URL httplwwwep-kade

[12] Matrix a PDM product from Adra Systems see URL httpwww adracom

[13] J-M Le Goff et al ldquoFrom Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systemsrdquo CERN CMS NOTE 19971104

[ 141 Objectivity an ob-ject-oriented database management system produced by Objectivity Inc httpwwwobj y co r (

[ 151 P H Feiler ldquoConfiguration Management Models in Commercial Environmentsrdquo Carnegie-Mellon Technical Reports CMUSEI-91-TR-7 ESD-91-TR-7 March 1991

[16] G Bachi amp A P Hameri ldquoWhat Should be Implemented at the Early Stages of a Lerge-Scale Projectrdquo CERN MT95-02 (DI) LHC Note 315

[17] M Fowler 2 K Scott ldquoUML Distilled - Applying the Standard Object Modeling Languagerdquo Addison-Wesley Longman Inc 1997

1995 ppll9-153

395

that may be allocated are termed Agents and are of the type Instrument Human or User-provided code Agents execute activities and are the initiators of the data gathering process The assembly sequence may itself be composite in nature ie one assembly sequence may be composed of other assembly sequences

Figure 4 shows that activities (in the case shown lsquoglue crystal with APDrsquo) act on products (crystal capsule) if certain start conditions are fulfilled The start conditions are the pre-requisites for the activity to take place and include (at least) the products involved in the activity and if relevant the resources required to carry out the activity The conditions are tested at the outset of the activity and the activity can only progress when all start conditions have been fulfilled Each activity has a description takes place at a specified Local Centre(s) (eg Rome or CERN) can be allocated resources (eg an Operator Agent) can have a specified minimum and maximum duration may optionally be repeatable and when completed produces an outcome which is tested against a set of so-called end conditions It is this test against end conditions which ensures quality control in the execution of activities If end conditions fail it will be possible for exceptional activities to be invoked All of this information is specified for a product type named by the Coordinator in the central ECAL construction database for a particular class of activities Each instance of the activity is actually run at the specified Local Centre and if successful physical characteristics are computed and stored against the tested product for future use in for example detector calibration and maintenance

StaftConditions Parts reauired for lis activiiy to start

Activitv Name Glue Crystal with APD Outcome amp

DescriDtion Place capsule on End glueing stand put glue on crystal Conditions and lower crystal on capsule IsActivityRepeatable No

Resources Operator glueing

1 crystal amp capsule Amlicable Centres Rome CERN

Duration tmin =22 h tmax = 24 h Results I toDB 1 Figure 4 An activity definition (including resources Centres start and end conditions and outcomes)

Management information regarding the definition configuration version performance and operational state of the distributed CRISTAL production line is stored at CERN in a central repository based on the Objectivity object- oriented database product [ 141 This object-based repository also stores the definitions of the parts that make up the detector together with the definitions of the instruments used to produce or take measurements of parts It stores descriptions of the life-cycle of each part (its so-called production scheme) and descriptions of the tasks and activities performed on the part This production scheme (or sequence of tasks) determines the order of tasks that can be applied to any part The task execution script will run and

will prompt the operator to perform manual operations or may automatically trigger networked instruments to take measurements of the chalracteristics of the part (eg the so- called ACCOS rig [IO] which is used to measure dimensions and optical properties of the ECAL crystals) All the details of the task operations and the associated measurements and instruments involved are eventually stored in the central repository

The data handling aspects of CRISTAL must be transparent to any users of the system irrespective of location throughout the system Part task production scheme and system configuration datal will be defined by the Coordinator at CERN thereby controlling the configuration and management of all the centres Once the production centres are configured parts will be produced and then shipped to further local centres for testing and assembly All users can browse part characteristics and navigate through production schemes centre coordinators can modify local production schemes and tasks In each centre Product Managers provide access to the database optimise part information retrieval handle operator and instrument agents update part histories and interact with the software component that duplicates all part data centrally to CERN (the DDM see figure 2)

Iv PRODUCT DATA MANAGEMENT The development lifecycle of a large high-energy physics

detector is much like my other large-scale construction activity in that it follows al design-prototype-implement cycle It does differ from industrial production however in that it is research-based and highly iterative and consequently dynamic in execution The overall performance of the detector is highly dependent on its design and therefore any changes in design need to be permeated through from conceptual design to physical construction as quickly as possible The importance of rapidly reflecting design changes in production activities is becoming increasingly typical of examples from manufacturing engineering

Conceptual design is a collaborative activity with designers checking-out and checking-in documents and diagrams of components under some policy of configuration management [15] The data vault aspects of a PDM lend themselves well to this creative design process Product breakdown structures (PBS) are often strictly hierarchical in form and attributes can be assigned to each (sub-)part The advantages of using a PDM are well-documented elsewhere [2 3 81 PDM system provide a document change management service which can be used by engineering applications to assess control and minimise the impact of product and process changes that occur in complex manufacturing lifecycles According to [ 161 any PDM system used for the engineering of large-scale one-of-a-kind facilities (such as CMS) should hold the descriptions of both the PBS and the work breakdown structure (WBS) The PBS saves information pertaining to projects sub-projects documents items etc The WBS holds information about the organisation of tasks (or activities) to be performed and the resources required for each task The WBS defines the activities which enable the engineers to build the production

396

line that can be viewed as a collection of (versioned) interface and an execution service provided by a so-called activities The WBS holds the definitions of the production workflow engine The engine is the component that executes line and can be mapped onto workflows (see following the static workflow (production) descriptions which were section)and instantiations of these definitions are used to defined through the programming interface Commercial form the PBS WfMS products normally subsume both the specification and

Essentially the purpose of building a PBS for industrial enactment Of workflOws applications in a PDM is to facilitate the capture of a design To achieve integration between the PDM and workflow hierarchy of parts As the product becomes more complex in components of CRISTAL the PDM can be used to store a set structure however a parts explosion can take place in the of definitions of both the parts and the tasks that need to be PDM and data management becomes an issue In CRISTAL executed on the parts The workflow definitions are mapped the CMS detector will be constructed from millions of onto the WBS and the workflow instances can therefore be individual parts sets of which will be identical in nature It is derived from their definitions residing in the PDM The PDM simply infeasible to enter and manage these parts individually acts as the reference database for the enactment services of in the PDM Instead a concept of meta-data management the production workflow system and manages (versions of) [13] can be followed where definitions of parts are captured the PBS and the WBS (see figure 6)

In practice the CRISTAL system sits alongside the PDM in the PDM and manages the execution of tasks upon parts CRISTAL is in effect the enactment service of instances of the workflows as defined in the WBS part of the PDM As a consequence of this CRISTAL accesses the PDM for part definitions and for the specification of assembly sequences CRISTAL consequently integrates a PDM and Wfh4 facility and tracks parts through the manufacturing life cycle from a PDM- resident design to the final construction and assembly of the detector

In the same way that the PBS has been modelled using meta-objects the workflow definitions (WBS) of CRISTAL can also be modeled Figure 5 shows the overall data model Each part definition has at most one activity definition assigned to it Activities can be reused for several part definitions A part definition is an item of the PBS the activity definition is the workflow meta-object (WBS) A

The left side of Figure 5 Shows a UML [I71 object m d e l workflow meta-object is an ordered collection of activity Of the Product bmddown structure which underlies definitions which will be executed on every part registered in CRISTAL Part definitions are specified as either elementary the CRISTAL system with its corresponding part definition Or composite in nature Composite Parts are made UP of Other Activity definitions can be either elementary or composite in Parts and the ComPositeMember object reflects membership nature composite activities can be composed of several other of a part in a composite In implementing this meta-datu activity definitions

The workflow definitions (WBS) of CRISTAL can be management the tree representation of assembly breakdown is transformed into a graph representation [ 131 Individual modeled using the same concept of meta-objects as was the

definitions thus easing data management The next section given activity definition and a named part definition The CRISTAL data model has been designed so that each investigates how these meta-objects also facilitate the assignment of a Part Definition to an Activity Definition is integration of PDMs with WfMSs

Figure 5 Simplified UML Object Model

parts (Of the same type) are then associated with Common PBS Figure 5 shows that there is an association between a

v INTEGRATING PRODUCT DATA AND WORKFLOW MANAGEMENT

A workflow management syatem (WFMS) [4 181 ia a system that completely defines manages and executes activities through the execution of software whose order of execution is driven by a computer representation of the workflow logic The main components of a workflow management system are a workflow application programming interface and a workflow enactment service The workflow application programming interface allows for the specification of workflows and workflow activities The workflow enactment service consists of an execution

decfared for a specific purpose In detector construction the assignment is made to indicate the activity to be instantiated for the assembly of a particular instance of a part of a given part definition Each assignment has associated with it some Conditions in detector construction the data model captures the definition of the construction conditions required for each assignment of an activity definition to a part definition

This technique can be generalised for other applications For example the association of a maintenance activity to a part will require quite different conditions to be captured than when the detector was constructed Also the association of a calibration activity to a part would require calibration- specific conditions to be captured In other words the identified association between the activity and part

397

Centralized Database for Models

description worlds carries rich semantics This method of integrating PDM and WfMS through the definition of meta- objects and their mutual assignment is very powerful It allows many other links to be made between aspects of the overall CRISTAL data model the same mechanism can be used to assign agents to activity definitions for the purposes of enactment or the assignment of agents to part definitions for the purposes of resource management

PDM CERN Engineering Data Management system for Design

Product Breakdown Structure (Part Specification) (Taskspecification)

Mapping from Design to production

(Workflow Models) (Workflow Enactment)

Distributed Database for Production Data

I I Figure 6 The relationship between CRISTAL and a PDM

VI CALIBRATION DATA AND CRISTAL The construction database was first released in the

summer of 1998 enabling ECAL physicists to collect the salient characteristics of each Barrel component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector construction phase must be arranged and processed to allow population of a calibration database This section details how the ECAL construction database can be used as the source of calibration data and by extension as a generalised physics data source

For the purposes of physics analysis CMS ECAL needs to produce a pre-calibration database which will hold all the physical parameters necessary for calculating the ECAL static calibration constants for a single ECAL supermodule However construction and calibration are two quite different views of ECAL detector data Physicists must be able to extract sets of data relevant for determining calibration coefficients from the ECAL construction database (figure 7) into a pre-calibration database The information required for calibration includes data for crystals capsules supermodules and electronics Crystal-specific information will include light yields attenuation lengths longitudinal and transverse transmission (as functions of wavelength) and any crystal non-uniformities Capsule information needed for calibration includes gain and dark current vs high voltage and nominal biases All of this information will be captured and recorded in the ECAL construction database as the detector is constructed step-wise from individual crystals through sub- units to super-modules

Production

Datasubset I Test Beam Data

Beam

subset I Data

Figure 7 The extraction of calibration data from the CRISTAL construction database

The step-wise construction procedure leads to a data organisation necessarily different to that required for calibration the structure of the construction database follows the assembly ordering of the calorimeter while the structure of the calibration database must follow the ECAL readout structure where the unit of detector that is normally considered is the readout channel Calibration constants need to be determined for each readout channel (eg a crystal plus its Capsule electronics (ADC etc) and fibres) so that ADC counts can be translated into energy deposited in a single readout channel Therefore the calibration system must be able to extract subsets of physics data from the ECAL construction database for the calibration of particular physics elements (or sets of detector components) even if these elements are specified in a manner orthogonal to that in which structures are defined in the ECAL construction database

All parameters needed for the calibration of a single supermodule will be extracted from the ECAL construction database and a matrix i j of readout channels (or so-called physics elements) will be Ixdt in the pre-calibration database To these data information from test beam slow controls data acquisition and monitoring is added to build a complete picture of the conditions under which calibration-data was taken and on completion of the calibration runs these data are copied back into the construction database (see figure 7) The construction database then accumulates this calibration data for each supermodule and acts as the source of calibration data for all of 1 he 36 supermodules comprising the ECAL Barrel The final static calibration constants for the complete ECAL are extracted from the construction database into a calibration database Fast and efficient access from physicist programmes is allowed both to the pre-calibration database for a single supermodule in the test beam and to the set of constants in the full calibration database (see figure 7)

398

As the CMS construction process evolves so more data and the relationships between different aspects of the data must be permanently recorded in the CRISTAL data database Different groups of users will require flexible ways to find access and share this construction data The actual information required will depend on the viewpoint and the role of the user in the organisation User groups may well require a maintenance a geometry an alignment or an experiment slow- controls viewpoint The physicist therefore needs to define a viewpoint in terms of lsquophysics elementsrsquo (or sets of detector components) which are derived from the tree of physical locations of detector components resulting from detector assembly Figure 8 shows how a matrix of physics elements can be extracted from the lsquoas-builtrsquo detector construction hierarchy by providing software which can traverse the construction tree and can query and extract the construction data for a selected set of detector components

Dataease for SuperModule SuperModule Precalition in testbeam

Subunit ampI) = A physics Element ex A Readout channel for ECAL

Capsule rystal

Treeregresentatiou established during assembly Matrix representation extracted fian with ~nsm~o Part physical

Figure 8 The derivation of physics elements from the construction lsquoas-builtrsquo tree

Meta-modeling will assist the extraction of data for physics elements provided an extraction mechanism is developed which can navigaqe the meta-model can interpret the structures in the database and can present the data in a form meaningful to the end-user The extraction facility comprises a set of software processes which can be invoked either by a viewpoint-specific application (eg calibration) or by a viewpoint non-specific application That is the processes either navigate a generalised meta-model to project out viewpoint- specific data (ie lsquolooking intorsquo the meta-model) or they navigate the meta-model to correlate effects between viewpoints (ie lsquolooking outrsquo from the data model) In the lsquolooking inrsquo (viewpoint-specific) case the processes perform traversal of the detector description and extract the relevant physics data for the application In the lsquolooking outrsquo case (viewpoint non-specific) the processes determine the effect of a system-wide change on individual viewpoints or sets of viewpoints for example a temperature variation on calibration and slow controls viewpoints An extraction facility is currently under development for CRISTAL [ 191

VII CONCLUSIONS The scale of LHC experiments requires that rigorous

production management techniques be employed for the first time in high energy physics detector construction The CRISTAL project of CMS has implemented a construction database and information system for managing the construction of the CMS ECAL detector The software is currently being evaluated for use in CMS HCAL and Tracker and there has been interest from other experiments and from the LHC magnet construction teams

The CRISTAL data model has been developed using UML [ 171 and has been designed with a view to capturing both PDM and WfMS data The PDM-component of the data model [20] is used to store the full specification for detector construction It will contain all the definitions of individual parts and will describe what is to be built and how it is to be built The specifications are defined and held centrally and releases of versions of the production scheme are issued from the central database The workflow specification is also centrally defined and it will define the process of part characterisation and the complete assembly process from single parts to full-scale detector

The first phase of CRISTAL prototyping and technology evaluation was initiated in the summer of 1996 and delivery of a working prototype is due in the summer of 1998 based on a distributed Objectivity database This prototype will enable ECAL physicists to collect all of the salient characteristics of each ECAL component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector assembly phase must be arranged and processed to allow population of for example a calibration database

The second phase of the project aims to adopt a strong meta-data approach as described in [19] in the expectation that it will produce a more adaptable system capable of interoperating with future systems This meta-modeling approach will facilitate further integration between product data management and workflow management thereby providing consistency between design and production and speeding up the process of implementing design changes in a production system

Since the CRISTAL projectrsquos inception a considerable amount of interest has been generated in meta-models and meta-object description languages Work is in progress within the OMG on the Meta Object Facility (MOF) [21] which is expected to manage all kinds of meta-models relevant to the OMG Architecture It is expected that future releases of CRISTAL will benefit from this standardisation effort The meta-modeling approach adopted in the design of CRISTAL could reasonably be applied to organisations developing technologies for lsquovirtual enterprisesrsquo (such as in [22] [23]) where collections of autonomous databases could be related via a central enterprise meta-model Furthermore inrsquothe area of standardisation the Meta Data Coalition is working towards a Metadata Interchange Specification [24] which aims to

399

tackle the exchange sharing and management of metadata in industry The experiences of CRISTAL in providing viewpoint access (with location and application transparency) to enterprise is clearly relevant to research in these areas

VIII GLOSSARY OF ACRONYMS To aid the readability of this paper the authors include here

a glossary of all the acronyms deployed throughout the body of this paperrsquos text ACCOS Automatic Crystal Control System ADC Analogue-to-Digital Converter APD Avalanche Photo-Diode CMS The Compact Muon Solenoid CRISTAL Cooperating Repositories and an Information System for Tracking Assembly Lifecycles DAcq Data Acquisition DCP Digital Control Panel DDM Data Duplication Manager ECAL Electromagnetic Calorimeter EDMS Engineering Document Management System HCAL Hadronic Calorimeter LHC Large Hadron Collider MOF Meta-Object Facility MSGC Micro-Strip Gas Chambers OMG The Object Management Group PBS Product Breakdown Structure PDM Product Data Management UML Unified Modeling Language WBS Work Breakdown Structure WfMS Workflows Management Systems

Ix ACKNOWLEDGEMENTS The authors take this opportunity to acknowledge the

support of their home institutes In particular the support of P Lecoq J-L Faure and M Pimia is greatly appreciated Richard McClatchey is assisted by CERN and by the Royal Academy of Engineering under their Technology Foresight Award scheme during 1998 and acknowledges their support Laszlo Varga and Marton Zsenei are supported by the OMFB Budapest for their activities in CRISTAL

X REFERENCES [ 11 CMS Technical Proposal The CMS Collaboration Jan

1995 ftpcmsdoccernchTprefmPhtml [2] M Philpotts ldquoAn Introduction to the Concepts Benefits

and Terminology of Product Data Managementldquo Industrial Management amp Data Systems 1996 4 ppll-17

[3] P Pikosz amp J Malmqvist ldquoPossibilities and Limitations when Using EDM Systems to Support the Product

Development Processrdquo Proceedings of NordDesignrsquo96 pp165-175 Espoo Fiinland 1996

[4] D Georgakopoulos M Hornick amp A Sheth ldquoAn Overview of Workdlow Management from Process Modeling to Infrastructure for Automationrdquo Journal of Distributed and Parallel Database Systems 3 (2) April

[5] J-M Le Goff et al ldquoDetector Construction Management and Quality Control Establishing and using a CRISTAL Systemrdquo CERN CMS NOTE 1998033 Available from ftpcrnsdoccerncNdocuments98note98~033 pdf

[6] R McClatchey et al ldquoA Distributed Workflow amp Product Data Management Application for the Construction of Large Scale Scientifiic Apparatusrdquo NATO ASI Series F Computer amp Systems Sciences Vol 164 ISBN 3-540- 64411-3 Springer Verlag 1998 Presented at the NATO AS1 workshop on lsquoAdvances in Workflow Management Systems amp Interoperabilityrsquo Istanbul Turkey 1997

[7] Z Kovacs J-M Le Goff amp R McClatchey ldquoSupport for Product data from Design to Operationrdquo Journal of Computer Integrated Manufacturing Vol 11 No 1-2 Elsevier Science Ltd in press

[8] A P Hameri ldquoEDN[S - Concepts Motivations and Basic requirementsrdquo Procleedings of the CERN School of Computing 96 CERN 96-08

[9] E Auffray et al ldquoCertifying Procedure for Lead Tungstate Crystal Parameters during the Mass Production for CMS ECALrdquo Poster N20-31 at IEEE Nuclear Science Symposium Toronto Canada November 1998

[lo] J-P Peigneux et al ldquoStudies and Proposals for an Automatic Crystal Control Systemrdquo CERN CMS NOTE 19971036

[ l l ] Cadim a PDM product developed by Eigner amp Partner see URL httplwwwep-kade

[12] Matrix a PDM product from Adra Systems see URL httpwww adracom

[13] J-M Le Goff et al ldquoFrom Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systemsrdquo CERN CMS NOTE 19971104

[ 141 Objectivity an ob-ject-oriented database management system produced by Objectivity Inc httpwwwobj y co r (

[ 151 P H Feiler ldquoConfiguration Management Models in Commercial Environmentsrdquo Carnegie-Mellon Technical Reports CMUSEI-91-TR-7 ESD-91-TR-7 March 1991

[16] G Bachi amp A P Hameri ldquoWhat Should be Implemented at the Early Stages of a Lerge-Scale Projectrdquo CERN MT95-02 (DI) LHC Note 315

[17] M Fowler 2 K Scott ldquoUML Distilled - Applying the Standard Object Modeling Languagerdquo Addison-Wesley Longman Inc 1997

1995 ppll9-153

396

line that can be viewed as a collection of (versioned) interface and an execution service provided by a so-called activities The WBS holds the definitions of the production workflow engine The engine is the component that executes line and can be mapped onto workflows (see following the static workflow (production) descriptions which were section)and instantiations of these definitions are used to defined through the programming interface Commercial form the PBS WfMS products normally subsume both the specification and

Essentially the purpose of building a PBS for industrial enactment Of workflOws applications in a PDM is to facilitate the capture of a design To achieve integration between the PDM and workflow hierarchy of parts As the product becomes more complex in components of CRISTAL the PDM can be used to store a set structure however a parts explosion can take place in the of definitions of both the parts and the tasks that need to be PDM and data management becomes an issue In CRISTAL executed on the parts The workflow definitions are mapped the CMS detector will be constructed from millions of onto the WBS and the workflow instances can therefore be individual parts sets of which will be identical in nature It is derived from their definitions residing in the PDM The PDM simply infeasible to enter and manage these parts individually acts as the reference database for the enactment services of in the PDM Instead a concept of meta-data management the production workflow system and manages (versions of) [13] can be followed where definitions of parts are captured the PBS and the WBS (see figure 6)

In practice the CRISTAL system sits alongside the PDM in the PDM and manages the execution of tasks upon parts CRISTAL is in effect the enactment service of instances of the workflows as defined in the WBS part of the PDM As a consequence of this CRISTAL accesses the PDM for part definitions and for the specification of assembly sequences CRISTAL consequently integrates a PDM and Wfh4 facility and tracks parts through the manufacturing life cycle from a PDM- resident design to the final construction and assembly of the detector

In the same way that the PBS has been modelled using meta-objects the workflow definitions (WBS) of CRISTAL can also be modeled Figure 5 shows the overall data model Each part definition has at most one activity definition assigned to it Activities can be reused for several part definitions A part definition is an item of the PBS the activity definition is the workflow meta-object (WBS) A

The left side of Figure 5 Shows a UML [I71 object m d e l workflow meta-object is an ordered collection of activity Of the Product bmddown structure which underlies definitions which will be executed on every part registered in CRISTAL Part definitions are specified as either elementary the CRISTAL system with its corresponding part definition Or composite in nature Composite Parts are made UP of Other Activity definitions can be either elementary or composite in Parts and the ComPositeMember object reflects membership nature composite activities can be composed of several other of a part in a composite In implementing this meta-datu activity definitions

The workflow definitions (WBS) of CRISTAL can be management the tree representation of assembly breakdown is transformed into a graph representation [ 131 Individual modeled using the same concept of meta-objects as was the

definitions thus easing data management The next section given activity definition and a named part definition The CRISTAL data model has been designed so that each investigates how these meta-objects also facilitate the assignment of a Part Definition to an Activity Definition is integration of PDMs with WfMSs

Figure 5 Simplified UML Object Model

parts (Of the same type) are then associated with Common PBS Figure 5 shows that there is an association between a

v INTEGRATING PRODUCT DATA AND WORKFLOW MANAGEMENT

A workflow management syatem (WFMS) [4 181 ia a system that completely defines manages and executes activities through the execution of software whose order of execution is driven by a computer representation of the workflow logic The main components of a workflow management system are a workflow application programming interface and a workflow enactment service The workflow application programming interface allows for the specification of workflows and workflow activities The workflow enactment service consists of an execution

decfared for a specific purpose In detector construction the assignment is made to indicate the activity to be instantiated for the assembly of a particular instance of a part of a given part definition Each assignment has associated with it some Conditions in detector construction the data model captures the definition of the construction conditions required for each assignment of an activity definition to a part definition

This technique can be generalised for other applications For example the association of a maintenance activity to a part will require quite different conditions to be captured than when the detector was constructed Also the association of a calibration activity to a part would require calibration- specific conditions to be captured In other words the identified association between the activity and part

397

Centralized Database for Models

description worlds carries rich semantics This method of integrating PDM and WfMS through the definition of meta- objects and their mutual assignment is very powerful It allows many other links to be made between aspects of the overall CRISTAL data model the same mechanism can be used to assign agents to activity definitions for the purposes of enactment or the assignment of agents to part definitions for the purposes of resource management

PDM CERN Engineering Data Management system for Design

Product Breakdown Structure (Part Specification) (Taskspecification)

Mapping from Design to production

(Workflow Models) (Workflow Enactment)

Distributed Database for Production Data

I I Figure 6 The relationship between CRISTAL and a PDM

VI CALIBRATION DATA AND CRISTAL The construction database was first released in the

summer of 1998 enabling ECAL physicists to collect the salient characteristics of each Barrel component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector construction phase must be arranged and processed to allow population of a calibration database This section details how the ECAL construction database can be used as the source of calibration data and by extension as a generalised physics data source

For the purposes of physics analysis CMS ECAL needs to produce a pre-calibration database which will hold all the physical parameters necessary for calculating the ECAL static calibration constants for a single ECAL supermodule However construction and calibration are two quite different views of ECAL detector data Physicists must be able to extract sets of data relevant for determining calibration coefficients from the ECAL construction database (figure 7) into a pre-calibration database The information required for calibration includes data for crystals capsules supermodules and electronics Crystal-specific information will include light yields attenuation lengths longitudinal and transverse transmission (as functions of wavelength) and any crystal non-uniformities Capsule information needed for calibration includes gain and dark current vs high voltage and nominal biases All of this information will be captured and recorded in the ECAL construction database as the detector is constructed step-wise from individual crystals through sub- units to super-modules

Production

Datasubset I Test Beam Data

Beam

subset I Data

Figure 7 The extraction of calibration data from the CRISTAL construction database

The step-wise construction procedure leads to a data organisation necessarily different to that required for calibration the structure of the construction database follows the assembly ordering of the calorimeter while the structure of the calibration database must follow the ECAL readout structure where the unit of detector that is normally considered is the readout channel Calibration constants need to be determined for each readout channel (eg a crystal plus its Capsule electronics (ADC etc) and fibres) so that ADC counts can be translated into energy deposited in a single readout channel Therefore the calibration system must be able to extract subsets of physics data from the ECAL construction database for the calibration of particular physics elements (or sets of detector components) even if these elements are specified in a manner orthogonal to that in which structures are defined in the ECAL construction database

All parameters needed for the calibration of a single supermodule will be extracted from the ECAL construction database and a matrix i j of readout channels (or so-called physics elements) will be Ixdt in the pre-calibration database To these data information from test beam slow controls data acquisition and monitoring is added to build a complete picture of the conditions under which calibration-data was taken and on completion of the calibration runs these data are copied back into the construction database (see figure 7) The construction database then accumulates this calibration data for each supermodule and acts as the source of calibration data for all of 1 he 36 supermodules comprising the ECAL Barrel The final static calibration constants for the complete ECAL are extracted from the construction database into a calibration database Fast and efficient access from physicist programmes is allowed both to the pre-calibration database for a single supermodule in the test beam and to the set of constants in the full calibration database (see figure 7)

398

As the CMS construction process evolves so more data and the relationships between different aspects of the data must be permanently recorded in the CRISTAL data database Different groups of users will require flexible ways to find access and share this construction data The actual information required will depend on the viewpoint and the role of the user in the organisation User groups may well require a maintenance a geometry an alignment or an experiment slow- controls viewpoint The physicist therefore needs to define a viewpoint in terms of lsquophysics elementsrsquo (or sets of detector components) which are derived from the tree of physical locations of detector components resulting from detector assembly Figure 8 shows how a matrix of physics elements can be extracted from the lsquoas-builtrsquo detector construction hierarchy by providing software which can traverse the construction tree and can query and extract the construction data for a selected set of detector components

Dataease for SuperModule SuperModule Precalition in testbeam

Subunit ampI) = A physics Element ex A Readout channel for ECAL

Capsule rystal

Treeregresentatiou established during assembly Matrix representation extracted fian with ~nsm~o Part physical

Figure 8 The derivation of physics elements from the construction lsquoas-builtrsquo tree

Meta-modeling will assist the extraction of data for physics elements provided an extraction mechanism is developed which can navigaqe the meta-model can interpret the structures in the database and can present the data in a form meaningful to the end-user The extraction facility comprises a set of software processes which can be invoked either by a viewpoint-specific application (eg calibration) or by a viewpoint non-specific application That is the processes either navigate a generalised meta-model to project out viewpoint- specific data (ie lsquolooking intorsquo the meta-model) or they navigate the meta-model to correlate effects between viewpoints (ie lsquolooking outrsquo from the data model) In the lsquolooking inrsquo (viewpoint-specific) case the processes perform traversal of the detector description and extract the relevant physics data for the application In the lsquolooking outrsquo case (viewpoint non-specific) the processes determine the effect of a system-wide change on individual viewpoints or sets of viewpoints for example a temperature variation on calibration and slow controls viewpoints An extraction facility is currently under development for CRISTAL [ 191

VII CONCLUSIONS The scale of LHC experiments requires that rigorous

production management techniques be employed for the first time in high energy physics detector construction The CRISTAL project of CMS has implemented a construction database and information system for managing the construction of the CMS ECAL detector The software is currently being evaluated for use in CMS HCAL and Tracker and there has been interest from other experiments and from the LHC magnet construction teams

The CRISTAL data model has been developed using UML [ 171 and has been designed with a view to capturing both PDM and WfMS data The PDM-component of the data model [20] is used to store the full specification for detector construction It will contain all the definitions of individual parts and will describe what is to be built and how it is to be built The specifications are defined and held centrally and releases of versions of the production scheme are issued from the central database The workflow specification is also centrally defined and it will define the process of part characterisation and the complete assembly process from single parts to full-scale detector

The first phase of CRISTAL prototyping and technology evaluation was initiated in the summer of 1996 and delivery of a working prototype is due in the summer of 1998 based on a distributed Objectivity database This prototype will enable ECAL physicists to collect all of the salient characteristics of each ECAL component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector assembly phase must be arranged and processed to allow population of for example a calibration database

The second phase of the project aims to adopt a strong meta-data approach as described in [19] in the expectation that it will produce a more adaptable system capable of interoperating with future systems This meta-modeling approach will facilitate further integration between product data management and workflow management thereby providing consistency between design and production and speeding up the process of implementing design changes in a production system

Since the CRISTAL projectrsquos inception a considerable amount of interest has been generated in meta-models and meta-object description languages Work is in progress within the OMG on the Meta Object Facility (MOF) [21] which is expected to manage all kinds of meta-models relevant to the OMG Architecture It is expected that future releases of CRISTAL will benefit from this standardisation effort The meta-modeling approach adopted in the design of CRISTAL could reasonably be applied to organisations developing technologies for lsquovirtual enterprisesrsquo (such as in [22] [23]) where collections of autonomous databases could be related via a central enterprise meta-model Furthermore inrsquothe area of standardisation the Meta Data Coalition is working towards a Metadata Interchange Specification [24] which aims to

399

tackle the exchange sharing and management of metadata in industry The experiences of CRISTAL in providing viewpoint access (with location and application transparency) to enterprise is clearly relevant to research in these areas

VIII GLOSSARY OF ACRONYMS To aid the readability of this paper the authors include here

a glossary of all the acronyms deployed throughout the body of this paperrsquos text ACCOS Automatic Crystal Control System ADC Analogue-to-Digital Converter APD Avalanche Photo-Diode CMS The Compact Muon Solenoid CRISTAL Cooperating Repositories and an Information System for Tracking Assembly Lifecycles DAcq Data Acquisition DCP Digital Control Panel DDM Data Duplication Manager ECAL Electromagnetic Calorimeter EDMS Engineering Document Management System HCAL Hadronic Calorimeter LHC Large Hadron Collider MOF Meta-Object Facility MSGC Micro-Strip Gas Chambers OMG The Object Management Group PBS Product Breakdown Structure PDM Product Data Management UML Unified Modeling Language WBS Work Breakdown Structure WfMS Workflows Management Systems

Ix ACKNOWLEDGEMENTS The authors take this opportunity to acknowledge the

support of their home institutes In particular the support of P Lecoq J-L Faure and M Pimia is greatly appreciated Richard McClatchey is assisted by CERN and by the Royal Academy of Engineering under their Technology Foresight Award scheme during 1998 and acknowledges their support Laszlo Varga and Marton Zsenei are supported by the OMFB Budapest for their activities in CRISTAL

X REFERENCES [ 11 CMS Technical Proposal The CMS Collaboration Jan

1995 ftpcmsdoccernchTprefmPhtml [2] M Philpotts ldquoAn Introduction to the Concepts Benefits

and Terminology of Product Data Managementldquo Industrial Management amp Data Systems 1996 4 ppll-17

[3] P Pikosz amp J Malmqvist ldquoPossibilities and Limitations when Using EDM Systems to Support the Product

Development Processrdquo Proceedings of NordDesignrsquo96 pp165-175 Espoo Fiinland 1996

[4] D Georgakopoulos M Hornick amp A Sheth ldquoAn Overview of Workdlow Management from Process Modeling to Infrastructure for Automationrdquo Journal of Distributed and Parallel Database Systems 3 (2) April

[5] J-M Le Goff et al ldquoDetector Construction Management and Quality Control Establishing and using a CRISTAL Systemrdquo CERN CMS NOTE 1998033 Available from ftpcrnsdoccerncNdocuments98note98~033 pdf

[6] R McClatchey et al ldquoA Distributed Workflow amp Product Data Management Application for the Construction of Large Scale Scientifiic Apparatusrdquo NATO ASI Series F Computer amp Systems Sciences Vol 164 ISBN 3-540- 64411-3 Springer Verlag 1998 Presented at the NATO AS1 workshop on lsquoAdvances in Workflow Management Systems amp Interoperabilityrsquo Istanbul Turkey 1997

[7] Z Kovacs J-M Le Goff amp R McClatchey ldquoSupport for Product data from Design to Operationrdquo Journal of Computer Integrated Manufacturing Vol 11 No 1-2 Elsevier Science Ltd in press

[8] A P Hameri ldquoEDN[S - Concepts Motivations and Basic requirementsrdquo Procleedings of the CERN School of Computing 96 CERN 96-08

[9] E Auffray et al ldquoCertifying Procedure for Lead Tungstate Crystal Parameters during the Mass Production for CMS ECALrdquo Poster N20-31 at IEEE Nuclear Science Symposium Toronto Canada November 1998

[lo] J-P Peigneux et al ldquoStudies and Proposals for an Automatic Crystal Control Systemrdquo CERN CMS NOTE 19971036

[ l l ] Cadim a PDM product developed by Eigner amp Partner see URL httplwwwep-kade

[12] Matrix a PDM product from Adra Systems see URL httpwww adracom

[13] J-M Le Goff et al ldquoFrom Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systemsrdquo CERN CMS NOTE 19971104

[ 141 Objectivity an ob-ject-oriented database management system produced by Objectivity Inc httpwwwobj y co r (

[ 151 P H Feiler ldquoConfiguration Management Models in Commercial Environmentsrdquo Carnegie-Mellon Technical Reports CMUSEI-91-TR-7 ESD-91-TR-7 March 1991

[16] G Bachi amp A P Hameri ldquoWhat Should be Implemented at the Early Stages of a Lerge-Scale Projectrdquo CERN MT95-02 (DI) LHC Note 315

[17] M Fowler 2 K Scott ldquoUML Distilled - Applying the Standard Object Modeling Languagerdquo Addison-Wesley Longman Inc 1997

1995 ppll9-153

397

Centralized Database for Models

description worlds carries rich semantics This method of integrating PDM and WfMS through the definition of meta- objects and their mutual assignment is very powerful It allows many other links to be made between aspects of the overall CRISTAL data model the same mechanism can be used to assign agents to activity definitions for the purposes of enactment or the assignment of agents to part definitions for the purposes of resource management

PDM CERN Engineering Data Management system for Design

Product Breakdown Structure (Part Specification) (Taskspecification)

Mapping from Design to production

(Workflow Models) (Workflow Enactment)

Distributed Database for Production Data

I I Figure 6 The relationship between CRISTAL and a PDM

VI CALIBRATION DATA AND CRISTAL The construction database was first released in the

summer of 1998 enabling ECAL physicists to collect the salient characteristics of each Barrel component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector construction phase must be arranged and processed to allow population of a calibration database This section details how the ECAL construction database can be used as the source of calibration data and by extension as a generalised physics data source

For the purposes of physics analysis CMS ECAL needs to produce a pre-calibration database which will hold all the physical parameters necessary for calculating the ECAL static calibration constants for a single ECAL supermodule However construction and calibration are two quite different views of ECAL detector data Physicists must be able to extract sets of data relevant for determining calibration coefficients from the ECAL construction database (figure 7) into a pre-calibration database The information required for calibration includes data for crystals capsules supermodules and electronics Crystal-specific information will include light yields attenuation lengths longitudinal and transverse transmission (as functions of wavelength) and any crystal non-uniformities Capsule information needed for calibration includes gain and dark current vs high voltage and nominal biases All of this information will be captured and recorded in the ECAL construction database as the detector is constructed step-wise from individual crystals through sub- units to super-modules

Production

Datasubset I Test Beam Data

Beam

subset I Data

Figure 7 The extraction of calibration data from the CRISTAL construction database

The step-wise construction procedure leads to a data organisation necessarily different to that required for calibration the structure of the construction database follows the assembly ordering of the calorimeter while the structure of the calibration database must follow the ECAL readout structure where the unit of detector that is normally considered is the readout channel Calibration constants need to be determined for each readout channel (eg a crystal plus its Capsule electronics (ADC etc) and fibres) so that ADC counts can be translated into energy deposited in a single readout channel Therefore the calibration system must be able to extract subsets of physics data from the ECAL construction database for the calibration of particular physics elements (or sets of detector components) even if these elements are specified in a manner orthogonal to that in which structures are defined in the ECAL construction database

All parameters needed for the calibration of a single supermodule will be extracted from the ECAL construction database and a matrix i j of readout channels (or so-called physics elements) will be Ixdt in the pre-calibration database To these data information from test beam slow controls data acquisition and monitoring is added to build a complete picture of the conditions under which calibration-data was taken and on completion of the calibration runs these data are copied back into the construction database (see figure 7) The construction database then accumulates this calibration data for each supermodule and acts as the source of calibration data for all of 1 he 36 supermodules comprising the ECAL Barrel The final static calibration constants for the complete ECAL are extracted from the construction database into a calibration database Fast and efficient access from physicist programmes is allowed both to the pre-calibration database for a single supermodule in the test beam and to the set of constants in the full calibration database (see figure 7)

398

As the CMS construction process evolves so more data and the relationships between different aspects of the data must be permanently recorded in the CRISTAL data database Different groups of users will require flexible ways to find access and share this construction data The actual information required will depend on the viewpoint and the role of the user in the organisation User groups may well require a maintenance a geometry an alignment or an experiment slow- controls viewpoint The physicist therefore needs to define a viewpoint in terms of lsquophysics elementsrsquo (or sets of detector components) which are derived from the tree of physical locations of detector components resulting from detector assembly Figure 8 shows how a matrix of physics elements can be extracted from the lsquoas-builtrsquo detector construction hierarchy by providing software which can traverse the construction tree and can query and extract the construction data for a selected set of detector components

Dataease for SuperModule SuperModule Precalition in testbeam

Subunit ampI) = A physics Element ex A Readout channel for ECAL

Capsule rystal

Treeregresentatiou established during assembly Matrix representation extracted fian with ~nsm~o Part physical

Figure 8 The derivation of physics elements from the construction lsquoas-builtrsquo tree

Meta-modeling will assist the extraction of data for physics elements provided an extraction mechanism is developed which can navigaqe the meta-model can interpret the structures in the database and can present the data in a form meaningful to the end-user The extraction facility comprises a set of software processes which can be invoked either by a viewpoint-specific application (eg calibration) or by a viewpoint non-specific application That is the processes either navigate a generalised meta-model to project out viewpoint- specific data (ie lsquolooking intorsquo the meta-model) or they navigate the meta-model to correlate effects between viewpoints (ie lsquolooking outrsquo from the data model) In the lsquolooking inrsquo (viewpoint-specific) case the processes perform traversal of the detector description and extract the relevant physics data for the application In the lsquolooking outrsquo case (viewpoint non-specific) the processes determine the effect of a system-wide change on individual viewpoints or sets of viewpoints for example a temperature variation on calibration and slow controls viewpoints An extraction facility is currently under development for CRISTAL [ 191

VII CONCLUSIONS The scale of LHC experiments requires that rigorous

production management techniques be employed for the first time in high energy physics detector construction The CRISTAL project of CMS has implemented a construction database and information system for managing the construction of the CMS ECAL detector The software is currently being evaluated for use in CMS HCAL and Tracker and there has been interest from other experiments and from the LHC magnet construction teams

The CRISTAL data model has been developed using UML [ 171 and has been designed with a view to capturing both PDM and WfMS data The PDM-component of the data model [20] is used to store the full specification for detector construction It will contain all the definitions of individual parts and will describe what is to be built and how it is to be built The specifications are defined and held centrally and releases of versions of the production scheme are issued from the central database The workflow specification is also centrally defined and it will define the process of part characterisation and the complete assembly process from single parts to full-scale detector

The first phase of CRISTAL prototyping and technology evaluation was initiated in the summer of 1996 and delivery of a working prototype is due in the summer of 1998 based on a distributed Objectivity database This prototype will enable ECAL physicists to collect all of the salient characteristics of each ECAL component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector assembly phase must be arranged and processed to allow population of for example a calibration database

The second phase of the project aims to adopt a strong meta-data approach as described in [19] in the expectation that it will produce a more adaptable system capable of interoperating with future systems This meta-modeling approach will facilitate further integration between product data management and workflow management thereby providing consistency between design and production and speeding up the process of implementing design changes in a production system

Since the CRISTAL projectrsquos inception a considerable amount of interest has been generated in meta-models and meta-object description languages Work is in progress within the OMG on the Meta Object Facility (MOF) [21] which is expected to manage all kinds of meta-models relevant to the OMG Architecture It is expected that future releases of CRISTAL will benefit from this standardisation effort The meta-modeling approach adopted in the design of CRISTAL could reasonably be applied to organisations developing technologies for lsquovirtual enterprisesrsquo (such as in [22] [23]) where collections of autonomous databases could be related via a central enterprise meta-model Furthermore inrsquothe area of standardisation the Meta Data Coalition is working towards a Metadata Interchange Specification [24] which aims to

399

tackle the exchange sharing and management of metadata in industry The experiences of CRISTAL in providing viewpoint access (with location and application transparency) to enterprise is clearly relevant to research in these areas

VIII GLOSSARY OF ACRONYMS To aid the readability of this paper the authors include here

a glossary of all the acronyms deployed throughout the body of this paperrsquos text ACCOS Automatic Crystal Control System ADC Analogue-to-Digital Converter APD Avalanche Photo-Diode CMS The Compact Muon Solenoid CRISTAL Cooperating Repositories and an Information System for Tracking Assembly Lifecycles DAcq Data Acquisition DCP Digital Control Panel DDM Data Duplication Manager ECAL Electromagnetic Calorimeter EDMS Engineering Document Management System HCAL Hadronic Calorimeter LHC Large Hadron Collider MOF Meta-Object Facility MSGC Micro-Strip Gas Chambers OMG The Object Management Group PBS Product Breakdown Structure PDM Product Data Management UML Unified Modeling Language WBS Work Breakdown Structure WfMS Workflows Management Systems

Ix ACKNOWLEDGEMENTS The authors take this opportunity to acknowledge the

support of their home institutes In particular the support of P Lecoq J-L Faure and M Pimia is greatly appreciated Richard McClatchey is assisted by CERN and by the Royal Academy of Engineering under their Technology Foresight Award scheme during 1998 and acknowledges their support Laszlo Varga and Marton Zsenei are supported by the OMFB Budapest for their activities in CRISTAL

X REFERENCES [ 11 CMS Technical Proposal The CMS Collaboration Jan

1995 ftpcmsdoccernchTprefmPhtml [2] M Philpotts ldquoAn Introduction to the Concepts Benefits

and Terminology of Product Data Managementldquo Industrial Management amp Data Systems 1996 4 ppll-17

[3] P Pikosz amp J Malmqvist ldquoPossibilities and Limitations when Using EDM Systems to Support the Product

Development Processrdquo Proceedings of NordDesignrsquo96 pp165-175 Espoo Fiinland 1996

[4] D Georgakopoulos M Hornick amp A Sheth ldquoAn Overview of Workdlow Management from Process Modeling to Infrastructure for Automationrdquo Journal of Distributed and Parallel Database Systems 3 (2) April

[5] J-M Le Goff et al ldquoDetector Construction Management and Quality Control Establishing and using a CRISTAL Systemrdquo CERN CMS NOTE 1998033 Available from ftpcrnsdoccerncNdocuments98note98~033 pdf

[6] R McClatchey et al ldquoA Distributed Workflow amp Product Data Management Application for the Construction of Large Scale Scientifiic Apparatusrdquo NATO ASI Series F Computer amp Systems Sciences Vol 164 ISBN 3-540- 64411-3 Springer Verlag 1998 Presented at the NATO AS1 workshop on lsquoAdvances in Workflow Management Systems amp Interoperabilityrsquo Istanbul Turkey 1997

[7] Z Kovacs J-M Le Goff amp R McClatchey ldquoSupport for Product data from Design to Operationrdquo Journal of Computer Integrated Manufacturing Vol 11 No 1-2 Elsevier Science Ltd in press

[8] A P Hameri ldquoEDN[S - Concepts Motivations and Basic requirementsrdquo Procleedings of the CERN School of Computing 96 CERN 96-08

[9] E Auffray et al ldquoCertifying Procedure for Lead Tungstate Crystal Parameters during the Mass Production for CMS ECALrdquo Poster N20-31 at IEEE Nuclear Science Symposium Toronto Canada November 1998

[lo] J-P Peigneux et al ldquoStudies and Proposals for an Automatic Crystal Control Systemrdquo CERN CMS NOTE 19971036

[ l l ] Cadim a PDM product developed by Eigner amp Partner see URL httplwwwep-kade

[12] Matrix a PDM product from Adra Systems see URL httpwww adracom

[13] J-M Le Goff et al ldquoFrom Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systemsrdquo CERN CMS NOTE 19971104

[ 141 Objectivity an ob-ject-oriented database management system produced by Objectivity Inc httpwwwobj y co r (

[ 151 P H Feiler ldquoConfiguration Management Models in Commercial Environmentsrdquo Carnegie-Mellon Technical Reports CMUSEI-91-TR-7 ESD-91-TR-7 March 1991

[16] G Bachi amp A P Hameri ldquoWhat Should be Implemented at the Early Stages of a Lerge-Scale Projectrdquo CERN MT95-02 (DI) LHC Note 315

[17] M Fowler 2 K Scott ldquoUML Distilled - Applying the Standard Object Modeling Languagerdquo Addison-Wesley Longman Inc 1997

1995 ppll9-153

398

As the CMS construction process evolves so more data and the relationships between different aspects of the data must be permanently recorded in the CRISTAL data database Different groups of users will require flexible ways to find access and share this construction data The actual information required will depend on the viewpoint and the role of the user in the organisation User groups may well require a maintenance a geometry an alignment or an experiment slow- controls viewpoint The physicist therefore needs to define a viewpoint in terms of lsquophysics elementsrsquo (or sets of detector components) which are derived from the tree of physical locations of detector components resulting from detector assembly Figure 8 shows how a matrix of physics elements can be extracted from the lsquoas-builtrsquo detector construction hierarchy by providing software which can traverse the construction tree and can query and extract the construction data for a selected set of detector components

Dataease for SuperModule SuperModule Precalition in testbeam

Subunit ampI) = A physics Element ex A Readout channel for ECAL

Capsule rystal

Treeregresentatiou established during assembly Matrix representation extracted fian with ~nsm~o Part physical

Figure 8 The derivation of physics elements from the construction lsquoas-builtrsquo tree

Meta-modeling will assist the extraction of data for physics elements provided an extraction mechanism is developed which can navigaqe the meta-model can interpret the structures in the database and can present the data in a form meaningful to the end-user The extraction facility comprises a set of software processes which can be invoked either by a viewpoint-specific application (eg calibration) or by a viewpoint non-specific application That is the processes either navigate a generalised meta-model to project out viewpoint- specific data (ie lsquolooking intorsquo the meta-model) or they navigate the meta-model to correlate effects between viewpoints (ie lsquolooking outrsquo from the data model) In the lsquolooking inrsquo (viewpoint-specific) case the processes perform traversal of the detector description and extract the relevant physics data for the application In the lsquolooking outrsquo case (viewpoint non-specific) the processes determine the effect of a system-wide change on individual viewpoints or sets of viewpoints for example a temperature variation on calibration and slow controls viewpoints An extraction facility is currently under development for CRISTAL [ 191

VII CONCLUSIONS The scale of LHC experiments requires that rigorous

production management techniques be employed for the first time in high energy physics detector construction The CRISTAL project of CMS has implemented a construction database and information system for managing the construction of the CMS ECAL detector The software is currently being evaluated for use in CMS HCAL and Tracker and there has been interest from other experiments and from the LHC magnet construction teams

The CRISTAL data model has been developed using UML [ 171 and has been designed with a view to capturing both PDM and WfMS data The PDM-component of the data model [20] is used to store the full specification for detector construction It will contain all the definitions of individual parts and will describe what is to be built and how it is to be built The specifications are defined and held centrally and releases of versions of the production scheme are issued from the central database The workflow specification is also centrally defined and it will define the process of part characterisation and the complete assembly process from single parts to full-scale detector

The first phase of CRISTAL prototyping and technology evaluation was initiated in the summer of 1996 and delivery of a working prototype is due in the summer of 1998 based on a distributed Objectivity database This prototype will enable ECAL physicists to collect all of the salient characteristics of each ECAL component Once the detector has been tested and assembled it is essential for event recognition programs to have access to detector characteristics in order for event reconstruction and calibration to take place The physics data collected during the detector assembly phase must be arranged and processed to allow population of for example a calibration database

The second phase of the project aims to adopt a strong meta-data approach as described in [19] in the expectation that it will produce a more adaptable system capable of interoperating with future systems This meta-modeling approach will facilitate further integration between product data management and workflow management thereby providing consistency between design and production and speeding up the process of implementing design changes in a production system

Since the CRISTAL projectrsquos inception a considerable amount of interest has been generated in meta-models and meta-object description languages Work is in progress within the OMG on the Meta Object Facility (MOF) [21] which is expected to manage all kinds of meta-models relevant to the OMG Architecture It is expected that future releases of CRISTAL will benefit from this standardisation effort The meta-modeling approach adopted in the design of CRISTAL could reasonably be applied to organisations developing technologies for lsquovirtual enterprisesrsquo (such as in [22] [23]) where collections of autonomous databases could be related via a central enterprise meta-model Furthermore inrsquothe area of standardisation the Meta Data Coalition is working towards a Metadata Interchange Specification [24] which aims to

399

tackle the exchange sharing and management of metadata in industry The experiences of CRISTAL in providing viewpoint access (with location and application transparency) to enterprise is clearly relevant to research in these areas

VIII GLOSSARY OF ACRONYMS To aid the readability of this paper the authors include here

a glossary of all the acronyms deployed throughout the body of this paperrsquos text ACCOS Automatic Crystal Control System ADC Analogue-to-Digital Converter APD Avalanche Photo-Diode CMS The Compact Muon Solenoid CRISTAL Cooperating Repositories and an Information System for Tracking Assembly Lifecycles DAcq Data Acquisition DCP Digital Control Panel DDM Data Duplication Manager ECAL Electromagnetic Calorimeter EDMS Engineering Document Management System HCAL Hadronic Calorimeter LHC Large Hadron Collider MOF Meta-Object Facility MSGC Micro-Strip Gas Chambers OMG The Object Management Group PBS Product Breakdown Structure PDM Product Data Management UML Unified Modeling Language WBS Work Breakdown Structure WfMS Workflows Management Systems

Ix ACKNOWLEDGEMENTS The authors take this opportunity to acknowledge the

support of their home institutes In particular the support of P Lecoq J-L Faure and M Pimia is greatly appreciated Richard McClatchey is assisted by CERN and by the Royal Academy of Engineering under their Technology Foresight Award scheme during 1998 and acknowledges their support Laszlo Varga and Marton Zsenei are supported by the OMFB Budapest for their activities in CRISTAL

X REFERENCES [ 11 CMS Technical Proposal The CMS Collaboration Jan

1995 ftpcmsdoccernchTprefmPhtml [2] M Philpotts ldquoAn Introduction to the Concepts Benefits

and Terminology of Product Data Managementldquo Industrial Management amp Data Systems 1996 4 ppll-17

[3] P Pikosz amp J Malmqvist ldquoPossibilities and Limitations when Using EDM Systems to Support the Product

Development Processrdquo Proceedings of NordDesignrsquo96 pp165-175 Espoo Fiinland 1996

[4] D Georgakopoulos M Hornick amp A Sheth ldquoAn Overview of Workdlow Management from Process Modeling to Infrastructure for Automationrdquo Journal of Distributed and Parallel Database Systems 3 (2) April

[5] J-M Le Goff et al ldquoDetector Construction Management and Quality Control Establishing and using a CRISTAL Systemrdquo CERN CMS NOTE 1998033 Available from ftpcrnsdoccerncNdocuments98note98~033 pdf

[6] R McClatchey et al ldquoA Distributed Workflow amp Product Data Management Application for the Construction of Large Scale Scientifiic Apparatusrdquo NATO ASI Series F Computer amp Systems Sciences Vol 164 ISBN 3-540- 64411-3 Springer Verlag 1998 Presented at the NATO AS1 workshop on lsquoAdvances in Workflow Management Systems amp Interoperabilityrsquo Istanbul Turkey 1997

[7] Z Kovacs J-M Le Goff amp R McClatchey ldquoSupport for Product data from Design to Operationrdquo Journal of Computer Integrated Manufacturing Vol 11 No 1-2 Elsevier Science Ltd in press

[8] A P Hameri ldquoEDN[S - Concepts Motivations and Basic requirementsrdquo Procleedings of the CERN School of Computing 96 CERN 96-08

[9] E Auffray et al ldquoCertifying Procedure for Lead Tungstate Crystal Parameters during the Mass Production for CMS ECALrdquo Poster N20-31 at IEEE Nuclear Science Symposium Toronto Canada November 1998

[lo] J-P Peigneux et al ldquoStudies and Proposals for an Automatic Crystal Control Systemrdquo CERN CMS NOTE 19971036

[ l l ] Cadim a PDM product developed by Eigner amp Partner see URL httplwwwep-kade

[12] Matrix a PDM product from Adra Systems see URL httpwww adracom

[13] J-M Le Goff et al ldquoFrom Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systemsrdquo CERN CMS NOTE 19971104

[ 141 Objectivity an ob-ject-oriented database management system produced by Objectivity Inc httpwwwobj y co r (

[ 151 P H Feiler ldquoConfiguration Management Models in Commercial Environmentsrdquo Carnegie-Mellon Technical Reports CMUSEI-91-TR-7 ESD-91-TR-7 March 1991

[16] G Bachi amp A P Hameri ldquoWhat Should be Implemented at the Early Stages of a Lerge-Scale Projectrdquo CERN MT95-02 (DI) LHC Note 315

[17] M Fowler 2 K Scott ldquoUML Distilled - Applying the Standard Object Modeling Languagerdquo Addison-Wesley Longman Inc 1997

1995 ppll9-153

399

tackle the exchange sharing and management of metadata in industry The experiences of CRISTAL in providing viewpoint access (with location and application transparency) to enterprise is clearly relevant to research in these areas

VIII GLOSSARY OF ACRONYMS To aid the readability of this paper the authors include here

a glossary of all the acronyms deployed throughout the body of this paperrsquos text ACCOS Automatic Crystal Control System ADC Analogue-to-Digital Converter APD Avalanche Photo-Diode CMS The Compact Muon Solenoid CRISTAL Cooperating Repositories and an Information System for Tracking Assembly Lifecycles DAcq Data Acquisition DCP Digital Control Panel DDM Data Duplication Manager ECAL Electromagnetic Calorimeter EDMS Engineering Document Management System HCAL Hadronic Calorimeter LHC Large Hadron Collider MOF Meta-Object Facility MSGC Micro-Strip Gas Chambers OMG The Object Management Group PBS Product Breakdown Structure PDM Product Data Management UML Unified Modeling Language WBS Work Breakdown Structure WfMS Workflows Management Systems

Ix ACKNOWLEDGEMENTS The authors take this opportunity to acknowledge the

support of their home institutes In particular the support of P Lecoq J-L Faure and M Pimia is greatly appreciated Richard McClatchey is assisted by CERN and by the Royal Academy of Engineering under their Technology Foresight Award scheme during 1998 and acknowledges their support Laszlo Varga and Marton Zsenei are supported by the OMFB Budapest for their activities in CRISTAL

X REFERENCES [ 11 CMS Technical Proposal The CMS Collaboration Jan

1995 ftpcmsdoccernchTprefmPhtml [2] M Philpotts ldquoAn Introduction to the Concepts Benefits

and Terminology of Product Data Managementldquo Industrial Management amp Data Systems 1996 4 ppll-17

[3] P Pikosz amp J Malmqvist ldquoPossibilities and Limitations when Using EDM Systems to Support the Product

Development Processrdquo Proceedings of NordDesignrsquo96 pp165-175 Espoo Fiinland 1996

[4] D Georgakopoulos M Hornick amp A Sheth ldquoAn Overview of Workdlow Management from Process Modeling to Infrastructure for Automationrdquo Journal of Distributed and Parallel Database Systems 3 (2) April

[5] J-M Le Goff et al ldquoDetector Construction Management and Quality Control Establishing and using a CRISTAL Systemrdquo CERN CMS NOTE 1998033 Available from ftpcrnsdoccerncNdocuments98note98~033 pdf

[6] R McClatchey et al ldquoA Distributed Workflow amp Product Data Management Application for the Construction of Large Scale Scientifiic Apparatusrdquo NATO ASI Series F Computer amp Systems Sciences Vol 164 ISBN 3-540- 64411-3 Springer Verlag 1998 Presented at the NATO AS1 workshop on lsquoAdvances in Workflow Management Systems amp Interoperabilityrsquo Istanbul Turkey 1997

[7] Z Kovacs J-M Le Goff amp R McClatchey ldquoSupport for Product data from Design to Operationrdquo Journal of Computer Integrated Manufacturing Vol 11 No 1-2 Elsevier Science Ltd in press

[8] A P Hameri ldquoEDN[S - Concepts Motivations and Basic requirementsrdquo Procleedings of the CERN School of Computing 96 CERN 96-08

[9] E Auffray et al ldquoCertifying Procedure for Lead Tungstate Crystal Parameters during the Mass Production for CMS ECALrdquo Poster N20-31 at IEEE Nuclear Science Symposium Toronto Canada November 1998

[lo] J-P Peigneux et al ldquoStudies and Proposals for an Automatic Crystal Control Systemrdquo CERN CMS NOTE 19971036

[ l l ] Cadim a PDM product developed by Eigner amp Partner see URL httplwwwep-kade

[12] Matrix a PDM product from Adra Systems see URL httpwww adracom

[13] J-M Le Goff et al ldquoFrom Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systemsrdquo CERN CMS NOTE 19971104

[ 141 Objectivity an ob-ject-oriented database management system produced by Objectivity Inc httpwwwobj y co r (

[ 151 P H Feiler ldquoConfiguration Management Models in Commercial Environmentsrdquo Carnegie-Mellon Technical Reports CMUSEI-91-TR-7 ESD-91-TR-7 March 1991

[16] G Bachi amp A P Hameri ldquoWhat Should be Implemented at the Early Stages of a Lerge-Scale Projectrdquo CERN MT95-02 (DI) LHC Note 315

[17] M Fowler 2 K Scott ldquoUML Distilled - Applying the Standard Object Modeling Languagerdquo Addison-Wesley Longman Inc 1997

1995 ppll9-153

400

[ 181 TSchall ldquoWorkflow Management Systems for Process Organisations rdquo Lecture Notes in Computer Science 1096 Springer Verlag 1996

[19] F Estrella Z Kovacs J-M Le Goff R McClatchey amp I Willers ldquoThe Design of an Engineering Data Warehouse Based on Meta-Object Structuresrdquo Proc of the Data Warehousing amp Data Mining workshop at ERrsquo98 Singapore November 1998 To be published in Lecture Notes in Computer Science Springer-Verlag 1998

[20] N Baker et al ldquoAn Object Model for Product and Workflow Data Managementrdquo Workshop proc at the 9amp Int Conf On Database amp Expert Systems Applications Vienna Austria August 1998

E211 Object Management Group Publications Common Facilities RFP-5 Meta-Object Facility TC Doc cf96-02- 01 R2 Evaluation Report TC Doc ad97-08-14

[22] M Hardwick et al ldquoSharing Manufacturing Information in Virtual Enterprisesrdquo Communications of the ACM 39

[23] B Gaines D Norrie amp A Lapsley ldquoAn Intelligent Information System Supporting the Virtual manufacturing Enterpriserdquo Roc of the IEEE Int Conf on Systems Man amp Cybernetics Vancouver Canada October 1995

[24] The Meta Data Interchange Specification (MDIS v11) Available from the Meta Data Coalition See URL http 207333206standardstoc html

(2) pp 46-54 1996