cs -framework - overview & status

8-Feb-2012 CS-Workshop 2012, Dietrich Beck

CSCS-Framework - Overview & Status-Framework - Overview & Status

• Intro

• Overview

• Status


OverviewOverview

• Same slides as last year…


StatusStatus

• Nothing new since CS Workshop 2011


ConclusionConclusion

Thank you for your attention!


there is one more thing...there is one more thing...


(


Towards the FAIR Timing System using Towards the FAIR Timing System using White RabbitWhite Rabbitss

• Towards FAIR• White Rabbit• Timing System


Towards FAIR…Towards FAIR…

8-Feb-2012 CS-Workshop 2012, Dietrich Beck 23 November 2011

8-Feb-2012 CS-Workshop 2012, Dietrich Beck 8 December 2011


Bild von heuteBild von heute

12 January 2012

8-Feb-2012 CS-Workshop 2012, Dietrich Beck 12 January 2012


General Machine Timing System @ FAIRGeneral Machine Timing System @ FAIR

• parallel execution of beam production chains

• cycles: 20ms to hours

• trigger and sync. equipment actions

• 1 µs precision in 99% of all cases

• few ns precision for kickers

• (ps for rf-systems: BuTiS)

• many rings

• > 2000 devices connected to timing system

• large distances

• robustness: lose at most one message per year


Idea: Timing System Based on TimeIdea: Timing System Based on Time

Not:

1km distance: 5s propagation time

1s precision: requires compensation for cable length

Timing Master Equipment 1do this

Equipment 2do this


Idea: Timing System Based on TimeIdea: Timing System Based on Time

Instead:

• equipment pre-programmed for autonomous action at a given time• pre-programming via timing-events• required: clock synchronization ns required distribution of information and execution of action are decoupled !!!distribution of information and execution of action are decoupled !!!• timing-events must be sent “early enough”timing-events must be sent “early enough”

• upper bound latency for transmission (e.g. 100upper bound latency for transmission (e.g. 100 s): real-time!s): real-time!

• lossless transmission: robustness!lossless transmission: robustness!

Timing Master Equipment 1

do this @ 15:03

Equipment 2

do this @ 15:03 do that

@ 15:01

"Oh dear! Oh dear! I shall be too late!"

???


“The White Rabbit (WR) project is a multi-laboratory and multi-company effort to bring together the best of the data transfer and timing worlds …” (Proc. ICALEPCS 2009)

www.ohwr.org )


White Rabbit and White Rabbit and CSCS

…


Now the Talk Starts (if you are still interested)Now the Talk Starts (if you are still interested)

• Intro

• Overview

• Status


Starting Point (Starting Point (2001)2001)

• Control System– Control devices (power supplies, function generators, timing, complex

experimental procedure…)– No process control– DAQ and data processing is only a minor issue– Integrate things that were not developed to work together as a system

• Ever changing set-up– The same experiment is never done twice– Reconfiguration of complete hardware (seconds!)– Integration of new hardware (next beam-time)

• Small and medium size experiments– No dedicated staff for control systems– Different experiments but similar problems to solve (configuration,

complex timing schemes, similar hardware,…)• … What about a “Framework”?


Different Points of View – A Framework is...Different Points of View – A Framework is...

Developer: User:

picture pocket knife

... a tool box ... something ready to use

(sth. to solve a problem) without additional work

(= the solution!)


Definition of the Definition of the CSCS-Framework-Framework

control system = framework + add-ons

EE/KS

and contributors

experiment

bug reports, new features requested

add-ons may become part of framework

bug fixes, new features, maintenance

CycleControl

AFGTiming+DAQ HV

GUI


PHELIX PHELIX (Petawatt High-Energy Laser for Heavy Ion Experiments)(Petawatt High-Energy Laser for Heavy Ion Experiments)

UNILAC

Z6

SIS

PHELIX

HHT

Injection box

Mai

n A

mp

lifi

erS

enso

rFaraday

Isolator

Switch Yard

Compressor

Target Chamber

fs Front End

Fiber ns Front End

Pre-amplifier2 x 19 mm heads1 x 45 mm head

X-ray Lab(low energy)

TW compressor

Target chamber

Z6 experimental

area

Heavy ions

Laser Bay

Double-pass 31.5 cm amplifier

70 m

(slide by S. Götte, GSI)

Up to 3 MJ electrical energy converted to 1kJ light energy or 0.5 Petewatt laser power. 2300 documented “shots” (Dec. 2010). About 10k process variables distributed on 15+ nodes in 2 buildings


Quad-Triple Trap Mass Spectrometer ISOLTRAPQuad-Triple Trap Mass Spectrometer ISOLTRAPHigh-precision mass determination of unstable nuclei with a Penning trap mass spectrometer at ISOLDE/CERN High-precision mass determination of unstable nuclei with a Penning trap mass spectrometer at ISOLDE/CERN

www.cern.ch/isoltrap

– Nuclear-, Astro-, Fundamental PhysicsNuclear-, Astro-, Fundamental Physics

m/m m/m 10 10-8 -8 (typically!)(typically!)

– via cyclotron frequency of stored ionsvia cyclotron frequency of stored ions

– ““sub-µs ion-juggling” every secondsub-µs ion-juggling” every second


Cooking Recipe for the Cooking Recipe for the CSCS Framework Framework

• One development tool LabVIEW

• Standardization object oriented approach, base classes

• Distribution to many nodes DIM (www.cern.ch/dim) – Event driven communication for everything

– Scaling to large systems by distribution

– Remote access

– …

• SCADA functionality (alarming, trending, …) LabVIEW DSC module

http://www.cern.ch/dim


(Over)view (Over)view CSCS-Framework-Framework

• Intro

• Overview

• Status


SourceForge: downloads, tracker (bugs, features,…)SourceForge: downloads, tracker (bugs, features,…)


SCC via SubVersion @ GSISCC via SubVersion @ GSI


Domain Management System: Domain Management System: Process Management in a Distributed EnvironmentProcess Management in a Distributed Environment


Process MonitoringProcess Monitoring


PackagingPackaging


Documentation via WikiDocumentation via Wiki


Class Documentation via UML ToolsClass Documentation via UML Tools


Lessons Learned…Lessons Learned…


Lessons Learned – What Users say about Lessons Learned – What Users say about CSCS(from a talk by Stefan Götte, at the NI BIG PHYSICS Round Table, Paris, 2009)

Stress Field of the Responsible Person:• The OS (XP) and/or the intranet are always unreliable,• the programming language (LabVIEW) never really fulfills the need,• the framework (CS) is only close to the requirement,• the classes of other CS collaborators are typically not usable,• there is no way to test things since there is no test system available

while the real system is always in use, • the users never define what the program has to do, but• are not pleased with what the programmer delivers, and• they misuse the system additionally.• Anyhow: The goal is an easy system where the happy user does not

realize what happens behind the scene, which works reliable and for ever (better: till the next LabVIEW version is installed).


““The OS (XP) and/or the intranet are always The OS (XP) and/or the intranet are always unreliable” (assumption for design phase)unreliable” (assumption for design phase)

Design principles of major importance• A control system must be designed to crash and• (auto-)recover from failures of all kinds

– Fileserver down– Computer down/reboot

• Installation of updates• Switched-Off/On by users• …

– Network down– Power cuts– …

• The system is stressed most, when– everything goes wrong (failures, alarms, sub-system crash…) and– all kinds of asynchronous signals fire at the same time (self-induced

DoS-Attack) and– availability is most important.


Lessons Learned – What Users say about Lessons Learned – What Users say about CSCS(from a talk by Stefan Götte, at the NI BIG PHYSICS Round Table, Paris, 2009)

Stress Field of the Responsible Person:• The OS (XP) and/or the intranet are always unreliable,• the programming language (LabVIEW) never really fulfills the need,• the framework (CS) is only close to the requirement,• the classes of other CS collaborators are typically not usable,• there is no way to test things since there is no test system available

while the real system is always in use, • the users never define what the program has to do, but• are not pleased with what the programmer delivers, and• they misuse the system additionally.• Anyhow: The goal is an easy system where the happy user does not

realize what happens behind the scene, which works reliable and for ever (better: till the next LabVIEW version is installed).


Lessons Learned - DocumentationLessons Learned - Documentation

The fundamental problem about documentation• Two lines of text are not enough, but• Two pages of docs – (almost) nobody will read that*

• Docs out of date …

Solutions• HOW-TOs, FAQs• Auto-generated documentation

– Class documentation, CS2JavaUML– Requires well documented code (again: coding conventions!)– Release notes

• Demo-system (delayed…)

* sometimes not even the first two lines…


Lessons Learned – Generic/Common ToolsLessons Learned – Generic/Common Tools

Generic tools (generic control system GUI, generic device GUIs)

• Sometimes difficult to understand ( documentation problem)

• Lot’s of work to maintain

• Don’t match the users’ needs exactly

• Either– least common denominator (missing features), or– include all requested features (too complex)

“Nobody likes them, but everybody uses them.”


Lessons Learned – Missing FeaturesLessons Learned – Missing Features

Identifying missing features sometimes

• requires telepathic skills

• results in cool solutions that are finally not used (nice to have versus really useful)– CS Access System– Sequencer– ObjectNets, PetriNets– some base classes

• …


Lessons Learned – CompatibilityLessons Learned – Compatibility

Backward Compatibility (“does my old code still work?”)

• Is maybe THE most important feature of a framework

• Often results in a impressive mix of different versions (core system v1, base classes v2, GUI v4, device classes v3,…)

• Makes it hard to correct wrong design decisions

• Requires clear public library routines or class methods, but…

• using conventions on what is public/private does not work


Lessons Learned – The “Saving-Time-Myth”Lessons Learned – The “Saving-Time-Myth”

Wikipedia: “…software frameworks … reducing overall development time” (?)

Amount of time for solving a problem• decreases dramatically, if a problem may be solved with

(generic) existing software. “configuration instead of coding”• does not change – but the solution is much better• may even increase (short-term), compared to a dedicated

solution not (!) using the framework (required: training, courses, understanding and application of conventions)

• decreases (long-term): framework maintained by others, re-usability of code, replacing hardware, coding conventions enforced, maintainability, common language, know-how transfer, …


Experiments using the Experiments using the CSCS framework... framework...

• require high flexibility

• have a large variety of hardware types

• have up to 10,000 (1M possible) process variables

• require fast ( 100 ns) timing control using dedicated hardware

PHELIXPHELIX

Motion CaveAMotion CaveASHIPTRAPSHIPTRAP

ISOLTRAPISOLTRAP

REXTRAPREXTRAP

LEBITLEBIT

GSI, GermanyGSI, Germany

Mainz, GermanyMainz, Germany

Greifswald, GermanyGreifswald, Germany

CERN, SwitzerlandCERN, Switzerland

MSU, USAMSU, USA

Lanzhou, ChinaLanzhou, China

data takingdata taking

developmentdevelopment

commissioningcommissioningFOPIFOPI

RISINGRISING

others ...others ...HITRAPHITRAP LPTLPT

TrigaTRAPTrigaTRAP

ClusterTRAPClusterTRAP

POLARIS, HIJPOLARIS, HIJ


Conclusion and OutlookConclusion and Outlook

• CS 3.21 released for LV2009.

• about 15 active applications

• 1,000,000 PVs demonstrated, an even larger number should be feasible

• 5,000 objects ( hardware devices) demonstrated, an even larger number should be feasible

• stability of a distributed CS system is better than a few hundred hours of continuous operation

• about 15 (60) hardware devices supported on SourceForge (Subversion)

• FAIR: MATS and parts of HITRAP have selected CS as control system framework.


Acknowledgements...Acknowledgements...

Holger Brand, Dietrich Beck, Mathias Richter, Alexander Schwinn, Falk Ziegler, Romain Savreux, Chabouh Yazidjian, Klaus Blaum, Dennis Neidherr, Melanie Wolf, Stefan Schwarz, Josh Savory, Stefan Götte, Maximilian Kugler, Tobias Habermann, Michael Block, Frank Herfurth, Christian Rauth, Stephen Koszudowski, Manas Mukherjee, Thomas Rechel, Martin Feldmann, IT@GSI, ……………...

cs -framework - overview & status

Documents