ABT Consolidation Review

Electronics & ControlsEtienne Carlier

18-03-2015

“My” Review Objectives

• Prioritize consolidation requests– What, When, Why

• Find the best compromise between safety, availability, maintenance and consolidation

• Review all consolidation budgets– Material (including FSU)– Fellow, TECH, TTE (not included in initial “25 years consolidation”

estimation) • Assess FTE for each consolidation work package• Define strategy for work that will be (can be) done during LS2• Identify possible consolidation activities that can be realized

during YETS or EYETS

General Status

SPS

PS-Complex

LHC

AD

Progressing but very slowly

Replacement of Obsolete / Old Hardware

New MKDV (Generators and) Controls Refurbish Horn Test BenchReplace ZS HV Generator

Renew RCPS Power SuppliesUpgrade ISOLDE Target DC Power Supplies

New MKDV (generators and) controls

• Electronics of SBDS partially consolidated during LS1– Full consolidation of slow control and electrical distribution– New HV power supplies for MKDV generators– Consolidation of MKDH Discharge and Earthing Switches– Consolidation Beam Energy Tracking System (BETS)

• Still to be consolidated – Fast interlocking (thyratron protection and machine protection)– Timing system (synchronization emergency dump)– Emergency Interlocking System (connection to BIS)– Power Triggering

• Next consolidation phase strongly linked to the ongoing reconfiguration of SBDS (MKDV)– Semi-conductor switches

LIU

Refurbish AD Horn Test Bench• Restart Horn test bench scheduled for EYETS with semi-conductor

switches instead of ignitron– Test bench will be installed in B195 instead of B393 as initially foreseen– Triggering system to be upgraded for semi-conductor switches– Slow and fast control to be consolidated– Infrastructure in B195 to be adapted

• Increase of the total pulse current from 380kA to 500kA (speed-up horn ageing) by putting more cubicles in parallel– EN/STI request– Not included in initial budget estimation– Missing power cables (and HV connectors)

• Replace operational power cables during LS2 (full dismantling and re-cabling of horn area)– Feasibility study to be done– No resources within ABT/EC for working at this (P + M)

Replace SPS-ZS (and PS SEH)HV generator

• Full refurbishment of “PANTAK” multiplier stage performed on exploitation budget– Consolidation project not approved while failure rate started to increase

(ageing effect) on operational generator– Performance OK after refurbishment

• Next weak point will be the control stage… – Enough spares available for the moment (good old LEP electronics)

• HV generator consolidation can be postponed to LS3, but…– Not an easy project (Commercially & Technically)– Approval to go ahead should be given before the end of LS2

• Consolidation of PS complex electrostatic septa remote control to be considered– Actually under TE/EPC responsibility– Not in any ABT consolidation plan for the moment

Renew RCPS Power Supplies (All Machines)

• Typical life duration for HV capacitor charging power supplies is ~15 years– Confirmed with recent failure rate increase of LHC-MKI power supplies– Typical MTBF for capacitor charging power supplies form manufacturer is

50000h– MTBF also depend on the number of charging cycles

• All SPS power supplies have been consolidated over the past 10 years– Should be OK up too LS2… at least– Consolidation needs after LS2 to be re-evaluated

• Major part of PS complex power supplies not yet consolidated (Sorensen, CCU…)– Longer life duration expected (LV power supplies type) but operated with a

higher capacitor charging rate– Maintenance done at CERN and start to be difficult

Consolidation should be started

Upgrade ISOLDE Target Power Supplies

• Replacement of old ASTEC power supplies– Reliability significantly decreased over the past years (>2 failures per

year for 2 power supplies in operation)– Maintenance fully done at CERN… and is time consuming– Obsolescence of some components is now critical

• Market Survey on-going – Not mandatory (order will be below 200kCHF) – Want to keep open the possibility to select firms for the Invitation to

Tender in view of the challenging technical requirements• Invitation to Tender

– Technical specification in preparation– Wait for first result of “Charge Pump” modulator prototype in order to

finalize specification– Will have to be validated by specification committee.

Upgrade / Replace Electronics

PS-Complex Septa & Kicker ControlsRenew SPS Kicker Timing Controls

PLC Based Control in SPSAD Kickers and Septa Electronics & Controls

Upgrade Oil Systems ControlsUpgrade Fast Thyratron Interlocking

PS-Complex Kickers & Septa Controls“The Dilemma”

• View from top (Request from BE/CO)– Migrate FEC from RIO3/LynxOS to MEN-A20/Linux– Migrate software from GM to FESA– Insource septa positioning system (G64 + MIL1553)

• View from the middle– Communication with upper layer still based on 6809/G64 hardware– Some electronics more than 25 years old

• View from the bottom (Request from HSE)– Electrical distribution not compliant anymore with latest electrical

safety norms– Clear identification during safety inspection of non conformities

• View from the rear– Paleolithic cabling

Full consolidation needed Staged approach difficult

• PXI solution validated for PI.KFA28 & ER.KFH is a good base for consolidation of BT.KFA, BE.KFA and PE.KFA71-79– Cover only top & middle layers consolidation needs– Solution for consolidation of electrical distribution still to be found– Cabling cleaning should be addressed at the same time

• Cost optimization of this approach to be addressed– Actual solution more expensive than anticipated

• BT.KFA, BE.KFA and PE.KFA71-79 not impacted by any LIU project– If consolidation cannot be done during LS2 next opportunity will be

LS3 (Not acceptable for BE/CO) • Consolidation not possible during YETS & EYETS

– “P” resource should be found for full consolidation during LS2• Looking for collaboration (not encouraging) or • Full subcontracting through FSU. Funding to found.

PS-Complex Kickers & Septa ControlsKickers Controls Consolidation

• Full expertise within the section– Inventory of the needs started within EC (Functional and engineering

specifications)– Generic & modular approach– No re-cabling campaign between machine and electronic racks

• Irradiated cables already replaced in the machine during LS1

• Progressive consolidation possible during YETS and EYETS (not too much linked to remote operation from CCC)– Planning to be defined– Try to keep this consolidation out of LS2 as much as if possible

• Major concern is the recalibration of positioning system after electronic replacement (ALARA)– Can we avoid to have to do it at the end of LS2?

Started

PS-Complex Kickers & Septa ControlsSepta Controls Consolidation

Renew SPS Kicker Timing System• Timing system consolidation finished

– MKP, MKQ, MKE4, MKE6– KiTS Software Framework

• Still to be done– SBDS timing system

• Synchronization emergency dump request (TSU-V3 + programmable dump)• Removal G64 Fine Delay

– Fast timing interlocking logic• Technical solution exists• Validated in CT

– Timing monitoring (all equipment)• Time stamping of all timing event• IPOC system as for MKI / LBDS

Ongoing

PLC based Controls in the SPS• All equipment concerned

– Kickers (MKP, MKQ, MKEs)– Septa (MST, MSE, ZS)

• First SIEMENS S7 PLCs put in operation in 1999 (MS2)!– 16 years of continued operation for the oldest– Progressive deployment

• Support still available from SIEMENS for hardware modules– Product discontinuity not yet announced for the majority of the modules that we are

using– Still 10 years of support ahead of us

• Software support less obvious– New STEP7 version don’t continue to support the full set of modules– Can rapidly become critical in case of modification

• Announced MTBF starts to be reached for some modules– Not yet identified a clear increase of failure rates– Some unexpected PROFIBUS communication error identified (linked?)

Strategy to be defined

AD Kickers and Septa Electronics & Controls

• Consolidation ongoing for kicker controls in the frame of Electronics and Controls displacement– Full timing consolidation (hardware + software)– Consolidation of thyratron protection

• Prototype on AD extraction kicker (and ELENA injection kicker)• Will be deployed on AD injection kicker if OK

• Consolidation not yet started for injection and extraction electro-magnetic septa– Will be done in phase with septa consolidation in PS-

Complex

Upgrade Oil Systems Controls• Oil systems fully consolidated (EC view)

– PSB injection and transfer lines kickers– PS Continuous Transfer kickers– AD kickers & horn

• Still to do– PS ion injection kicker (KFA28)– PS proton injection (KFA45)– PS fast extraction (KFA71-79)– ISOLDE

• Missing functional and engineering specifications– Close collaboration needed between FPS & EC for this type of consolidation

• Complex systems that require detailed specification

– FPS is the “Maitre d’Oeuvre”• Planning to be synchronized with other consolidation activities

(PE.KFA71-79) or project (PI.KFA45)

Upgrade Fast Thyratron Interlocking• Good old LEP electronics (1992) used for all thyratron protection in

SPS (and LHC injection kickers)– Clear identification of missing functionalities– Weaknesses identified when used in the LHC– Maintenance not possible anymore (obsolescence of components and

embedded software programming environment)• Same functionalities needed for PS-Complex kicker systems

– Try to find a common modular approach that can be used for all our systems• Critical project for equipment and machine protection

– Deployment and commissioning strategy to be defined• Project started

– Different approaches under consideration (prototyping)– “Meeting point” in about 1 year for critical cross-comparison of the different

ideas and decision on the final solution– Final solution will be available for full deployment during LS2

Safety ImprovementElectronics & Controls

Kicker Discharge and Earthing SwitchesKicker Fire Prevention

Improve Capacitor Safety

Kicker Discharge and Earthing Switches• Consolidation launched after a “just missed” fire event in MKP

(BA1)– One electrical earthing switch blocked in CLOSE position with no

interlock to HV power supply • Upgrade of “Discharge and Earthing Switches” design done by EC

– Review mechanical design– Use of fail-safe actuators and sensors (SIL3) instead of relays & micro-

switches– Design of a new controller

• Status– MKDH, MKE4 and MKE6 already consolidated– MKP and MKQ still to be done

• Can be scheduled during EYETS– Small cabling campaign needed

Kicker Fire Prevention• Solution found by FPS– Integration in EC infrastructure still to be validated– Test to be scheduled in 867 (SBDS?)

• Integration into the “AUE” safety chain – Trip MAINS powering in case of activation– Extension (dedicated) ASi-bus to sensor locations

• Small cabling campaign– Possible during YETS / EYEST

• To be included in MKE4 re-configuration process foreseen during next YETS

Improve PS –Complex Capacitor Safety

• Capacitor bank unit not conform anymore with CERN safety rules– Missing automated capacitor discharge and manual earthing

mechanism– No cable interlocking on capacitor charging and discharging

circuit (banana plugs based system)• Design of a first prototype scheduled for 2015• Progressive deployment possible (YETS & EYETS)– One to one replacement

• Integration within kicker control architecture (interlocking) still to be studied. – Can be done in a second phase

Additional Consolidation Requests (up to end LS2) and Candidate Items for Post LS2

SPS Tune Kicker ReconfigurationLHC Consolidation

Electronics & Controls for Test Cages in 867

SPS MKQ Reconfiguration• Four non PPM resonant charging power supplies

– Two small capacitor banks for use at low energy– Two big capacitor banks for use at high energy

• Tune system (Beam Instrumentation) can only do one measurement per USER– Two big capacitor banks only used once in the same USER for dynamic aperture

measurements (Frank Schmidt)… unsuccessful and finally done with old MKE6 upstream kicker system

• Reconfigure system with only two fully PPM resonant charging power supplies– One small and one big capacitors banks– Significant reduction in hardware

• Earthing / Discharge switches• Power supplies• Thyristor units

• Already discussed a few time with OP and ABP. – Never got a go/no-go– Probably to be proposed at IEFC

LHC Consolidation• Strategy for LHC Consolidation to be defined

– Ageing effect starts to be visible on some LHC equipment– Maximum possible performance reached for some sub-systems

• Pending consolidation requests– MKI HV power supplies– MKI thyratron heating power supplies

• New consolidation request identified after LS1– LBDS Slow controls (PLC) (CPU capacity now used at 95% & Communication capacity used

at 90%)• Consolidation requests identified in the frame of HL-LHC

– LBDS power triggering and re-triggering systems (faster detection and reaction time)– Injection kicker consolidation (fast interlocking and triggering)

• Consolidation strategy to be defined for medium term– LBDS power supplies

• Stage approach?• Start to increase the number of spares

– Fast acquisition system (IPOC…)• Homogenous w.r.t. heterogeneous approach• Drivers w.r.t. OS evolution

Electronics & Controls for Test Cages in 867

• A lot of old electronics used for test cages in 867– SALEVE side

• Consolidation not possible through exploitation budget

• “P” resource not available• Not dependent of YETS, EYEST and LS2

Next

Next

• Alain Antoine will coordinate “Consolidation” projects for EC up to LS2 (included)

• Possible major consolidation activities to be done during forthcoming YETS & EYEST and LS2 identified for the section

• Pending validation• An “EC consolidation Coordinator” nominated

for each consolidation project

“EC Consolidation Coordinator” Role

• Capture requirements• Define work sub-packages• Write engineering & functional specifications• Estimate resources (FSU)• Setup & follow-up planning• Coordinate project with FPS & SE• Report to section meeting & TCM as required

Major Consolidation ActivitiesEquipment Responsible Due date

PSB-CONS 1 BT.KFA / BE.KFA Thierry Gharsa LS2 3

PSB-CONS 1 Septa positioning Marco Pimentel YETS & EYETS

PSB-CONS 1 Septa control Felipe Cordobes YETS & EYETS

PS-CONS 1 Septa positioning Marco Pimentel YETS & EYETS

PS-CONS 1 Septa control Felipe Cordobes YETS & EYETS

PS-CONS 1 PE.KFA71-79 Jan Schipper LS2 3

PS/SPS/LHC-CONS 1 Fast Interlocking Pieter Van Trappen LS2 2

AD-CONS 1 Horn test bench Gregor Grawer EYETS 2016

1 : Coordination Alain Antoine2 : Equipment to be defined3 : Manpower to be found