Future Fast Orbit Feedback for SOLEILMTCA workshop 2021

Romain BROUCQUART

December 9th 2021

SOLEIL Synchrotron

• 3rd generation light source

• France, 20km south of Paris

• In operation since 2006

• 29 beamlines

Energy 2,75 GeV

Circumference 355 m

Revolution period 1,18 µs

Number of Cells 16

2

• SOLEIL Upgrade

• TDR ongoing

• Shutdown scheduled for 2027

Actual FOFB system

• SOLEIL’s FOFB runs on Libera Electron.

• DLS Communication Controller

• PSU UART communication

• Error signal PI corrector + Matrix (row) multiplication

• Dedicated fast network

• 122 Libera Electron

• 2 Gbps Fiber, DLS CC

• Libera Synchronized by Machine Clock

• Shortcuts and link redundancy

• Fast magnet correctors

• 50 correctors H & V, connected to Libera Electrons

• 1,25 Mbps RS422

• Feedback correction

• PI controller

• Inverted response matrix (SVD + Tikhonov reg.)

• Other features

• Sniffer archiver : 2 weeks rolling buffer of 10 kHz positions

• Interface to TANGO Controls

• Strong interaction with SOFB (set point adjustment)

Actual FOFB

# BPM 122

# Corr. PSU 50 H & V

Data rate 10 kHz

Correction

Bandwidth

150 Hz

Latency (communication

+ computation)

100 µs

Stability 10% of beam size

20 µm H ; 0,8 µm V

Algorithm PI, 122x50 matrix from

SVD + Tikhonov

3

BPM electronic : Libera Electron

Actual FOFB system

Position measure122 BPM electronics, 10 kHz data rate

Position exchange60µs to complete the entire exchange

Correction computationError to set point, Matrix (row) multiplication, accumulator

Correction application1,25 Mbps, RS422

4

BPM electronic : Libra Electron

RF

Steerer magnet and BPM block

mA

RS422

Fiber Ring

Magnet PSU

Fiber ring network

topology

11

22

33

44

44

33

2211

FOFB Upgrade

• Follow machine evolutions

• eBPM, timing system, machine upgrade

• Future algorithms

• Performance improved

• Bandwidth to 1 kHz (latency reduced)

• Data rate to 100 kHz

• Precision to 2-5% of beam size

• New functionalities

• Fast BBA, fast response matrix

• Record correction data

• FOFB and SOFB on same platform (?)

• Angle interlock (?)

• XBPM in FOFB (?)

Timeline

OFB

Upgrade

eBPM

Upgrade

MACHINE UPGRADE

Corrector

Upgrade

2023

… and everything

still works

Actual FOFB Future FOFB

# BPM 122 ~250

# Corr. PSU 50 H & V ?

Data rate 10 kHz 100 kHz

Correction

Bandwidth

150 Hz 1 kHz

Latency (communication

+ computation)

100 µs 10 µs

Stability 10% of beam size

20 µm H ; 0,8 µm V

2-5% of beam size

50 nm H & V

Algorithm PI, 122x50 matrix from

SVD + Tikhonov

?

More

BPMsNew

algorithm ? 5

Libera Electron now obsolete

→ Separate Position measure from feedback system

2027

Timing

Upgrade

Future topology

6

Layered network

• Lower network

• Ring, star, chain …

• BPM and Correctors

• dedicated protocol

• low data throughput

• Evolve with eBPM and Correctors

• Higher network

• Star network with a Central Node

• Cell nodes aggregate data

• standard protocol (Ethernet)

• high data throughput (10 Gbps)

Future BPM (Red) might be

connected in grapes, in

chain…

Actual BPM can be

connected in ring

Cell nodes aggregate data

of N eBPM and are

connected to M correctors

The Central Node runs the

algorithm and archive data.

Future platform

• nVent/NAT NATIVE-R1

• 1U simple rack

• Embedded eMCH, no PCIe switch

• Point to point backplane

• DESY/CAENels DAMC-FMC2ZUP

• FPGA Zynq UltraScale +

• 4 GB DDR4

• 2 slots FMC

• Timing interface (2 signals or White rabbit)7

Same platform for cell nodes and central nodes

• MTCA standard

• Flexible

• Industrial grade

• More and more present in applications

• FPGA AMC with FMC

• Speed computation

• Customize and upgrade interfacesnVent/NAT Native-R1

DAMC-FMC2ZUP

Interface to eBPM

8

• SFP interface: actual eBPM, future eBPM

• Connect to a maximum of BPM

• High density interface

• leverage by topology (ring, chain, grouped…)

• Use both FMC slots ?

• DESY/CAENels FMC-4SFP

• 4 Interfaces

• Simple Direct Attach Cable (short reach) or Fiber (long reach)

• Hitech Global HTG-FMC-2xQSFP+ with breakout DAC

• 8 Interfaces

• With breakout cable

• Samtec Firefly MTP (to be tested)

• 10 interfaces

• Need a breakout cassette

FMC-4SFP

HTG-FMC-2xQSFP

QSFP+ copper and fiber

breakout cables

Firefly FMC

MTP cable and LC

breakout cassette

LC Fiber and SFP+ module

• DLS CC Protocol

• Tested on DAMC-FMC2ZUP

• GT wizard + home designed Protocol decoder

Interface to Corrector PSUs

9

• Future Corrector PSUs : unknow

• Actual Corrector PSUs

• RS422, 1.25 Mbps, UART

• AMC

• 16 or 32 links (Vadatech AMC304 or AMC082)

• AMC304: Communication via backplane PCIe

• AMC082 On board FPGA

• FMC

• 16 links (Vadatech FMC156)

• Would take one FMC slot

• Custom RTM (leaning to this solution)

• RS422 line drivers

• 32 links to 8 RJ45 connectors

FMC156

AMC082AMC304

Custom RTM

Interface Cell Nodes – Central Node

10

• SFP+ : Use same solution as eBPM communication

• On Cell Nodes Use one of the SFP+/LC port of the

BPM interface FMC

• 10 GbE Protocol

• Nice to have Standard protocol

• Latency consideration : Tested

• No IP/UDP : only ethernet frames

• Allow usage of ethernet switches

FMC FMC

To BPMs

Cell Node

To Cell

Nodes

FMC FMCCentral Node

FPGA Firmware architecture

11

• Use AXIS

• Improve Compatibility (Xilinx,

DESY framework)

• GT wrapper depends on the

FMC board/Slot

10GbE tests

12

Loopback test

128 bytes payload

200 kHz sending rate

→ 0,5 µs Latency (one way)

10GbE tests

13

Through Ethernet Switch

128 bytes payload

200 kHz sending rate

→ 1,3 µs Latency (one way)

10GbE tests

14

Through Ethernet Switch

N to 1 communication

128 bytes payload

200 kHz sending rate

→ 1,3 µs Latency

Project status

15

Prototypes

Mass storage

FMC QSFP, SFP

eBPM communication

10 GbE communication

FMC MTP

Interface to timing system

Next steps

Software architecture : Driver, interface to Tango Controls…

→ Used Python interpreter on the on-board Linux

FPGA Prototype → clean/packaged IP

Connect to actual eBPM network : data spy

Troubles on the way

DAMC-FMC2ZUP shipping delay

Corrector PSU communication solution

Thank you !

16