lcls llrf distributed control system

1 Dayle Kotturidayle@slac.stanford.edu

1LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System

EPICS Collaboration Meeting, Vancouver 1 May 2009

LCLS LLRF Distributed Control System

Dayle KotturiControls Department

SLAC National Accelerator Lab

OutlineScope

Global Overview

General stability requirements

Principal motivator

Solutions

Throughput measurement

Conclusions

Additional resources

The low level RF controls system consists of RF phase and amplitude controls at these locations:

LaserGunL0-A (a.k.a. L0-1)L0-B (a.k.a. L0-2)L0 Transverse cavityL1-SL1-XL2 – using 2 klystrons to control avg phase/ampl of L2L3 Transverse cavityL3 - here is a bit different (lots of klystrons!)

LLRF Global Overview

For LCLS, the general RF stability requirements are: 0.1 deg phase and 0.1% amplitude in L0 and L1 for S band.

Principal motivator

Placing the digitizers next to the low noise RF components eliminates transmission of low noise analog signals outside the chassis.

I and Q Demo-dulator

Thermocouple system

VME Crate

L0-AL0-B

L1-SL1-XT Cav

Beam-based longitudinal

fast feedback gigabit

ethernet

Controls gigabit ethernet (interface to MCC)

Eth recvr

Private ethernet8 kBytes at 120 Hz

I and Q Modulator

1 trigger for 4

channels of 1k

samples

Solution 2: Multiple VME crates with in-house modules

F Refe

k dist

ribute

d to a

ll 30 s

s in t

RF Reference/4 = 119 MHzstabilized to 50 fs jitter

RF Reference*6 = 2856 MHzstabilized to 50 fs jitter

Controller with

ethernet

Controller with

ethernet

Local trigger

Possibly combined into one module

Slow adjustments to allow rotation of the

reference phase

Other waveformsFast, but not 119

MHz. 59.5 MHz ok

Global longitudinal beam-based

feedback VME crate

L2: in sector 24, there are 3 stations to adjust in order to accurately control phase and amplitude for long, beam-based fast feedback

Sector 25 T Cav (new 4/2005)

RF Phase and Amplitude correction at 120 Hz for:laser, gun, L0-A, L0-B, L1-S, L1-X, T cav, L2 and S25 Tcav

10' accelerator

IQ Modulator: a phase shifter

and an attenuator

1 kW 1 kW

100 mW

Solid State Sub Booster

Klystron

SLED cavity

HPRF240 MW

All except laser RF

100 mW

119 MHz Laser

Oscillator

GunNB: For the gun, SLED

cavity is shorted out

119 MHz120 Hz

photodiode

1 trigger to travel up to ½ sector

DAC (slow)Updates

@120 Hz;stable 20 µs before

beam arrives and able to distinguish beamcodes

feedback VME crate

IQ Modulator (driving sub-

booster klystron in sectors 29)

RF Phase and Amplitude correction at 120 Hz for: L3

Beam-based longitudinal fast feedback gigabit

ethernet

istrib

klystron

10' accelerator

1 kW 60 MW

HPRF240 MW

SLED cavity

klystron

10' accelerator

1 kW60 MW

HPRF240 MW

SLED cavity

Sub Booster Klystron Sub Booster Klystron

100 mW 100 mW

1 kW1 kW

8 copies8 copies

Eth recvr

VME Crate at S20 running

longitudinal, beam-based

feedback.

PADPADPAD

RF Dist’n

PADPAD

PACL0-Tcav

PADPAD

PACL0-A

PADPAD

PAD PAC

Beam Phase Monitor

SPACSPACSPAC

SPACPAD

Fast PACs: Slow PACs (SPACs): PADs: VME crates:

Eth recvr

PACL24-2

PACL24-3

PACTcav L24-8

PADPAD

RF phase and amplitude correction and global feedback at 120 Hz for LCLS LINAC

PADPADPAD

feedback.

Indicates may be needed

128212

S20 S24 Total

The maybe is included in counts below

Indicates located in RF HutOtherwise at Klystron

FPGADAC

1 trigger

Coldfire CPU

running RTEMS

and EPICS

Coldfire CPU

running RTEMS

and EPICS

Example of generic LLRF control system instance

VMETemperature monitors Temperature monitors

1 trigger

IQ Modulator gives phase

and amplitude control

2 channels of 1k

samples

4 channels of 1k

samples

Phase/Amplitude Detector -> VME

Coldfire

LLRF PAD (Phase and amplitude detector)

ADCChannel

LLRF VME(Handles phase and amplitude

conversions, feedback calculations and setting of PAC)

mvme6100

LCLSCA network

MCC workstation running lclshome GUI

Change PAD channels’ size and offsets

Set PDES, ADES and control RF feedback

Use secondary ethernet interface to send 4 channels of PAD data via UDP broadcast via private

switch. Time required: 100 nsec

- Receive I and Q averages- Convert to 4 channels of Phase and amplitude- Compute weighted averages- Trigger BSA - Do feedback calculation- Send I and Q adjustment to PAC

EVR timing trigger

Update GUIs with actual phases amplitudes, feedback

corrections, etc, every 2 seconds

Update GUIs with PAD

waveforms, temperature,

I and Q actuals every

2 seconds

Private gigabit switch

OS, BSP and EPICS versions

PAD: rtems4.9.1m68k uC5282epics-R3.14.10

VME: rtems4.9.1powerpc beatnik (mvme5500/mvme6100)epics-R3.14.8.2

PAC: rtems4.9.1 m68k uC5282epics-R3.14.10

PAD waveform readout

PAD IOC Stats

VME Feedback Calculation

VME IOC Stats

PAC waveform control

Throughput Time: PAD->VME->PAC

Throughput steps: PAD->VME->PAC

Event (#

matches

diagram)

Absolute time after trigger (μsec)

Description

- 0.0 Trigger delivered to phase/amplitude detector digitizer (PAD)

1 7.2 ISR: signals “data ready” to wake up DAQ task

2 102.0 Channel0 readout and data processing begins (40 Is and 40 Qs)

6 1963.2 PAD starts stream of processed values to VME over private net

7 2349.6 PAD streaming completed. VME parses, does feedback, sends.

8 2508.0 Phase/amplitude controller (PAC) receives new setpts from VME

9 2524.0 PAC writes to FPGA which calcs new WF to send next trigger

10 2529.2 Data is ready

Conclusions

At 120 Hz operation, time budget=8.333 ms

LLRF PAD->VME->PAC throughput measured=2.529 ms for 4 channels of 40 points each, with no offsets,

adjust: subtract 2.5 μsec per pair of IRQ raise/lower calls (8 pairs = 20 μsec)

adjust: one socket sends to multiple PACs; add switching time

Acknowledgements

Thanks to Ron Akre and Klystron Department for setting up hardware, scopes and signal generators

Thanks to SLAC NAL Controls Group

Additional Information

Details of the PAD->VME transferDetails of the VME->PAC transferRF stability measurementPADPAD Block diagramLCLS LLRF website: http://www.slac.stanford.edu/grp/lcls/controls/global/subsystems/llrf

Details of the PAD->VME transfer

http://www.slac.stanford.edu/grp/lcls/controls/global/sw/epics/epics%20team%20meetings/presentations/lanIpBasic.pdfRaw ethernet packets with IP and UDP headers. Similar to BSD sockets.Solution is for low end CPU on small LAN.Requirement: ship 1 KB of data in ~200 μsecVME initializes, starts and stops PAD streamingWhen PAD is streaming, device support for waveform on VME parses out the values and uses them in the feedback calculations of new setpoints.

Details of the VME->PAC transfer

On VME, a subroutine record that has calculated new setpoints calls a driver routine that sends the values to the PAC via udp socket

PAC is has thread waiting to receive packet

When packet arrives, it parses out the setpoints and puts them into mem mapped FPGA

LCLS Jitter Specification for 2 Seconds is 0.14% Amplitude and 0.14 degree Phase

Feedback ON 20 Second Plot showsPhase Jitter 0.043 degreesAmplitude Jitter 0.022%

Short Term RF Jitter Specification for L0B are well Exceeded.This is as good as it gets – Don’t tell Physicists or they will expect it. Ron Akre 2007

About the PAD

CPU is MCF5282 (64MHz)The digitizer used is the Linear Technologies LTC2208. It was the first 16 bit digitizer chip on the market capable of running at 119MHz, it is specified to run up to 130MHz.At SLAC NAL, PAD digitizer used for RF, beam position monitors, beam charge monitors and bunch length monitors.Pohang Light Source is also using PAD for new RF system.

Phase/amplitude detector (PAD) Block

5VDCLM340Reg

4 X 16 bit ADC102MHz ClockLTC2208Transformer Coupled Inputs

Chan. 2

Chan. 1 FIFO 64k wordsWCLK

16bit DATA

CHAN 1RF INPUTRP N

CHAN 2RF INPUTRP N

ZX60-4016E

LORFIF

RF Board25.5MHz IF

Control Board

Control

FIFO 64k words

16bit DATA

12VDCLM340Reg5 ohm

FRONT PANEL LEDs

10 ohm

1 A F B

110VAC

1 1 0 V A CP o w e r M o d uleC o rc o m6 E H L 1 S C GRN

BRNBLU

YELBLK

REDVIO

TE62071-ND 2x9V 1.94A

5VDCLM340Reg

12VDCLM340Reg

5VDCLM340Reg

GRNRED

BRNBLU

YELBLK

REDVIO

TE62054-ND 2x18V 0.417A

CHAN 4RF INPUTRP N

CHAN 3INPUTRP BNC

Chan. 4

Chan. 3

QSPI5VDC0.8A x 2 Analog

5VDC0.5A DigitalLO OUTPUT

2830.5MHzFP N

24Bit Analog InputBoard

20 pin ribbonTRIG MonFP BNC

ANALOG IN ANALOG IN

CLOCK Mon102MHzFP N

TRIG In120HzRP BNC

CLOCK IN102MHzRP N

ZN4PD1-50

CS/CLK

16 bitDATA

CONTROL /Arcturus uC5282Microcontroller Modulewith 10/100 Ethernet

LO INPUT2830.5MHzRP N

-2dBm 10dBm

10dBm-8dBm

3x 75mA

FILTER 25.5MHz BP

5VD 5VA2 5VA1 12V2 12V1

Ron Akre

OutlineScope

Principal motivator

Solutions

Throughput measurement

Conclusions

Additional resources

The low level RF controls system consists of RF phase and amplitude controls at these locations:

LaserGunL0-A (a.k.a. L0-1)L0-B (a.k.a. L0-2)L0 Transverse cavityL1-SL1-XL2 – using 2 klystrons to control avg phase/ampl of L2L3 Transverse cavityL3 - here is a bit different (lots of klystrons!)

For LCLS, the general RF stability requirements are: 0.1 deg phase and 0.1% amplitude in L0 and L1 for S band.

Principal motivator

Placing the digitizers next to the low noise RF components eliminates transmission of low noise analog signals outside the chassis.

Thermocouple system

VME Crate

L0-AL0-B

L1-SL1-XT Cav

Beam-based longitudinal

fast feedback gigabit

ethernet

Eth recvr

I and Q Modulator

1 trigger for 4

channels of 1k

samples

Solution 2: Multiple VME crates with in-house modules

F Refe

k dist

ribute

d to a

ll 30 s

s in t

Controller with

ethernet

Controller with

ethernet

Local trigger

Possibly combined into one module

Slow adjustments to allow rotation of the

reference phase

Other waveformsFast, but not 119

MHz. 59.5 MHz ok

feedback VME crate

L2: in sector 24, there are 3 stations to adjust in order to accurately control phase and amplitude for long, beam-based fast feedback

Sector 25 T Cav (new 4/2005)

RF Phase and Amplitude correction at 120 Hz for:laser, gun, L0-A, L0-B, L1-S, L1-X, T cav, L2 and S25 Tcav

10' accelerator

IQ Modulator: a phase shifter

and an attenuator

1 kW 1 kW

100 mW

Solid State Sub Booster

Klystron

SLED cavity

HPRF240 MW

All except laser RF

100 mW

119 MHz Laser

Oscillator

GunNB: For the gun, SLED

cavity is shorted out

119 MHz120 Hz

photodiode

1 trigger to travel up to ½ sector

DAC (slow)Updates

@120 Hz;stable 20 µs before

beam arrives and able to distinguish beamcodes

feedback VME crate

RF Phase and Amplitude correction at 120 Hz for: L3

Beam-based longitudinal fast feedback gigabit

ethernet

istrib

klystron

10' accelerator

1 kW 60 MW

HPRF240 MW

SLED cavity

klystron

10' accelerator

1 kW60 MW

HPRF240 MW

SLED cavity

Sub Booster Klystron Sub Booster Klystron

100 mW 100 mW

1 kW1 kW

8 copies8 copies

Eth recvr

feedback.

PADPADPAD

RF Dist’n

PADPAD

PACL0-Tcav

PADPAD

PACL0-A

PADPAD

PAD PAC

Beam Phase Monitor

SPACSPACSPAC

SPACPAD

Fast PACs: Slow PACs (SPACs): PADs: VME crates:

Eth recvr

PACL24-2

PACL24-3

PACTcav L24-8

PADPAD

RF phase and amplitude correction and global feedback at 120 Hz for LCLS LINAC

PADPADPAD

feedback.

Indicates may be needed

128212

S20 S24 Total

The maybe is included in counts below

Indicates located in RF HutOtherwise at Klystron

FPGADAC

1 trigger

Coldfire CPU

running RTEMS

and EPICS

Coldfire CPU

running RTEMS

and EPICS

Example of generic LLRF control system instance

VMETemperature monitors Temperature monitors

1 trigger

IQ Modulator gives phase

and amplitude control

2 channels of 1k

samples

4 channels of 1k

samples

Phase/Amplitude Detector -> VME

Coldfire

LLRF PAD (Phase and amplitude detector)

ADCChannel

LLRF VME(Handles phase and amplitude

conversions, feedback calculations and setting of PAC)

mvme6100

LCLSCA network

MCC workstation running lclshome GUI

Change PAD channels’ size and offsets

Set PDES, ADES and control RF feedback

Use secondary ethernet interface to send 4 channels of PAD data via UDP broadcast via private

switch. Time required: 100 nsec

- Receive I and Q averages- Convert to 4 channels of Phase and amplitude- Compute weighted averages- Trigger BSA - Do feedback calculation- Send I and Q adjustment to PAC

EVR timing trigger

Update GUIs with actual phases amplitudes, feedback

corrections, etc, every 2 seconds

Update GUIs with PAD

waveforms, temperature,

I and Q actuals every

2 seconds

Private gigabit switch

OS, BSP and EPICS versions

PAD: rtems4.9.1m68k uC5282epics-R3.14.10

VME: rtems4.9.1powerpc beatnik (mvme5500/mvme6100)epics-R3.14.8.2

PAC: rtems4.9.1 m68k uC5282epics-R3.14.10

Measuring Throughput: PAD->VME->PAC

Throughput steps: PAD->VME->PAC

Event (#

matches

diagram)

Absolute time after trigger (μsec)

Description

- 0.0 Trigger delivered to phase/amplitude detector digitizer (PAD)

1 7.2 ISR: signals “data ready” to wake up DAQ task

6 1963.2 PAD starts stream of processed values to VME over private net

7 2349.6 PAD streaming completed. VME parses, does feedback, sends.

8 2508.0 Phase/amplitude controller (PAC) receives new setpts from VME

9 2524.0 PAC writes to FPGA which calcs new WF to send next trigger

10 2529.2 Data is ready

Conclusions

At 120 Hz operation, time budget=8.333 ms

LLRF PAD->VME->PAC throughput measured=2.529 ms for 4 channels of 40 points each, with no offsets,

adjust: subtract 2.5 μsec per pair of IRQ raise/lower calls (8 pairs = 20 μsec)

adjust: one socket sends to multiple PACs; add switching time

Acknowledgements

Thanks as always to Ron Akre and Klystron Department for setting up hardware, scopes and signal generators

Thanks to SLAC NAL Controls Group

Additional Information

Details of the PAD->VME transferDetails of the VME->PAC transferRF stability measurementPADPAD Block diagramLCLS LLRF website: http://www.slac.stanford.edu/grp/lcls/controls/global/subsystems/llrf

Details of the PAD->VME transfer

http://www.slac.stanford.edu/grp/lcls/controls/global/sw/epics/epics%20team%20meetings/presentations/lanIpBasic.pdfRaw ethernet packets with IP and UDP headers. Similar to BSD sockets.Solution is for low end CPU on small LAN.VME initializes, starts and stops PAD streamingWhen PAD is streaming, device support for waveform on VME parses out the values and uses them in the feedback calculations of new setpoints.

Details of the VME->PAC transfer

On VME, a subroutine record that has calculated new setpoints calls a driver routine that sends the values to the PAC via udp socket

PAC is has thread waiting to receive packet

When packet arrives, it parses out the setpoints and puts them into mem mapped FPGA

LCLS Jitter Specification for 2 Seconds is 0.14% Amplitude and 0.14 degree Phase

Short Term RF Jitter Specification for L0B are well Exceeded.This is as good as it gets – Don’t tell Physicists or they will expect it. Ron Akre 2007

About the PAD

The digitizer used is the Linear Technologies LTC2208. It was the first 16 bit digitizer chip on the market capable of running at 119MHz, it is specified to run up to 130MHz.At SLAC NAL, PAD digitizer used for RF, beam position monitors, beam charge monitors and bunch length monitors.Pohang Light Source is also using PAD for new RF system.

Phase/amplitude detector (PAD) Block

5VDCLM340Reg

4 X 16 bit ADC102MHz ClockLTC2208Transformer Coupled Inputs

Chan. 2

Chan. 1 FIFO 64k wordsWCLK

16bit DATA

CHAN 1RF INPUTRP N

CHAN 2RF INPUTRP N

ZX60-4016E

LORFIF

RF Board25.5MHz IF

Control Board

Control

FIFO 64k words

16bit DATA

12VDCLM340Reg5 ohm

FRONT PANEL LEDs

10 ohm

1 A F B

110VAC

1 1 0 V A CP o w e r M o d uleC o rc o m6 E H L 1 S C GRN

BRNBLU

YELBLK

REDVIO

TE62071-ND 2x9V 1.94A

5VDCLM340Reg

12VDCLM340Reg

5VDCLM340Reg

GRNRED

BRNBLU

YELBLK

REDVIO

TE62054-ND 2x18V 0.417A

CHAN 4RF INPUTRP N

CHAN 3INPUTRP BNC

Chan. 4

Chan. 3

QSPI5VDC0.8A x 2 Analog

5VDC0.5A DigitalLO OUTPUT

2830.5MHzFP N

24Bit Analog InputBoard

20 pin ribbonTRIG MonFP BNC

ANALOG IN ANALOG IN

CLOCK Mon102MHzFP N

TRIG In120HzRP BNC

CLOCK IN102MHzRP N

ZN4PD1-50

CS/CLK

16 bitDATA

CONTROL /Arcturus uC5282Microcontroller Modulewith 10/100 Ethernet

LO INPUT2830.5MHzRP N

-2dBm 10dBm

10dBm-8dBm

3x 75mA

FILTER 25.5MHz BP

5VD 5VA2 5VA1 12V2 12V1

Ron Akre

lcls llrf distributed control system

Documents

tuner and llrf

lcls-ii linac llrf control system – l1, bc1 zheqiao geng...

lcls-ii controls machine protection system ·...

llrf cavity simulation for spl

llrf phase reference system the lcls linac is broken down...

llrf workshop 2013 towards the next generation llrf...

mark reichanadter lcls fac meetingreich@slac.stanford.edu...

a llrf hardware testbench for lcls-ii€¦ · llrf...

lcls-i/lcls-ii timing system low level

lcls timing

lcls lcls-ii survey & alignment

llrf ilc gde meeting feb.6,2007 shin michizono llrf -...

lcls-ii project overview - stanford university ·...

llrf for chopper

lcls llrf system

design & implementation of the llrf system for lcls-ii

lcls-ii linac llrf control system – l2, l3 zheqiao geng...

llrf design report - university college...

llrf world wide

ess llrf system 2015-04-23 anders j johansson. llrf at ess...