ssd readout upgrade

M.J. LeVine 1STAR HFT meeting, Sept 27-28, 2011

STAR

SSD readout upgrade

M. LeVine, R. Scheetz -- BNL Ch. Renard, S. Bouvier -- Subatech

J. Thomas -- LBNL

M.J. LeVine 2STAR HFT meeting, Sept 27-28, 2011

STAR

DAQ PC

DDL

DAQ roomOuter support

coneSouth platform

VME crate

RDO (1 of 8)

SlaveFPGA

SlaveFPGA

SlaveFPGA

SlaveFPGA

SlaveFPGA

Master FPGA

DA

Q in

terfa

ceTR

G in

terfa

ce

VME FPGA

Fiber links

Ladder cards

VM

E in

terfa

ce

Readout components

M.J. LeVine 3STAR HFT meeting, Sept 27-28, 2011

STAR Prototype ladder board (inner side)

M.J. LeVine 4STAR HFT meeting, Sept 27-28, 2011

STAR Prototype ladder card (outer side)

Debug connector

Optical transceiver

Flex cable connectors to ladder modules

M.J. LeVine 5STAR HFT meeting, Sept 27-28, 2011

STAR Interposer

FPGA layout error• Choice of redoing the PCB• or designing an interposer

between the PCB and FPGA• Interposer design, fabrication,

and assembly: 6 months• Present status: installed and

operational

M.J. LeVine 6STAR HFT meeting, Sept 27-28, 2011

STAR Debug cards (Subatech)

• USB port• JTAG (configuration)• JTAG (slow controls)

M.J. LeVine 7STAR HFT meeting, Sept 27-28, 2011

STAR Fake static source

M.J. LeVine 8STAR HFT meeting, Sept 27-28, 2011

STAR Analog level shifting

M.J. LeVine 9STAR HFT meeting, Sept 27-28, 2011

STAR Ladder card commissioning

• Using debug card– JTAG FPGA configuration– JTAG slow controls – USB simple protocol

• Using fake static source– Map analog response

• ADCs and level shifting circuitry– Verify packing of ADC data working correctly

M.J. LeVine 10STAR HFT meeting, Sept 27-28, 2011

STAR Mapping analog response

• Software– Python script driving– Multiple .exe (C code)

• Time to map response for 1 ADC: 30 sec• Time to map all 16 ADCs: 20 minutes

– Disconnect/connect flex cable• Basis for future slow controls software• SC uses JTAG header on debug card

– Will be replaced by fiber protocol

M.J. LeVine 11STAR HFT meeting, Sept 27-28, 2011

STAR Verification of packing code• USB output for ADC data• Install USB spy at output of FIFO

module

module

X16

5 MHz

5 MHz

adc 12bit16 bit serial output

adc 12bit16 bit serial output

80 MHz

80 MHz

register

16

50 MHz

FIFO

2

JTAG

TDO

s

20

40 MHz

serializer to fiber

16 bit width4 words temp

Write enable:true on 10 clocks only

USB output USB output

M.J. LeVine 12STAR HFT meeting, Sept 27-28, 2011

STAR

-1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.40

128

256

384

512

640

768

ADCs vs. calculated

adc0 adc1 adc2 adc3 adc4 adc5 adc6 adc7 adc8adc9 adc10 adc11 adc12 adc13 adc14 adc15 calc

Analog response for all ADCs

M.J. LeVine 13STAR HFT meeting, Sept 27-28, 2011

STAR Analog response

• Non-linear behavior of N-face needs to be understood

• Discovered we are sensitive to PS fluctuations via DAC– Will be separately regulated in production

version

M.J. LeVine 14STAR HFT meeting, Sept 27-28, 2011

STAR RDO proposed layout

VME driver

Opt

ical

xc

vr

Opt

ical

xc

vr

Opt

ical

xc

vr

Opt

ical

xc

vr

Opt

ical

xc

vr

241 124 241 124241 124 241 124 241 124

Slave FPGA484 FBGA

EPCS4

VME FPGAVME

EEPROM

VME P2 connector (TRG: rows A,C) VME P1 connector

VME driver VME driver VME driver

X X

PS

header

Slave FPGA484 FBGAEPCS4

Slave FPGA484 FBGAEPCS4

Slave FPGA484 FBGA

EPCS4

Slave FPGA484 FBGA

EPCS4

GB

ICKeep top layer clear for LVDS serial lanes (master to/from each slave)

SlaveEEPROM

AddressSW 1

AddressSW 2

Master FPGAMasterEEPROM

Reb

oot P

B

Res

et P

B

SIU

40m

m x

16

0mm

SIU connectors

Opt

ical

xc

vr

M.J. LeVine 15STAR HFT meeting, Sept 27-28, 2011

STAR QRDO layout

Opt

ica

l xcv

r241 124

Slave FPGA484 FBGA

EPCS4

VME P1 connector

X X

PS

header

SlaveEEPROM

Reb

oot P

B

Res

et

PB

DC regulator1.2V2.5V3.3V

+5V, GND only

USBconn

FT24

5R

US

B

9.1 MHz oscillator

Test header (32 signals)

+5V external pwr iinput

EPCS16

LEM

O c

onne

ctor

M.J. LeVine 16STAR HFT meeting, Sept 27-28, 2011

STAR QRDO assembled

M.J. LeVine 17STAR HFT meeting, Sept 27-28, 2011

STAR QRDO commissioning

• Orsay USB protocol implemented– Message layer on top of byte pipe– Goal: replace VME (4-byte messages)

• Problems –– Bad synthesis by Synopsys tool !!– Now resolved

• Message protocol working

M.J. LeVine 18STAR HFT meeting, Sept 27-28, 2011

STAR USB message protocol

4-byte write

4-byte read

6-bit subaddress used for register addressing

M.J. LeVine 19STAR HFT meeting, Sept 27-28, 2011

STAR Integration of ladder card/QRDO

• Generate test patterns on ladder card• Spy on incoming data via fiber with logic

analyzer

M.J. LeVine 20STAR HFT meeting, Sept 27-28, 2011

STAR ADC data received in QRDO

DREADY: ADC/STATUSWORD1: Synch

Test pattern as sent (almost)

Birds-eye view

Zoom in on data phase

M.J. LeVine 21STAR HFT meeting, Sept 27-28, 2011

STAR Status words received in QRDO

Word   Value Comment (from Table 52, master document)  0 07000 configured, OK, serdes clock used  1 04000 deserializer lock OK  2 00000 (no optical transceiver problems)  3 05000 usb present, debug present  4 00000  5 00000 6 01000   7 00000 ladder 0 (not yet assigned by QRDO)

serial #1 (agrees with hardware assignment on board)

M.J. LeVine 22STAR HFT meeting, Sept 27-28, 2011

STAR Master FPGA development (CR)

• Original plan for communication– 5 High speed serial lanes

• 5 slaves master– As implemented by Altera

• Requires more PLLs than available

• CR implemented his own serial receivers– Share common PLL

• Validation of new scheme in simulation– In progress– See next slides

M.J. LeVine 23STAR HFT meeting, Sept 27-28, 2011

STAR Simulation of ADC acquisition (CR)

M.J. LeVine 24STAR HFT meeting, Sept 27-28, 2011

STARSimulation of status words in RDO (CR)

M.J. LeVine 25STAR HFT meeting, Sept 27-28, 2011

STAR RDO roadmap

• Slave FPGA – code completed• VME FPGA – code completed• Master (TRG/DAQ) FPGA

– In late stages of simulation @ Subatech• PCB layout expected to start 12/11• Assemble prototypes 3/12

M.J. LeVine 26STAR HFT meeting, Sept 27-28, 2011

STAR Status: documentation

M.J. LeVine 27STAR HFT meeting, Sept 27-28, 2011

STAR Summary of present status• Ladder card

– 3 prototypes functional– Analog behavior characterized– Data packing verified– Beginning to look at exported data

• QRDO– Message protocol working in USB– Fiber functional– Still a work in progress

M.J. LeVine 28STAR HFT meeting, Sept 27-28, 2011

STAR Status (cont’d)

• RDO– Layout will begin after

• master FPGA simulation is complete• and QRDO debugging completed

• DAQ PC– Delivered– 2 DRORCs installed

M.J. LeVine 29STAR HFT meeting, Sept 27-28, 2011

STAR Comments

• We were delayed 6 months due to ladder card layout error and subsequent fix with interposer.

• We have had working ladder cards only since June, 2011

• Amazing progress since then