twepp 2016 topical workshop on electronics for particle ... · the bram content was downloaded...

| TWEPP 2016 | KARLSRUHE |

Kintex-7 Irradiation, test bench and results

HARDWARE Vlad-Mihai PLĂCINTĂ

Lucian-Nicolae COJOCARIU Ovidiu-Emanuel HUȚANU

FIRMWARE Cristian-Andy TĂNASE

Lukas ARNOLD Stephen WOTTON

PHYSICS Florin MACIUC

Mihai STRATICIUC

TWEPP 2016 Topical Workshop on Electronics for Particle Physics


OUTLINE

Introduction KINTEX-7 FPGA

TEST BENCH STRUCTURE POWER SUPPLY BOARD KINTEX-7 TEST BOARD DAQ SYSTEM LABVIEW GUI

Irradiation facilities UC Louvain ion irradiation Legnaro NL ion irradiation Paul Scherrer Institute proton irradiation Conclusions and future plans

1

TWEPP 2016 - Topical Workshop on Electronics for Particle Physics


Introduction

2

RICH photodetectors to be redesigned to work at 40 MHz trigger rate;

Digital Board FPGA -- SRAM based, KINTEX-7 family; Device considered for the Digital Board: XC7K70T-

FBG676; Device Under Test: XC7K70T-FBG484C6. (Details)

LHCb detector https://lhcb-public.web.cern.ch/lhcb-public/

Elementary cell (EC-R)

(4 x 64 read-out channels)


The LHCb Collaboration, LHCb Particle Identification Upgrade Technical Design Report, CERN/LHCC, November 2013, available at: link.

https://lhcb-public.web.cern.ch/lhcb-public/





https://cds.cern.ch/record/1624074


Introduction radiation environment

3

Region TID [krad]

Neutrons: 1 MeV neq [cm−2]

Hadrons: >20 MeV [cm−2]

RICH1 200(30) 3 x 𝟏𝟎𝟏𝟐(1.5 x 𝟏𝟎𝟏𝟐) 1.2 x 𝟏𝟎𝟏𝟐 (0.5 x 𝟏𝟎𝟏𝟐)

RICH2 80(22) 1.5 x 𝟏𝟎𝟏𝟐(0.8 x 𝟏𝟎𝟏𝟐) 0.5 x 𝟏𝟎𝟏𝟐(2.8 x 𝟏𝟎𝟏𝟏)

FLUKA simulation --> Total Ionizing Dose (TID) and neutron equivalent for 50 1/fb; A safety factor used on worse and best case “scenarios”; FPGA exposed to a maximum of 200 krads (2 kGy) over Upgrade phase.



KINTEX-7 FPGA

4

SRAM FPGA family Flip Chip design using 28 nm High-K Metal Gate technology; KINTEX-7 best price to performance ratio; In order to allow the heavy ions to penetrate the dice to the bottom active layer:

• The FPGA is lidless;

• the wafer was thinned from 250 to about 60 µm (typical O18 ion penetration depths of 100 µm for available beams at the laboratories from the list).

DUTs were thinned(details in backup).



KINTEX-7 FPGA ~Test bench architecture~

5



KINTEX-7 FPGA ~Power supply board~

6

Top view of the power supply board

Power supply board is designed around ADP5050 integrated circuit from Analog Devices(datasheet);

80x80 𝐦𝐦𝟐-- 4 layers.

Board Output voltages

1V~4.4A

1V~1.2A

1.5V~4.4A

1.2V~0.2A

1.8V~1.2A

3.3V~1.5A

ADP5050 features

Wide input voltage range

Overheat protection

Overcurrent protection

High efficiency

Soft start/enable functions

Small, 48-lead 7 x 7 mm


http://www.analog.com/media/en/technical-documentation/data-sheets/ADP5050.PDF


KINTEX-7 FPGA ~KINTEX-7 TEST BOARD~

7

Top view of the FPGA board http://www.xilinx.com/support/documentation/data_sheets/ds180_7Series_Overview.pdf

FPGA working in JTAG mode w/o external memory --> reconfigure at each power up/cycle;

In our tests we use 2 clocking sources: A single ended 50 MHz clock, for the SEM IP core; External clock which ranges from 500 KHz up to

12 MHz. 80x80 𝐦𝐦𝟐-- 8 layers. We included in all firmware versions the SEM IP core

from Xilinx as a mitigation technique for CRAM. The SEM IP core was configured to work in the repair and enhanced repair modes;


https://www.xilinx.com/products/intellectual-property/sem.html







KINTEX-7 FPGA ~DAQ SYSTEM~

8

A custom DAQ system monitors most important 4 voltage rails of the Kintex-7 FPGA. The LabVIEW GUI allow us to:

Read and plot the parameters; Save the parameters in to ASCII files for later analyses; Perform power cycle for the entire FPGA or only on VCC_AUX rail.

All the parameters are sampled every 50 ms.

DaQ system schematic



KINTEX-7 FPGA ~LABVIEW GUI~

9

Allow us to reconfigure FPGA and also to do a blind scrubbing procedure; We can see the SEM IP core response; We can control the functionality of the DUT board, by controlling its logic frequency, and some other features; We can read the BRAM content and also we compared it with a golden copy; All the data is saved in ASCII files for later analyses.


| TWEPP 2016 | KARLSRUHE | 10


Irradiation facilities

Heavy ions: 𝐎𝟏𝟖 at 126 MeV (LET=2.85 𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠) and 𝐅𝟏𝟗 at 118 MeV (LET=3.67

𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠) at SIRAD facility served by a 14 MV TANDEM accelerator from Legnaro National Laboratories, in Italy; (July 2015)

Heavy Ion Facility at Cyclotron Resource Center at Louvain-la-Neuve, Universite Catholique de Louvain Belgium (UCL), the following ions were used (June 2016): o 𝐂𝟏𝟑 at 131 MeV (LET=1.3 𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠); o 𝐍𝐞𝟐𝟐 at 238 MeV (LET=3.3 𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠); o 𝐀𝐫𝟒𝟎 at 379 MeV (LET=10 𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠); (different inclination angles

wrt the beam: 0, 30 and 50 degrees)

o 𝐍𝐢𝟓𝟖 at 582 MeV (LET=20.4 𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠); o 𝐊𝐫𝟖𝟑 at 769 MeV (LET=32.4 𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠);

Protons: Paul Scherrer Institute (at PIF) using 200 MeV protons (LET of 0.0036

𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠); (August 2016)


UC Louvain ion irradiation

11

After the Legnaro irradiation we redesigned totally the firmware architecture, which now allowed us to test specific resources from FPGA like: BRAM; CRAM; Flip-Flops; SLR32.

Two error mitigation techniques were introduced in the new firmware versions: TMR for logic mitigation and SEM IP core provided by Xilinx for CRAM mitigation;

Implemented blind scrubbing procedure to allow full device reconfigure when triggered by SEU rate (user defined limit).


VACUUM VESSEL WHERE THE DUT WAS PLACED


In most cases we tested each resource as a individual firmware, but we tested also some firmware with more than one resource at one time;

We implemented a fault injector module for offline tests, in order to simulate the behavior of the firmware under radiation conditions.





With 𝐍𝐞𝟐𝟐 beam (LET=3.3 𝐌𝐞𝐕 ∗ 𝐜𝐦𝟐/𝐦𝐠): results - no Latch-ups, significant SEU rate though SEU in essential bits low;

However, we saw 100mA jumps (micro-Latch-ups) in the 1.8 V rail when we used 𝐀𝐫𝟒𝟎 beam with the DUT placed at 50 degrees wrt to the beam. (LET=15.57

𝐌𝐞𝐕 ∗ 𝐜𝐦𝟐/𝐦𝐠 ); In fact, the threshold for the SEL is 15.57 𝐌𝐞𝐕 ∗ 𝐜𝐦𝟐/𝐦𝐠 , which is consistent with

the literature.



UC Louvain ion irradiation - 𝐊𝐫𝟖𝟑

When we used 𝐊𝐫𝟖𝟑 beam at a LET of 32.4 𝐌𝐞𝐕 ∗ 𝐜𝐦𝟐/𝐦𝐠 beside SEL, we saw 2 SEFI-like events.

Left image: core (and BRAM) supply current(red) and voltage(blue) during irradiation; Right image: 1.8 V rail (includes VCC_AUX and also some I/O bank) current(red) and

voltage(blue) during irradiation.



UC Louvain ion irradiation Cross-sections for SEU

Single-bit SEU cross-section (preliminary values), corrected, classified and reported by SEM IP core;

Total number of CRAM is 24 Mbits, hence if needed cross-section/bit just divide.



UC Louvain ion irradiation Cross-sections for SEL


Data from 𝐀𝐫𝟒𝟎 are missing, because there are some differences in the electrical parameters from 2 samples, and we are still look in to that.


UC Louvain ion irradiation - 𝐂𝟏𝟑

BRAM logic was tested using 𝐂𝟏𝟑 beam with LET=1.3 𝐌𝐞𝐕 ∗ 𝒄𝒎𝟐/𝐦𝐠 ; A golden copy of BRAM content was compared with the content downloaded after 38 s of

irradiation in beam; 141 flipped bits out of 4.86 Mbits x 55% occupancy were highlighted; The BRAM content was downloaded using a UART communication implemented in

Kintex-7; The configuration file for BRAM tests had included also SEM IP core for CRAM mitigation.



Cross-section of logic failure at various LET values for 𝐎𝟏𝟖 and 𝐅𝟏𝟗 beams:

Lack of scrubbing and inefficient mitigation technique led to more than 4 orders of magnitude difference between our values comparing to literature similar measurements.

Legnaro NL ion irradiation



Paul Scherrer Institute proton irradiation

We tested mostly the same resources from Louvain, but this time we included a first test of the I/O Banks;

For these tests we had about 15 hours of beam time; Three samples were tested for 0.5 Mrad of total Ionization dose each.


METALIC SUPPORT WHERE THE DUT WAS PLACED




During one irradiation step we saw some spikes (some of them in large values) in the current consumption for core and BRAM, which after reconfiguring the FPGA they disappeared. Definitely not SEL but due to CRAM failure.;

The same effects we saw in our offline tests in the laboratory, when we injected some errors in the CRAM using SEM IP tool.



We implemented 4 ring oscillator for each 5 from all 6 I/O Bank of the FPGA; The oscillation frequency of each ring oscillator, ranges from 1MHz to 11 MHz, and

depend on the number of I/O pins used and the voltage which supplies the I/O bank. (The ring oscillator is a voltage controlled oscillator)

1 2 3 4 n

OUT (to oscilloscope) IN

Fatima Zahra Tazi et al., “On Delay Faults Affecting I/O Blocks of an SRAM-Based FPGA Due to Ionizing Radiation”, IEEE Transactions on Nuclear Science 61(6) 3138-3145 (2014), DOI:10.1109/TNS.2014.2369417.



Paul Scherrer Institute proton irradiation ~preliminary conclusions~

We didn’t see any SEL; Large SEU rates still an issue even for proton low LET value because 200 MeV

protons are displacing silicon ions from the lattice plus spallation; Erratic behavior of some I/O banks -- currently under study; The 0.5 Mrad dose is 2.5 higher that the highest TID expected in the LHCb RICH

sub-detectors; We saw some modifications in current consumption for the VCCINT VCCBRAM rail

and VCC_AUX rail, but after a day of annealing at room temperature (around 30-35 degrees) they disappeared.



Conclusion and future plans

23

Advance stage of investigating device behavior in radiation environment - especially CRAM induced behavior;

Preliminary results are confirmed by existing literature; Study of I/O behavior delayed by device decontamination after PSI beam test; Data collected during (ion/proton) beam tests is being analyzed; Preliminary results after proton irradiation do not rule out this DUT;

back-up solution: antifuse FPGA to be tested; Plans to test GTx transceivers -- need to procure adequate device and design

dedicated board and test bench; Fully assembled LHCb RICH digital board prototype to be tested at CHARM facility in

CERN; Two more irradiations with ions and protons are approved for the next year.




Parameters Highest from ARTIX-7 XC7A200T

Lowest from KINTEX-7 XC7K70T

Logic Cells 215,360 65,600

DSP Slices 740 240

Memory 13,140 4,860

Transceivers 16 GTP 8 GTX

I/O Pins 500 300

https://www.xilinx.com/products/silicon-devices/fpga/artix-7.html#productTable

https://www.xilinx.com/products/silicon-devices/fpga/kintex-7.html https://www.xilinx.com/products/technology/high-speed-serial.html

KINTEX-7 vs ATRIX-7

Max performance [Gb/s] Peak Bandwidth [Gb/s] (combined TX & RX)

GTP 6.6 211

GTX 12.5 800







https://www.xilinx.com/products/silicon-devices/fpga/kintex-7.html






https://www.xilinx.com/products/technology/high-speed-serial.html









Literature results XC7K325T


David S. Lee, Michael Wirthlin, Gary Swift, and Anthony C. Le, "Single-Event Characterization of the 28 nm Xilinx Kintex-7 Field-Programmable Gate Array under Heavy Ion Irradiation“, IEEE Radiation Effects Data Workshop (REDW) (2014), DOI:10.1109/REDW.2014.7004595.

http://ieeexplore.ieee.org/document/7004595/









KINTEX-7 family


https://www.xilinx.com/products/silicon-devices/fpga/kintex-7.html#productTable







twepp 2016 topical workshop on electronics for particle ... · the bram content was downloaded...

Documents