high speed link design - cern
TRANSCRIPT
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 1
High speed link design
J.-P. Cachemiche, F. Hachon, F. Rethore
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 2
Signal integrity (and designer's life) evolution
1990'
2010'
randomjitter
group speedpropagation mode
conversionvia stub
odd modedielectric
losses attenuationskin effect
even modenoise
returncurrent
path
reflectionscoper
roughness
sinusoidaljitter fiber weave
near endcrosstalk
far endcrosstalk
inter symbolinterferences
impedancematching
deterministicjitter
referenceplane
discontinuities
viainductance
via resonance
groundbounce
Crosstalk
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 3
Methodology questions
Can we still design at high speed with thumb rules ?
Or do we need to use simulators ?
Many tools and simulators available now
− 2D : Cadence Allegro SI, MentorGraphics Hyperlynx, ...− 3D : Ansoft HFSS, MentorGraphics Hyperlynx Full Wave solver, ...
Many modelisation techniques
− IBIS,− Spice,− IBIS-AMI,− S-Parameter models
All work fine (on paper !)
Do we need to build specific test mock-ups and make measurements ?
Is there a best compromise between speed, low cost and secure design ?
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 4
Outline
Quick tour of SI issues
Simulations
Measurements
Example design
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 5
Main signal integrity issuesin high speed designs
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 6
What degrades your signals
Eye diagramme = qualitative approach to the quality of your signals
Important to understand why an eye closes
Vertical closure : − Distorsions due to reflections, crosstalk, …− Attenuation due to losses, …
Horizontal closure:− Same + noise, thermal effects,
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 7
Distorsion by reflections
Main causes
Impedance mismatchNon ideal signal path:
− Stubs− Connectors− Non continuous return path− And at highest frequencies :
via resonance !
No via
Via back drilled
Thru via
Design File: Line_with_vias_near_end_back_drilledHyperLynx LineSim V8.0
U3
TX_diff2
1
TL1
56.7 ohms59.024 ps1.000 cmCoupled StackupTL2
56.7 ohms59.024 ps1.000 cmCoupled Stackup
TL5
60.5 ohms553.355 ps8.000 cmCoupled Stackup
TL6
60.5 ohms553.355 ps8.000 cmCoupled Stackup
R5
100.0 ohms
TL7
56.7 ohms59.024 ps1.000 cmCoupled StackupTL8
56.7 ohms59.024 ps1.000 cmCoupled Stackup
V2TOP
Inne...
V3TOP
Inne...
U4
RX_dif f2
1
Thru hole via
Back drilled via
Current
Current
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 8
Distorsions by mode conversion
Mode conversion for differential signaling
Asymmetries in differential lines are converted to common modeas a peak traveling with the signal.
In principle a differential receiver is not sensitive to common signals however ...− Loss of energy in differential mode,− Differential signal terminated by a resistor, but common signal sees an open
and reflects back signal,− EMI generation
V
t
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 9
Attenuation
Main causes− Skin effect losses− Dielectric losses
Given by:
dB=4,34RacZ c
Gleakage Z c
Varies with f Varies with f
Source GigaTest Labs
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 10
Consequence of attenuation
Inter Symbol Interference (ISI)
Higher frequencies more attenuated than low frequencies/
Rise time increases
When rise time is no more negligiblevs period, high or low level cannot bereached
Signal 480 Mbits/s same pattern 4800 Gbits/s
D = 0cm, Rt=40 ps D = 40cm, Rt=167 ps
Attenuation + jitter
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 11
Jitter
Random
jitterRandom
jitterDeterministic
jitterDeterministic
jitter
Sinusoidaljitter
Sinusoidaljitter
Data dependentjitter
Data dependentjitter
Duty cycledistorsion
Duty cycledistorsion
IntersymbolinterferenceIIntersymbolinterferenceI
Boundeduncorelated
jitter
Boundeduncorelated
jitter
Totaljitter
Totaljitter
Gaussian Bounded
Source: flickerThermal noise
Source: Rise andFalltime mismatch
Source: Losses
Source: Crosstalk
Source: Ground bouncePower supply variation
RX TXTXCH
Average value Average value
Crossing 0 Crossing 1
Bit interval
Histogram 0 Histogram 0
Samplingtime
Probability of error
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 12
Importance of decoupling
Careful decoupling needed at high speed
Spreading out a set of capacitorsall around your chip is not enough
Inductance of the path between a capacitorand the chip can dominate the capacitive effect
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 13
Mitigation of attenuation
Pre-emphasis or equalization compensate high frequencies attenuation
Modern devices embed complex filters
Ex: 3 taps FIR in Stratix IX GX
One tap pre-emphasis principle Equalization principle
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 14
Mitigation of attenuation requires tools
Optimization of emission and reception stages
Many parameters. Example: Stratix IV GX from Altera
− VOD :
8 values− Pre-emphasis parameters :
3 taps FIR filter 16 x 32 x 16 possible values− Equalization parameters: 16 values− Matching resistor : 4 values out, 4 values in
131072 combinations for pre-emphasis/equalization !
16 Millions for total
Simulation mandatory !Optimization tolls as well.
Emission side
Reception side
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 15
SimulationThe Power of SimulationPractical product design is about balancing three different requirements: 1) acceptable performance, 2) developed on a tight schedule, 3) at the lowest cost.It’s easy meeting any two of these requirements; it is a challenge meeting all three at the same time.
Eric Bogatin
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 16
Pros and cons of simulation
Pros
+ Allows understand problems and their relative importance
+ Simulation allows to explore rapidly several design hypothesis at no cost
+ Allows to explore the domain of correct functioning :Valuable help for defining constraints for automatic routing
+ Avoid design errors
+ Allows to design at the right cost
+ Sometimes the only solution to optimize efficiently
Cons
- Not everything can be simulated
- Tools far from perfect : many bugs
- Quite some time need to be invested to thrust simulation results
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 17
Comparison between simulation and measurements
Differences difficult to explain
Exemple : 1.6 Gbits/s design
Simulation Measurement
Unexplained distortion
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 18
New simulation
Increased length track to simulatepath in the case
Measurement
Hypothesis : path to the die notpresent in the encrypted model
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 19
Simulation trends (1)
Components models
IBIS models were widely used because they made impossible reverse-engineering.But not adapted to high speed links termination:
− no possibility to integrate equalization or pre-emphasis features
Encrypted Spice models have apeared but simulations takes days
IBIS-AMI (IBIS Algorithmic Modeling Interface) is a modeling standard for SerDes PHYs that enables fast, accurate, statistically significant simulation of multi-gigabit serial links
− Simulator agnostic− Modeling of Equalization− “Executable” blackbox (DLL)− 10 million bit simulations run in 10 minutes or less
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 20
Simulation trends (2)
Modeling paths
As frequency increases, problems get close to those of RF designers.RF designers tools appear more and more in simulators
In particular S-Parameters
Scatered-parameters are a frequencyrepresentation of time domain signals
As they have been progressively integratedin all simulation tools, they become a handyway to export or import designs betweenheterogeneous tools.
Their analysis gives precious information on:− which kind of perturbation you have, − the importance of it,− the frequency at which they occur
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 21
S-parameters in a nutshell
A single ended lossy line can be represented as a two ports network:
Output waves are a linear function of input waves at each port:
b1 = a
1 s
11 + a
2 s
12
b2 = a
1 s
21 + a
2 s
22
Where:
− s11
is the port-1 reflection coefficient: s11
= b1/a
1 when a
2 = 0
− s22
is the port-2 reflection coefficient: s22
= b2/a
2 when a
1 = 0
− s21
is the forward transmission coefficient: s21
= b2/a
1 when a
2 = 0
− s12
is the reverse transmission coefficient: s12
= b1/a
2 when a
1 = 0
wherea
1 is the wave into port 1
b1
is the wave out of port 1a
2 is the wave into port 2
b2
is the wave out of port 2
Port 1 Port 2
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 22
S-Parameters often used
Return loss (S11
)fraction of sine wave that reflects back from an interconnect when the sourcedriving it and the far end termination are each 50 ohms
Must be small ! (negative dB)
Insertion loss (S21
)fraction of sine wave that is transmittedthrough an interconnect, also when it isdriven by and terminated in 50 ohms
Must be close to 1 (0 dB)
S21
S11
S-parameters example
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 23
Differential S-parameters
Principe is the same for differential lines: 4 ports and 16 parameters
With the following numbering convention
Parameters are:
SingleEnded
SingleEnded
1
3
2
4
Differential
1
Differential
2
Source Agilent
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 24
Most used parameters
SDD21
and SDD11
: insertion and return losses for differential signals
SCD21
is a measure of the transmitted mode conversion. Indicates how much of a differential signal, launched into the channel, is converted into a common signal by the end of the channel.
Gives information on any asymmetries between the two lines.
SCD11
is a measure of the common signal that reflects back to the source when a differential signal is incident to the channel.
Converted back to time domain, gives information about the location of the asymmetry.
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 25
Datamining S-parameters
Analyzing parameters is a powerful means of understanding relative importance of perturbations at the transmission frequency
No via
Via back drilled
Thru via
Comparison of S21 parameter for via resonance
S11 parameter shows than more energy is reflectedby a via for high frequencies
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 26
Limits to simulation
Things are difficult to simulate accurately : Ex. Surface roughness
Irregularities of copper added to skin effect increase the actual pathAt 2.5 Gbits/s, skin depth below 2 µm !
T
Source Nanya CCL Ltd
0,00E+00 5,00E+02 1,00E+03 1,50E+03 2,00E+03 2,50E+03 3,00E+031,00
10,00
100,00
1000,00
10000,00
Skin effect
Epaisseur peau en µm
Frequence en MhzE
pais
seur
pea
u en
µm
18 µm
5 µm
Not easy to get data from the manufacturersSimulators use imperfect estimations
5 µmcurre
nt
copper
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 27
Measurements
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 28
Additional reasons for measuring
Models issues
Models are not always well adapted to your needs, in particular connector models:− Wrong size or model
Or they simply do not exist.
Real implementation of your board differs from expected
Track width (irregular etching).
Copper and dielectric thickness.
Copper roughness.
Malfunctioning
There is a problem, you don't know where it is.
It is essential to compare simulations with measurements
Trust the models or trust the measurements.
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 29
Modelling from measurement
S-parameters can be obtained from 2 types of devices
Time Domain Reflectometer (TDR)
Operate in time domain.Allow to see the line profile and have a raw idea of importance of defaults as well as their location.
Vector Network Analyzer (VNA)
Operate in frequency domainBetter dynamic range than TDR (~110 dB)Not significant in signal integrity issueswhere range is limited to 40-50 dB.40 dB ~ less than 1%
Both device give similar resultsfor S-parameters
Source Lecroy
Source Agilent
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 30
Method 1
Build a separate set-up
Implement different vias and track topologies,same connectors, same dielectric andstack-up as your future design
Needs SMA connectors for signal launchand on receiving end
Advantages
Can be done in parallel with actual design: allows to explore
SMAs allow easy connection with TDR or VNA
Drawbacks
Perturbation due to SMA connectors and tracks leading to the part to be measured.
Comparison with simulations on such setup requires a high speed pattern generator
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 31
Method 2
Build an additional PCB
Mount only connectors on a second PCBwith no components.
Advantages
Exact realistic paths.
All the paths are there: allows to investigate in case of problem.
Allows a fine tuned pre-emphasis and equalization parameters optimization.
Drawbacks
Available late.
Connection quite difficult.
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 32
De-embedding
S-parameters matrix can be converted into Transmission-parameters and vice versa
T-parameters are cascadable
If T1 is the transfer function for the launching probe, and T3 the transfer function of the receiving probe, they can be mathematically removed. Only measurement of the DUT remains. This is called de-embedding
b1
b2=S 11 S12
S 21 S 22∗a1
a2 a1
b1=T 11 T 12
T 21 T 22∗a2
b2S-parameters T-parameters
Fixture DUT Fixture
T1 T2 T3
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 33
De-embedding in practice
What you need to de-embed
− The cables− The probes− The onboard track connection to the portion
to be measured
Methods
Cables : calibration procedure (SOLT: Short Open Load Thru)
Probes if any : it is possible to calibrate or you can measure it !
Track connection:− Model the track, get the S-parameters then de-embed− Gating: you remove the signal up to the DUT part
All these methods are available with TDRs and VNAs
Ex. : Software PLTS from AgilentGating
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 34
Simulation with S-parameters
Few issues
Very often simulation does not converge because of insufficient quality of S-parameters.Most often due to bad elimination of test fixtures
Not easy to detect because models are quite different from physical device measured.2 main issues:
− Passivity : the system must not provide energy− Causality
Some tools propose to detect and correct the modelsNot recommended because means model is wrongBetter to find the cause
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 35
Example designGoing from 8 to 10 Gbits/s
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 36
System architecture
SNAP12
SNAP12
SFP+
Stratix IVFPGA
Stratix IVFPGA
Crossbar144 x 144
max6.5 Gbits/s Cyclone III
FPGA
1 Gigabit Ethernet transceiver
Acquisition board (AMC)
NAT MCH (Tongues 1 and 2)
Supervision
NIOS
Standard backplane(dual star layout)
Stratix IVFPGA
Stratix IVFPGA
Quad transceiver
Output board (AMC)
NIOS
10 boards
6
6
2 to MCH1
1mezzanine board
mezzanine board
Switch board (Tongues 3 and 4)
GigabitEthernetSwitch
CPU
Ethernet link
SMAClockdistribution
Tongue 1
Tongue 2
Tongues 3 & 4
External clock
2 to MCH2
2 to MCH12 to MCH2
2
2
2 from MCH12 from MCH2
2 from MCH12 from MCH2
PHY
6
6
Tracks to simulate
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 37
AMC board and mezzanine
AMC board
Mezzanine
Mezzanine mounted
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 38
Theorical approach
Mezzanine link
Checking the simulation− Eye diagram measurements at 8 Gbits/s to get a reference− Check of coherence between measures and simulations
Feasibility− Simulation at 10 Gbits/s− Optimization of pre-emphasis and equalization filters
Improvement of models− TDR measurements to obtain model of full data path− Extraction of S-parameters− Simulation at 10 Gbits/s− New optimization of filters
Backplane link
No model of backplane nor connectors !− TDR measurements to obtain model of full data path− Extraction of S-parameters− Simulation at 6.5 Gbits/s− Check of coherence between measures and simulations− Optimization of filters
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 39
Comparison between simulations and measurements at 8 Gbits/s
One way simulation between FPGA and optical device
Measured opening at 10-16 : 0.78 UISimulated opening at 10-16 : 0.67 UI
Simulation more pessimistic but approximate connector model
Design File: GT_TX_RXtstHyperLynx LineSim V8.1
U1
stratix4_gtstratix4_gt_tx_N
2
1
R2
100.0 ohms
J15
TX_SERIAL_TO_...
Port1 Port2
Port3 Port4
J10
sivgx_pkg_tx.s4p
Port1
Port2
Port3
Port4
J16
ELDO_seam035_s...SEAx10mm_conn
SEA... SEA...
SEA... SEA...
J17
TD6_c.s4p
Port1 Port2
Port3 Port4
Stratix IV GX track trackconnector
SimulationMeasurement
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 40
Simulation and preemphasisat 10 Gbits/s
Without preemplasisSimulated opening at 10-16 : 0.31 UI
With preemphasisSimulated opening at 10-16 : 0.63 UI(was 0.67 UI at 8 Gbits/s without error)
Preemphasis cancels the eye closure
No pre-emphasis
3 taps pre-emphasis
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 41
S-parameters extraction
Positioners
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 42
Connection issues
Card sawed to get accessto bottom side of mezzanine
Bad contactdifficult to see
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 43
Final setup
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 44
Issues met
Whole measurement chain has been tested
But quality of model insufficientSlight shift between measuring channels
Apparently a problem of calibration
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 45
Backplane Model
Nodel required to optimize pre-emphasisand equalization
Need another method to makemeasurements on the backplane
Build small size AMC cards with SMAConnectors
Backplane link at 4.8 Gbits/s
CERN 15 february 2011 High speed design seminar IN2P3/CPPM 46
Conclusion
No universal recipe for high speed design, but rather a mix of several tools and methods
Thumb rules can be used … but not sufficient
Simulation allows to explore several hypothesis and specially to weight the relative importance of each source of perturbation but …
− You can simulate everything but not at a reasonable cost nor in a reasonable time− It is not accurate until you have crosschecked it with real measurements
Measurements can give powerful models you don't find from the manufacturer and understand the source of dysfunction.
Not an easy road
Simulations and measurements are difficult
All this has a high cost− Steep learning curve− Expensive measurement devices− Time
The main challenge is to find the best compromise between safe design, tight schedule and final cost: very difficult exercise !.