ethernet over sonet - internet2 · ethernet vs sonet feature ethernet ... – lan-phy or wan-phy at...
TRANSCRIPT
Ethernet over SONET
Dr. John S. GrahamUniversity of London Computer
Centre
Ethernet vs SONET
Feature Ethernet SONETGuaranteed Delivery
No (Packet Switched)
Yes (Circuit Switched)
Guaranteed BW No Yes
Performance Monitoring CRC Count BER
Protection None (STP?) APS
Fault Location None AIS
Access Costs Low High
SONET/SDH Model
RRMUX DXC MUX
Path Layer Connection
Line LayerLine Layer
Section Layer Section Layer Section LayerSection Layer
SONET Network Elements
SONET STS-1 Frame
Section
Line
PTRSPE
3
5
86 columns
Path
30 59 9041
Payload Capacity = 756 Bytes = 48.384 Mb/s
Section Overhead Bytes
Data ComD3
Data ComD2
Data ComD1
UserF1
OrderwireE1
BIP-8B1
STS-1 IDC1
FramingA2
FramingA1
Section
Line Overhead Bytes
Line
OrderwireE2
Line FEBEZ2
Sync StatZ1
Data ComD12
Data ComD11
Data ComD10
Data ComD9
Data ComD8
Data ComD7
Data ComD6
Data ComD5
Data ComD4
APSK2
APSK1
BIP-8B2
Path Overhead Bytes
Path
Multi-frameH4
User ChannelF2
Path StatusG1
Signal LabelC2
BIP-8B3
Path TraceJ1
Accurate Bit-Level Timing• Line Coding
– AMI, AMI RZ– HDB3
• Block Coding– 4B/5B (FDDI)– 5B/6B (E3)– 8B/10B (Gigabit Ethernet)
• Scrambling– Frame Synchronous– Self Synchronous
8B/10B Encoding (1/2)• Ensures equal number of 1’s and 0’s for
– Clock recovery– DC Balance
• Detection of single bit transmission errors• Predefined ‘comma’ (0011111 and
1100000) characters
8B/10B Encoding (2/2)• Data (D) and commands (K) encoded separately• Octet split into 5 MSB and 3 LSB• Code groups generated from lookup tables
DecodedA B C D E F G H0 1 0 0 0 0 0 0
2 0
Encoded
0010101101jHGFiEDCBA
D2.0
Control Words (‘Ordered Sets’)Symbol Encoding Purpose
F K28.5 D21.5 LINK_NOT_AVAILABLEC K28.5 D10.5 config_reg LINK_CONFIGURATION
I1 K28.5 D5.6 Idle
I2 K28.5 D16.2 Idle
S K27.7 SoF
T K29.7 EoF
R K23.7 CARRIER_EXTEND
H K30.7 ERROR
Scrambling in SONET/SDH
SDH STM-1 uses x7 + x6 +1 polynomial
ConcatenationSTS-3 155 Mb/s
145.152 Mb/s
STS-3c/VC-4 155 Mb/s149.76 Mb/s
STS-12 622 Mb/s580.608 Mb/s
VC-4-4 622 Mb/s599 Mb/s
STS-12c 622 Mb/s599.04 Mb/s
Provisioning ExampleA
D
B
E C
F3 - 5
7
9
10 -12
1
???
1 – 12 (622 Mb/s)
Bandwidth Fragmentation
STS-3c
STS-12
Blocked!
Drop Interface (Node F)
Ring
Virtual Concatenation
STS-3c
STS-12
STS-1-3v
Drop Interface (Node F)
Ring
The Magic of VCAT
Protocol Client Rate
Contiguous Concatenation
Virtual Concatenation
Fast Ethernet 100 STS-3c 65% STS-1-2v 98%
ESCON 160 STS-12c 26% STS-1-4v 78%
Fibre Channel 850 STS-48c 34% STS-3c-6v 92%
Gigabit Ethernet 1000 STS-48c 40% STS-3c-7v 92%
Fibre Channel 2 1700 STS-48c 66% STS-3c-12v 92%
Differential Delay
TX Ty
Source
STS-1-12v
SinkCore Network(Partial Mesh)
Multiframe IndicatorBit 1 Bit 2 Bit 3 Bit 4 Bit 5 Bit 6 Bit 7 Bit 8
MFI2 MSBs (bits 1 - 4) 0 0 0 0MFI2 LSBs (bits 5 – 8) 0 0 0 1
CTRL 0 0 1 0GID 0 0 1 1
0 1 0 00 1 0 1
CRC-8 0 1 1 0CRC-8 0 1 1 1MST 1 0 0 0MST 1 0 0 1
1 0 1 01 0 1 11 1 0 01 1 0 1
SQ MSBs (bits 1 – 4) 1 1 1 0SQ LSBs (bits 5 – 8) 1 1 1 1
VCAT ExampleGID-a, SQ#=01GID-a, SQ#=12GID-a, SQ#=23
654
1211
GID-b, SQ#=310GID-b, SQ#=29GID-b, SQ#=18GID-a, SQ#=37
STS-1-4v STS-1-3v
STS-12
VCAT Puzzle• An STS-1-2v between Chicago and
London• One STS-1 sent via geostationary satellite• The other STS-1 sent via transatlantic
fibre• Will it work? ;-)
VCAT: Buffer RequirementsVC Path Type
Transport Signal
Number Paths Buffer Size
STS-1VC-3
STS-12STM-4 12 18 MB
STS-1VC-3
STS-48STM-16 48 71 MB
STS-3cVC-4
STS-12STM-4 4 18 MB
STS-3cVC-4
STS-48STM-16 12 55 MB
Link Capacity Adjustment Scheme• Modify membership of a VCG• On-demand and hitless bandwidth
changes• Automatic removal of failed VCG members• Interworking of LCAS-enabled VCG with
non-LCAS VCG• Old-style ‘bridge and roll’ requires double
bandwidth
LCAS Messaging• Member Status (MST)• Re-Sequence Acknowledge (RS-Ack)• Control (CTRL)
– Fixed– Add– Norm– EOS– Idle– DNU
• Group ID• CRC
LCAS Signalling Flows (1/3)
CTRL = ADD
SQ = 255
Source PTE STS-1 #1
CTRL = EOS
SQ = 2
CTRL = NORM
SQ = 1
MST = OK (#3) CTRL = NORM
SQ = 2
CTRL = NORM
SQ = 1
STS-1 #2 STS-1 #3
CTRL = NORM
SQ = 1
CTRL = EOS
SQ = 2
CTRL = EOS
SQ = 3
CTRL = IDLE
SQ = 255CTRL = NORM
SQ = 1
CTRL = EOS
SQ = 2
RS-ACKCTRL = NORM
SQ = 2
CTRL = NORM
SQ = 1
CTRL = EOS
SQ = 3
MST = FAIL (#3)
LCAS Signalling Flows (2/3)
CTRL = EOS
SQ = 3
Source PTE STS-1 #1
CTRL = NORM
SQ = 2
CTRL = IDLE
SQ = 255
CTRL = EOS
SQ = 3
CTRL = NORM
SQ = 2
CTRL = NORM
SQ = 1
MST = FAIL (#1)CTRL = EOS
SQ = 2
CTRL = NORM
SQ = 1
STS-1 #2 STS-1 #3
RS-ACKCTRL = EOS
SQ = 2
CTRL = NORM
SQ = 1
LCAS Signalling Flows (3/3)
CTRL = IDLE
SQ = 255
Source PTE STS-1 #1
CTRL = NORM
SQ = 2
CTRL = EOS
SQ = 3
CTRL = NORM
SQ = 2
CTRL = NORM
SQ = 1
MST = FAIL (#3)CTRL = EOS
SQ = 2
CTRL = NORM
SQ = 1
STS-1 #2 STS-1 #3
RS-ACKCTRL = EOS
SQ = 2
CTRL = NORM
SQ = 1
CTRL = NORM
SQ = 1
Gigabit Ethernet Implementations
Gigabit Ethernet (802.3z)
FC-0 Interface & Media
FC-1 Encode/Decode
FC-2 Signalling
FC-3 Common Services
FC-4 Upper Layer Mapping
IEEE 802.3 PHY
IEEE 802.3 CSMA/CD
IEEE 802.2 LLC
ANSI X3T11 Fibre Channel IEEE 802.3 Ethernet IEEE 802.3z Gigabit Ethernet
MDL
IEE 802.3 MAC
IEEE 802.2 LLC
PMD
PMA
PCS
1000BASE-LX/SX
Sequence of EventsLINK_NOT_AVAILABLE LINK_NOT_AVAILABLE~ LINK_CONFIGURATION
~ LINK_CONFIGURATION IDLE ~ IDLE SOP PREAMBLE
SFD DA LLC DataSA LENGTH
PADDING FCS
Generic Framing Procedure• ITU-T G.7041 and ANSI T1.105.02• Defines mapping for many types of service
onto SONET/SDH or OTN:– Ethernet, IP/PPP– GbE, Fibre Channel (inc. DVB), FICON etc
• Excellent bandwidth utilization; efficiency tailored to suit different client types
• Simple delineation and robust error control• Extensible
Motivation for GFP (1/2)• ATM
– Cell overhead causes 10% bandwidth inflation– Adaptation functions needlessly complex
• Packet over SONET (POS)– Requires all frames to be converted to PPP
over HDLC– Byte stuffing causes non-deterministic
bandwidth inflation– QoS hard to monitor or guarantee
Motivation for GFP (2/2)• Minimal overhead• Transport of client PDUs in native format• Designed for optimized processing• Easy aggregation of frames from multiple
client and multiple protocols into shared bandwidth channels
• Low latency capabilities for SAN
GFP: Types of Frame
GFP
Client Frames Control Frames
Data Frames Management Frames Idle Frames OA&M
Frames
GFP: Functional Model
Source = IEEE Communications Magazine, May 2002
GFP: Frame Structure
Source = IEEE Communications Magazine, May 2002
Payload TypesValue of UPI Field Protocol GFP Mode0x01 Ethernet F0x02 PPP (IP & MPLS) F
TTTT
F
0x03 Fibre Channel0x04 FICON0x05 ESCON0x06 GbE0x07 reserved0x08 MAPOS0x09 – 0xFE reserved0x00 & 0xFF unavailable
GFP: Client-Independent Processes
• Synchronization– Bit Level– Frame Delineation
• Scrambling• Multiplexing
– GFP Frame– Client PDU
GFP: Synchronization
Hunt Presync
SynccHEC M
ismatch 2n
d cHEC
Mat
ch
cHEC Match
No 2nd cHEC Match
cHEC Match
No CHEC Match
GFP: Frame Delineation
PayloadPLI cHEC PLI cHEC Payload
Payload PLI cHEC PayloadPLI cHEC PayloadPLI cHEC
PayloadPLI cHEC
GFP: Client PDU Multiplexing
Source Sink
STS-3c-2vGFP Framer
1
2
3
GFP Framer
1
2
31
2
3
GFP-F Encapsulation (1/2)
Preamble
SFD
Destination
Source
Length
Data
FCS
PLI (MSB)
PLI (LSB)
cHEC (MSB
cHEC (LSB)
Payload Header
Payload
FCS1 8
Bit Transmission Order
Transmission O
rder
7
1
6
6
2
4 4
X = 4 to 64
0 to 65,535 - X
Ethernet MAC Frame
GFP-F Encapsulation (2/2)
Flag (0x7E)
Address
Control
PPP Type
Data
FCS
PLI (MSB)
PLI (LSB)
cHEC (MSB
cHEC (LSB)
Payload Header
Payload
FCS1 8
Bit Transmission Order
Transmission O
rder
4
X = 4 to 64
0 to 65,535 - X
PPP/HDLC Frame
1
1
2
4
1
GFP-T: 64B/65B Payload
D1 K1 D2 D3 K2D4 D5 D6
000 001 010 011 101 110 111F
65B Sequence 1 001 C1 D2 D3 D4 D5 D61 0 101 C2 D1
100Octet Number
64B Sequence
000 001 010 011 100 101 110 111Octet Number
GFP-T Error Detection• Leading flag bit is errored• Error affects ‘Last control-code’ indicator• Control-code location address received in
error• Error causes 4-bit control code to be
modified
GFP-T: Superblocks
1,1 1,2 … 1,7 1,8
8,1 8,2 8,7… 8,8
CRC (MSB)CRC (LSB)
1,11,2
…
1,71,8
8,18,2
…
8,78,8
536-bit Byte-Aligned
GFP-T: Bandwidth Requirements
Client Signal
Client Signal
Bandwidth
Minimum Transport Channel
Size
Nominal Transport Channel
Bandwidth
Number Superblocks
per GFP Frame
ESCON 160 Mb/sSTS-1-4vVC-3-4v
193.536 Mb/s 1
Fibre Channel 850 Mb/s STS-3c-4v
VC-4-6v 898.56 Mb/s 13
Gigabit Ethernet 1000 Mb/s STS-3c-7v
VC-4-7v 1.04832 Gb/s 95
Pros and Cons: GFP-F
Higher bandwidth efficiency
Higher LatencyMore buffer memory requiredCore header fields must be calculated
Pros and Cons: GFP-TSupports many protocolsLow latencyIngress core header fields need not be calculated
Less bandwidth efficientMore logic required
10 G Ethernet – 802.3ae
10GBASE-R 10GBASE-W
S = 850 nm MM 300 m
L = 1310 nm SM 10 km
E = 1550 nm SM 40 km
Full Duplex 802.3 MAC
64B/66B PCS
Serial PMASerial PMA
WIS
E L SE L S
XGMII
PMDPMD
10 GbE Overview• Usual 802.3 MAC and frame format
– Jumbo frames not included in standard• Full duplex only
– No shared media– No CSMA/CD
• Only optical fibre– No copper interface– LAN-PHY or WAN-PHY at various reaches
The STS-192c WIS FrameJ1B3C2G1F2H4Z3Z4Z5
Fixed Stuff
Synchronous Payload Envelope (SPE) = 16704 Columns
63 columns
9.58464 Gbs-1 Capacity
802.3 FrameIdle Idle
802.3 FrameIdle Idle
…
…
Section & Line OverheadA1B1D1
A1 … A1 A2 … A2
Section
A2E1D2
J0F1D3
Z0 … Z0
Overhead …H1B2D4D7D10
S1
H1B2
Z1
……
…
H1B2
Z1
H2K1D5D8D11
Z2
…
…
H2
Z2
H2
M1
H3K2D6D9D12
E2
…H3 H3
Line Overhead …
1 2 192 193 194 384 385 386 576
Defined by WAN PHY as Fixed Value
Supported by WAN PHY Unused by WAN PHYOptional for SONET/SDH
Unused by WAN PHYUndefined for SONET/SDH
NetherLight Connectivity
UKLight – NetherLight Today
MetroDirector
UKLight
15454
NetherLight
HDXc
NetherLight
15454
CERN
CoreDirector
UKLight
MetroDirector
UKLight
UKLight – NetherLight Tomorrow?
HDXc
NetherLight
CoreDirector
UKLight
MetroDirector
UKLight
15454
CERN
Requires that MSPPsare GFP-F enabled
15454
NetherLight
Peering Using 10 G WAN-PHY
Force10
StarLight
CoreDirector
UKLight10GBASE-SW
CoreDirector
UKLightS
TM-64
10GBASE-SW Switch
UKLight
London
Chicago