washington washington university in st louis gigabit ethernet interface for the msr fred kuhns...

WashingtonWASHINGTON UNIVERSITY IN ST LOUIS

[email protected]

Gigabit Ethernet Interface for the MSR

Fred KuhnsApplied Research Laboratory

Washington UniversitySt. Louis Mo.

2WashingtonWASHINGTON UNIVERSITY IN ST LOUIS

Fred Kuhns - 05/14/23

Overview• Two Example Ethernet Scenarios• Relevant Ethernet and ARP specifications

– General standards– ATM AAL5 and Ethernet Frame Formats– Ethernet Addressing– The Address Resolution Protocol

• Routing in the MSR• GigE Interface protocol processing requirements

– packet processing– ARP



Ethernet Forwarding Scenario 1

EthernetSwitch

Host

IP: 192.163.204.2MAC: 08:00:20:7C:E3:25

Host

IP: 192.163.204.3MAC: 08:00:20:7C:F2:45

RouterPort 0:IP: 192.163.204.4MAC: 00:01:03:7C:23:03Port 1:IP: 192.163.150.1MAC: 00:01:03:7C:56:34

EthernetSwitch

Port 1:IP: 192.163.204.2MAC: 00:00:5E:04:00:01

MSR P1

HostIP: 192.163.150.2MAC: 00:40:33:A3:4C:04

P0P1

Host

IP: 192.163.150.3MAC: 08:00:20:54:6C:4A

P3

Use the Address Resolution Protocol to Map 192.168.204.2

to 08:00:20:7C:E3:25. Encapsulation datagram in Ethernet frame and send.

Destination Addr:192.168.204.2

IP hdrdata

Packet arrives with destination host on local

network. Output port must map destination IP address

to MAC address.



Ethernet Forwarding Scenario 2

EthernetSwitch

Host

IP: 192.163.204.2MAC: 08:00:20:7C:E3:25

Host

IP: 192.163.204.3MAC: 08:00:20:7C:F2:45

RouterPort 0:IP: 192.163.204.4MAC: 00:01:03:7C:23:03Port 1:IP: 192.163.150.1MAC: 00:01:03:7C:56:34

EthernetSwitch

Port 1:IP: 192.163.204.2MAC: 00:00:5E:04:00:01

MSR P1

HostIP: 192.163.150.2MAC: 00:40:33:A3:4C:04

P0P1

Host

IP: 192.163.150.3MAC: 08:00:20:54:6C:4A

P3 Forwards to final destination host

Destination Addr:192.168.150.2

IP hdrdata

Packet arrives with destination host NOT on locally attached network. Output port must send to

the next hop router.

Next hop router IP address must be used in the ARP

request: Map 192.168.204.4 to 00:01:03:7C:23:03.

Encapsulate datagram in Ethernet frame and send.



What is required?• To simplify the overall system design, Ethernet

specific processing confined to Link Interface• Interface must:

– Bridge between ATM and Ethernet networks– Map IP addresses to corresponding Ethernet Addresses:

• send ARP requests and maintain an ARP cache– Respond to ARP requests from other hosts

• send ARP replies in response to requests

• We are not supporting– IEEE 802.1P/Q: VLANS, priorities etc.



Related Specifications• RFC 1122 - Requirements for Internet Hosts

– Must send and receive using RFC-894 - compliant– Should receive RFC-1042 mixed with RFC-894 - we do not– May send using RFC-1042 - we do not– Must use ARP - compliant– Must flush out-of-date ARP cache entries - not compliant– Must prevent ARP floods - we only try once– Should have configurable ARP cache timeout - no– Should save at least one (latest) unresolved (by ARP) packet - no– Must report broadcasts to IP layer - compliant– IP layer Must pass TOS to link layer - via the header– Must Not report no ARP entry as “destination unreachable” -

compliant



Related Specifications - continued• RFC-826 : Address Resolution Protocol

– Maps <protocol, address> to Ethernet address– Minor differences in suggested algorithm

• RFC 1700 : Assigned Numbers - Now an online database– Managed by the Internet Assigned Numbers Authority (IANA)– Ethertype values and IP to ethernet multicast address mapping

• RFC-1812 : Requirements for IPv4 Routers– Must not believe ARP reply if contains multicast or broadcast

address - not compliant– Must be compliant with RFC 1122 - Partial

• Support Ethernet V2 only– RFC 894: IP encapsulation in Ethernet V2 - Supported– RFC 1042: IP encapsulation in 802.3 frames - Not Supported



IP over ATM (rfc 791 and 2684)IP

Hea

der

AAL5

Tra

iler

IP D

atagram

Fragment offsetVersionH-length TOS Total length

Identification flagsTTL protocol Header checksum

Source AddressDestination Address

Options ??

IP data (transport header and transport data)

AAL5 padding (0 - 40 bytes)

CPCS-UU (0) CPCS-UU (0) Length (IP packet + LLC/SNAP)CRC



Ethernet Frame Format

Transport Header

Fragment offsetVersionH-length TOS Total length

Identification FlagsTTL Protocol IP Header checksum

IP Source AddressIP Destination Address

Destination Address cont.Destination (6 B)

Source Address cont.Source Address - (6 B)

Ether Type (2 B)

IPH

eade

rE

ther

net

Hea

der

IPD

atag

ram



IP Encapsulation in Ethernet Frames

FCS (4)Data (46-1500)type0800

src address (6)dst address (6)

len(2)

src address (6)dst address (6) FCS (4)Data (38 - 1492)

DSAPAA

SSAPAA

ctl03

Org Code00

type0800

802.2 LLC 802.2 SNAP

802.2 LLC/SNAP

• Ethernet frame size: 64 - 1518 Bytes• if type 1500, then IEEE frame, otherwise Ethernet V2.Ethernet Encapsulation, RFC 894 - Support

IEEE 803.2/802.2 encapsulation, RFC 1042 - Do Not Support

0 len 1500

Pad(0-46)

Pad(0-46)



Ethernet Assigned Numbers• RFC1700 obsoleted by online database at IANA:

– http://www.iana.org/assignments/ethernet-numbers• Ethernet Address - 6 octets:

– 3 high-order octets = Organizationally Unique Identifier (OUI)

– 3 low-order octets = the interface number• Multicast bit = lsb of the MSB (xxxx xxx1)

– first byte odd => multicast or broadcast– first byte even => unicast address– multicast address = ((OUI | 0x0100) << 24) & Group_ID

• Ethernet Broadcast: FF:FF:FF:FF:FF:FF



IP and Ethernet Multicast• IANA has allocated address block with OUI = 00:00:5E

– Used for unicast addresses for ”IETF standard track protocols “

– Half of Multicast addresses reserved for IP, remaining for “special use”. Leaves 23 bits for multicast addresses:

• 01:00:5E:00:00:00 to 01:00:5E:7F:FF:FF– Could use this block for our interface, see ethernet numbers

• IP Multicast– Class D address, 0xE0000000 + 28 Bit Group ID– 224.0.0.0 to 239.255.255.255 (0xE0000000 - 0xEFFFFFFF)

• IP to Ethernet Mapping– RFC1112 - Host Extensions for IP Multicasting – Non-unique mapping: 28 bit IP group to 23 bit Ethernet group

• 32 IP multicast groups per mapped ethernet multicast address.



Multicast: IP to Ethernet Mappings• Network Byte Ordering, Internet Standard Bit order:

(Big-Endian)

0000 0001 0000 0000 0101 1110 0xxx xxxx xxxx xxxx xxxx xxxx47240

Multicast Bit Internet BitMSB LSB

lsbmsb 1110 xxxx xxxx xxxx xxxx xxxx xxxx xxxx

Class D (Multicast)Not Used in IP-to-Ethernet Mapping

Block of Ethernet Multicast Address0 8

LSB

23 bits



IP Broadcast• No Direct Impact on GigE Interface• IP Broadcast : default, we will not forward directed

broadcasts.– limited versus:

• {-1, -1}. Must not be forwarded, Destination address only– Directed broadcast:

• {Network-Number, -1}, destination address only.– Subnet Directed Broadcast:

• {Network-Number, Subnet-Number, -1}– Directed Broadcast to all subnets:

• {Network-Number, -1, -1}



Unicast - We can use the IANA Block

0000 0000 0000 0000 0101 1110 0000 0100 xxxx xxxx xxxx xxxx47230

Multicast Bit set to 0

MSB LSB

IANA Block of Ethernet Addresses 16 bitsARL Interface Number

WUARL MAC: 00:00:5E:04:XX:XX



ARP FrameDestination Address (6B)Source Address (6B)Ether Type (2B)Hardware Address Space (2B) Protocol Address Space (2B) Byte length of Hardware address = 6 (1B)Byte length of Protocol address = 4 (1B)

Hardware Address of Sender (6 B)Protocol Address of Sender (4 B)Hardware Address of Destination (6 B)Protocol Address of Destination (4 B)

Operation Code 1|2(2B)



ARP Message Formats

ARP Requesttype806

src address<eth-A>

dst address ff:ff:ff:ff:ff:ff

FCSxx

has1

pas0800

hl6

pl4

op01

sha<eth-A>

spa<ip-A>

tha<??>

tpa<ip-B>

type806

src address<eth-B>

dst address <eth-A>

FCSxx

has1

pas800

hl6

pl4

op02

sha<eth-B>

spa<ip-B>

tha<eth-A>

tpa<ip-A>

ARP Reply

Host B Eth<eth-B>

Reply (02)

Request (01)

Host A Eth<eth-B>

Host A IP<ip-A>

Host B IP<ip-A>

Ethernet Header (14 B)

pad

pad

ARP Message (28 Bytes for Request or Reply)

Ethernet Data - Pad with zeros to 46 BytesFCS(4B)

Ethernet Frame with ARP Request/Reply - 64 Bytes

18 Byte Pad



SPC

shimupdate

shimdemuxWUGS

...40 ... 47

(out port +40)

...40 ... 47

(in port + 40)

Ingress Egress

From previous

hop routeror endsystem

add shim

rem shimFIPL

shimproc.

FPX FPX

SPC

shimdemux

shimupdate

Outbound VC = SPI + 500 <= SPI<= 3

Link Interface L

ink

Inte

rfac

e

IP eval: IP processing for FPX.

1. Broadcast and Multicast destination address

2. IP options3. Packet not recognized

Current VCI Support:1) 8 Ports (PN)2) 4 sub-ports (SP)FIPL

IPproc

plugins

GiGE interface will use allfour Sub-Port identifier

(i.e.four VCI values)

GiGE Interface willonly send on one VCIvalue (currently = 50)

FPX_VCI FPX_VCI

Packet Routing, SPC and FPX

FIPL

IPproc

plugins



Routing in the MSR• Route tables must map a given destination address to output

port and sub-port identifier (i.e. Virtual Interface Number or VIN).– Route table entry: {prefix/length, Output_VIN}

• 192.168.204.0/24, 41 (Port 1, Subport 1)• Output_VIN = {Port # (10 bits), Sub-port # (6 bits)}.

• At input port, packet is sent to the indicated output port:– VCI = 40 + Port number

• At output port, the sub-port is mapped to an output VCI value:– VCI = 50 + Sub-port Identifier



MSR Routing ExampleControl

Processor

Switch Fabric

. . .OutputPortProc.

Flow/RouteLookup

Dist. Q. Ctl.Dist. Q. Ctl.

FlowLookup

InputPortProc.

Flow/RouteLookup

Dist. Q. Ctl.Dist. Q. Ctl.

FlowLookup

SPC SPC

IP hdrdata

packet received atinput port

IP hdrdata

(4, 0)

remove shim, calculate VCI for subport 0:VCI = 50 + 0 = 50

IP hdrdataSend to next

hop/endsystem

route lookup returns <port = 4, subport = 0>

IP hdrdata

IP hdrdata

(4, 0)

insert shim with OutVIN, send onVCI = 40 + 4 = 44



Supporting Ethernet• We can leverage the sub-port identifier to facilitate

IP to ethernet address resolution.– if packet received on VCI = 50 (subport 0), then use the

IP destination address in the header– otherwise (subports 1-3), lookup VCI value in a table to

obtain the next hop IP address.• Once we have the IP address we must map it to the

corresponding Ethernet address.– We can then implement a simple version of ARP in the

GigE interface card.



GigE Link Interface

ARP Table(Simplified)

MACIP

IP1 MAC1

IPM MACM

......

Pkt VC = 50

Endsystem, Broadcastor Multicast address

if VC != 50,Lookup VC in

VIN tablereturns IP used for ARP lookup

Send to pkt->dstif bcast or mcast

map to eaddrelse unicast

resolve w/ARP

IP Header

data

AAL5 trailer

IP Header

data

Ethernet

Add Ethernet header using the derived destination address and our corresponding source address. Software creates

VIN table at boot time by writing to

interface.

To Next Hopor

Endstation

To a next hop routerNH #0 = Base + 1 = 51NH #1 = Base + 2 = 52NH #2 = Base + 3 = 53

VIN Table(Simplified)

Map multicast or broadcast toethernet address

If ARP table lookup fails, send ARP request to broadcast address, drop packet. No retries are made. No ARP entry aging!

FromFPX/SPC

2 NhIP2

Entry NhIP

1 NhIP1

0 NhIP0



GigE Link Interface

Base VCto FPX/SPCif ( An ARP packet)

update Mapping in ARP tableif not for us then dropif (ARP Request)

swap source and target infoset operation to Replyset ether headersend reply

else if (An IPv4 packet)remove ethernet “stuff”add AAL5 trailer/paddingsend on default Base VC

else drop packet

IP Header

data

AAL5 trailer

From

Nex

t Hop

or

End

stat

ion

IP Header

data

Ethernet

ARP

Ethernet

ARP Table(Simplified)

MACIP

IP1 MAC1

IPM MACM

......



Some Details• Packet Received on ATM interface:

– If received on VCI 50 (i.e. the base VCI) then • Map IP destination in header (ip->dst_addr) to ethernet MAC address.• Unicast uses ARP table, multicast and broadcast use appropriate mapping.

– Otherwise (VCI = {51, 52 or 53}), • lookup VC in VIN table: Table entry index = RX_VC - Base_VC - 1.

For example, packet received on VCI = 53, Index = 53 - 50 - 1 = 2• ARP the resulting Next Hop IP address.

– This permits a simple mechanism for “directing” traffic to a gateway. This allows us to support directed broadcast and provides a convenient mechanism for testing.

• Packet Received on Ethernet interface: – if IPv4 then send all (unicast, multicast and broadcast) to input

port processor on VCI 50 (i.e. the Base VCI)



ARP Cache• IP Address = Network_Prefix.Host or simply Net.Host

– Assume a prefix length of at least 24 bits, leaves 8 bits for the host– An interface can have at most 3 unique IP addresses

• Interface may communicate with at most 256 hosts per network• Implement ARP cache as a table with 768 entries (3 * 256)• See next slide

VIN TablePrefixMask

Local IPAddress

Next HopIP Address

Mask0 MyIP0 NH0

Mask1 MyIP1 NH1

Mask2 MyIP2 NH2

EntryNumber

012

EthernetIPIP0,0

......IP0,255 Ether0,255

Ether0,0

IP1,0

......IP1,255 Ether1,255

Ether1,0

IP2,0

......IP2,255 Ether2,255

Ether2,0

ARP Table

Net 0

Net 1

Net 2Net 0 = Mask0 & MyIP0

Net 1 = Mask1 & MyIP1

Net 2 = Mask2 & MyIP2



‘get next packet’:// received frame from ATM interfaceif (RX_VC == Base_VC)

ipdst = ip->dst_addr;else

ipdst = VIN_Table[RX_VC- Base_VC - 1].NextHop// ipdst == IP Address of host we must send packet to// determine network, Using the VIN tablefor (i = 0; i < 3; i++) {

// i corresponds to the Network Number (0 - 2)if ((ipdst & Maski) == (MyIPi & Maski)) {

index = (i << 8) | ((ipdst & ~Maski) & 0xffffff00)break; }

if i == 3 ; drop packet, goto get next packet// lookup in ARP tableif (ArpTable[index].EtherAddress != 00:00:00:00:00:00) {

construct ethernet frame send packet goto ‘get next packet’

} else {send ARP Request for ipdstdrop packet, goto ‘get next packet’}

Implementing the ARP TableVIN Table

EthernetIPIP0,0

......IP0,255 Ether0,255

Ether0,0

IP1,0

......IP1,255 Ether1,255

Ether1,0

IP2,0

......IP2,255 Ether2,255

Ether2,0

ARP Table

index

PrefixMask

Local IPAddress

Next HopIP Address

Mask0 MyIP0 NH0

Mask1 MyIP1 NH1

Mask2 MyIP2 NH2

EntryNumber

012

don’t need to store IP address



Notes and Issues• GigE Control Interface for Software configuration.

1. Reset interface to defaults2. Clear ARP cache3. Read ARP table4. Read VIN table5. Read ethernet address6. set VIN table entries and other registers

• Set BASE VC (currently 50)• Set Entries in the VIN table• Add static ARP entries



Hardware and Status• Software Simulation completed• Hardware implementation and status: Dave ...

washington washington university in st louis gigabit ethernet interface for the msr fred kuhns...

Documents