nexus 7000 test plan vpc v1

Customer Proof of Concept Lab

Data Center Business UnitNexus 7000 vPC Test Plan - version 1.0

Activity Start Date End Date

Customer Arrival/Testing

January 27, 2009 January 27, 2009

Name Company Position

Jeff Raymond Cisco Systems, Inc. Director, Technical Marketing

Roberto Mari Cisco Systems, Inc. Technical Marketing Engineer

Arkadiy Shapiro Cisco Systems, Inc. Tech Project SE

<SE_NAME> Cisco Systems, Inc. System Engineer

<CUSTOMER_NAME>

<CUSTOMER_NAME>

Note: Lab operation hours are 9 a.m. to 5 p.m. Monday to Friday.

Table of Contents

INTRODUCTION.........................................................................................................................................................3

EQUIPMENT - HARDWARE AND SOFTWARE...................................................................................................4

CISCO DEVICES...........................................................................................................................................................4Hardware...............................................................................................................................................................4Software & Licenses..............................................................................................................................................4

THIRD PARTY DEVICES...............................................................................................................................................4Hardware...............................................................................................................................................................4Software.................................................................................................................................................................4

DESIGN AND TOPOLOGY DIAGRAM...................................................................................................................5

TESTING.......................................................................................................................................................................6

PART I – PRELIMINARY CONFIGURATION...................................................................................................................6Test 1 – Spanning Tree Configuration...................................................................................................................6Test 2 – Enable HSRP feature...............................................................................................................................8Test 3 – Enable LACP feature.............................................................................................................................10Test 4 – Enable global UDLD.............................................................................................................................12Test 5 – Verify Layer 2 Connectivity...................................................................................................................12

PART II – BRING UP VPC ON NETWORK....................................................................................................................12Test 6 – Enable vPC feature, perform consistency and connectivity checks.......................................................12

PART III - VPC FAILURE SCENARIO AND CONVERGENCE TESTS...............................................................................22Test 7 - shutdown the vPC peer-link port-channel on one switch.......................................................................23Test 8 - no shutdown the vPC peer-link port-channel on the switch...................................................................24Test 9 – Shutdown in sequence the two vPC port-channel members on one switch............................................26Test 10 - shut/no shutdown the vPC peer-keepalive link.....................................................................................27Test 11 - Fail peer-link followed by peer-keepalive link.....................................................................................29Test 12 - Fail peer-keepalive link followed by peer-link...................................................................................30Test 13 - SSO switchover test during traffic.......................................................................................................32

CONCLUSION............................................................................................................................................................33

EXTERNAL REFERENCES.....................................................................................................................................33

INTERNAL REFERENCES......................................................................................................................................33

Acronyms......................................................................................................................................................................34

Introduction

The following presents a sample comprehensive test plan for Nexus 7000 testing. This focuses on testing environments (general configs, general streams) and procedures. Testing notes cover additional comments regarding procedure and procedure examples. For more specific information on testing environment, configurations and results, see Nexus 7000 case studies.

Note:

This document is a work in progress. It will be updated as more test procedures are validated in CPOC lab.

Equipment - Hardware and Software

Following is a list of hardware and software required for a typical Nexus 7000 testing setup.

Cisco Devices

Hardware

Quantity Device Description

AGGREGATION/CORE

2 N7K-C7010 10 Slot Chassis

6N7K-C7010-FAB-1

Nexus 7000 – 10 Slot Chassis – 46Gbps/Slot Fabric Module

6 N7K-AC-6.0KW Nexus 7000 - 6.0KW AC Power Supply Module

4 N7K-SUP1 Nexus 7000 - Supervisor

2N7K-148GT-11

Nexus 7000 - 48 Port 10/100/1000, RJ-45, 40G Fabric

4 N7K-132XP-12 Nexus 7000 - 32 Port 10GbE, 80G Fabric

16 SFP-10G-SR 10GBASE-SR-SFP+ Module

ACCESS

1 Nexus 5020 Cisco Nexus5020 Chassis ("40x10GE/Supervisor")

Software & Licenses

Software Description Version

N7KS1K9-41.3 NX-OS software for Sup 1 4.0(1) or higher

N7KS1K9-401A1.1NX-OS Base License is only required for vPC N/A

Third Party Devices

Hardware

Port Quantity Description

2 Agilent N2X / Spirent Test Center 1GigE Traffic Generator8 Agilent N2X / Spirent Test Center 10GigE Traffic Generator

Software

Software Description Version

- - -

Design and Topology Diagram

The diagram below shows the “typical” network topology that was used to validate/test vPC feature and interoperability with the Nexus 5000.

e1/1-2 e1/3-4

20

e1/9-10

mgmt

20

e1/9-10

n5k-rhs-1

10

rhs-7k-2rhs-7k-1

TestingNote: this is organized by general functional areas of Nexus 7000 testing. As well the tests are ordered by priority as seen based on some customer and account team requests.

Part I – Preliminary Configuration

Test 1 – Spanning Tree Configuration

Purpose

Spanning Tree is still needed with vPC. STP is acts as a failsafe and backup mechanism when vPC configuration is removed or misconfigured. On links which are part of the vPC the Spanning Tree state machine runs but do not take any action in changing port states. Rapid Spanning Tree implementation will be used throughout this Test Plan.

Environment and configurations

Setup layer 2 links and wire connections between the two nexus 7000 boxes and the Nexus 5000 and configure roots in your network, to achieve a deterministic behavior in the STP topology.

Procedures

Configure Layer 2 and Spanning Tree

1. Configure and unshut VLANs that will be transported from access to the aggregation. rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# vl 11rhs-7k-1(config-vlan)# no shrhs-7k-1(config-vlan)# vl 12rhs-7k-1(config-vlan)# no shrhs-7k-1(config-vlan)# vl 13rhs-7k-1(config-vlan)# no shrhs-7k-1(config-vlan)# vl 14rhs-7k-1(config-vlan)# no shrhs-7k-1(config-vlan)# exitrhs-7k-1(config)# exit

Check if the VLANs are up and running and repeat above steps for all three switches in the testbed setup (7k-1, 7k-2, 5k).

rhs-7k-1# sh vlan briefVLAN Name Status Ports---- -------------------------------- --------- -----------------------------1 default active2 VLAN0002 active3 VLAN0003 active4 VLAN0004 active11 VLAN0011 active12 VLAN0012 active13 VLAN0013 active14 VLAN0014 active

2. Configure interconnecting Trunks to carry the layer 2 VLANs defined at step 1, on access and distribution switches.

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.

rhs-7k-1(config)# interface ethernet 1/9, e 1/10rhs-7k-1(config-if-range)# switchportrhs-7k-1(config-if-range)# switchport mode trunkrhs-7k-1(config-if-range)# switchport trunk allowed vlan 11-14rhs-7k-1(config-if-range)# no shrhs-7k-1(config-if-range)# exitrhs-7k-1(config)# exit

Check if Trunks are up and running and repeat above steps for all three switches in the testbed setup (7k-1, 7k-2, 5k).

rhs-7k-1# show interface trunk vlan 11-14

--------------------------------------------------------------------------------Port Native Status Port Vlan Channel----------------------------------------------------------------------------------------------------------------------------------------------------------------Port Vlans Allowed on Trunk----------------------------------------------------------------------------------------------------------------------------------------------------------------Port Vlans Err-disabled on Trunk--------------------------------------------------------------------------------Eth1/1 noneEth1/2 noneEth1/9 noneEth1/10 none--------------------------------------------------------------------------------Port STP Forwarding--------------------------------------------------------------------------------Eth1/1 11-14Eth1/2 noneEth1/9 11-14Eth1/10 11-14

3. Configure Spanning Tree priorities in order to make one of the two aggregation switches the Primary Root of this Layer 2 domain (e.g. N7k-1), the other aggregation switch the secondary root (e.g. N7k-2) and the Nexus 5000 the third in the priority list (e.g. assigning the maximum priority). In order to achieve this use the Spanning Tree CLIs to change the bridge priority for the interested VLANs.

Primary Root: rhs-7k-1(config)# spanning-tree vlan 11-14 root primary

Secondary Root:rhs-7k-2(config)# spanning-tree vlan 11-14 root secondary

Access Switch (higher STP priority numerical value)

n5k-rhs-1(config)# spanning-tree vlan 11-14 priority ? <0-61440> bridge priority in increments of 4096

n5k-rhs-1(config)# spanning-tree vlan 11-14 priority 61440

Check for the Spanning Tree Roots through the appropriate show commands:

rhs-7k-1# show spanning-tree root Root Hello Max FwdVlan Root ID Cost Time Age Dly Root Port---------------- -------------------- ------- ----- --- --- ----------------VLAN0011 24587 0022.5579.7641 0 2 20 15 This bridge is rootVLAN0012 24588 0022.5579.7641 0 2 20 15 This bridge is rootVLAN0013 24589 0022.5579.7641 0 2 20 15 This bridge is rootVLAN0014 24590 0022.5579.7641 0 2 20 15 This bridge is root

Test 2 – Enable HSRP feature

Purpose

HSRP should be enabled on the SVIs to handle remote failures and establish a fault-tolerant default gateway.


Enable HSRP and configure it to have HSRP primary to follow the Spanning Tree primary Root. Use the HSRP priority to define the active as the STP Root. Define policy preemption to ensure that the high priority HSRP session is ready to take over every time becomes active.

Procedures

1. Configure HSRP on both the nexus 7000 aggregation switches

rhs-7k-1# sh run int vl 11version 4.1(3)

interface Vlan11 no shutdown ip address 192.168.202.2/24 hsrp 1 preempt priority 255 ip 192.168.202.1 hsrp 2


interface Vlan12 no shutdown ip address 192.168.203.2/24 hsrp 2 preempt priority 255 ip 192.168.203.1

2. Repeat this configuration on the other Nexus 7000 aggregation switch.


interface Vlan11 no shutdown ip address 192.168.202.3/24 hsrp 1 ip 192.168.202.1


interface Vlan12 no shutdown ip address 192.168.203.3/24 hsrp 2 ip 192.168.203.1

Check for the HSRP configuration final configuration and make sure the HSRP session are up and secondary HSRP can be seen.

rhs-7k-1# sh hsrp brief P indicates configured to preempt. |Interface Grp Prio P State Active addr Standby addr Group addrVlan11 1 255 P Active local 192.168.202.3 192.168.202.1

Vlan12 2 255 P Active local 192.168.203.3 192.168.203.1

rhs-7k-2# show hsrp brief P indicates configured to preempt. |Interface Grp Prio P State Active addr Standby addr Group addrVlan11 1 100 Standby 192.168.202.2 local 192.168.202.1

Vlan12 2 100 Standby 192.168.203.2 local 192.168.203.1

3. Define Layer 3 SVI interface on the Nexus 7000 for a layer 3 connectivity test.

n5k-rhs-1(config)# conf tn5k-rhs-1(config)# int vl 11^% invalid command detected at '^' marker.n5k-rhs-1(config)# feature interface-vlann5k-rhs-1(config)# int vl 11n5k-rhs-1(config-if)# 192.168.202.4/24^% invalid command detected at '^' marker.% Invalid commandn5k-rhs-1(config-if)# ip add 192.168.202.4/24n5k-rhs-1(config-if)# no shn5k-rhs-1(config-if)# int vl 12n5k-rhs-1(config-if)# ip add 192.168.203.4/24n5k-rhs-1(config-if)# no shn5k-rhs-1(config-if)# exit

4. Enable the ICMP debugs on both the aggregation switches and ping the virtual HSRP address from the access switch. You should observe the active answering to the ping packets.

n5k-rhs-1# ping 192.168.202.1PING 192.168.202.1 (192.168.202.1): 56 data bytesRequest 0 timed out64 bytes from 192.168.202.1: icmp_seq=1 ttl=254 time=2.116 ms64 bytes from 192.168.202.1: icmp_seq=2 ttl=254 time=1.318 ms64 bytes from 192.168.202.1: icmp_seq=3 ttl=254 time=1.357 ms64 bytes from 192.168.202.1: icmp_seq=4 ttl=254 time=1.421 ms

rhs-7k-1# 2009 Jan 23 18:09:35.900358 netstack: [3553] (default) Rcvd packet on Vlan12 (prty 0): s=192.168.203.4, d=192.168.203.1, proto=1 (icmp), ip_len=84, id=0b21, ttl=2542009 Jan 23 18:09:35.900480 netstack: [3553] (default) Send packet (prty 0): s=192.168.203.1, d=192.168.203.4, proto=1 (icmp), ip_len=84, id=0b21, ttl=2552009 Jan 23 18:09:36.913280 netstack: [3553] (default) Rcvd packet on Vlan12 (prty 0): s=192.168.203.4, d=192.168.203.1, proto=1 (icmp), ip_len=84, id=0b22, ttl=2542009 Jan 23 18:09:36.913403 netstack: [3553] (default) Send packet (prty 0): s=192.168.203.1, d=192.168.203.4, proto=1 (icmp), ip_len=84, id=0b22, ttl=2552009 Jan 23 18:09:37.923325 netstack: [3553] (default) Rcvd packet on Vlan12 (prty 0): s=192.168.203.4, d=192.168.203.1, proto=1 (icmp), ip_len=84, id=0b23, ttl=2542009 Jan 23 18:09:37.923446 netstack: [3553] (default) Send packet (prty 0): s=192.168.203.1, d=192.168.203.4, proto=1 (icmp), ip_len=84, id=0b23, ttl=2552009 Jan 23 18:09:38.933378 netstack: [3553] (default) Rcvd packet on Vlan12 (prty 0): s=192.168.203.4, d=192.168.203.1, proto=1 (icmp), ip_len=84, id=0b24, ttl=2542009 Jan 23 18:09:38.933502 netstack: [3553] (default) Send packet (prty 0): s=192.168.203.1, d=192.168.203.4, proto=1 (icmp), ip_len=84, id=0b24, ttl=2552009 Jan 23 18:09:39.943509 netstack: [3553] (default) Rcvd packet on Vlan12 (prty 0): s=192.168.203.4, d=192.168.203.1, proto=1 (icmp), ip_len=84, id=0b25, ttl=2542009 Jan 23 18:09:39.943629 netstack: [3553] (default) Send packet (prty 0): s=192.168.203.1, d=192.168.203.4, proto=1 (icmp), ip_len=84, id=0b25, ttl=255

Notes

Active HSRP as STP Roots are defined to follow the vPC Primary placement. This is not must but allows a better observation of the network behavior in case of different failure scenarios.

Test 3 – Enable LACP feature

Purpose

LACP is used to configure port-channels between access and distribution switches and between distribution switches.


Make sure that LACP rather than ON mode is used and use active/active mode on the port-channel peers. The Port-Channel on the access device must have a unique ID in order to be recognized and used by the vPC. The access device will have a single port-channel spanning across two Nexus 7000 devices.

Procedures

Enable the LACP feature on all three switches and define L2 Trunk port-channels as Port-channel 10 (ISL channel-group) and Port-channel 20 (downstream port-channel).

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# feature lacp

rhs-7k-2# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-2(config)# feature lacp

n5k-rhs-1# conf tn5k-rhs-1(config)# feature lacp

Define layer 2 port-channels carrying interconnecting VLANs and repeat these operations for both aggregation switches.

rhs-7k-1# sh run | be port-channel | be 10

interface port-channel10 switchport switchport mode trunk switchport trunk allowed vlan 11-14

interface port-channel20 switchport switchport mode trunkswitchport trunk allowed vlan 11-14

Bundle interfaces interconnecting the Network devices to the above predefined port-channels and repeat this operation for both aggregation switches.

rhs-7k-1: conf t int e 1/1 channel-group 10 mode active int e 1/2 channel-group 10 mode active exit

rhs-7k-1# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed

S - Switched R - Routed U - Up (port-channel)-----------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel-----------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SD) Eth NONE --

rhs-7k-2: conf t int e 1/1 channel-group 10 mode active int e 1/2 channel-group 10 mode active exit

rhs-7k-2# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SD) Eth NONE --

n5k-rhs-1:

n5k-rhs-1(config)# interface port-channel 20n5k-rhs-1(config-if)# switchportn5k-rhs-1(config-if)# switchport mode trunkn5k-rhs-1(config-if)# switchport trunk allowed vl 11-14n5k-rhs-1(config-if)# no sh

n5k-rhs-1# conf tn5k-rhs-1(config)# int ethernet 1/1-4n5k-rhs-1(config-if-range)# channel-group 20 mode activeEthernet1/1 Ethernet1/2 Ethernet1/3 Ethernet1/4 added to port-channel20

n5k-rhs-1(config-if-range)# end

n5k-rhs-1# show port-channel summaryFlags: D - down U - up in port-channel I - Individual S - suspended H - Hot-standby (LACP only) R - Module-removed

------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel------------------------------------------------------------------------------20 Po20(D) Eth LACP Eth1/1(I) Eth1/2(I) Eth1/3(I) Eth1/4(I)

Notes

Ports in the port-channel on the access switch are not bundled as we can not build a channel spanning the two Nexus 7000 unless we configure vPC first.

Test 4 – Enable global UDLD

Purpose

Enable UDLD globally to monitor for link unidirectional failures and connectivity issues on the inter-connecting links.


Configure the UDLD feature and leave it at its default (enabled as normal).

Procedures

Enable UDLD globally and verify neighbors, repeat the same operation on the other aggregation Nexus 7000 switch.

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.

rhs-7k-1(config)# feature udldrhs-7k-1(config)# exit

rhs-7k-1# sh udld neighborsPort Device Name Device ID Port ID Neighbor State--------------------------------------------------------------------------Ethernet1/1 TBM12326982 1 Ethernet1/1 bidirectionalEthernet1/2 TBM12326982 1 Ethernet1/2 bidirectional

Notes

To address some particular failures, loss of bidirectional connectivity UDLD in aggressive mode can also be used.

Test 5 – Verify Layer 2 Connectivity

Purpose

Verify that basic Layer 2 connectivity can get established.


Use Ping/Traceroute/traffic generator or a generic client to generate a test stream

Procedures

Verify that devices can ping and see each other through CDP. From the Peer boxes ping an SVI on the access Nexus 5000 box.

rhs-7k-1# ping 192.168.202.4PING 192.168.202.4 (192.168.202.4): 56 data bytesRequest 0 timed out64 bytes from 192.168.202.4: icmp_seq=1 ttl=254 time=8.63 ms64 bytes from 192.168.202.4: icmp_seq=2 ttl=254 time=4.87 ms64 bytes from 192.168.202.4: icmp_seq=3 ttl=254 time=0.984 ms64 bytes from 192.168.202.4: icmp_seq=4 ttl=254 time=0.96 ms

hs-7k-1# show cdp neighborsCapability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge S - Switch, H - Host, I - IGMP, r - Repeater, V - VoIP-Phone, D - Remotely-Managed-Device, s - Supports-STP-Dispute

Device-ID Local Intrfce Hldtme Capability Platform Port IDrhs-c2-1 mgmt0 150 S I WS-C2960-24TC Fas0/20

rhs-7k-2(TBM12326982) Eth1/1 169 R S I s N7K-C7010 Eth1/1


FLC12310020 Eth1/9 171 S I s N5K-C5020P-BF Eth1/1



rhs-7k-2# ping 192.168.202.4PING 192.168.202.4 (192.168.202.4): 56 data bytesRequest 0 timed out64 bytes from 192.168.202.4: icmp_seq=1 ttl=254 time=1.157 ms64 bytes from 192.168.202.4: icmp_seq=2 ttl=254 time=0.763 ms64 bytes from 192.168.202.4: icmp_seq=3 ttl=254 time=0.707 ms64 bytes from 192.168.202.4: icmp_seq=4 ttl=254 time=0.715 ms

--- 192.168.202.4 ping statistics ---5 packets transmitted, 4 packets received, 20.00% packet lossround-trip min/avg/max = 0.707/0.835/1.157 ms

rhs-7k-2# show cdp neighborsCapability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge S - Switch, H - Host, I - IGMP, r - Repeater, V - VoIP-Phone, D - Remotely-Managed-Device, s - Supports-STP-Dispute

Device-ID Local Intrfce Hldtme Capability Platform Port IDrhs-c2-1 mgmt0 145 S I WS-C2960-24TC Fas0/21






Part II – Bring up vPC on network

Test 6 – Enable vPC feature, perform consistency and connectivity checks

Purpose

Enable vPC on the network testbed and perform basic connectivity verification.


o Support for vPC is on Nexus-OS 4.1(3) or later on the Nexus 7000 platform.

o vPC peer link must be on a N7K-M132XP-12, dedicated mode port is recommended

o vPC peer-keepalive link is supported on any L3 reachable interface (e.g., front-panel port in a, separate VRF, etc.). A front panel gigabit interface can be used.

Procedures

1. Enable vPC feature on both aggregation Nexus 7000 switches.

rhs-7k-1# sh feature | in vpc

vpc 1 disabledrhs-7k-1# sh proc | in vpcPID State PC Start_cnt TTY Type Process - NR - 0 - VL vpc

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# feature vpcrhs-7k-1(config)# exit

rhs-7k-1# sh proc | in vpc26669 S b749640d 1 - VL vpc

rhs-7k-1# sh feature | in vpcvpc 1 enabled

2. Create peer-keepalive-vrf, migrate at least one port into the vrf, repeat the operation on the other peer and make sure they can ping.

vrf context peerkeepalivevlan 1,11-14

rhs-7k-1# sh run int e 2/14version 4.1(3)

interface Ethernet2/14 vrf member peerkeepalive ip address 192.168.100.1/24 no shutdown

rhs-7k-1# show vrf peerkeepalive interfaceInterface VRF-Name VRF-IDEthernet2/14 peerkeepalive 3

rhs-7k-1# ping 192.168.100.2 vrf peerkeepalivePING 192.168.100.2 (192.168.100.2): 56 data bytes64 bytes from 192.168.100.2: icmp_seq=0 ttl=254 time=1.292 ms64 bytes from 192.168.100.2: icmp_seq=1 ttl=254 time=0.711 ms64 bytes from 192.168.100.2: icmp_seq=2 ttl=254 time=0.719 ms64 bytes from 192.168.100.2: icmp_seq=3 ttl=254 time=0.599 ms64 bytes from 192.168.100.2: icmp_seq=4 ttl=254 time=0.599 ms


3. Create vPC domain, same domain on both peer switches and verify this step

7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# vpc domain 10rhs-7k-1(config-vpc-domain)# exitrhs-7k-1(config)# exit

rhs-7k-1# show vpc briefLegend: (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 10Peer status : peer link not configuredvPC keep-alive status : DisabledConfiguration consistency status: failedConfiguration consistency reason: vPC peer-link does not existsvPC role : none established

4. Configure peer-keepalive in vPC domain, repeat this operation on both Nexus 700 switches and verify the configuration. Be sure to use correct source/destination IP addresses and VRF name.

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# vpc domain 10rhs-7k-1(config-vpc-domain)# peer-keepalive destination 192.168.100.2 source 192.168.100.1 vrf peerkeepalive

rhs-7k-2# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# vpc domain 10rhs-7k-1(config-vpc-domain)# peer-keepalive destination 192.168.100.1 source 192.168.100.2 vrf peerkeepalive

rhs-7k-1# show vpc peer-keepalive

vPC keep-alive status : peer is alive--Destination : 192.168.100.2--Send status : Success--Receive status : Success--Last update from peer : (0 ) seconds, (176 ) msec


vPC keep-alive status : peer is alive--Destination : 192.168.100.1--Send status : Success--Receive status : Success--Last update from peer : (0 ) seconds, (338 ) msec

5. Configure vPC Role to administratively determine the primary vPC Peer. A lower role priority means that the vPC peer is going to be the primary. Until the peer link is not configured both Nexus 7000 think to be the primary vPC Peer devices.

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# vpc domain 10 rhs-7k-1(config-vpc-domain)# role priority 10

Warning: !!:: vPCs will be flapped on current primary vPC switch while attempting role change ::!!Note: --------:: Change will take effect after user has re-initd the vPC peer-link ::--------

rhs-7k-2(config)# vpc domain 10rhs-7k-2(config-vpc-domain)# role priority 200

Warning: !!:: vPCs will be flapped on current primary vPC switch while attempting role change ::!!Note: --------:: Change will take effect after user has re-initd the vPC peer-link ::--------

rhs-7k-1# sh vpc briefLegend: (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 10Peer status : peer link not configuredvPC keep-alive status : peer is aliveConfiguration consistency status: failedConfiguration consistency reason: vPC peer-link does not exists

vPC role : primary


vPC domain id : 10Peer status : peer link not configuredvPC keep-alive status : peer is aliveConfiguration consistency status: failedConfiguration consistency reason: vPC peer-link does not existsvPC role : primary

6. Move ISL port-channels (between the 2 N7Ks) into peer-link mode. Now the role determination will kick in and elect the primary vPC peer.

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# interface port-channel10rhs-7k-1(config-if)# vpc peer-linkWarning: vPC peer-link is configured as STP network port typerhs-7k-1(config-if)# end

rhs-7k-2# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-2(config)# interface port-channel 10

rhs-7k-2(config-if)# vpc peer-linkWarning: vPC peer-link is configured as STP network port type


vPC domain id : 10Peer status : peer adjacency formed okvPC keep-alive status : peer is aliveConfiguration consistency status: successvPC role : primary

vPC Peer-link status---------------------------------------------------------------------id Port Status Active vlans-- ---- ------ --------------------------------------------------1 Po10 up 11-14


vPC domain id : 10Peer status : peer adjacency formed okvPC keep-alive status : peer is aliveConfiguration consistency status: successvPC role : secondary


7. Configure/carry same VLANs on the peer-link, and on the vPC member ports. Perform this action on both the vPC Peer links and verify that bundling occurs. Note first two LACP ports on one Peer vPC peer will be

bundled correctly and other two will be in suspended state as vPC membership has not been assigned yet for downstream port-channels.

rhs-7k-1# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SD) Eth NONE --

rhs-7k-1(config-if)# int e 1/9-10rhs-7k-1(config-if-range)# channel-group 20 mode activerhs-7k-1(config-if-range)# exitrhs-7k-1(config)# exit

rhs-7k-1# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SU) Eth LACP Eth1/9(P) Eth1/10(P)




rhs-7k-2# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SD) Eth NONE --

rhs-7k-2# conf t

Enter configuration commands, one per line. End with CNTL/Z.rhs-7k-2(config)# int e 1/9-10rhs-7k-2(config-if-range)# channel-group 20 mode activerhs-7k-2(config-if-range)# end

rhs-7k-2# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SD) Eth LACP Eth1/9(s) Eth1/10(s)

n5k-rhs-1# sh port-channel summaryFlags: D - down U - up in port-channel I - Individual S - suspended H - Hot-standby (LACP only) R - Module-removed

--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------20 Po20(U) Eth LACP Eth1/1(U) Eth1/2(U) Eth1/3(S) Eth1/4(S)

8. Move downstream port channels into vPC mode and verify all links comes up fine on the port-channel spanning across the two peer switches.

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# int port-channel 20rhs-7k-1(config-if)# vpc 20rhs-7k-1(config-if)# end

rhs-7k-2# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-2(config)# interface port-channel 20rhs-7k-2(config-if)# vpc 20rhs-7k-2(config-if)# end

rhs-7k-1# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SU) Eth LACP Eth1/9(P) Eth1/10(P)

rhs-7k-2# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)

--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SU) Eth LACP Eth1/9(P) Eth1/10(P)

n5k-rhs-1# sh port-channel summaryFlags: D - down U - up in port-channel I - Individual S - suspended H - Hot-standby (LACP only) R - Module-removed

--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------20 Po20(U) Eth LACP Eth1/1(U) Eth1/2(U) Eth1/3(U) Eth1/4(U)

9. Perform vPC sanity, verify consistency and check connectivity.




vPC status----------------------------------------------------------------------id Port Status Consistency Reason Active vlans-- ---- ------ ----------- -------------------------- ------------20 Po20 up success success 11-14

rhs-7k-1# show vpc consistency-parameters interface port-channel 10

Legend: Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value------------- ---- ---------------------- -----------------------STP Mode 1 Rapid-PVST Rapid-PVSTSTP Disabled 1 None NoneSTP MST Region Name 1 "" ""STP MST Region Revision 1 0 0STP MST Region Instance to 1 VLAN MappingSTP Loopguard 1 Disabled DisabledSTP Bridge Assurance 1 Enabled EnabledSTP Port Type 1 Normal NormalSTP MST Simulate PVST 1 Enabled EnabledAllowed VLANs - 11-14 11-14



Name Type Local Value Peer Value------------- ---- ---------------------- -----------------------STP Port Type 1 Default DefaultSTP Port Guard 1 None NoneSTP MST Simulate PVST 1 Default Defaultlag-id 1 [(7f9b, [(7f9b, 0-23-4-ee-be-a, 8014, 0-23-4-ee-be-a, 8014, 0, 0), (8000, 0, 0), (8000, 0-d-ec-a3-9f-0, 13, 0, 0-d-ec-a3-9f-0, 13, 0, 0)] 0)]mode 1 active activeSpeed 1 10 Gb/s 10 Gb/sDuplex 1 full fullPort Mode 1 trunk trunkNative Vlan 1 1 1MTU 1 1500 1500Allowed VLANs - 11-14 11-14







Name Type Local Value Peer Value------------- ---- ---------------------- -----------------------STP Mode 1 Rapid-PVST Rapid-PVSTSTP Disabled 1 None NoneSTP MST Region Name 1 "" ""STP MST Region Revision 1 0 0STP MST Region Instance to 1 VLAN MappingSTP Loopguard 1 Disabled DisabledSTP Bridge Assurance 1 Enabled EnabledSTP Port Type 1 Normal NormalSTP MST Simulate PVST 1 Enabled EnabledAllowed VLANs - 11-14 11-14



Name Type Local Value Peer Value------------- ---- ---------------------- -----------------------STP Port Type 1 Default DefaultSTP Port Guard 1 None NoneSTP MST Simulate PVST 1 Default Defaultlag-id 1 [(7f9b, [(7f9b, 0-23-4-ee-be-a, 8014, 0-23-4-ee-be-a, 8014, 0, 0), (8000, 0, 0), (8000, 0-d-ec-a3-9f-0, 13, 0, 0-d-ec-a3-9f-0, 13, 0, 0)] 0)]mode 1 active activeSpeed 1 10 Gb/s 10 Gb/sDuplex 1 full fullPort Mode 1 trunk trunkNative Vlan 1 1 1MTU 1 1500 1500Allowed VLANs - 11-14 11-14

Ping the HSRP Virtual IP Addresses on the from the Nexus 5000 switch

n5k-rhs-1# ping 192.168.202.1PING 192.168.202.1 (192.168.202.1): 56 data bytes64 bytes from 192.168.202.1: icmp_seq=0 ttl=254 time=1.109 ms64 bytes from 192.168.202.1: icmp_seq=1 ttl=254 time=1.037 ms64 bytes from 192.168.202.1: icmp_seq=2 ttl=254 time=1.24 ms64 bytes from 192.168.202.1: icmp_seq=3 ttl=254 time=1.138 ms64 bytes from 192.168.202.1: icmp_seq=4 ttl=254 time=1.275 ms


n5k-rhs-1# ping 192.168.203.1PING 192.168.203.1 (192.168.203.1): 56 data bytes64 bytes from 192.168.203.1: icmp_seq=0 ttl=254 time=1.131 ms64 bytes from 192.168.203.1: icmp_seq=1 ttl=254 time=1.237 ms64 bytes from 192.168.203.1: icmp_seq=2 ttl=254 time=1.091 ms64 bytes from 192.168.203.1: icmp_seq=3 ttl=254 time=1.207 ms64 bytes from 192.168.203.1: icmp_seq=4 ttl=254 time=1.357 ms


10. Check if Spanning Tree gives active/active paths (no blocked downstream links)

rhs-7k-1# show spanning-tree summarySwitch is in rapid-pvst modeRoot bridge for: VLAN0011-VLAN0014Port Type Default is disableEdge Port [PortFast] BPDU Guard Default is disabledEdge Port [PortFast] BPDU Filter Default is disabledBridge Assurance is enabledLoopguard Default is disabledPathcost method used is short

Name Blocking Listening Learning Forwarding STP Active---------------------- -------- --------- -------- ---------- ----------VLAN0011 0 0 0 2 2VLAN0012 0 0 0 2 2VLAN0013 0 0 0 2 2VLAN0014 0 0 0 2 2

---------------------- -------- --------- -------- ---------- ----------4 vlans 0 0 0 8 8

rhs-7k-1# show spanning-tree vlan 11

VLAN0011 Spanning tree enabled protocol rstp Root ID Priority 24587 Address 0022.5579.7641 This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 24587 (priority 24576 sys-id-ext 11) Address 0022.5579.7641 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type---------------- ---- --- --------- -------- --------------------------------Po10 Desg FWD 1 128.4105 (vPC peer-link) Network P2pPo20 Desg FWD 1 128.4115 (vPC) P2p

rhs-7k-2# show spanning-tree summarySwitch is in rapid-pvst modeRoot bridge for: nonePort Type Default is disableEdge Port [PortFast] BPDU Guard Default is disabledEdge Port [PortFast] BPDU Filter Default is disabledBridge Assurance is enabledLoopguard Default is disabledPathcost method used is short

Name Blocking Listening Learning Forwarding STP Active---------------------- -------- --------- -------- ---------- ----------VLAN0011 0 0 0 2 2VLAN0012 0 0 0 2 2VLAN0013 0 0 0 2 2VLAN0014 0 0 0 2 2---------------------- -------- --------- -------- ---------- ----------4 vlans 0 0 0 8 8

rhs-7k-2# show spanning-tree vlan 11

VLAN0011 Spanning tree enabled protocol rstp Root ID Priority 24587 Address 0022.5579.7641 Cost 1 Port 4105 (port-channel10) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 28683 (priority 28672 sys-id-ext 11) Address 0022.5579.7341 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type---------------- ---- --- --------- -------- --------------------------------Po10 Root FWD 1 128.4105 (vPC peer-link) Network P2pPo20 Desg FWD 1 128.4115 (vPC) P2p

n5k-rhs-1# show spanning-tree summarySwitch is in rapid-pvst modeRoot bridge for: VLAN0001-VLAN0004Port Type Default is disableEdge Port [PortFast] BPDU Guard Default is disabledEdge Port [PortFast] BPDU Filter Default is disabled

Bridge Assurance is disabledLoopguard Default is disabledPathcost method used is short

Name Blocking Listening Learning Forwarding STP Active---------------------- -------- --------- -------- ---------- ----------VLAN0001 0 0 0 4 4VLAN0002 0 0 0 4 4VLAN0003 0 0 0 4 4VLAN0004 0 0 0 4 4VLAN0011 0 0 0 1 1VLAN0012 0 0 0 1 1VLAN0013 0 0 0 1 1VLAN0014 0 0 0 1 1---------------------- -------- --------- -------- ---------- ----------8 vlans 0 0 0 20 20

n5k-rhs-1# show spanning-tree vl 11

VLAN0011 Spanning tree enabled protocol rstp Root ID Priority 24587 Address 0022.5579.7641 Cost 1 Port 4115 (port-channel20) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 61451 (priority 61440 sys-id-ext 11) Address 000d.eca3.9f01 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type---------------- ---- --- --------- -------- --------------------------------Po20 Root FWD 1 128.4115 P2p

Notes

When configuring initially downstream port-channels some LACP ports on one Peer vPC peer will be bundled correctly and others will be in suspended state as vPC membership has not been assigned yet on downstream port-channels. Effectively here LACP detects that the channel is split and that is because no vPC configuration has been performed yet.

Part III - vPC Failure scenario and convergence tests

Purpose

The purpose of this section is to run failure scenario with vPC and verify that system behaves correctly and converges within expected time limit.

Test 7 - shutdown the vPC peer-link port-channel on one switch

Purpose

Verify downstream links on the secondary vPC and toward the access will be suspended to avoid the split brain condition.


Run continuous traffic between end hosts and start an unlimited ping from access to the aggregation/core and vice versa to verify the impact on packet forwarding.

Procedures

1. Shut the vPC peer-link port-channel on one Peer vPC device and check that standby vPC links gets disabled.

Notes

No traffic disruption is observed, Procedure example follows:

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# int po 10 ^Crhs-7k-1(config)#rhs-7k-1# show vpc briefLegend: (*) - local vPC is down, forwarding via vPC peer-link




rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# int po 10rhs-7k-1(config-if)# shrhs-7k-1(config-if)# exit

rhs-7k-1(config)# sh vpc briefLegend: (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 10Peer status : peer link is downvPC keep-alive status : peer is aliveConfiguration consistency status: successvPC role : primary

vPC Peer-link status---------------------------------------------------------------------id Port Status Active vlans-- ---- ------ --------------------------------------------------1 Po10 down -


rhs-7k-2# 2009 Jan 24 02:20:48 rhs-7k-2 %VPC-2-VPC_SUSP_ALL_VPC: Peer-link going down, suspending all vPCs on secondary

rhs-7k-2# show popolicy-map port-channel port-monitor

rhs-7k-2# show portport-channel port-monitorrhs-7k-2# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SD) Eth LACP Eth1/1(D) Eth1/2(D)20 Po20(SD) Eth LACP Eth1/9(D) Eth1/10(D)


vPC domain id : 10Peer status : peer link is downvPC keep-alive status : peer is aliveConfiguration consistency status: successvPC role : secondaryDual Active Detected


vPC status----------------------------------------------------------------------id Port Status Consistency Reason Active vlans-- ---- ------ ----------- -------------------------- ------------20 Po20 down failed Peer-link is down -

Test 8 - no shutdown the vPC peer-link port-channel on the switch

Purpose

Verify downstream links on the secondary vPC and toward the access will be re-enabled after the peer-link recovers.



Procedures

2. Un-Shut the vPC peer-link port-channel on one Peer vPC device and check that standby vPC links gets enabled.

Notes

No packet loss should be observed, follows procedure example:

rhs-7k-1(config)#rhs-7k-1(config)# int po 10rhs-7k-1(config-if)# no shrhs-7k-1(config-if)# exitrhs-7k-1(config)# exit

... vPC recovers all links in few seconds









Test 9 – Shutdown in sequence the two vPC port-channel members on one switch

Purpose

Verify that connectivity toward the core is not lost and that impact on the traffic is expected



Procedures

Shutdown in two steps the two ports member of the downstream port-channel

Notes

A small packet loss may occur due to load balancing algorithm on the nexus 5000 readjusting its hashing scheme due to a link failed in the channel. Anyway the traffic keeps flowing, here follows the procedure example:

rhs-7k-1# sh port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SU) Eth LACP Eth1/9(P) Eth1/10(P)rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# int e 1/9rhs-7k-1(config-if)# sh

rhs-7k-1# show port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SU) Eth LACP Eth1/9(D) Eth1/10(P)rhs-7k-1#

1 Ping from the access device is being lost.

64 bytes from 192.168.202.1: icmp_seq=3628 ttl=254 time=1.042 ms64 bytes from 192.168.202.1: icmp_seq=3629 ttl=254 time=1.055 ms64 bytes from 192.168.202.1: icmp_seq=3630 ttl=254 time=1.043 msRequest 3631 timed out64 bytes from 192.168.202.1: icmp_seq=3632 ttl=254 time=1.223 ms

rhs-7k-1# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-1(config)# int e 1/10rhs-7k-1(config-if)# shrhs-7k-1(config-if)# no shrhs-7k-1(config-if)# sh port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SU) Eth LACP Eth1/9(D) Eth1/10(P)

rhs-7k-1(config-if)# shrhs-7k-1(config-if)# exi

rhs-7k-1(config)# sh port-channel summaryFlags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel)--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel-------------------------------------------------------------------------------- 10 Po10(SU) Eth LACP Eth1/1(P) Eth1/2(P)20 Po20(SD) Eth LACP Eth1/9(D) Eth1/10(D)

rhs-7k-1(config)# show vpc briefLegend: (*) - local vPC is down, forwarding via vPC peer-link



vPC status----------------------------------------------------------------------id Port Status Consistency Reason Active vlans-- ---- ------ ----------- -------------------------- ------------20 Po20 down* success success -

rhs-7k-1(config)#

Test 10 - shut/no shutdown the vPC peer-keepalive link

Purpose

Shutdown the peer-keepalive and demonstrate that this has no impact on traffic and vPC operational state. A syslog should be produced to advertise the loss of the peer-keepalive link.



Procedures

1. Shutdown the peer-keepalive link and observe vPC states and traffic

Notes

Procedure example:

rhs-7k-2# sh vrf peerkeepaliveVRF-Name VRF-ID State Reasonpeerkeepalive 3 Up --rhs-7k-2#rhs-7k-2#rhs-7k-2# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-2(config)# exirhs-7k-2#

rhs-7k-2#rhs-7k-2# sh vrf peerkeepalive ? <CR> > Redirect it to a file >> Redirect it to a file in append mode detail Display VRF detail information interface Display interface VRF information | Pipe command output to filter

rhs-7k-2# sh vrf peerkeepalive interfaceInterface VRF-Name VRF-IDEthernet2/14 peerkeepalive 3

rhs-7k-2# conf tEnter configuration commands, one per line. End with CNTL/Z.rhs-7k-2(config)# int e 2/14rhs-7k-2(config-if)#rhs-7k-2(config-if)#rhs-7k-2(config-if)# shutrhs-7k-2(config-if)# 2009 Jan 24 03:12:59 rhs-7k-2 %VPC-2-PEER_KEEP_ALIVE_RECV_FAIL: In domain 10, VPC peer keep-alive receive has failed


vPC keep-alive status : peer is not alive--Destination : 192.168.100.1--Send status : Success--Receive status : Failed--Last update from peer : (57 ) seconds, (488 ) msecrhs-7k-2# show vpc briefLegend: (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 10Peer status : peer adjacency formed okvPC keep-alive status : peer is not aliveConfiguration consistency status: successvPC role : secondary




vPC domain id : 10Peer status : peer adjacency formed okvPC keep-alive status : peer is not aliveConfiguration consistency status: successvPC role : primary

vPC Peer-link status---------------------------------------------------------------------

id Port Status Active vlans-- ---- ------ --------------------------------------------------1 Po10 up 11-14


Test 11 - Fail peer-link followed by peer-keepalive link

Fail the peer-link and subsequently the peer-keepalive link observe that existing flows are not impacted.



Procedures

1. Shut the peer-link from the primary vPC Peer. 2. Shut the peer-keepalive link from the primary vPC Peer.3. Verify the vPC operational status and check for traffic impact on existing flows

Notes

Procedure example:

rhs-7k-1(config-if)# int po 10rhs-7k-1(config-if)# sh

rhs-7k-2(config-if)# 2009 Jan 24 03:46:18 rhs-7k-2 %VPC-2-VPC_SUSP_ALL_VPC: Peer-link going down, suspending all vPCs on secondary

rhs-7k-1(config-if)# sh vrf peerkeepalive interfaceInterface VRF-Name VRF-IDEthernet2/14 peerkeepalive 3

rhs-7k-1(config-if)# conf trhs-7k-1(config)# int e 2/14rhs-7k-1(config-if)# shutrhs-7k-1(config-if)# end

rhs-7k-2(config-if)#2009 Jan 24 03:46:59 rhs-7k-2 %VPC-2-PEER_KEEP_ALIVE_RECV_FAIL: In domain 10, VPC peer keep-alive receive has failed


vPC domain id : 10Peer status : peer link is downvPC keep-alive status : peer is not aliveConfiguration consistency status: successvPC role : primary



rhs-7k-2(config-if)#rhs-7k-2(config-if)# sh vpc briefLegend: (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 10Peer status : peer link is downvPC keep-alive status : peer is not aliveConfiguration consistency status: successvPC role : secondary


vPC status----------------------------------------------------------------------id Port Status Consistency Reason Active vlans-- ---- ------ ----------- -------------------------- ------------20 Po20 down failed Peer-link is down -

Test 12 - Fail peer-keepalive link followed by peer-link

Fail the peer-keepalive link and subsequently the peer-link and observe that existing flows are not impacted.



Procedures

1. Shut the peer-keepalive link from the primary vPC Peer.2. Shut the peer-link from the primary vPC Peer. 3. Verify the vPC operational status and check for traffic impact on existing flows

Notes

Procedure example:

rhs-7k-1(config-if)# conf trhs-7k-1(config)# int e 2/14rhs-7k-1(config-if)# sh

rhs-7k-2# debug vpc peer-link 2009 Jan 24 04:11:04 rhs-7k-2 %VPC-2-PEER_KEEP_ALIVE_RECV_FAIL: In domain 10, VPC peer keep-alive receive has failed


vPC keep-alive status : peer is not alive--Destination : 192.168.100.1--Send status : Success--Receive status : Failed--Last update from peer : (17 ) seconds, (4294402) msecrhs-7k-2# sh vpc briefLegend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 10Peer status : peer adjacency formed okvPC keep-alive status : peer is not aliveConfiguration consistency status: successvPC role : secondary



rhs-7k-1(config-if)# int po10rhs-7k-1(config-if)# shutrhs-7k-1(config-if)# exit


vPC domain id : 10Peer status : peer link is downvPC keep-alive status : Suspended (Destination IP not reachable)Configuration consistency status: successvPC role : secondary, operational primary



Test 13 - SSO switchover test during traffic

Perform an SSO switchover with traffic running and verify this does not impact vPC operations and traffic flows.



Procedures

1. Check for vPC session to be up and running 2. Issue a system Supervisor switchover 3. Verify vPC operational status and check for traffic impact on existing flows

Notes

Few packet losses are observed as we are pinging the supervisor HRSP virtual address. In this case ICMP responses are processed by the control plane of the supervisor which is swapped in the process. A more meaningful variant of this test could be to ping a remote hosts. These packets will be hardware switched and for these no loss should be seen. Procedure example:





rhs-7k-1# system switchoverrhs-7k-1# Read from remote host rhs-7k-1: Connection reset by peerConnection to rhs-7k-1 closed.[demo@virtualab ~]$[demo@virtualab ~]$[demo@virtualab ~]$ ssh admin@rhs-7k-1User Access VerificationPassword:Cisco Nexus Operating System (NX-OS) SoftwareTAC support: http://www.cisco.com/tacCopyright (c) 2002-2009, Cisco Systems, Inc. All rights reserved.The copyrights to certain works contained in this software areowned by other third parties and used and distributed underlicense. Certain components of this software are licensed underthe GNU General Public License (GPL) version 2.0 or the GNULesser General Public License (LGPL) Version 2.1. A copy of eachsuch license is available athttp://www.opensource.org/licenses/gpl-2.0.php andhttp://www.opensource.org/licenses/lgpl-2.1.phprhs-7k-1# sh vpc briefLegend: (*) - local vPC is down, forwarding via vPC peer-link



vPC status----------------------------------------------------------------------id Port Status Consistency Reason Active vlans

-- ---- ------ ----------- -------------------------- ------------20 Po20 up success success 11-14

n5k-rhs-1# ping 192.168.202.1 interval 1 count unlimitedPING 192.168.202.1 (192.168.202.1): 56 data bytes64 bytes from 192.168.202.1: icmp_seq=0 ttl=254 time=1.245 ms64 bytes from 192.168.202.1: icmp_seq=1 ttl=254 time=1.099 ms64 bytes from 192.168.202.1: icmp_seq=2 ttl=254 time=1.129 ms64 bytes from 192.168.202.1: icmp_seq=3 ttl=254 time=1.185 ms64 bytes from 192.168.202.1: icmp_seq=4 ttl=254 time=1.105 ms64 bytes from 192.168.202.1: icmp_seq=5 ttl=254 time=1.061 ms64 bytes from 192.168.202.1: icmp_seq=6 ttl=254 time=1.261 ms64 bytes from 192.168.202.1: icmp_seq=7 ttl=254 time=1.405 ms64 bytes from 192.168.202.1: icmp_seq=8 ttl=254 time=1.183 ms64 bytes from 192.168.202.1: icmp_seq=9 ttl=254 time=1.099 msRequest 10 timed out <<< Control plane switchover !!Request 11 timed out64 bytes from 192.168.202.1: icmp_seq=12 ttl=254 time=1.342 ms64 bytes from 192.168.202.1: icmp_seq=13 ttl=254 time=1.229 ms64 bytes from 192.168.202.1: icmp_seq=14 ttl=254 time=1.198 ms64 bytes from 192.168.202.1: icmp_seq=15 ttl=254 time=1.211 ms64 bytes from 192.168.202.1: icmp_seq=16 ttl=254 time=50.733 ms

ConclusionThis test plan provides an overview of Nexus 7000 / NX-OS testing and should server as a guideline in developing a custom test plan for a particular customer testing / demonstration. For more information, consult NX-OS configuration guides, TME documents or CPOC case studies.

External References http://www.cisco.com/en/US/products/ps9402/tsd_products_support_series_home.html http://www.cisco.com/en/US/products/ps9402/prod_literature.html http://wwwin-eng.cisco.com/Eng/DCBU/DC3/ http://www.cisco.com/cdc_content_elements/flash/dataCenter/ciscofeaturenavigator/

index.html

Internal References http://bock-bock.cisco.com/wiki/N7K http://wwwin.cisco.com/dss/dcbu/nexus7000/ http://bock-bock.cisco.com/wiki/N7K:tech_resources:vpc http://bock-bock.cisco.com/wiki/N7K:FAQ:vPC http://wwwin.cisco.com/vod/tac/np220208_dc3_nexus/

Acronyms

vPC Feature: Virtual Port Channel FeaturevPC peer: a vPC switch, one of a pairvPC member port: one of a set of ports (port channels) that form a vPCvPC Instance: the combined port channel between the vPC peers and the downstream devicevPC peer-link: Link used to synchronize state between vPC peer devices, must be 10GbEvPC peer-keepalive link: the fault tolerant link between vPC peer devices, i.e., backup to the vPC peer-linkCFS: Cisco Fabric Services protocol, used for state synchronization & configuration validation between vPC peersHSRP: Hot Standby Router ProtocolUDLD: UniDirectional Link DetectionVLAN: Virtual Local Area NetworkVRRP: Virtual Router Redundancy Protocol

http://wwwin.cisco.com/vod/tac/np220208_dc3_nexus/

http://bock-bock.cisco.com/wiki/N7K:FAQ:vPC

http://bock-bock.cisco.com/wiki/N7K:tech_resources:vpc

http://wwwin.cisco.com/dss/dcbu/nexus7000/

http://bock-bock.cisco.com/wiki/N7K

http://www.cisco.com/cdc_content_elements/flash/dataCenter/ciscofeaturenavigator/index.html

http://www.cisco.com/cdc_content_elements/flash/dataCenter/ciscofeaturenavigator/index.html

http://wwwin-eng.cisco.com/Eng/DCBU/DC3/

http://www.cisco.com/en/US/products/ps9402/prod_literature.html

http://www.cisco.com/en/US/products/ps9402/tsd_products_support_series_home.html

nexus 7000 test plan vpc v1

Documents

vpc peerlink portchannel

typical nexus

vpc peerkeepalive link27test

vpc test plan version

vpc portchannel members

test procedures

validatetest vpc feature

vpc failure scenario