learner guide troubleshooting hp networks 1041 no watermark

108
Troubleshooting HP Networks Learner Guide Version 10.41

Upload: -

Post on 13-Apr-2015

150 views

Category:

Documents


13 download

TRANSCRIPT

Page 1: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

Learner Guide

Version 10.41

Page 2: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Copyright 2010 Hewlett-Packard Development Company, L.P.

The information contained herein is subject to change without notice. The only warranties for HP products and

services are set forth in the express warranty statements accompanying such products and services. Nothing

herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial

errors or omissions contained herein.

This is an HP copyrighted work that may not be reproduced without the written permission of HP. You may not

use these materials to deliver training to any person outside of your organization without the written permission

of HP.

Troubleshooting HP Networks

Learner Guide

Rev 10.41

Page 3: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Rev 10.41 i

Contents Module 1: Troubleshooting Methodologies and Practices ................................. 1 - 1

Troubleshooting Methodology ................................................................ 1 - 2

Problem Solving Methodology ............................................................... 1 - 4

Identification and Analysis .................................................................... 1 - 6

Hypothesis and Validation ..................................................................... 1 - 8

Implementation and Verification ............................................................ 1 - 10

Summary ........................................................................................... 1 - 11

Module 2: Layer 1 (Physical Layer) Troubleshooting and Problem Resolution ....... 2 - 1

“It’s the cable” ..................................................................................... 2 - 2

Physical Layer Symptoms ....................................................................... 2 - 3

Module 3: Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution ..... 3 - 1

Switching ............................................................................................ 3 - 2

VLANs ................................................................................................ 3 - 3

Switch VLAN port types ........................................................................ 3 - 4

Link Aggregation .................................................................................. 3 - 9

LACP – Link Aggregation Control Protocol .............................................. 3 - 14

Configurable LACP States ..................................................................... 3 - 14

Static vs. Dynamic Link Aggregation ...................................................... 3 - 15

Spanning Tree .................................................................................... 3 - 16

Basic IRF Concepts .............................................................................. 3 - 21

How IRF simplifies networks ..................................................................3 - 23

Lab 4: VLAN Switching ....................................................................... 3 - 29

Module 4: Layer 3 (Network Layer) Troubleshooting and Problem Resolution ...... 4 - 1

Forwarding between VLANs .................................................................. 4 - 2

VRRP Basics ......................................................................................... 4 - 5

OSPF Basics ........................................................................................ 4 - 7

External and internal Border Gateway Protocol (BGP) .............................. 4 - 12

Network Address Translation (NAT) ....................................................... 4 - 14

Static and Dynamic NAT ...................................................................... 4 - 16

Lab 5: Layer 3 Practice and Tools .......................................................... 4 - 17

Lab 6: OSPF Routing Issues .................................................................. 4 - 18

Lab 7: Addressing Issues ...................................................................... 4 - 19

Lab 8: Inter-VLAN and Routing ............................................................. 4 - 20

Module 5: Layer 4 (Transport Layer) Troubleshooting and Problem Resolution ..... 5 - 1

Troubleshooting TCP/UDP ..................................................................... 5 - 2

Firewalls.............................................................................................. 5 - 7

Firewall types ....................................................................................... 5 - 9

Network address translator (NAT) ......................................................... 5 - 11

Module 6: Layer 5 (Application Layer) Troubleshooting and Problem Resolution .. 6 - 1

QoS process flow ................................................................................. 6 - 2

802.1p traffic prioritization .................................................................... 6 - 8

Traffic marking by an end station .......................................................... 6 - 11

Retaining priority between VLANs ......................................................... 6 - 12

Normal priority data traffic ................................................................... 6 - 14

Lab 10: Quality of Service .................................................................... 6 - 15

Page 4: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

ii Rev 10.41

Module 7: Troubleshooting an End-to-End Complex, Integrated Multi-Protocol

Network .................................................................................................... 7 - 1

Lab 11: Final lab ................................................................................... 7 - 2

Page 5: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Rev 10.41 1 –1

Troubleshooting Methodologies and Practices Module 1

No network or networking technology operates smoothly all of the time. Every network technician will be required at some time to troubleshoot issues in network configuration and performance. This module introduces basic techniques for network troubleshooting.

After completing this module, you will be able to:

Describe a framework for basic network troubleshooting

Page 6: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

1 –2 Rev 10.41

Troubleshooting Methodology Network troubleshooting benefits from having:

Methodology

A discipline for evaluating, analyzing and investigating problem conditions

Includes determining the scope of the problem, developing a hypothesis, testing it out, and if successful, implementing a resolution

Skill sets

Familiarity with network devices, how they operate and how they are managed

Technical tools that may be useful for investigating and verifying problems; from CLI commands and protocol analyzers

Good Q&A skills

Experience

Over time, applying a methodology and the technical tools helps develop your own “library” of problem recognition capabilities and yields a more efficient problem resolution process

The basics of troubleshooting any kind of networking trouble might be succinctly stated as “keep eliminating obvious causes until the real cause presents itself.” But understanding what this means requires a systematic approach and real discipline when attempting to identify causes from symptoms and apply the right fixes or workarounds.

Troubleshooting is a skill that all networking professionals learn by trial and error. But skipping some of the more painful or obvious errors can make your learning somewhat less trying than it might be otherwise. The most important characteristic to cultivate when solving problems is calmness. If you can keep a clear head when things fail or start degrading seriously, you’ll be better able to assess your situation and better equipped to solve whatever problems you discover.

Methodology Development of problem solving techniques is often an on-the-job acquisition process. Few of us can expect much along the lines of formal network troubleshooting training in our job positions for a number of reasons. These reasons may include:

The relatively fast pace of the day-to-day job tasks and challenges yields little time to pursue formal training on troubleshooting aspects such as technical tools like a protocol analyzer.

Page 7: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting Methodologies and Practices

Rev 10.41 1 –3

Few business environments provide the luxury of a “test lab” and the time to hone your skills where a progression of test problems can be examined, worked through, and resolutions tried out.

In the absence of a more ideal situation, a problem solving methodology can increase the effectiveness of support staff by standardizing the approach used to some extent. With a fairly modest amount of discipline, network technicians can improve their problem resolution efficiency in terms of the effort needed and the number of other people that must be directly involved.

Skill Sets There are a variety of skill sets that can enhance a network technician’s success in problem solving. Some of these skills are purely technical in nature. For instance, it is important to understand the fundamentals of how network devices operate and how they are managed. Having proficiency in reading logs or interpreting a protocol analyzer display are examples of having familiarity with the potential tools you may need to call upon from your “toolbox”.

Other skills are much less technical, but still very important. As part of the problem investigation process, a network technician may need to talk with various levels of staff. The staff may include non-technical end-users and business unit managers to software and hardware vendor support people. Having sufficient interpersonal skills coupled with good investigative reporter-like skills can expedite the isolation of a problem and eliminate the “noise” that often conceals the real problem.

Proactive IT support groups tend to spend time on developing procedures and tools to facilitate problem resolutions. Some examples of technical tools used by the network technicians are:

Device logs—Archived instances of the logs as well as the current one may provide hints of where the problem may be. At the very least, familiarity with a log file’s typical contents helps you differentiate normal from abnormal situations.

Device statistics and status information—Being able to determine the health of a system or the network is important for gathering the “vital” signs. This type of information can include anything from port statistics and CPU utilization to network reachability results.

Protocol analyzer—Although this may not be a frequently used tool, it can be invaluable for examining what conversations are or are not occurring between communicating devices.

A problem solving methodology that is refined over time can be very beneficial to network technicians. Being methodical and learning from the macro and micro levels of mistakes can help network technicians improve problem recognition capabilities and yield a more efficient application of a problem resolution process.

Page 8: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troublesh

1 –4

Prob

hooting HP Ne

blem So

A probleAlthoughsituationin the effprobabiligeneral ptoday’s c

There aremust be methodoor possib

The six st

Identechpotemult

Analtools

tworks

olving

m solving mh there is no s, a general forts made tity of a succeproblem solvontemporar

e six steps toexecuted in

ology state tbly return to

teps are:

ntification—Unical perspe

ential probletiple perspec

lysis—Evaluas, product do

Metho

methodologyone specificframework co solve a proessful resoluving methodry network e

o the problemorder startinhat if a step the top leve

Understand aective. Somem is before ctives.

ate the situatocumentatio

dology

Figure 1

y is a processc model thatcan provide oblem. Apption. This g

dology that hnvironments

m solving mng with idenfails, you m

el step.

and documeetimes it is psearching fo

tion by inveson and user

y

1

s for managit may be useguidelines a

plying a methraphic illustrhas many aps.

methodology ntification. Tust return to

ent the probpossible to loor a cause wh

stigating usiinput.

ing problemeful for all prand help enshodology carates the frapplications, i

y outlined heThe rules of to the preced

blem from boose sight of when we don’

ing problem

Rev 1

resolution. roblem sure efficiencan improve tmework for ncluding in

ere. The stepthe

ding step abo

oth a user anwhat the ’t consider

m resolution

10.41

cy he a

ps

ove

nd

Page 9: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting Methodologies and Practices

Rev 10.41 1 –5

Hypothesis—Develop possible resolutions based on the analysis and document a possible resolution. This documentation may be fairly informal, but it is important to be able to explain it in writing. Doing so can reveal a hypothesis that is unclear and for which a possible resolution may not be plausible.

Validation—Run a validation process to prove or disprove the hypothesis. This may not be particularly feasible, for example, if you have no test lab equipment to try out your hypothesis. At the very least, performing a walk-through of the hypothesis in an articulate manner with other team members may help.

Implementation—Develop an implementation plan along with a back-out plan, just in case, and then implement the resolution. For example, have a backup configuration and software image readily available.

Verification—Verify the success or failure of the implementation. If it fails, implement the back-out plan.

Page 10: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troublesh

1 –6

Iden

hooting HP Ne

ntificati

The first process. assumingevaluatinanalysis o

For examhost, a gwhat thehosts, buin the samissue maconcerniperformsdefault g

The iden

Docudepe

tworks

on and

step of the s Try to obseg somethingng and resolvof symptoms

mple, it is improup of hosty have in co

ut not remotme VLAN cay be a logicang connectivs Layer 3 forwgateway is do

tification pro

umenting thending on th

d Analy

six-step metrve everythi

g. Because nving connects to determi

portant to dets, or the entmmon. For e hosts, veri

an communical problem wvity with thewarding on one using La

ocess consis

he physical sehe problem s

ysis

Figure 2

hodology is ng, not just

network troutivity issues, ne the scop

etermine whtire networkinstance, if afy connectivcate with locwith the defae default gatebehalf of loc

ayer 2 addre

ts of doing t

ettings. Thescenario, bu

2

identificatiothe apparenbleshooting the generale of the issu

hether the pr. If many hoa host can c

vity with its dcal hosts, buault gatewayeway. Althocal hosts, thessing.

tasks that ca

e specifics wit some exam

on, which is ant problem, ag primarily inl procedure

ue.

roblem is affosts are affecommunicate

default gatewut not remotey or a physicough the defeir commun

an include:

ill of course mples are the

Rev 1

an observatiand avoid nvolves begins with

fecting a sincted, determe with local way. If all hoe hosts, the al problem

fault gatewayication with

vary e following:

10.41

ion

an

gle mine

osts

y the

Page 11: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting Methodologies and Practices

Rev 10.41 1 –7

What client, server and network device hardware and software are in use?

What is the network topology between the client and server?

Where are the applications and services located? Determine the effects the problem has on the user/customer and the business.

Developing a problem definition—Document probable failures.

Prioritizing the problem—Prioritize based on defined user/customer policies. Is this a problem that must be investigated immediately or can it wait until you can assemble a strategy using the problem solving methodology?

Step 2 is analysis. Analysis is the process of isolating the problem with the objective to narrow down the different possibilities.

The analysis process considers such factors as the following:

Does the system work without the problem

Previous changes to the system

Something new, such as networking equipment, that may have been introduced

Any changes to peripheral equipment that may have been made

Whether the hardware or software is being used correctly

With the scope of the problem having been narrowed down, that can help suggest the type of network troubleshooting tools you may want to use to test probable causes. For example, the problem investigation may involve using simple network reachability tools, such as traceroute or ping, or examination of the logs of multiple switches, or even use of a protocol analyzer.

Page 12: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troublesh

1 –8

Hyp

hooting HP Ne

othesis

Step 3 is informatcauses.

Some thi

Wha

You youron thproc

Evenimm

Validatiohypothesfact be re

tworks

s and V

hypothesis. ion acquired

ings to keep

at is the tech

need a validr intuition mhat primarilycess.

ntual resolutmediately obv

n, step 4, typsis is reasonaesolved after

Validati

The hypothd from the an

in mind are

hnical reason

dation proceday prove to y makes it di

ion of the prvious.

pically involvable. It increr implement

on

Figure 3

hesis step invnalysis step t

e:

n for the bus

dure for the be correct aifficult for m

roblem could

ves experimeeases the cotation of a po

3

volves the evto determin

iness proble

hypothesis at times, in thanagement

d create side

entally deternfidence levotential solu

valuation of e a number

em?

to be usablehe business to feel confi

e effects, som

rmining whevel that the pution.

Rev 1

the of probable

e. Althoughworld, relyinident about

me that are

ether the problem will

10.41

ng the

not

in

Page 13: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting Methodologies and Practices

Rev 10.41 1 –9

The validation step involves:

Testing each hypothesis until you validate a probable cause with a high degree of certainty. The objective is not necessarily to be 100% sure, but to balance the time criticality of resolving a problem with the information you have available.

If validation fails for all probable causes you developed, then you may need to return to the problem definition phase and start over. Despite what may appear to be time wasted, you will likely have improved your awareness of the problem situation and will have some additional facts to use when you attempt to redefine the problem.

Page 14: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troublesh

1 –10

Imp

hooting HP Ne

lement

Implemeor netwoimplemestate.

The plan

Deve

Devesucc

Deveremo

Verificatiand deteeffects arphase is

Upon sucproblem documen

tworks

tation a

entation, stepork fix or montation fails,

ning involve

elopment of

elopment ofcessful.

elopment ofoved, if it fai

on, step 6, isermining thatre unacceptaexecuted.

ccessful comresolution s

ntation can l

and Ve

p 5, requires dification alo, you must b

es:

f a specific im

f a verificatio

f a back-out ls. It should

s the processt any side efable, the bac

mpletion, theshould be doead to lengt

rificatio

Figure 4

planning foong with prebe able to re

mplementati

on process to

plan to ensud also addres

s of proving ffects are accck-out plan d

e user or custocumented ithy resolutio

on

4

or installationeparation fostore the sys

on plan.

o prove the i

ure the impless how to ha

the implemceptable. If developed in

tomer must n a trouble l

on for recurri

n of some for failure. If astem to a pr

implementa

ementation andle side ef

mentation waverification n the implem

be informedlog. Lack of ing problem

Rev 1

orm of systeman revious stabl

tion was

can be ffects.

as successfulfails or side

mentation

d and the f

ms.

10.41

m

e

Page 15: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting Methodologies and Practices

Rev 10.41 1 –11

Summary Network troubleshooting benefits from having a methodology, skill sets and

experience

General problem solving methodology consists of six steps:

Identification: Develop a problem statement

Analysis: Narrow the scope

Hypothesis: Define procedures to validate

Validation: Test probable causes

Implementation: Make changes with back-out plan ready

Verification: Ensure that changes resolve problem without side effects

Page 16: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

1 –12 Rev 10.41

This page left blank intentionally.

Page 17: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Rev 10.41 2 –1

Layer 1 (Physical Layer) Troubleshooting and

Problem Resolution Module 2

In this module, various layer 1 problems will be discussed.

The technologies include:

Cable / Link problems

Link Errors

Page 18: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

2 –2 Rev 10.41

“It’s the cable”

Figure 2.1: Cables

Some of the most common Layer 1 problems can be isolated to the cable.

Common physical layer problems:

Bad Cables can be terminated improperly or have physical breaks in one or

more conductors, etc.

Mis-wired cables can be terminated in the wrong order. A common symptom

here is that a cable works with 10 or 100Mbps links but not 1 Gig links because

of the extra conductors required for Gigabit. It is also common to have fiber

links mis-wired so that transmit is connected to transmit and receive is connected

to receive.

Interference is mostly a problem with unshielded copper cables. This can be

due to running data cable alongside power cable.

Wrong cable types could be using a CAT3 cable with a Gigabit link or a

multimode fiber cable with transceivers that require single mode, etc.

Page 19: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution

Rev 10.41 2 –3

Physical Layer Symptoms These are some common symptoms of layer 1 issues:

No link

Link on one end only

Errors on link

To troubleshoot these issues, the switches port counters and event logs can be very

useful.

A-Series commands

display interface <INT-ID>

display interface brief

or

display brief interface

display logbuffer reverse

E-Series commands

show interfaces <INT-ID>

show interfaces brief

log –r

Here are some examples of these commands.

[4800G]display interface GigabitEthernet 1/0/2

GigabitEthernet1/0/2 current state: UP

IP Packet Frame Type: PKTFMT_ETHNT_2, Hardware Address: 0022-

5782-fec2

Description: GigabitEthernet1/0/2 Interface

Loopback is not set

Media type is twisted pair

Port hardware type is 1000_BASE_T

1000Mbps-speed mode, full-duplex mode

Link speed type is autonegotiation, link duplex type is

autonegotiation

Flow-control is not enabled

The Maximum Frame Length is 1522

Broadcast MAX-pps: 3000

Unicast MAX-ratio: 100%

Multicast MAX-ratio: 100%

Forbid jumbo frame to pass

PVID: 1

Mdi type: auto

Link delay is 0(sec)

Port link-type: access

Tagged VLAN ID : none

Untagged VLAN ID : 1

Port priority: 0

Peak value of input: 279 bytes/sec, at 2000-04-26 12:09:54

Peak value of output: 78 bytes/sec, at 2000-04-26 12:09:59

Last 300 seconds input: 1 packets/sec 115 bytes/sec 0%

Page 20: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

2 –4 Rev 10.41

Last 300 seconds output: 0 packets/sec 78 bytes/sec 0%

Input (total): 916 packets, 136158 bytes

186 unicasts, 79 broadcasts, 651 multicasts

Input (normal): 916 packets, - bytes

186 unicasts, 79 broadcasts, 651 multicasts

Input: 0 input errors, 0 runts, 0 giants, 0 throttles

0 CRC, 0 frame, - overruns, 0 aborts

- ignored, - parity errors

Output (total): 199 packets, 35587 bytes

146 unicasts, 10 broadcasts, 43 multicasts, 0 pauses

Output (normal): 199 packets, - bytes

146 unicasts, 10 broadcasts, 43 multicasts, 0 pauses

Output: 0 output errors, - underruns, - buffer failures

0 aborts, 0 deferred, 0 collisions, 0 late collisions

0 lost carrier, - no carrier

[4800G]display brief interface

The brief information of interface(s) under route mode:

Interface Link Protocol-link Protocol type Main IP

NULL0 UP UP(spoofing) NULL --

Vlan1 UP UP ETHERNET 16.1.1.50

The brief information of interface(s) under bridge mode:

Interface Link Speed Duplex Link-type PVID

GE1/0/1 DOWN auto auto access 1

GE1/0/2 UP 1G(a) full(a) access 1

GE1/0/3 DOWN auto auto access 1

---- More ----

[4800G]display logbuffer reverse

Logging buffer configuration and contents:enabled

Allowed max buffer size : 1024

Actual buffer size : 512

Channel number : 4 , Channel name : logbuffer

Dropped messages : 0

Overwritten messages : 0

Current messages : 166

%Apr 26 13:54:59:803 2000 4800G LLDP/2/CREREM:Port

GigabitEthernet1/0/2 (IfIndex 9437185):Created new neighbor,

chassis ID: 001c-2e96-8900, port ID: 1.

%Apr 26 13:54:58:908 2000 4800G MSTP/2/PFWD:Instance 0's

GigabitEthernet1/0/2 has been set to forwarding state!

%Apr 26 13:54:58:907 2000 4800G IFNET/4/UPDOWN:

Line protocol on the interface Vlan-interface1 is UP

%Apr 26 13:54:58:907 2000 4800G IFNET/4/LINK UPDOWN:

Vlan-interface1: link status is UP

%Apr 26 13:54:58:873 2000 4800G IFNET/4/LINK UPDOWN:

GigabitEthernet1/0/2: link status is UP

%Apr 26 13:54:56:209 2000 4800G IFNET/4/UPDOWN:

Line protocol on the interface Vlan-interface1 is DOWN

---- More ----

Page 21: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution

Rev 10.41 2 –5

E3500yl# show interfaces 23

Status and Counters - Port Counters for port 23

Name :

MAC Address : 001c2e-968929

Link Status : Up

Totals (Since boot or last clear) :

Bytes Rx : 1,821,092 Bytes Tx :

304,614

Unicast Rx : 1626 Unicast Tx : 1938

Bcast/Mcast Rx : 10,253 Bcast/Mcast Tx : 503

Errors (Since boot or last clear) :

FCS Rx : 0 Drops Tx : 0

Alignment Rx : 0 Collisions Tx : 0

Runts Rx : 0 Late Colln Tx : 0

Giants Rx : 0 Excessive Colln : 0

Total Rx Errors : 0 Deferred Tx : 0

Others (Since boot or last clear) :

Discard Rx : 0 Out Queue Len : 0

Unknown Protos : 0

Rates (5 minute weighted average) :

Total Rx (bps) : 5,001,008 Total Tx (bps) : 3,010,520

Unicast Rx (Pkts/sec) : 0 Unicast Tx (Pkts/sec) : 0

B/Mcast Rx (Pkts/sec) : 0 B/Mcast Tx (Pkts/sec) : 0

Utilization Rx : 00.50 % Utilization Tx : 00.30 %

E3500yl# show interfaces brief

Status and Counters - Port Status

| Intrusion MDI Flow Bcast

Port Type | Alert Enabled Status Mode Mode Ctrl Limit

----- ----- + ------ ------- ------ ------- ----- ---- -----

1 1000 | No Yes Up 1000FDx MDI off 0

2 1000 | No Yes Down 1000FDx Auto off 0

3 1000 | No Yes Down 1000FDx Auto off 0

4 1000 | No Yes Down 1000FDx Auto off 0

5 1000 | No Yes Down 1000FDx Auto off 0

6 1000 | No Yes Down 1000FDx Auto off 0

-- MORE --, next page: Space, next line: Enter, quit: Control-C

E3500yl# log -r

Keys: W=Warning I=Information

M=Major D=Debug E=Error

---- Reverse event Log listing: Events Since Boot ----

I 10/22/10 17:52:38 00561 ports: port 1 Applying Power to PD.

I 10/22/10 17:52:38 00560 ports: port 1 PD Detected.

I 10/22/10 17:52:36 00076 ports: port 1 is now on-line

I 10/22/10 17:52:35 00565 ports: port 1 PD Removed.

I 10/22/10 17:52:34 00561 ports: port 1 Applying Power to PD.

I 10/22/10 17:52:34 00560 ports: port 1 PD Detected.

I 10/22/10 17:52:31 00565 ports: port 1 PD Removed.

I 10/22/10 17:52:30 00077 ports: port 1 is now off-line

-- MORE --, next page: Space, next line: Enter, quit: Control-C

Page 22: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

2 –6 Rev 10.41

Troubleshooting no link

Step 1: Determine if one or two fibers are in use. BX (bi-directional) transceivers use

only one fiber for both transmit and receive. There are two "flavors" of BX

transceiver. One is a "D" (downstream), the other is a "U" (upstream). You must

connect a "D" to a "U". You cannot connect a "D" to a "D", and you cannot

connect a "U" to a "U".

Is this a BX transceiver link?

Action: If BX, try using the other "flavor" (D or U). Or try a connection to a nearby

device, ensuring D connects to U.

Step 2: Roll (swap) transmit and receive fibers at only one place; for BX ensure "D"

connects to"U".

Does link come up?

Step 3: If no link after rolling the fibers, try connecting to a nearby device with

crossover fiber.

NOTE: Fiber must be "crossover", meaning transmit at one end connects to receive

at the far end. Many fiber patchcords are mis-labeled. Do not rely on color-coding

of strain relief, or "A" and "B" labels on the patchcord, to determine if patchcord is

crossover. (Those can be wrong.) Instead, use manufacturer's lettering on outside of

fiber to identify which strand is which. With connector nub facing up on each end,

and with each connector pointing the same direction, be sure lettering is on left at

one end, and on right at other end, as shown here:

With both connectors facing

same direction, crossover fiber

has lettering on left fiber at one

end, and lettering on right fiber

at other end.

Does link come up?

Action: If no link occurs using crossover fiber to nearby device with known-good

transceiver, then validate with physical inspection that this is a genuine HP

transceiver.

Page 23: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution

Rev 10.41 2 –7

Troubleshooting Errors on link

HP switches keep per-port statistics (counters) that help us diagnose problems on the

link or on the network. In addition to "normal" errors like an occasional bad packet

received (with incorrect FCS/CRC for example), HP switches alert users to abnormal

or "excessive" errors.

"Excessive" errors and FFI

FFI (Find, Fix, Inform) is a feature of HP switches that informs the user when the switch detects a

large number of errors in a short period of time, with specific parameters defined by the HP

Switch Lab. The feature was originally called "Fault-finder", and is a good indicator of

problems on the link or network. Here are the FFI messages and explanations from the "Help"

text in the menu-based event log. Description is what the switch detected. Possible causes are

documented, as are user Actions to resolve the problem.

Too many undersized/giant packets

Description: A device on this port is transmitting packets shorter than 64 bytes or

longer than 1518 bytes (longer than 1522 bytes if tagged), with valid CRCs.

Possible Causes: A misconfigured NIC or a malfunctioning NIC, NIC driver, or

transceiver.

Actions:

a. Check the NIC for a misconfiguration.

b. Update the NIC driver software.

c. Replace the malfunctioning NIC or transceiver.

d. Check for a short-circuit in the cable path connected to this port.

Excessive jabbering

Description: A device on this port is incessantly transmitting packets ("jabbering" is

detected as oversized packets with CRC errors).

Possible Causes: A misconfigured NIC, or a malfunctioning NIC or transceiver. It

could also be caused by a short-circuit in the network cable path.

Actions:

a. Check the NIC for a misconfiguration.

b. Update the NIC driver software.

c. Replace the NIC or transceiver.

d. Check for a short-circuit in the cable path connected to this port.

Excessive CRC/alignment errors

Description: A high percentage of data errors was detected on this port.

Possible Causes: Faulty cabling or topology, half/full duplex mismatch, a

misconfigured NIC, or a malfunctioning NIC, NIC driver, or transceiver.

Page 24: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

2 –8 Rev 10.41

Actions:

a. If this port is 100Base-T, make sure the cable, connectors, punch-down

blocks, and patch panels connecting to the port are Category 5 or better.

Verify the correctness of the installation using a Category 5 test device.

b. Check the directly-connected device for mismatches in half/full duplex

operation (half duplex on the switch and full duplex on the connected

device, or the reverse).

c. Update the NIC driver software.

d. Verify that the network topology conforms to IEEE 802.3 standards.

e. Replace or relocate the cable. Also check wiring closet components,

transceivers, and NICs for proper operation.

Excessive late collisions

Description: Late collisions (collisions detected after transmitting ~64 bytes) were

detected on this port.

Possible Causes: An overextended LAN topology, half/full duplex mismatch, or a

misconfigured or faulty device connected to the port.

Actions:

a. Verify that the network topology conforms to IEEE 802.3 standards. Insert

bridges or switches, if needed, to extend the network topology.

b. Check the directly-connected device for mismatches in half/full duplex

operation (half duplex on the switch and full duplex on the connected

device).

c. If this port is 100Base-T, make sure the cable connecting to that port is

Category 5 or better.

d. Check for faulty cabling, transceivers, and NICs.

High collision or drop rate

Description: A large number of collisions or packet drops have occurred on the

port.

Possible Causes: An extremely high level of traffic on this port, half/full duplex

mismatch, a misconfigured or malfunctioning NIC or transceiver on a device

connected to this port, or a topology loop in the network.

Actions:

a. Use a network monitoring device or application to determine the traffic

levels on the affected segment. If needed, consider subdividing that

segment with switches or bridges, or moving high-traffic devices to their

own switch ports.

Page 25: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution

Rev 10.41 2 –9

b. Check the directly-connected device for mismatches in half/full duplex

operation (half duplex on the switch and full duplex on the connected

device).

c. Check for a misconfigured NIC or transceiver (such as a transceiver

configured for "loopback test" or "SQE test").

d. Verify that there are no topology loops in your network. If not enabled,

you may also enable spanning.

Excessive broadcasts

Description: An excessively high rate of broadcast packets was received on the

port. This degrades the performance of all devices connected to this switch.

Possible Causes: This is usually caused by a network topology loop, but can also be

due to a malfunctioning device, NIC, NIC driver, or software application.

Actions:

a. Verify that there are no topology loops in your network.

b. Find and correct any malfunctioning devices or NICs on the segment.

c. Find and correct any malfunctioning applications on devices on the

segment.

Page 26: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

2 –10 Rev 10.41

This page left blank intentionally.

Page 27: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Rev 10.41 3 –1

Layer 2 (Data Link Layer) Troubleshooting and

Problem Resolution Module 3

In this module, various layer 2 technologies will be reviewed and common problems

will be discussed.

The technologies include:

Layer 2 switching

VLANs

Link Aggregation

Spanning Tree

IRF

Page 28: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –2 Rev 10.41

Switching

Figure 3.1: Switching

Today’s switches forward frames in two ways. They flood frame and they switch

frames. Frames are flooded if their destination is unknown. That is, the destination

doesn’t have an entry in the MAC address table. This is also the biggest difference

between hubs and switches. Hubs do not maintain a MAC address table.

When the destination address is known, then a frame is only forwarded towards that

destination. This has the effect of reducing traffic on a network because traffic is not

sent out on all links.

Page 29: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –3

VLANs Virtual LAN – A logical broadcast domain

VLANS are used to divide a network segment to smaller sub networks to :

Reduce the overhead of layer 2 broadcast.

Increase security.

Improve management of network infrastructure

VLANs are created through software configuration.

Type of VLANS

Port-based VLANs

MAC address-based VLANs

Protocol-based VLANs

IP-subnet-based VLANs

Policy-based VLANs

A virtual LAN (VLAN) is a collection of network nodes that are logically grouped

together to form a separate broadcast domain. A VLAN has the same general

attributes as a physical LAN, but it allows all nodes for a particular VLAN to be

grouped together, regardless of physical location. One advantage of using VLANs is

design flexibility.

VLANs allow individual users to be grouped based on business needs.

Connectivity within a VLAN is established and maintained through software

configuration. The list above is a partial list of supported VLAN types.

A-Series switches also support Voice VLANs and policy-based VLANs, which are

used with 802.1X authentication. This security technology is covered in the

Accredited Systems Engineer (ASE) certification track.

Page 30: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –4 Rev 10.41

Switch VLAN port types Access ports:

Belong to one VLAN – Port is untagged

Trunk ports:

Carry multiple VLANs on a single physical link

VLANs are 802.1Q tagged

The native VLAN is untagged

Hybrid ports:

Belongs to multiple VLANs

Multiple VLANs can be untagged and tagged

Typically used for IP phone connection

Also in conjunction with protocol VLANs, IP subnet VLANs

A-Series switches

By default, VLAN 1 is the native VLAN. To define a trunk:

interface gi 1/0/1

port link-type trunk

port trunk permit [all | vlan ids]

port trunk pvid vlan [id] (Defines Native VLAN.)

In this case, VLAN 1 will be tagged if still carried. The undo port trunk permit vlan 1

command undoes VLAN 1 assignment.

Control plane info, including BPDU and LLDP frames, is sent untagged. To configure

multiple ports, define port groups:

[switch] port-group manual [port-group-name]

[switch] group-member [port names]

[switch] port link-type [trunk | hybrid | access]

Access ports are ports that belong to a single VLAN and the traffic is sent and

received untagged. There are two methods to define access ports.

Add access ports to VLAN for PCs

[SW-A]vlan 100

[SW-A-vlan100]port gigabitethernet 1/0/1 to gig 1/0/20

OR in interface configuration mode, set interface as an access port in VLAN 100

[SW-A]interface gi 1/0/1

[SW-A-GigabitEthernet1/0/1]port link-type access

[SW-A-GigabitEthernet1/0/1]port access vlan 100

Page 31: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –5

Use these commands to view VLAN membership.

display vlan [vid]

display vlan all

Hybrid Ports

Hybrid ports are used mostly for IP phones. Hybrid ports can be assigned to multiple

VLAN as tagged or untagged.

To set Hybrid ports using a port group:

[SW]port-group manual phones-1

[SW-port-group-manual-phones-1]group-member gi 1/0/11 to gi

1/0/20

[SW-port-group-manual-phones-1]port link-type hybrid

To set Data VLAN 100 as the native VLAN:

[SW-port-group-manual-phones-1]port hybrid PVID 100

Note: Hybrid port is still part of VLAN 1. Then Remove hybrid port from VLAN 1

[SW-port-group-manual-phones-1]undo port hybrid vlan 1 untagged

To set VLAN 200 as voice VLAN:

[SW-port-group-manual-phones-1]voice vlan 200 enable

This makes VLAN tagged on port and auto-QoS if Phone SNMP OUI Is detected.

Voice VLAN command will dynamically:

Allocate the voice vlan as a tagged vlan with auto qos if a predefined phone

SNMP OUI is detected.

Add OUI with voice OUI command at system view.

Hybrid ports can be set as untagged in one or more VLANs. Here is an example of

configuration on a hybrid port to use with protocol VLAN:

[SWA]vlan 2

[SWA-vlan2]Description IP and ARP VLAN

[SWA-vlan2]protocol-vlan mode ethernetii etype 0800

[SWA-vlan2]protocol-vlan mode ethernetii etype 0806

[SWA-vlan2]vlan 3

[SWA-vlan2]Description Novell IPX VLAN

[SWA-vlan3]protocol-vlan ipx llc

[SWA-vlan3]interface gigabit 1/1/1

[SWA-gigabit1/1/1]description Access port Separate IP and IPX

traffic

[SWA-gigabit1/1/1]port link-type hybrid

[SWA-gigabit1/1/1]undo port hybrid vlan 1

[SWA-gigabit1/1/1]port hybrid vlan 2 3 untagged

Page 32: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –6 Rev 10.41

[SWA-gigabit1/1/1]port hybrid protocol-vlan vlan 2 all

[SWA-gigabit1/1/1]port hybrid protocol-vlan vlan 3 all

[SWA-vlan3]interface gigabit 1/1/23

Hybrid ports can be assigned to multiple VLAN as tagged or untagged.

To set Hybrid ports using a port group

[SW]port-group manual phones-1

[SW-port-group-manual-phones-1]group-member gi 1/0/11 to gi

1/0/20

[SW-port-group-manual-phones-1]port link-type hybrid

Ts set Data VLAN 100 as the native VLAN

[SW-port-group-manual-phones-1]port hybrid PVID 100

Note

Hybrid port is still part of VLAN 1. Then Remove hybrid port from VLAN 1.

[SW-port-group-manual-phones-1]undo port hybrid vlan 1 untagged

To set VLAN 200 as voice VLAN

[SW-port-group-manual-phones-1]voice vlan 200 enable

This makes VLAN tagged on port and auto-qos if Phone SNMP OUI Is detected.

[SWA-gigabit1/1/23]description Trunk port Separate IP and IPX

traffic

[SWA-gigabit1/1/23]port link-type trunk

[SWA-gigabit1/1/23]port trunk permit vlan 2 3

Trunk Ports

On trunk 802.1Q ports: one VLAN at most is untagged, all other VLANs are tagged

To configure the trunk interfaces & allow the VLANs:

[SW-A]interface gi 1/0/23

[SW-A-GigabitEthernet1/0/23]port link-type trunk

[SW-A-GigabitEthernet1/0/23]port trunk permit vlan 100 200

[SW-A]interface gi 1/0/24

[SW-A-GigabitEthernet1/0/24]port link-type trunk

[SW-A-GigabitEthernet1/0/24]port trunk permit vlan all

List trunk ports:

[SW-A]display port trunk

Interface PVID VLAN passing

GE1/0/23 1 1, 100, 200

GE1/0/24 1 1, 100, 200

On edge switches you can set the uplinks as trunk ports carrying all VLANs:

Page 33: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –7

port link-type trunk

port trunk permit vlan all

Note

Do not confuse ―trunk‖ ports with the link aggregation ports that are called trunk

ports on HP E-Series switches.

On distribution/core switches, set exactly what VLANs should be carried on

downlinks to edge switches:

port link-type trunk

port trunk permit vlan 100 200

Note

VLAN 1 is set by default.

To change the native VLAN to VLAN 99

[SW-A-GigabitEthernet1/0/23]port trunk PVID 99

This forces the interface to be tagged on VLAN 1. If VLAN 1 is not desired on port,

remove it

[SW-A-GigabitEthernet1/0/23]undo port trunk permit vlan 1

List trunk ports

[SW-A]display port trunk

Interface PVID VLAN passing

GE1/0/23 99 99, 100, 200

GE1/0/24 99 99, 100, 200

VLAN 1 is not necessary on A-Series switches. For example: BPDUs for STP, LLDP or

LACP are sent untagged whatever is the setup of VLANs on the link. BPDUs are

accepted by a receiving switch because their destination MAC address matches the

list of MAC addresses on the ports. In other words, because the protocols (LLDP, STP,

LACP) are enabled on port and global levels.

Page 34: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –8 Rev 10.41

E-Series switches

E-Series switches do not use the same terminology as the A-Series. On E-Series

devices, VLAN membership is configured from the VLAN context with the tagged

and untagged commands. A port can be considered to be a VLAN trunk port if it is

assigned to more than one VLAN. Similarly, a port can be considered to be an

access port if it is only assigned to one VLAN for untagged traffic.

To configure a port to be an untagged member of a VLAN (access port):

E-Series(config)# vlan 100

E-Series(vlan-100)# untagged a1-a12

To configure a port to be a tagged member of a VLAN (trunk port):

E-Series(config)# vlan 100

E-Series(vlan-100)# tagged a1-a12

E-Series(vlan-100)# vlan 200

E-Series(vlan-200)# tagged a1-a12

E-Series(vlan-100)# vlan 5

E-Series(vlan-5)# untagged a1-a12 (This is optional)

To configure a port to be a voice VLAN:

E-Series(config)# vlan 100

E-Series(vlan-100)# voice

Page 35: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –9

Link Aggregation Link aggregation is called trunking on HP E-Series switches.

E-Series switches support two trunking methods:

HP Port Trunking—HP has supported port trunking since its first offering of

switches in the mid-1990s. The original HP port trunking technology remains an

option on ProCurve switches. HP port trunking is the default on E-Series switches.

For proper trunk operation, all links in the same trunk group must have the same

speed, duplex, and flow control

Link Aggregation Control Protocol (LACP)—The IEEE standard for link

aggregation. HP’s implementation of LACP supports both active and passive

configuration of trunking.

These link-aggregation methods impose a similar set of requirements and restrictions.

However, LACP imposes an additional restriction—the links must operate in full-

duplex mode. This is rarely a concern because trunks consist of point-to-point links

between switches, and these links will usually negotiate up to full duplex operation.

HP port trunking does not have this requirement.

Both methods for port trunking share one important limitation in the area of load

sharing—they are static methods. They do not adjust to reflect traffic volume on the

links or evaluate an individual conversation to determine which link would be best at

a given moment. Instead, all methods distribute the conversations evenly across all

links with the expectation that the load generally is balanced. The benefits of

trunking are always best realized in the presence of many source and destination

points on each side of the trunk.

Configuring Port Trunking on E-Series Devices

To enable static port trunking from the CLI, you use the trunk command. At the global

configuration level, issue the trunk command followed by a list of the ports that will

be aggregated, a name for the trunk, and the type of trunk (HP trunk or LACP). The

ports need not be contiguous, although the example above shows four contiguous

ports. A list of ports is separated by commas, for example: trunk a1,a7,b1,b24 trk1

LACP.

Note

The 2500 series switches support only one trunk. If the trunk is statically defined,

it will be named ―Trk1.‖

The trunk configuration must be performed on both sides of the trunk before the

redundant links are connected.

Edge_1(config)# trunk ?

[ethernet] PORT-LIST Specify the ports that are to be

added to/removed from a trunk.

Edge_1(config)# trunk c1,c2 ?

Page 36: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –10 Rev 10.41

trk1 Trunk group 1

trk2 Trunk group 2

...

Edge_1(config)# trunk c1,c2 trk1 ?

trunk Do not use any protocol to create or maintain

the trunk.

lacp Use IEEE 802.1ad Link Aggregation protocol.

<cr>

Edge_1(config)# trunk c1,c2 trk1 lacp

The trunk command is used to create an HP port trunk or LACP port trunk

trk1, trk2, etc. are fixed label names for trunks

On the 8100fl series, trunks are referred to as Link Aggregation Groups

Configuring Link Aggregation on A-Series Devices

Static Link Aggregation

1. Create VLAN 10 and aggregate interface 1, and assign the aggregate interface

to VLAN 10.

<DeviceA> system-view

[DeviceA] vlan 10

[DeviceA-vlan10] quit

[DeviceA] interface bridge-aggregation 1

[DeviceA-Bridge-Aggregation1] port access vlan 10

[DeviceA-Bridge-Aggregation1] quit

2. Assign ports GE4/0/1 through GE4/0/3 to link aggregation group 1 and

VLAN 10 one at a time.

[DeviceA] interface gigabitethernet 4/0/1

[DeviceA-Gigabitethernet4/0/1] port link-aggregation group 1

[DeviceA-Gigabitethernet4/0/1] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

[DeviceA-Gigabitethernet4/0/1] quit

[DeviceA] interface gigabitethernet 4/0/2

[DeviceA-Gigabitethernet4/0/2] port link-aggregation group 1

[DeviceA-Gigabitethernet4/0/2] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

Page 37: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –11

[DeviceA-Gigabitethernet4/0/2] quit

[DeviceA] interface gigabitethernet 4/0/3

[DeviceA-Gigabitethernet4/0/3] port link-aggregation group 1

[DeviceA-Gigabitethernet4/0/3] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

[DeviceA-Gigabitethernet4/0/3] quit

3. Configure Device A to perform load sharing based on source and destination

MAC addresses for link aggregation groups.

[DeviceA] link-aggregation load-sharing mode source-mac

destination-mac

Dynamic Link Aggregation

1. Create VLAN 10 and aggregate interface Bridge-aggregation 1, configure the

link aggregation mode as dynamic, and assign the aggregate interface to

VLAN 10.

<DeviceA> system-view

[DeviceA] vlan 10

[DeviceA-vlan10] quit

[DeviceA] interface bridge-aggregation 1

[DeviceA-Bridge-Aggregation1] link-aggregation mode dynamic

[DeviceA-Bridge-Aggregation1] port access vlan 10

[DeviceA-Bridge-Aggregation1] quit

2. Assign ports GE4/0/1 through GE4/0/3 to link aggregation group 1 and

VLAN 10 one at a time.

[DeviceA] interface gigabitethernet 4/0/1

[DeviceA-Gigabitethernet4/0/1] port link-aggregation group 1

[DeviceA-Gigabitethernet4/0/1] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

[DeviceA-Gigabitethernet4/0/1] quit

[DeviceA] interface gigabitethernet 4/0/2

[DeviceA-Gigabitethernet4/0/2] port link-aggregation group 1

[DeviceA-Gigabitethernet4/0/2] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

[DeviceA-Gigabitethernet4/0/2] quit

Page 38: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –12 Rev 10.41

[DeviceA] interface gigabitethernet 4/0/3

[DeviceA-Gigabitethernet4/0/3] port link-aggregation group 1

[DeviceA-Gigabitethernet4/0/3] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

[DeviceA-Gigabitethernet4/0/3] quit

3. Configure Device A to perform load sharing based on source and destination

MAC addresses for link aggregation groups.

[DeviceA] link-aggregation load-sharing mode source-mac

destination-mac

Load Sharing Mode

1. Create VLAN 10.

<DeviceA> system-view

[DeviceA] vlan 10

[DeviceA-vlan10] quit

2. Create aggregate interface Bridge-aggregation 1, configure the source MAC-

based load sharing mode for the link aggregation group, and assign the

aggregate interface to VLAN 10.

[DeviceA] interface bridge-aggregation 1

[DeviceA-Bridge-Aggregation1] link-aggregation load-sharing mode

source-mac

[DeviceA-Bridge-Aggregation1] port access vlan 10

[DeviceA-Bridge-Aggregation1] quit

3. Assign ports GE4/0/1 and GE4/0/2 to link aggregation group 1 and VLAN

10.

[DeviceA] interface gigabitethernet 4/0/1

[DeviceA-Gigabitethernet4/0/1] port link-aggregation group 1

[DeviceA-Gigabitethernet4/0/1] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

[DeviceA-Gigabitethernet4/0/1] quit

[DeviceA] interface gigabitethernet 4/0/2

[DeviceA-Gigabitethernet4/0/2] port link-aggregation group 1

[DeviceA-Gigabitethernet4/0/2] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

Page 39: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –13

[DeviceA-Gigabitethernet4/0/2] quit

4. Create aggregate interface Bridge-aggregation 2, configure the destination

MAC-based load sharing mode for the link aggregation group, and assign the

aggregate interface to VLAN 10.

[DeviceA] interface bridge-aggregation 2

[DeviceA-Bridge-Aggregation2] link-aggregation load-sharing mode

destination-mac

[DeviceA-Bridge-Aggregation2] port access vlan 10

[DeviceA-Bridge-Aggregation2] quit

5. Assign ports GE4/0/3 and GE4/0/4 to link aggregation group 2 and VLAN

10.

[DeviceA] interface gigabitethernet 4/0/3

[DeviceA-Gigabitethernet4/0/3] port link-aggregation group 2

[DeviceA-Gigabitethernet4/0/3] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

[DeviceA-Gigabitethernet4/0/3] quit

[DeviceA] interface gigabitethernet 4/0/4

[DeviceA-Gigabitethernet4/0/4] port link-aggregation group 2

[DeviceA-Gigabitethernet4/0/4] port access vlan 10

Warning: This port is a member of the link aggregation group. If

configuration of the whole group is required to be modified, please

configure it under the aggregation interface view. Otherwise, this

operation may interrupt network traffic.Continue?[Y/N]: y

[DeviceA-Gigabitethernet4/0/4] quit

Page 40: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –14 Rev 10.41

LACP – Link Aggregation Control Protocol Link Aggregation Control Protocol (LACP) is another option for creating ―port trunk

groups‖ on HP switches. LACP is defined by the IEEE standard 802.3ad. LACP was

standardized to allow a switch to automatically recognize coterminous, full duplex,

same-speed links between itself and another LACP-compliant switch.

Although LACP can automatically recognize links that are capable of aggregation,

the activation of an LACP trunk requires some configuration. You can’t simply connect

four links between the same two switches and expect them to act as a trunk.

When using dynamic LACP, you must define the trunk on one side, which is known

as the ―active‖ side. The active side sends Bridge Protocol Data Units (BPDUs) across

every link that has LACP defined statically.

Although a complete description of the fields in the BPDU is beyond the scope of this

course, a few BPDU fields relevant to dynamic operation are worth noting. They are:

A system identifier, which is the switch’s MAC address.

A priority value, which is a permutation of the MAC address.

A port identifier, which contains a port number.

When a switch receives BPDUs through multiple ―passive‖ LACP ports that have the

same system identifier, it knows that those ports are linked to the same switch. If the

links are the same speed, the switch sends BPDUs to the ―active‖ partners on the

other side of the links, and the two switches agree to load share across the group of

links. Passive LACP ports only ―speak‖ when ―spoken‖ to; a passive LACP port sends

BPDUs only after it has received BPDUs from a connected switch.

Configurable LACP States HP switches offer three possible options for LACP configuration:

Passive

Active

Disabled - (default state)

LACP is configured on a per-port basis. When a port is configured for a passive

LACP state, it will be blocked for approximately five seconds when the switch is

initialized. This is appropriate for ports that are linked to active LACP partners

because it provides the ports with time to discover the LACP topology before

forwarding any traffic. However, this delay can be unacceptable for normal switch

operation.

Consequently, HP recommends that LACP remain in the default state of disabled for

all ports that will not participate in dynamic link aggregation.

If you define a trunk using the trunk command described earlier in this module, the

no lacp command is automatically executed and included in the configuration for

Page 41: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –15

the ports specified in the trunk command’s port list. Static and dynamic port trunking

cannot be simultaneously active on the same port.

Finally, is the case of 802.1X (Port-Based Access Control) being configured on a Port.

To maintain security, LACP is not allowed on ports configured for 802.1X

authenticator operation. If you configure port security on a port on which LACP

(active or passive) is configured, the switch removes the LACP configuration, displays

a notice that LACP is disabled on the port(s), and enables 802.1X on that port.

Static vs. Dynamic Link Aggregation One important advantage of dynamic link aggregation is its ability to recognize and

use trunk standby links. When two switches detect more than four coterminous, same

speed links, they aggregate the four links with the lowest port numbers. The

remaining links are used as standby links.

While dynamic LACP is the only way to set up standby links in a trunk, its

disadvantage is that in certain circumstances it can give you less control.

The primary disadvantage of static link aggregation is its lack of support for standby

links. Switches configured for static link aggregation cannot automatically detect new

members of the trunk group and, therefore, cannot use standby links.

On the other hand, static aggregation enables administrators to retain more control

of the operation of the trunk ports.

Page 42: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –16 Rev 10.41

Spanning Tree

Figure 3.1: Spanning tree

Multiple Spanning Tree Protocol (MSTP) enables the configuration of VLAN-aware

Spanning Tree topologies. As described in IEEE 802.1S, multiple spanning trees

allow frames assigned to different VLANs to follow different data routes within

administratively established regions of the network.

In this way, MSTP enables the configuration of Multiple Spanning Trees within a

physical topology, which provides significant improvement in the utilization of

redundant links. Furthermore, the standard notes that an MST configuration probably

will provide simple and full connectivity for frames even in the presence of

administrative errors in the allocation of VLANs to Spanning Trees.

MSTP should not be confused with another VLAN-aware Spanning Tree protocol

known as Per VLAN Spanning Tree (PVST). In PVST configurations, a separate

Spanning Tree instance is created for each VLAN. BPDUs are transmitted with tags

that identify the STP instance and VLAN ID to which they belong. While this enables

the use of redundant links if you apply priorities and costs intelligently, it can be a

CPU-intensive process if there are many VLANs.

MSTP, on the other hand, enables the creation of multiple Spanning Tree instances

that are specifically mapped to VLANs. It is not necessary to literally have a one-to-

one correspondence between Spanning Trees and VLANs. In this way, MSTP

combines the best of two extremes—the single Spanning Tree configurations of STP

and RSTP and the Spanning Tree per VLAN configuration of PVST.

MSTP Features

MSTP is the default protocol when Spanning Tree is enabled

MSTP allows for multiple instances of a redundant path for a set of VLANs within

the bridged network

Each Spanning Tree instance has its own Root Bridge

Traffic is distributed across redundant links

Page 43: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –17

MSTP follows the same basic principles as STP

and RSTP

Compatible and interoperable with STP and RSTP

Emulates STP and RSTP behaviors when encountering switches that do NOT

support MSTP

Because MSTP implements the same basic principles as the earlier Spanning Tree

protocols, it is completely interoperable and compatible with STP and RSTP.

Furthermore, MSTP will emulate STP and RSTP behaviors when encountering devices

that do not support MSTP.

MSTP is the latest iteration of Spanning Tree, and is the default Spanning Tree

protocol on most switches. Check the release notes or manuals for a specific switch to

determine its default.

Comparing RSTP, PVST and MSTP

Table 2.1: Comparing RSTP, PVST and MSTP

Before the release of the MSTP standard, the only IEEE-standardized way to combine

VLANs and Spanning Tree was to resolve loops within the topology without regard to

VLAN configuration.

Cisco Systems Inc. developed PVST—and later PVST+—to enable the configuration of

VLAN-aware Spanning Trees. PVST enables administrators to configure Bridge and

Port Priority settings and path costs so that any two paths between a pair of switches

can both be used. With PVST enabled, some Spanning Tree instances will take one

path while other instances take another path. However, each of the Spanning Tree

instances is separately configured, which results in more overhead than the simpler

Page 44: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –18 Rev 10.41

RSTP solution. Furthermore, the scalability of PVST is limited because of the increased

CPU utilization described earlier in this module.

MSTP, on the other hand, enables the configuration of fewer Spanning Tree

instances, typically between 1 and 16, with each VLAN mapped to the appropriate

instance.

Spanning Tree for Instance 1

Figure 3.3: Multiple spanning tree (1)

With MSTP, Spanning Tree instances are associated with VLAN IDs, not with

individual links. Because a separate Root Bridge is elected for each MST instance,

each instance uses a different set of links as the active path.

As with STP and RSTP, backup—or Blocking State—ports are not used in the primary

active path, but they enable the quick restoration of connectivity in the event of link

failure.

In the graphic above, Edge_1 was elected as the Root Bridge for MST Instance 1,

which resulted in the topology shown. Instance 1 includes VLANs 2 to 10. The next

slide illustrates the Spanning Tree topology for MST Instance 2.

Page 45: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –19

Spanning Tree for Instance 2

Figure 3.4: Multiple spanning tree (2)

In the diagram above, Edge_2 has been elected as the Root Bridge for MST Instance

2. Instance 1 includes VLANs 11 to 20.

Because of this election, the state of the physical links is different than in MST

Instance 1, shown on the previous slide.

MST Regions

A group of switches that collectively define multiple Spanning Tree instances is

known as an MST region

Each switch can belong to only one region

All switches in a region must have identical configuration attributes:

Alphanumeric configuration name

Configuration revision number

Associations between VLANs and Spanning-Tree instances

A switch defines a region boundary if it receives BPDUs from:

A switch with different configuration attributes, or

An STP or RSTP switch

MST Instances Within a Single Switch

When MST is initially enabled, the default conditions are as follows:

Page 46: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –20 Rev 10.41

Each switch defines its MAC address as its configuration name and ―0‖ as

its configuration revision number

All of the VLANs defined on a switch belong to the Internal Spanning Tree

(IST) instance

To cause the switch to interact correctly with other switches in the MST region,

you must define common configuration attributes

Any VLAN not explicitly mapped to a user-defined instance remains associated

with the IST

VLAN 1 is often associated with the IST

Immediately after MSTP is enabled, all the VLANs configured on a switch are part of

the Internal Spanning Tree (IST), which is an RSTP instance that exists within the MST

region. As you add new instances and associate them with VLANs, the VLANs are

removed from the IST. However, the IST remains in place, even if no VLANs are

explicitly mapped to it.

In most cases, user-defined VLANs are associated with user-defined instances

configured identically on all switches in the MST region. The default VLAN (VLAN ID

1) remains associated with the IST. This provides an important benefit: if the VLAN-to-

instance mappings are misconfigured, you can still access the switch because the

IST’s association with VLAN 1 ensures that connectivity is not completely disrupted.

Page 47: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –21

Basic IRF Concepts

Figure 3.5: IRF concepts

The devices that form an IRF virtual device are called IRF member devices. A member

device assumes the role of master or slave. An IRF stack contains only one master,

which manages the IRF virtual device. All other members operate as slaves and as

backups for the master. When the master fails, the IRF virtual device automatically

elects a new master from one of the slaves. Master and slaves are selected through

the role election mechanism. The details of the role election mechanism will be

covered later in this module.

A logical IRF port is a logical port dedicated to the internal connection of an IRF

virtual device. These ports cannot act as access, trunk or hybrid ports. An IRF port is

effective only when it is bound to a physical IRF port.

Physical ports used for connecting members of an IRF virtual device are called

physical IRF ports. Typically, an Ethernet port or optical port forwards frames to the

network. When a physical port is bound to an IRF port, it acts as a physical IRF port

and forwards data traffic such as IRF-related negotiation frames and data traffic

among members.

As shown in the figure above, an IRF stack can have a daisy chain topology or a

ring topology. A ring connection is more reliable than the daisy chain connection. In

a daisy chain topology, the failure of one link can cause the IRF virtual device to

partition into two independent IRF virtual devices, which can disrupt connectivity as

well as IRF functioning. The failure of a link in a ring connection results in a daisy

chain connection, and does not affect IRF services.

Page 48: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –22 Rev 10.41

IRF application scenario: Increasing port density

Figure 3.6: IRF increases port density

IRF provides a simple, cost-effective solution to the issues that arise when use

population exceeds the available network ports. With IRF deployed, you can add

new members to your virtual IRF device, adding port density with minimal

configuration of the new switches.

IRF application scenario: Expanding system processing

capabilities

Figure 3.7: IRF expands system processing capabilities

When the forwarding capability of the core switch cannot satisfy users’ needs, you

can add a switch to form an IRF stacking system with the original core switch. If the

forwarding capability of one switch is 64 Mpps, the forwarding capability of the

whole stack system is 128 Mbps after another switch is added. Note that this

increases the forwarding capability of the entire stacking system, not a single switch.

Page 49: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –23

IRF application scenario: Expanding bandwidth

Figure 3.8: IRF expands bandwidth

You can increase the uplink bandwidth of an edge switch by adding another switch

to form a stacking system with the existing edge switch. You can configure multiple

physical links of the member devices as an aggregation group to increase the

bandwidth of the link to the core switch. In the IRF configuration in the above Figure ,

four links (two from each switch) are aggregated to double the bandwidth from the

edge to the core. Adding a second edge switch without IRF would add more

throughput to the core, but the bandwidth would be divided between the edge

switches and their corresponding clients. To the core switch, the number of edge

switches does not change. The original edge switch will back up the current

configurations to the newly added switch in batches, having minimal effect on

network planning and configuration.

How IRF simplifies networks

Figure 3.9: IRF simplifies networks (1)

Page 50: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –24 Rev 10.41

This network topology provides redundant links between the edge and the

distribution layer. MSTP is required to prevent loops introduced by these redundant

links.

VRRP is a protocol for providing router redundancy. For each of the two segments in

the configuration shown, one router in the distribution layer acts as the master and

does the actual routing and the other acts as a backup. If the master fails, the

backup can take over the routing. In enterprise networks, VRRP is often combined to

add Layer 3 redundancy to the Layer 2 redundancy provided by MSTP.

Figure 3.10: IRF simplifies networks (2)

In this solution, all four of the distribution layer switches are combined into one IRF

stack. All of the switches have the same routing table and can route packets received

from the edge switches. The IRF master will run the routing protocol for the entire

virtual device.

When configured as an IRF stack, the distribution layer switches now act as a single

virtual switch. Loops can still occur, however between an edge switch and the IRF

virtual switch. In order to retain the redundant links between the edge and

distribution layers, the redundant links can be combined in a link aggregation,

creating a single logical link that spans two physical devices in the IRF virtual switch.

Advantages of this topology The IRF topology is simpler to configure and maintain

than the MSTP/VRRP solution. In the IRF implementation, the virtual switch is

configured as if it were a single device. If the same switches were running MSTP and

VRRP, each switch would need a distinctly different configuration to ensure the correct

election of MSTP Root Bridge and VRRP Master. Furthermore, each switch would

need to be configured separately for all routing and switching functions.

Architecture: Operational Planes

Plane Functions

Page 51: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –25

Management

Management interfaces (console, Telnet, SNMP, FTP,

TFTP, etc.) Internal/hardware monitoring: temperature, fan status,

module and power management, etc. File system including: Configuration File

Control

Layer 2 protocols: LACP, RSTP, MSTP Layer 3 Protocols: RIP, OSFP, BGP, ISIS, etc. Routing Table ACLs and QoS Policies

Forwarding FIB (Forwarding Information Base) and Local ACLs and

QoS Policies Frame/packet forwarding and handling

Modern Switches and Routers segregate their functions into different groups called

―operational planes‖ or simply ―planes‖.

Operational Planes in Standalone Switches

Figure 3.11: Operational planes in standalone switches

Modern Switches and Routers segregate their functions into different groups called

operational planes or simply planes.

The most common planes are:

Control Plane: this group includes all internal monitoring and control functions

related to power, temperature, and hardware state in general.

Management plane: this functional group is where the user interface is located

and where and all protocols run, for example STP in Layer 2 and OSPF in layer

3.

It is in this plane that the routing table is built.

Functions in this plane are software based to allow for upgrades.

Forwarding Plane: this group of functions includes L2 and L3 forwarding, packet

filtering and QoS policies.

Page 52: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –26 Rev 10.41

It is in this plane that the routing table is actually used.

Functions in this plane are hardware based because of speed requirements.

Operational Planes in IRFv2

Figure 3.12: Operational planes in IRFv2

In stackable switches, the distribution of these planes is simple: a general purpose

CPU runs the management and control planes and one or two ASICs are in charge

of actual packet processing and forwarding.

In the case of chassis, the management and control plane are centralized in SRPUs

(Switching and Routing Processing Units) and the forwarding plane is distributed in

two or more LPUs (Line Processing Units). All chassis have the option of installing two

SRPUs for redundancy.

Operational Planes in IRFv2

When connecting several units to form an IRF, the management and control planes of

one of the units become active and those of the other units stay in standby.

In the case of chassis, today only two of them can be connected in an IRF. If each

has 2 SRPUs, one of these SRPUs is going to become active and the other three will

stay in standby.

In other words, an IRF system acts like a chassis with centralized management and

control planes and a distributed forwarding plane.

Page 53: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –27

IRF-ports

Figure 3.13: IRF ports

To build an IRF-stack its member devices must be connected. This connection requires

the configuration of IRF-ports. An IRF-port is a logical entity composed of one or more

standard 10GbE ports. In other words, physical 10 GbE ports are bound to an IRF-

port.

By allowing the configuration of standard 10GbE ports as IRF ports, HP offers the

possibility of having:

Local IRF-stacks, in which all members are in the same room

Geographically distributed IRF-stacks

Important: IRF-port 1 can only be connected to IRF-port 2 of the next device in the

IRF-stack.

By allowing the configuration of regular 10GbE ports as IRF ports, H3C offers the

possibility of having:

Local IRF systems, in which all members are in the same room and

Geographically distributed IRF systems, for Data Center redundancy.

Local connections can be built using inexpensive copper cables:

with CX4 and XFP ports, CX4 local connection cables can be used

with SFP+ ports special IRF cables can be used

In both cases, cables of 50, 100 and 300cm are available.

For geographically distributed IRFs, the 10GbE technology required will depend on

the distance.

IRF Member ID

Devices forming an IRF-stack must have a different IRF Member ID. This number

is equivalent to the slot number in a chassis.

Page 54: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –28 Rev 10.41

Switches A5120 and A5500 support dynamic Member ID allocation: when there

is a ―member-id collision‖ one of the devices changes it Member ID

automatically.

In all other A-Series switches the Member ID must be configured manually. This

step is the first step required when building an IRF-stack

By default: IRF Member ID = 1

Page 55: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Rev 10.41 3 –29

Lab 4: VLAN Switching Lab 4 is design to ensure you can use a structured troubleshooting methodology to

resolve VLAN switching problems. There are three trouble tickets in this lab. Refer to

your lab guide for instructions on how to do this lab.

Page 56: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

3 –30 Rev 10.41

This page left blank intentionally.

Page 57: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Rev 10.41 4 –1

Layer 3 (Network Layer) Troubleshooting and

Problem Resolution Module 4

In this module, various layer 3 technologies will be reviewed and common problems

will be discussed.

The technologies include:

IPv4 Routing and Addressing

Inter-VLAN Routing

VRRP

OSPF

iBGP/eBGP

NAT

Page 58: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –2 Rev 10.41

Forwarding between VLANs

Figure 4.1: forwarding between VLANs

As is shown in the example above, IP address 10.1.2.1 with the 24-bit mask

(255.255.255.0) defines a range of local IP addresses between 10.1.2.0 and

10.1.2.255. When using this mask, the first 24 bits of the IP address are recognized

as the "network" portion; the addresses of all the hosts in this range have the same

value in the network portion.

Layer 3 forwarding - host to router

Figure 4.2: Layer 3 forwarding

The router has traditionally been a tool for interconnecting networks. As a layer 3

device, it uses layer 3 information to make forwarding decisions and requires that

Page 59: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –3

each interface leads to a different network. The diagram above illustrates layer 3

forwarding.

When Host 1 wants to talk to Host 2, it first determines whether Host 2 is local to its

own network. Host 1 uses its own IP address and mask to determine the range of

addresses that are local. In the example above, Host 2 is not in the same address

range as Host 1. The local range of Host 1 is 10.1.2.0 – 10.1.2.255.

Since the intended destination is remote, Host 1 sends the traffic to the MAC address

of its configured default gateway, which is a local router interface. All traffic

destined for address ranges other than the local network are directed toward the

default gateway. While Host 1 maintains an ARP cache that contains information

about local hosts, including the default gateway, it has no knowledge of layer 2

addresses on the other side of the router.

Layer 3 forwarding – router to host

Figure 4.3: Forwarding router to host

A router is not transparent to end stations; IP hosts are configured with a local

router's address as a default gateway and they send to the router all traffic destined

for hosts on other networks or subnetworks.

The router performs a lookup operation on the packet's destination IP address

against the entries in a routing table or cache. A successful lookup returns an

outbound interface.

The router performs an ARP cache lookup operation to resolve the layer 2 address of

the destination IP host. In the slide below, the destination host is on a network that is

directly connected to the router. If the destination network is not directly attached to

the router, it sends the packet to another router that leads toward the destination

network.

Page 60: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –4 Rev 10.41

The router encapsulates the outbound IP datagram in a new layer 2 header and

forwards it to Host 2. Unlike the switched frame, which is forwarded without

modification, a routed frame is always changed by the router.

E-Series – Enable routing between VLANs

Figure 4.4: Routing between VLANs

To forward IP traffic between VLANs on the HP 5400zl switch, you need to add the

global configuration level command: ip routing. When you enable routing, the IP

addresses that are defined within the context of the VLANs are used as router

interfaces that provide default gateway service for end stations

The members of the VLANs may be tagged and/or untagged ports. Note that in the

diagram above two of the ports on the routing switch, ports C1 and C2, lead to

switches that support two port-based VLANs. Although these VLANs completely

overlap from the perspective of the 5400zl switch, they have two different IP

addresses; each VLAN has its own IP address that is within the range of the hosts in

that VLAN.

Also note that two ports on the switch, ports C3 and C4, lead to layer 2 switches

whose ports are all within the same VLAN. Although there are multiple physical

ports within this VLAN, there is only one IP address assigned to the group of ports.

All of the hosts within the address range 10.1.4.0/24 will use the same IP address

(10.1.4.1) as their default gateway.

Also note that, although none of the layer 2 switches have active ports in VLAN 1,

they do have an IP address within VLAN 1 for management purposes. The 2524

switches use VLAN 1 for management by default, called the primary VLAN.

A-Series devices have routing enabled by default.

Page 61: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –5

VRRP Basics

Figure 4.5: VRRP basics

Basic default gateway redundancy operation

Common goals for default gateway redundancy methods:

Enable continuity for off-network communication despite the failure of the

primary default gateway

Provide for automatic failover from primary to backup default gateway

within typical session timeout intervals

Common technologies and implementation methods:

Routers use shared IP address (virtual address or interface on one router)

that is the default gateway address for hosts

Backup router takes over forwarding if Master router fails or is unavailable

VRRP: automatic failover for default gateway

Virtual Router Redundancy Protocol (VRRP) provides automatic failover for default

gateways

Specified in IETF RFC 3768

Enables load sharing in designs that coordinate VRRP and MSTP

Provides industry standard for default gateway provisioning

Implemented on all HP Networking E-Series ProVision ASIC switches

VRRP terminology review

A virtual router consists of a set of router interfaces on the same network that

share:

A virtual router identifier (VRID)

A virtual IP address

One router in the group becomes the VRRP Master; other routers are VRRP

Backup(s)

Page 62: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –6 Rev 10.41

The VRRP Master router periodically sends advertisements to a reserved

multicast group address

VRRP Backup routers listen for advertisements and assumes Master role if

necessary

A VRRP router can support many virtual router instances, each with a unique

VRID/IP address combination

Client interacts with virtual router

Figure 4.6: Client interacts with virtual router

Hosts on VRRP-protected networks learn the default gateway’s virtual MAC

address from the Master’s via ARP request

Hosts send all off-network traffic to the local virtual MAC address without

knowing it is not a physical address

Automatic failover

Figure 4.7: Automatic fallover

If the Owner fails, the non-Owner (backup) begins forwarding traffic addressed

to the VRID 2 virtual MAC address (same as the Router 1 virtual MAC address)

Host does not require any configuration changes or session restarts

Host is unaware that a different router is forwarding its off-network traffic

Page 63: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –7

OSPF Basics Benefits

Offers faster convergence than RIP

Scales to meet the needs of very large intranets

Characteristics

OSPF routers advertise the state of connected links

Flood advertisements to neighbors, who flood to other neighbors

Depends on router adjacency, formal relationship used to share routing

information

Intelligent path selection based on bandwidth-sensitive link costs

Divide large domain into smaller areas to enhance efficiency

Careful design can avoid router overload

As described in IP Routing Foundations, OSPF is a sophisticated routing protocol

designed to scale to meet the needs of very large enterprise networks. OSPF offers

several important advantages over the older Routing Information Protocol (RIP),

including faster convergence times as well as scalability.

OSPF uses hierarchical areas to enhance efficiency. By making sound decisions

when defining area borders, network designers can develop routing hierarchies that

scale readily without placing undue load on the routers.

This module will describe the design, deployment, and configuration of OSPF

networking using the E-Series ProVision ASIC switches.

Figure 4.8: OSPF basics

OSPF provides a hierarchical routing structure based on multiple areas

Backbone area (Area 0) required

Page 64: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –8 Rev 10.41

Other area types include stub and NSSA

Router roles:

Area Border Router (ABR)

Autonomous System Boundary Router (ASBR)

As described in IP Routing Foundations, OSPF provides a hierarchical routing

structure that can scale to meet enterprise needs. The graphic, adapted from IRF,

illustrates some basic elements of the OSPF topology.

For more detail, consult IRF.

Enabling OSPF

Figure 4.9: Enabling OSPF

Before enabling OSPF on an IP router, it is advisable to statically define a Router ID.

If no Router ID is configured, the switch will assign one automatically. On the E-

Series ProVision ASIC switches, the choice of ID will depend on other configuration

items. Five possible cases are:

1. A single loopback interface and multiple VLANs with addresses

The loopback interface will be used as ROUTER ID.

2. A single loopback interface with multiple IP addresses

The lowest loopback IP address will be used as Router ID.

3. Multiple loopback interfaces with multiple IP addresses

The lowest loopback number and lowest loopback IP address will be used as

Router ID.

4. Multiple VLANs with a single IP Address in each VLAN

The IP address of the VLAN that becomes active first will be used as a Router ID.

Typically, on E-Series switches, the lowest number VLAN becomes active first.

Consequently, if an address is defined in VLAN 1, it will become the Router ID.

Enabling OSPF

Rev. 10.417

E5406_A

10.1.65.0/30 10.1.67.3024

5406zl_A(config)# router ospf

5406zl_A(ospf)# area 0

Enable OSPF on each VLAN and the loopback interface; area ID defaults to Area 0

Enable OSPF and create Area 0

5406zl_A(ospf)# vlan 10

5406zl_A(vlan-10)# ip ospf [area 0]

5406zl_A(vlan-10)# ip ospf passive

5406zl_A(vlan-10)# vlan 30

5406zl_A(vlan-30)# ip ospf

5406zl_A(vlan-30)# ip ospf passive

5406zl_A(vlan-30)# vlan 65

5406zl_A(vlan-65)# ip ospf

5406zl_A(vlan-65)# vlan 67

5406zl_A(vlan-67)# ip ospf

5406zl_A(vlan-67)# interface loopback 0

5406zl_A(lo-0)# ip ospf all

5406zl_A(config)# ip router-id 10.1.0.3 Define Router ID

Student VLAN 30

Server VLAN 10

Optionally, define stub networks as ―passive‖

Page 65: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –9

If VLAN 1 is down, the switch will use the next lowest number VLAN IP address

as the Router ID.

5. Multiple VLANs with multiple IP addresses in each VLAN

The lowest IP address of the first active VLAN will be used as a Router ID. In

most cases, this will be a default VLAN IP address.

After the ID is defined, two separate commands are required to enable OSPF

globally on the E-Series ProVision ASIC switches. In the first, you simply enable OSPF

by issuing the router ospf command. In the second, you define at least one area.

To form adjacencies, which are fundamental to OSPF operation, two OSPF routers

must agree on an area ID, among other items.

Note that the configuration for the loopback interface must include an argument

specifying which IP addresses will be included in OSPF advertisements. In the

example on the previous page, ―all‖ indicates that all addresses will be included.

Alternatively, the administrator could specify any address configured on the interface

as this argument.

On the E-Series ProVision ASIC switches, configuration of OSPF at the global and

interface level is dynamic. Enabling OSPF on an interface may cause the router to:

1. Begin sending Hello packets through this interface in an effort to establish

adjacencies.

2. Include the network address range associated with this interface in its Router

LSA.

To minimize OSPF processing overhead, interfaces with no neighboring routers, such

as VLANs 10 and 30 in the example on the previous page, may be defined as

―passive.‖ The router does not send Hello messages over a passive interface, which

means it can never form an adjacency and will never send Link State Updates over

this type of interface.

Page 66: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –10 Rev 10.41

Verifying OSPF status

Figure 4.10

After assigning each IP interface to an OSPF area, you can verify the status of

configured OSPF interfaces by issuing the show ip ospf interface command. In

the example shown in the figure above, only the backbone area has been defined,

and all interfaces are associated with the backbone area.

All of these interfaces were configured with default settings for authentication type,

cost, and priority. OSPF interfaces 10.1.10.1/24 and 10.1.30.1/24 were defined as

passive. The ―State‖ column indicates the relationship each OSPF interface has with

neighboring routers. Note that the passive interfaces have the Designated Router

state. The interfaces assume this role even though the router does not expect to find

neighbors on these networks.

This router has a neighbor on the network 10.1.65.0/30, which is indicated in the

output from the OSPF neighbor table. The entry in this table shows the neighbor’s

Router ID, its IP address on the network it shares with E5406_A, and the state of the

neighbor relationship. In this case, the neighbor is the Backup DR of the network

10.1.65.0/30. The next slide will provide more detail on the OSPF neighbors table.

Verifying OSPF Status

– View status of OSPF interfaces

– View state of the router’s neighbors

Rev. 10.418

5406zl_A(config)# show ip ospf interface

OSPF Interface Status

IP Address Status Area ID State Auth-type Cost Pri Passive

----------- --------- --------- ------ --------- ---- --- -------

10.1.0.3 enabled backbone LOOP none 1 1 no

10.1.10.1 enabled backbone DR none 1 1 yes

10.1.30.1 enabled backbone DR none 1 1 yes

10.1.65.2 enabled backbone DR none 1 1 no

10.1.67.2 enabled backbone DR none 1 1 no

5406zl_A(config)# show ip ospf neighbor

OSPF Neighbor Information

Router ID Pri IP Address NbIfState State Rxmt QLen Events

----------- ---- ---------- --------- ------ ---------- ------

10.1.0.1 1 10.1.65.1 BDR FULL 0 6

Page 67: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –11

Viewing OSPF neighbor states

Figure 4.11: OSPF neighbor states

The figure on the previous page showed how information from the OSPF interface

and neighbor tables can be combined to learn the state the router interfaces on a

given network. In the figure above, the neighbor table from a different router,

E8212_A, which has three neighbors. Because all of E8212_A’s neighbors have

Router IDs that are higher than E8212_A’s Router ID, which is 10.1.0.1, all three

neighbors have assumed the role of Designated Router on their respective networks.

If you were to view the OSPF interface table, you would see that E8212_A has the

Backup DR state for the three networks that support its full adjacencies.

As shown, the neighbor table identifies each adjacent router by its Router ID and the

IP address on the interface where the adjacency has formed. The table also

indicates each neighbor’s priority and state. Use the OSPF neighbor table to

troubleshoot routing problems that may arise from the failure to form an adjacency.

Viewing OSPF neighbor states

– E8212_A has full adjacency with one

neighbor on each of the following

networks:

• 10.1.64.0/30

• 10.1.65.0/30

• 10.1.68.0/30

– With equal interface priorities, the OSPF

router with the highest router ID becomes

the Designated Router

Rev. 10.419

E8212_A(config)# show ip ospf neighbor

OSPF Neighbor Information

Router ID Pri IP Address NbIfState State Rxmt QLen Events

--------------- --- --------------- --------- -------- --------- ----------

10.1.0.2 1 10.1.64.2 DR FULL 0 6

10.1.0.3 1 10.1.65.2 DR FULL 0 6

10.1.0.4 1 10.1.68.2 DR FULL 0 7

10.1.65.0/30

E5406_A10.1.0.3

E5406_B10.1.0.4

E8212_A10.1.0.1

E8212_B10.1.0.2

10.1.64.0/30

10.1.67.0/30 10.1.68.0/30

10.1.66.0/30

Page 68: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –12 Rev 10.41

External and internal Border Gateway Protocol

(BGP) BGP uses the Transmission Control Protocol (TCP) as its transport protocol, using port

179 for establishing connections. Running over a reliable transport protocol

eliminates the need for BGP to implement update fragmentation, retransmission,

acknowledgment, and sequencing.

The Internet is organized in a multitude of administratively independent networks

called domains or Autonomous Systems (AS). For example, an AS can be an Internet

Service Provider (ISP), a University campus or a corporate network.

The Border Gateway Protocol is an inter-Autonomous System routing protocol. The

primary function of a BGP speaking system is to exchange network reachability

information with other BGP systems. This network reachability information includes

information on the list of Autonomous Systems (ASs) that reachability information

traverses. This information is sufficient to construct a graph of AS connectivity from

which routing loops may be pruned and some policy decisions at the AS level may

be enforced

The route to each destination is called the AS path, and the additional route

information is included in path attributes. BGP uses the AS path and the path

attributes to completely determine the network topology, detect and eliminate routing

loops and it can enforce administrative preferences and routing policy decisions.

Contrasting eBGP and iBGP

BGP-4 provides a new set of mechanisms for supporting CIDR. These mechanisms

include support for advertising an IP prefix and they eliminate the concept of network

"class" within BGP. BGP-4 also introduces mechanisms which allow aggregation of

routes, including aggregation of AS paths.

Once BGP speakers are connected they exchange messages to start a BGP session

with a neighbor. This initial message identifies the sender’s AS number and BGP

identifier. Dependent upon whether the two speakers are in the same AS or different

will govern the session type. There are two basic session types for BGP, interior and

exterior.

While there are many similarities between exterior and interior BGP, the most

important difference is that the BGP speakers in an interior BGP peer session are in

the same AS. Interior BGP is used within a transit AS, as is shown in the diagram

below.

Page 69: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –13

Figure 4-12: Contrasting eBGP and iBGP

Note that BGP routers at the "edge" of a domain will support both interior BGP

peers and exterior BGP peers.

BGP messages and route selection

Routers send open messages to each other to open or establish a BGP connection.

The two routers must first establish a TCP connection between them. After which the

sending of the Open Message is bidirectional.

Routers send Open Messages out and wait until they receive an Open Message from

their peer before continuing. Once the BGP peer is established, routers can

exchange routing information. This routing information is contained in Update

Messages

Once the connection has been established, the routers send incremental updates that

include summarized address ranges and AS numbers. (Messages vary somewhat

dependent upon whether they are between interior or exterior BGP speakers.) They

also send ―keepalives‖ to maintain the session. The router builds a graph or table of

the destinations and the attributes. BGP uses the AS or Autonomous System number

to select the shortest path to route data and avoid routing loops.

The two routers use UPDATE messages to add new routes, replace existing routes,

withdraw invalid routes, and communicate attributes.

BGP Notification Messages are an error message. The router selects the error type,

and puts it into the Notification Message and sends it to the peer. It then tears down

the peer connection

Notification Messages consist of multiple pieces, including the BGP header, error

code, error sub-code, and data that describes the error. This is important as it helps

the Notification Message recipient router to troubleshoot BGP peering problems

Page 70: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –14 Rev 10.41

Network Address Translation (NAT)

Figure 4.13: Network address translation (NAT)

Network Address Translation (NAT) was originally created as a solution to the limited

number of public IP addresses. Internet Protocol version 4 (IPv4) uses four octets (32

bits) of address space—which does not provide enough IP addresses for the current

demand—and IPv6 is not yet widely implemented. NAT can provide an alternative

to obtaining a large block of registered addresses. With NAT implemented on the

network, a company does not need a public IP address for each of its computers.

NAT uses a device (a router, firewall, or computer) as an agent between the trusted

network and the untrusted network. When a packet destined for the untrusted

network reaches this device, the sender’s private IP address is translated into either

the company’s one public IP address or one of a limited range of such addresses

assigned to that company.

NAT also provides security: you give away nothing about your company’s internal

network if you use NAT when communicating with untrusted networks. The NAT-

enabled device adds an entry to its address translation table that maps the internal

address it replaced with the new public IP address. When the destination computer

sends a reply packet back through the router, the router uses the table to identify the

original internal IP address and sends the reply back to the appropriate computer on

the trusted network.

The following sections discuss the various types of NAT technology available. These

include single IP address translation, static NAT, dynamic NAT, Port Address

Translation (PAT), and NAT Traversal (NAT T).

Page 71: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –15

Single IP Address Translation

Figure 4.12: Single address translation

Single IP address translation allows one public IP address to be used by a full IP

network. In this version of NAT, the available port numbers of the NAT-enabled

gateway (router) are assigned to different private IP addresses. This allows multiple

simultaneous TCP/IP sessions to occur using only the router’s public IP address.

How It Works

When an internal computer sends a packet (containing the source IP address, source

port, destination IP address, and destination port), the packet must travel through the

NAT-enabled router. At this point, the router rewrites the packet header so that it

contains the router’s public IP address instead of the source IP address. The router

then encapsulates the package to send to its destination.

When the router rewrites the packet, it adds an entry into the address translation

table that maps the internal address it replaced with its own public IP address. When

the destination computer sends a reply packet back through the router, the router

identifies its original internal IP address from the address translation table and sends

the reply back to the appropriate computer. The above figure illustrates this process.

Page 72: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –16 Rev 10.41

Static and Dynamic NAT

Figure 4.13: Static and dynamic NAT

Static NAT maps an internal IP address to a public IP address on a one-to-one basis.

That is, static NAT will always assign a particular computer the same public IP

address. For example, it will always assign the computer with IP address

192.168.45.10 the public IP address 213.18.121.110.

Dynamic NAT maps an internal IP address to a public IP address from a range of

public addresses assigned to that company. A computer on the trusted network is

dynamically assigned a random IP address depending on which addresses are

available at a given time. For example, NAT can assign a computer public IP

address 213.18.121.110 one time and then assign that same computer IP address

213.18.121.116 the next time that computer tries to send a packet to the untrusted

network.

Static NAT is particularly useful when a device needs to be accessible from outside

the network. Conversely, implementing dynamic NAT automatically creates a firewall

of sorts between a company’s internal network and untrusted networks: NAT only

allows connections that originate from the trusted network. Essentially, this means

that a computer in an untrusted network cannot connect to a computer in the trusted

network unless the trusted host initiates contact first.

Page 73: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –17

Lab 5: Layer 3 Practice and Tools Lab 5 is design to ensure you can use the troubleshooting methodology and

troubleshooting tools. There are three trouble tickets in this lab. Refer to your lab

guide for instructions on how to do these labs.

Page 74: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –18 Rev 10.41

Lab 6: OSPF Routing Issues Lab 6 is designed to ensure you can use a structured troubleshooting methodology to

resolve OSPF routing problems. There is one trouble ticket in this lab. Refer to your

lab guide for instructions on how to do this lab.

Page 75: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Rev 10.41 4 –19

Lab 7: Addressing Issues Lab 7 is design to ensure you can use a structured troubleshooting methodology to

resolve IP addressing problems. There is one trouble ticket in this lab. Refer to your

lab guide for instructions on how to do this lab.

Page 76: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

4 –20 Rev 10.41

Lab 8: Inter-VLAN and Routing Lab 8 is design to ensure you can use a structured troubleshooting methodology to

resolve inter-VLAN routing issues. There is one trouble ticket in this lab. Refer to your

lab guide for instructions on how to do this lab.

Page 77: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Rev 10.41 5 –1

Layer 4 (Transport Layer) Troubleshooting and

Problem Resolution Module 5

This module focuses on troubleshooting at the transport layer 4. Upper layer

protocols such as TCP, UDP, HTTP, FTP and Telnet run on top of the IP layer 3.

Figure 5.1: The 5 layer IETF model

In the course the five Layer IETF model is used to describe a layered approach to

networking. The TCP/IP model consists of four Layers. Even though there are some

architectural differences, both models have interchangeable transport and network

layers and their operation is based upon packet-switched technology.

Page 78: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

5 –2 Rev 10.41

Troubleshooting TCP/UDP The Host-to-Host (Transport) Layer contains two protocols; Transmission Control

Protocol (TCP) and User Datagram Protocol (UDP). TCP and UDP are used to

transmit datagrams.

Figure 5.2: Contrasting TCP and UDP

Below is a description of major differences between TCP and UDP.

Reliable/Connection-Oriented

TCP is a connection-oriented protocol. When a file or message send it will get

delivered unless connections fails. If connection lost, the server will request the

lost part. There is no corruption while transferring a message.

Unreliable/connectionless

UDP is connectionless protocol. When you a send a datagram or message, you

don't know if it'll get there, it could get lost on the way. There may be

corruption while transferring a message

Ordered

Each message is sent with a sequence number, so that even if they arrive out of

order, they can be reassembled in the correct order.

Not Ordered

If you send two messages out, and they arrive out of order, the application itself

would be responsible for reassembly in the proper order.

Heavyweight

When the low level parts of the TCP "stream" are lost, resend requests have to

be sent, and all the out of sequence parts have to be put back together, so

requires a bit of work to piece together.

Lightweight

No ordering of messages, no tracking connections, etc. This means it's a lot

quicker, and the network card / OS have to do very little work to translate the

data back from the packets.

Streaming

Page 79: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Rev 10.41 5 –3

Data is read as a "stream," with nothing distinguishing where one packet ends

and another begins.

Datagram

Packets are sent individually and are guaranteed to be whole if they arrive.

The TCP header occupies quite a large space in the Ethernet frame.

Figure 5.3: TCP message segment format

Source Port: 16 bits - The source port number.

Destination Port: 16 bits - The destination port number.

Sequence Number: The sequence number of first data octet in the segment

(except when SYN is present). If SYN is present the sequence number is the

initial sequence number (ISN) and the first data octet is ISN+1.

Acknowledgment Number: If the ACK control bit is set this field contains the

value of the next sequence number the sender of the segment is expecting to

receive. Once a connection is established this is always sent.

Data Offset: The number of 32 bit words in the TCP Header. This indicates

where the data begins. The TCP header (even one including options) is an

integral number of 32 bits long.

Reserved: 6 bits - Reserved for future use. Must be zero.

Flags: 6 bits and contains:

URG: Urgent Pointer field significant

ACK: Acknowledgment field significant

PSH: Push Function

RST: Reset the connection

SYN: Synchronize sequence numbers

FIN: No more data from sender

Page 80: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

5 –4 Rev 10.41

Window: 16 bits - The number of data octets beginning with the one indicated

in the acknowledgment field which the sender of this segment is willing to

accept.

Checksum: 16 bits

The TCP Length: The TCP header length plus the data length in octets (this is not

an explicitly transmitted quantity, but is computed).

Urgent Pointer: This field communicates the current value of the urgent pointer as

a positive offset from the sequence number in this segment. The urgent pointer

points to the sequence number of the octet following the urgent data. This field

is only be interpreted in segments with the URG control bit set.

Options: Options may occupy space at the end of the TCP header and are a

multiple of 8 bits in length. All options are included in the checksum. An option

may begin on any octet boundary. There are two cases for the format of an

option:

A single octet of option-kind.

An octet of option-kind, an octet of option-length, and the actual option-

data octets.

The option-length counts the two octets of option-kind and option-length as well

as the option-data octets. Note that the list of options may be shorter than the

data offset field might imply.

Data: variable - The actual user data is included after the end of the header

To troubleshoot TCP and UDP it is often necessary to analyze TCP segments using a

network analyzer tool such as Wireshark. The TCP Packet capture shown in the

figure below is a request-response message sequence carried over TCP.

Notice the fields discussed above: Source Port, Destination Port, Sequence number,

Window size, Flags, Checksum and options.

Page 81: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Rev 10.41 5 –5

Figure 5.4: TCP packet capture

UDP does not ensure that the data bytes sent will arrive at the other site. Thus, UDP

imposes less network overhead than TCP.

Source Port: The 16-bit port number of the process that originated the UDP

message on the source device. This will normally be an ephemeral (client) port

number for a request sent by a client to a server, or a well-known/registered

(server) port number for a reply sent by a server to a client.

Destination Port: The port number of the process that is the ultimate intended

recipient of the message on the destination device. This will usually be a well-

known/registered (server) port number for a client request, or an ephemeral

(client) port number for a server reply.

Length: The length of the entire UDP datagram, including both header and Data

fields.

Checksum: An optional checksum computed over the entire UDP datagram plus

a special “pseudo header” of fields. See below for more information.

Data: The encapsulated higher-layer message to be sent.

Page 82: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

5 –6 Rev 10.41

Figure 5.5: UDP message segment format

Below is a picture of a packet capture of the UDP section of the Ethernet frame. Note

that the UDP packet capture shows the Source port, Destination port, Length and

Checksum

Figure 5.6: UDP packet capture

Page 83: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Rev 10.41 5 –7

Firewalls Layer 4 protocols are subject to packet filters and firewalls. It is possible to have IP

connectivity between the network components but certain packets are unable to

traverse between a source and destination address. These types of connectivity

issues may cause by problems with:

Firewalls

Packet filters

Servers

Authentication and authorization

Application software interoperability

Operating system interoperability

In this section we are going to look at troubleshooting firewall and packet filter

issues.

Firewall configurations

You have many options when deciding where or how to implement your firewall. The

configuration typically includes a combination of routers, gateways, and servers on

the edge of a trusted network. Firewalls can be configured in (but are not limited to)

the following architectures shown in the picture below.

Figure 5.7: Firewall configurations

Page 84: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

5 –8 Rev 10.41

Denying or permitting packets

A firewall is a collection of components configured to enforce a specific access

control policy between your internal (trusted) network and any other (untrusted)

network. As the above figure shows, a firewall protects your company’s internal

network from the Internet.

A firewall filters incoming and outgoing packets to ensure only authorized packets

pass. You must set up a clearly defined security policy that delineates authorized

traffic. For example, you can configure rules in which the firewall drops packets from

specific untrusted servers that you identify by IP address.

Essentially, you can use one of two principles when implementing rules for your

company’s firewall:

Deny everything except that which is explicitly permitted

Permit everything except that which is explicitly denied

Page 85: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Rev 10.41 5 –9

Firewall types Firewalls fall into one or more of the following categories:

Packet-filtering firewall:

Must establish a predefined table of rules against which a packet-filtering

firewall compares the full association of the packets.

Must specify which packets should be accepted and which denied.

Can create rules that will drop packets from specific untrusted servers, which you

identify by IP address.

Can also create rules that permit particular types of connections (such as FTP

connections) only if they are using the appropriate trusted servers (such as the

FTP server).

Circuit-level gateway

Acts as a proxy server to establish a circuit with the internal computers.

All outgoing packets from the trusted clients appear to have the proxy server’s

source IP.

After a connection is established, the circuit-level gateway simply copies and

forwards packets back and forth without filtering them further.

Application-level gateway

Acts as a proxy server between a trusted client and an untrusted host.

Only accept packets generated by services they are designed to copy, forward,

and filter.

For example, only a telnet proxy can copy, forward, and filter telnet traffic.

Stateful-inspection firewall

Combines all of the above.

Filtering all incoming and outgoing packets based on source and destination IP

addresses and port numbers.

Ensuring packets in a session are appropriateEvaluates the contents of each packet

up through the application layer and ensures that these contents match the rules

in your company’s network security policy.

Algorithms compare packets against known bit patterns of authorized packets.

Page 86: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

5 –10 Rev 10.41

Table 5.1: Contrasting firewall types

Few firewalls belong in only one of these categories, and fewer still exactly match the

definition for any one category. These categories, however, do reflect the key

capabilities that differentiate one firewall from another.

Figure 5.8: Stateful-inspection firewalls

In a specific firewall implementation, the various types can be combined to create

complex, sophisticated solutions. For example, a dual-homed host can be either a

circuit-level gateway or an application-level gateway. A screened subnet includes at

least two packet-filtering firewalls.

Page 87: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Rev 10.41 5 –11

Network address translator (NAT) There are various types of NAT technology available. These include

Single IP address translation

Static NAT and dynamic NAT

Port Address Translation (PAT)

NAT Traversal (NAT T)

Network address translation (NAT) was discussed in an earlier module. This module

extends this discussion to include Port Address Translation (PAT).

Often, a company’s global address pool does not contain enough public IP

addresses to ensure all hosts in the trusted network can be mapped to an Internet

address when they need to be. In this situation, the company should implement Port

Address Translation (PAT). PAT maps each host in the trusted network to a global IP

address and also to a unique TCP or UDP port number on the NAT-enabled router.

In this way, PAT can map the same global IP address to a number of private IP

addresses; it uses the unique port number to distinguish between them.

Figure 5.9: Port address translation (PAT)

The router stores the original IP address and port against the new IP address and

port in the address translation table. When the destination computer on the

untrusted network sends a reply packet back through the router, the router identifies

the recipient on the trusted network using the address translation table and routes the

packet appropriately.

Enabling PAT NAT

PAT must be enables when you configure NAT, that is:

Configure a basic or advanced ACL for each range of private addresses for

which you want to provide NAT.

Configure a pool for each consecutive range of Internet addresses to which you

want NAT to be able to map the private addresses specified in the ACLs. Each

pool must contain a range with no gaps. If your Internet address space has

Page 88: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

5 –12 Rev 10.41

gaps, configure separate pools for each consecutive range within the address

space.

Associate a range of private addresses (specified in a basic or advanced ACL)

with a pool.

Enable the Port Address Translation feature if you have more private addresses

that might need NAT than the Internet address pools contain.

Enable outbound NAT on the interface connected to global addresses. The following

commands configure a basic ACL for the private subnet 10.10.10.x/24, then enable

inside NAT for the subnet. This example has Port Address Translation Enabled.

# acl number 2001

rule permit source 10.10.10.0 0.0.0.255

#

nat address-group 1 209.157.1.2 209.157.1.254

)# interface Serial 5/0

nat outbound 2001 address-group 1

Page 89: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Rev 10.41 6 –1

Layer 5 (Application Layer) Troubleshooting and

Problem Resolution Module 6

In this module, the common issues around troubleshooting application layer 5 issues

will be reviewed and common problems will be discussed. The most common

application layer problems evolve around QoS. The focus of this module is therefore

around QoS.

Page 90: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

6 –2 Rev 10.41

QoS process flow QoS attributes include:

QoS parameters

Congestion

QoS mechanisms

Switch QoS configurations

Traffic handling techniques

QoS policies

QoS parameters

The reason that networks exist is to enable users to access and run their

applications. Applications include web browsing, file transfers, video streaming,

email exchange, and voice conversations. These applications have different

Quality of Service requirements, where Quality of Service defines the level of

service that the application requires from the network. Quality of Service (QoS)

parameters may include minimum data rates, packet error rates, jitter and latency.

When making a QoS scheme, a network administrator must consider the

characteristics of various applications to balance the interests of diversified users

and fully utilize network resources.

In addition, enterprises today are experiencing increased voice and video traffic

over their networks, and many have fully migrated their voice traffic from a

separate PBX network to run over their IP networks. Voice and video have different

network requirements such that the voice and video quality will be perceived

acceptable by the users:

Video and voice are both sensitive to jitter.

The variation in intervals between the arrival of packets.

Can cause dead spots in real-time transmission.

Voice is sensitive to delay, sometimes called “latency”.

Relates to the amount of time that passes between the sending of a

transmission and its arrival at the receiving station.

Switches and routers can be configured to support these QoS needs.

Congestion

When the rate at which traffic arrives at a device exceeds the rate that the

devicecan forward the traffic on a specific interface then congestion occurs. As

such the interface that forwards packets is a basic network resource.

TCP applications such as web browsing can tolerate congestion

Page 91: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Rev 10.41 6 –3

Acknowledgement and flow-control mechanisms

Lost packets retransmitted

Back off procedure when congestion is detected

UDP applications such as voice and video are more susceptible

No acknowledgement or flow control at the transport layer

Applications may provide acknowledgement and flow control

No back off when congestion is detected

Single application might monopolize link

Queuing processes

Congestion management uses queuing and scheduling algorithms to classify and

sort traffic leaving a port. Each queuing algorithm addresses a particular network

traffic problem, and has a different impact on bandwidth resource assignment,

delay, and jitter. Queue scheduling processes packets by priority, and

preferentially forwards high-priority packets. Queuing processes include:

Strict Priority (SP) queuing.

SP queuing is specially designed for mission-critical applications, which must be

served first to reduce response delays when congestion occurs. SP queuing

classifies eight queues on an A-Series switch port into eight classes, numbered 7

to 0 in descending priority order.

SP queuing schedules the eight queues strictly according to the descending

order of priority. It sends packets in the queue with the highest priority first.

When the queue with the highest priority is empty, it sends packets in the queue

with the second highest priority, and so on. Thus, you can assign mission-critical

packets to the high priority queue to ensure that they are always served first and

common service packets to the low priority queues and transmitted when the

high priority queues are empty.

The disadvantage of SP queuing is that packets in the lower priority queues

cannot be transmitted if there are packets in the higher priority queues. This may

cause lower priority traffic to be starved and never be transmitedt.

Weighted Round Robin (WRR) queuing

WRR queuing schedules all the queues in turn to ensure that each can be served

for a certain time. Assume there are eight output queues on a port. WRR assigns

each queue a weight value (represented by w7, w6, w5, w4, w3, w2, w1, or

w0) to decide the proportion of resources assigned to the queue. On a 100

Mbps port, you can configure the weight values of WRR queuing to 5, 5, 3, 3,

1, 1 1, and 1 (corresponding to w7, w6, w5, w4, w3, w2, w1, and w0

respectively). In this way, the queue with the lowest priority is assured, thus

avoiding the disadvantage of SP queuing that packets in low-priority queues

may fail to be served for a long time.

Page 92: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

6 –4 Rev 10.41

Another advantage of WRR queuing is that while the queues are scheduled in

turn, the service time for each queue is not fixed, that is, if a queue is empty, the

next queue will be scheduled immediately. This improves bandwidth resource

use efficiency.

Weighted Fair Queuing (WFQ)

The only difference between WFQ and WRR is that: WRR schedules certain

number of packets from a queue in each cycle of scheduling, while WFQ

schedules certain number of bytes from a queue in each cycle of scheduling.

Additionally, WFQ can work with the minimum guaranteed bandwidth

mechanism. You can configure a minimum guaranteed bandwidth for each

WFQ queue, so that each WFQ queue is guaranteed of the bandwidth when

congestion occurs. The assignable bandwidth (total bandwidth – the sum of the

minimum guaranteed bandwidth for each queue) is allocated to queues based

on queue priority.

Because WFQ can balance delay and jitter among congested flows, it can be

applied in certain special scenarios. For example, WFQ is used for the assured

forwarding (AF) services of the Resource Reservation Protocol (RSVP). In Generic

Traffic Shaping (GTS), WFQ schedules buffered packets.

SP+WRR queuing.

By assigning some queues on the port to the SP scheduling group and the others

to the WRR scheduling group (group 1), you implement SP + WRR queue

scheduling on the port. Packets in the SP scheduling group are scheduled

preferentially. When the SP scheduling group is empty, packets in the WRR

scheduling group are scheduled. Queues in the SP scheduling group are

scheduled with the SP queue scheduling algorithm. Queues in the WRR

scheduling group are scheduled with WRR.

QoS mechanisms

QoS mechanisms enable network administrators to manage the use of network

resources, enabling mission critical applications to receive priority access to

network resources over lower priority traffic.

Traffic arriving at the device is separated into flows via a process referred to as

“Classification.”

Classification

Recognize traffic that should be prioritized

Assign an internal traffic class (internal forwarding priority)

The device maps priority values to its internal queues and forwards appropriately.

If transmitting host does not mark its own traffic, devices can apply policies to

inbound traffic

Marking

Page 93: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Rev 10.41 6 –5

Indicates within the header how traffic should be handled – for the

benefit of other devices

Layer 2 marking: IEEE 802.1p

Layer 3 marking: IP Precedence or Differentiated Services Code

Point (DSCP)

Scheduling algorithms determine the packets and the rate of the packets that will

be forwarded on the interface.

Scheduling / traffic shaping

Place traffic in queues based on traffic class

Allocate sufficient percentage of outbound bandwidth for high priority

traffic

Figure 6.1: QoS mechanism

Switch QoS configuration

As such when configuring the switch to provide QoS to application traffic, you

must configure the QoS interface parameters including:

Classifications

Behavior

Number of queues

Traffic filtering

You can filter in or filter out a class of traffic by associating the class with a traffic

filtering action. For example, you can filter packets sourced from a specific IP

Page 94: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

6 –6 Rev 10.41

address according to network status. By using ACL rules configured with a time

range for traffic classification, you can implement time-based traffic filtering.

Class of Service (CoS) is:

The process of classifying traffic based on:

Layer 2: IEEE 802.1p

Layer 3: IP Precedence or DSCP

A classification method only

A tool used by scheduling (queuing) mechanisms to limit delay

To illustrate traffic filtering, below is an example configuration for a host connected

to interface GigabitEthernet 1/0/1 of the switch. The requirement is to configure

traffic filtering to filter the packets whose TCP source port number is 21 received on

the interface

# Create advanced ACL 3000, and configure a rule to match packets whose

source TCP port number is 21.

[DeviceA-acl-basic-3000] rule 0 permit tcp source-port eq 21

[DeviceA-acl-basic-3000] quit

# Create a class named classifier_1, and use ACL 3000 as the match criterion in

the class.

[DeviceA] traffic classifier classifier_1

[DeviceA-classifier-classifier_1] if-match acl 3000

[DeviceA-classifier-classifier_1] quit

# Create a behavior named behavior_1, and configure the traffic filtering action

for the behavior to

drop packets.

[DeviceA] traffic behavior behavior_1

[DeviceA-behavior-behavior_1] filter deny

[DeviceA-behavior-behavior_1] quit

# Create a policy named policy, and associate class classifier_1 with behavior

behavior_1 in the policy

[DeviceA] qos policy policy

[DeviceA-qospolicy-policy] classifier classifier_1 behavior behavior_1

[DeviceA-qospolicy-policy] quit

# Apply the policy named policy to the incoming traffic on interface

GigabitEthernet 1/0/1.

[DeviceA] interface gigabitethernet 1/0/1

Page 95: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Rev 10.41 6 –7

[DeviceA-GigabitEthernet1/0/1] qos apply policy policy inbound

Page 96: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

6 –8 Rev 10.41

802.1p traffic prioritization Traffic handling techniques generally involve the host sending time-sensitive traffic

modifies bits in packet header. Examples of protocols that do this include:

DSCP or IP Precedence: Priority field within IP datagram header

IEEE 802.1p: Priority field within 802.1Q tag

802.1p is a layer 2 marking that is used in many LANs. 802.1p defines a field in

the MAC Ethernet header that carries one of eight priority values as shown in the

picture below.

IEEE 802.1p reserves a three-bit field in the 802.1Q tag

Some end stations set priorities for their traffic

Figure 6.2: 802.1p priority tag

Switches can retain or modify markers for prioritized traffic forwarded over tagged

links.

The table below provides an example of parameters that can be configured in a E-

Series switch.

Minimum percentages shown below are configurable per port

If all waiting traffic has the same priority level (e.g. normal) in a given time

period, 100% of the bandwidth is given to that traffic.

Page 97: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Rev 10.41 6 –9

Table 6.1: Illustration of 802.1p switch settings

Configuring QoS policy

Switches can act as QoS policies enforcement points (PEP) to control access. PEPs

determine whether traffic can be admitted.

Figure 6.3: Configuring QoS

A QoS policy can be applied to:

An interface, the policy takes effect on the traffic sent or received on the

interface.

Page 98: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

6 –10 Rev 10.41

A user profile, the policy takes effect on the traffic sent or received by the online

users of the user profile.

A VLAN, the policy takes effect on the traffic sent or received on all ports in the

VLAN.

Globally, the policy takes effect on the traffic sent or received on all ports.

Default QoS example

The picture below shows an example of the E-Series switch supporting both a data

VLAN and a voice VLAN.

Figure 6.4: Default QoS on A-Series 3500 switches

Page 99: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Rev 10.41 6 –11

Traffic marking by an end station Many IP phones mark their traffic for high-priority handling. In this illustration:

1. Phone marks priority level in IEEE 802.1Q tag

2. The edge switch

a. Classifies traffic based on priority marker in tag

b. Schedules packet for delivery by placing it in queue associated with

traffic class

Figure 6.5: IP phone illustration

Display the QoS policy applied to VLAN

Below is an example of how to display the parameters of VLAN 2.

# show qos vlan-policy vlan 2

VLAN priorities

VLAN ID Apply rule | DSCP Priority

------- ----------- + ------ -----------

1 No-override | No-override

500 No-override | No-override

Page 100: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

6 –12 Rev 10.41

Retaining priority between VLANs Continuing the previous example:

3. The core switch classifies traffic based on priority marker in tag

4. The core switch

a. Marks priority in 802.1p field of outbound packet‟s 802.1Q tag

b. Schedules packet for delivery by placing it in appropriate queue

c. Classifies and schedule delivery

Figure 6.6: Continuation of IP phone illustration

Configuring port priority

Below is an example of how port priorities can be set per VLAN or per interface.

# vlan 500

(vlan-500)# qos priority

(vlan-500)# qos

dscp Specify DSCP policy to use.

priority Specify priority to use.

(vlan-500)# qos priority

0

1

2

3

Page 101: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Rev 10.41 6 –13

4

5

6

7

# interface A1

(eth-A1)# qos priority

(eth-A1)# qos

dscp Specify DSCP policy to use.

priority Specify priority to use.

(eth-A1)# qos priority

0

1

2

3

4

5

6

7

Page 102: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

6 –14 Rev 10.41

Normal priority data traffic This last illustration show the normal priority data traffic. In this example the edge

switch uplink (port 50) is a tagged member of VLAN 10; 802.1p field in tag contains

“0” value.

The steps are:

1. The user‟s data traffic is sent untagged, with no priority marker

2. The edge switch

a. Classifies the traffic as „normal‟

b. Then marks “0” value in 802.1p field of outbound packet‟s 802.1Q tag

c. Schedules packet for delivery, assigning it to the queue associated with

“normal” traffic

Figure 6.7: Normal priority data traffic

Page 103: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Rev 10.41 6 –15

Lab 10: Quality of Service Lab 10 is designed to ensure you can use a structured troubleshooting methodology

to resolve Quality of Service problems. There is one trouble ticket in this lab. Refer to

your lab guide for instructions on how to do this lab.

Page 104: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

6 –16 Rev 10.41

This page left blank intentionally.

Page 105: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Rev 10.41 7 –1

Troubleshooting an End-to-End Complex,

Integrated Multi-Protocol Network Module 7

This module brings all the lessons from the previous modules and challenges you to

resolve a complex multi-protocol problem.

Stable network operations are critical to most enterprises. Failure of the network

results in productivity and revenue losses. Troubleshooting multiprotocol networks

can be complex and formidable, however following a structured approach diagnosis

and resolution can help resolve problems quickly and effectively.

In this lab you will solve a trouble ticket that has several problems. To do this lab,

you should use a structured approach to troubleshooting and document your steps.

Page 106: Learner Guide Troubleshooting HP Networks 1041 No Watermark

Troubleshooting HP Networks

7 –2 Rev 10.41

Lab 11: Final lab Lab 11 is designed to ensure you can use a structured troubleshooting methodology

to resolve problems at multiple protocol layers. There is one trouble ticket in this lab

that contains several problems. Refer to your lab guide for instructions on how to do

this lab.

Page 107: Learner Guide Troubleshooting HP Networks 1041 No Watermark
Page 108: Learner Guide Troubleshooting HP Networks 1041 No Watermark

To learn more about HP Networking, visit

www.hp.com/networking

© 2010 Hewlett-Packard Development Company, L.P. The information contained herein is

subject to change without notice. The only warranties for HP products and services are set forth

in the express warranty statements accompanying such products and services. Nothing herein

should be construed as constituting an additional warranty. HP shall not be liable for technical

or editorial errors or omissions contained herein.