netflow(6500 specific)

© Cisco Systems 2005

Netflow Technical Updateon the Catalyst 6500 Carl SolderTechnical Marketing Engineer, ISBU



NetflowIntroduction - Where are we at today?

• Tracks statistics for traffic flows through the system• IPv4 statistics entries created in NetFlow table when new flows

start• IPv6 flows are created in Netflow table but cannot be exported• For Exported records, flow removal is timer based• Full collection by default

–Also support sampled NetFlow • Flow statistics can be exported using NetFlow Data Export (NDE)

– NetFlow v5 and v7– NetFlow aggregation with NetFlow v8– Netflow v9 [12.2(18)SXF is Sup720 only]

• Theoretical maximum utilization versus effective utilization–Varies based on hardware implementation and hash efficiency


NetflowDisplaying flows on the system

C6500#show mls netflow ip

Displaying Netflow entries in Supervisor Earl

DstIP SrcIP Prot:SrcPort:DstPort Src i/f :AdjPtr

-----------------------------------------------------------------------------

Pkts Bytes Age LastSeen Attributes

---------------------------------------------------

10.102.130.213 10.214.39.79 tcp :46528 :www :0x0

7 3766 17 15:47:37 L3 - Dynamic

10.230.215.148 10.155.22.221 tcp :51813 :45912 :0x0

25 21329 47 15:47:39 L3 - Dynamic

10.97.36.200 10.17.64.177 tcp :65211 :www :0x0

9 7664 17 15:47:38 L3 - Dynamic

10.90.33.185 10.46.13.211 tcp :27077 :60425 :0x0

10 5734 17 15:47:38 L3 - Dynamic

<…>


NetflowFlow Masks

The Catalyst 6500 supports the following flow masks - these are used to identify which pieces of information in the header will be used as input into generating a key for flow lookups…


NetflowRecord Types

The following record types have been defined as part of the Netflow specification…


NetflowRecord Types - v5 and v7


NetflowConfiguring the Netflow Export Record Version

C6500(config)#mls netflow

C6500(config)#mls flow ip ? destination destination flow keyword destination-source destination-source flow keyword full full flow keyword interface-destination-source interface-destination-source flow keyword interface-full interface full flow keyword source source only flow keyword

C6500(config)#mls nde sender version ? 5 7

C6500(config)#mls nde interface

C6500(config)#ip flow-export destination 10.66.231.10

C6500(config)#interface g1/1C6500(config-if)#ip route-cache flow

Enable NetflowEnable NetflowOptionally set the flow maskOptionally set the flow mask

Set the Netflow Record Version on PFC Set the Netflow Record Version on PFC

Set the Netflow Export DestinationSet the Netflow Export Destination

Enable Netflow on the interfaceEnable Netflow on the interface

Populate interface field in NDE packetPopulate interface field in NDE packet


NetflowRecord Types - v8

Netflow v8 flow export uses separate aggregation caches to group flow records allowing it to store a subset of the information contained in a version 5 record - this has the added benefit of reducing bandwidth requirements for exporting records and improving export record scalability - eleven aggregation methods are available…



Each of the aggregation schemes

contains a slightly different

representation of the data contained within

a full Netflow v5 record…

Each of the aggregation schemes

contains a slightly different

representation of the data contained within

a full Netflow v5 record…

NOTE - the “green” ToS shows the ToS

version of that aggregation scheme (i.e. AS aggregation

scheme by itself does not contain the ToS

information)

NOTE - the “green” ToS shows the ToS

version of that aggregation scheme (i.e. AS aggregation

scheme by itself does not contain the ToS

information)


NetflowConfiguring the Netflow v8 Aggregation Cache

C6500(config)#ip flow-aggregation cache ? as AS aggregation as-tos AS-TOS aggregation bgp-nexthop-tos BGP nexthop TOS aggregation destination-prefix Destination Prefix aggregation destination-prefix-tos Destination Prefix TOS aggregation prefix Prefix aggregation prefix-port Prefix-port aggregation prefix-tos Prefix-TOS aggregation protocol-port Protocol and port aggregation protocol-port-tos Protocol, port and TOS aggregation source-prefix Source Prefix aggregation source-prefix-tos Source Prefix TOS aggregation

Configuration of the v8 aggregation cache on the Catalyst 6500 is enabled with the following command…



Support for Netflow v9 was added in IOS

12.2(18)SXF - this version of Netflow provides a more

flexible format in that the sequence of data records is defined by

a template that is inherently built into the exported record

itself…


Netflowv9 Template Flow set Field DescriptorsBuilt within each Flow Set Template are a number of field descriptors which can be used to define the records within the Data Flow set records


NetflowRecord Types - v9 with Options Template

More information at http://www.cisco.com/warp/public/cc/pd/iosw/prodlit/tflow_wp.htmMore information at http://www.cisco.com/warp/public/cc/pd/iosw/prodlit/tflow_wp.htm

Option Templates can be used to provide information about the Netflow process itself - an example could be the sampling rate (i.e. one in x) used on a given interface on the Catalyst 6500…


NetflowRecord Types - v9 with Multicast support

http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123newft/123_1/nfmultic.htmhttp://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123newft/123_1/nfmultic.htm


NetflowConfiguring Netflow v9 on the Catalyst 6500

C6500(config)# mls nde sender

C6500(config)# mls flow ip interface-full

C6500(config)# ip flow-export version 9

C6500(config)# ip flow-export destination 10.10.10.1 2111

Set flow maskSet flow mask

Enable Netflow on the PFCEnable Netflow on the PFC

Set Export DestinationSet Export Destination

Enable Netflow v9Enable Netflow v9

If you wanted to enable v9 export of Multicast data, you can enable this as follows

C6500(config)# interface gigabitethernet 3/1C6500(config-if)# ip multicast netflow ingress

C6500(config-if)# interface gigabitethernet 3/2C6500(config-if)# ip multicast netflow egress

Enable Netflow v9 ingressMulticast collection

Enable Netflow v9 ingressMulticast collection

Enable Netflow v9 egressMulticast collection

Enable Netflow v9 egressMulticast collection


NetflowSampled Netflow

The Catalyst 6500 supports both full and sampled Netflow record collection - both options are configurable on the switch - Sampled Netflow on the Sup720 uses a full interface flow mask

Sampling Rate options - one in every 64, 128, 256, 512, 1024,

2048, 4096 or 8192

Sampling Rate options - one in every 64, 128, 256, 512, 1024,

2048, 4096 or 8192


NetflowNetflow Capacities across the Supervisor family

Each of the Supervisors support for Netflow yields a different number of flows that can be stored in the Netflow tables - the table below provides a summary of the Netflow capacities for each of the Supervisors…

Table Size Hash Efficiency Effective Size Hash Key Size

Sup2 128K 25% 32K 17 bits

Sup720 128K 50% 64K 36 bits

Sup720-3B 128K 90% 115K 36 bits

Sup720-3BXL 256K 90% 230K 36 bits

Sup32-8GE 128K 90% 115K 36 bits

Sup32-10GE 128K 90% 115K 36 bits

Sup720-10GE-3C 128K 90% 115K 36 bits

Sup720-10GE-3CXL 256K 90% 230K 36 bits


NetflowArchitecture

Built within the PFC on the Supervisor are multiple sets of specialized memory each dedicated to storing different pieces of information - for the purposes of Netflow there is a TCAM and two sets of SRAM that, in combination, provide the ability to store information about flows in the system…


NetflowArchitecture

IPv4Key Table

Entry

Protocol/Mask

4

VLAN/VPN12

Protocol Type

8

IP DA

32

IP SA

32

SRC Port16

DST PORT

16

Xtag

4

VPN Valid

1

Re-Circ

1

CentralRewrite

1

PrimaryInput

1

First Packet Seen

FIN/RST Create Time

Last Seen timestamp

Byte Count

Packet Count

Threshold Exceeded

Count

Bucket Count

RPF Fail Cache Update

Control Bits

1 1 22 24 40 32 39 25 1 1 10

Netflow Key Table EntryNetflow Key Table Entry

Netflow Stats Table EntryNetflow Stats Table Entry

Netflow Key Table Entry Record

Netflow Statistics Table Entry Record


NetflowNetflow Step by Step on the PFC3


NetflowNetflow Hash Collision


NetflowUtilization of Netflow TCAM and SRAM Resources

C6500#show mls netflow table-contention detailed

Earl in Module 6

Detailed Netflow CAM (TCAM and ICAM) Utilization

================================================

TCAM Utilization : 100%

ICAM Utilization : 0%

Netflow TCAM count : 130944

Netflow ICAM count : 0

Netflow Creation Failures : 270274

Netflow CAM aliases : 0

If a flow hashes to the same location as an existing flow, while the packet is still switched, the flow record is not created. Netflow tables are a finite resource, and as such need to be managed to avoid the situation where flow records are not kept…


NetflowNetflow Aging

Tuning of Netflow aging parameters is a solution to managing the Netflow Table resource

Aging is used to define when flows are to flushed from the Netflow tables…

Three aging parameters to consider

Normal - fixed idle time for flowsFast - Threshold based aging for flowsLong - Maximum lifetime for flows

NOTE - Normal and Long Aging enabled by default: Fast aging is disabled by default

Timers are by default CONSERVATIVE


NetflowNetflow Aging

C6500#show mls netflow aging enable timeout packet threshold ------ ------- ----------------normal aging true 300 N/Afast aging false 32 100 long aging true 1920 N/A

Feature AgingFeature Pattern Agetime------- ------- -------NAT_INGRESS 4 300 NAT_EGRESS 4 300 NAT_INGRESS 3 300 NAT_EGRESS 3 300

C6500#


NetflowWhat can you do with the information? Answer = Plenty!!


NetflowLets look at the Device List


NetflowLets choose the Catalyst 6500


NetflowTraffic is broken up by interface - Lets inspect VLAN 64…


NetflowTraffic now broken up as IN/OUT traffic


NetflowWe can zoom in on a specific time interval…


NetflowNow I can see individual Source IP Address info…


NetflowAnother mouse click away and more info…


NetflowGetting to the specifics…


NetflowWe can also zoom in on specific SRC address info…


NetflowHere is who 10.66.236.94 has been talking to…


Netflow Case StudyTracking the Hacker at a University customer…


NetflowInternal Netflow Resources to check out

• Netflow on the Catalyst 6500 White Paper (Marco Foschiano)(includes updated section on Netflow v9)http://wwwin-eng.cisco.com/Eng/ISBU/TME/Netflow_6500_7600.pdf

• Netflow Performance on the Sup720-3BXLhttp://bock-bock/~icox/presentations/Netflow_Performance_May_2005_subset.ppt

• Netflow on the Catalyst 6500 and Cisco 7600 Presentationhttp://bock-bock/~icox/presentations/CCIE_Nov_2003-NDE-WAN_white.ppt

• Netflow on CISCO.COMhttp://www.cisco.com/go/netflow

netflow(6500 specific)

Documents