nat deployment in cloud networksd2zmdbbm9feqrf.cloudfront.net/2014/usa/pdf/brkdct-2448.pdf · how...

NAT Deployment in Cloud Networks

BRKDCT-2448

Jason Yang – CCIE #10467

Technical Marketing Engineer

© 2014 Cisco and/or its affiliates. All rights reserved. BRKDCT-2448 Cisco Public

Session Goals

• NAT is becoming the critical component of the Cloud Gateway, customers are thirsty for recommendations and best practices to design NAT with high scalability and high availability in the Hosted Cloud Networks.

• This session will share

1. How VRF Aware Network Address Translation (NAT) enables Cloud Gateway Architecture

2. Cloud Gateway High Availability Design

3. Performance, Scalability & Operation Best Practice* *This section will focus on ASR 1000 as the Cloud Gateway platform

3


Agenda

• Cloud Gateway Architecture enabled by VRF-Aware NAT

• Cloud Gateway HA Design

• Perf/Scale & Operation Best Practice

• Summary and Take Away

4

Cloud Gateway Architecture enabled by VRF-Aware NAT


MPLS VPN

Cloud Gateway Architecture

Apps

Internet

Partners

AAA

Location

Hosted Cloud Services

Multi-tenant

• VRF Aware

• VRF Scale

Private/Overlapping

Addressing access

Common Services

• Network Address Translation

• NAT Scale

Inter-VRFs

Communication

• VRF Aware Service Infrastructure (VASI)

High Availability

• Dual Box Design

• Stateless Redundancy

• Stateful Redundancy

PE GW

6


• VRF NAT supports MPLS/VPN for

– Communication between remote hosts in different VPNs and Internet common servers.

– Intra-VPN communication.

• VRF-Aware Service Infrastructure (VASI) for

– Traffic flows and routing exchange across different VRFs

– VASI is implemented by using virtual interface pairs (vasileftx, vasirightx), where each of the interfaces in the pair is associated with a different VRF instance.

– Apply services such as NAT, ACL, Policing, ZBFW, IPsec, PBR.

VRF Aware NAT & VASI

7


Connectivity Model Summary

Model 1 Model 2 Model 3

Cloud Gateway

Autonomous System

GW and PE are in different

BGP AS – Cloud Services is

managed outside the business

VPN network

GW and PE are in different

BGP AS - Cloud Services is

managed outside the

business VPN network

GW and PE are in the same

BGP AS – Cloud Services is

managed as part of

business VPN network

Connectivity to the

VPN network

Inter-AS Option A (eBGP + back

to back VRF)

NAT Inside interface

Inter-AS Option B (eBGP +

label)

NAT inside interface

MP-iBGP

NAT inside interface

(a) Connectivity to

the Cloud in Global

NAT outside interface NAT outside interface N/A

(b) Connectivity to

the Cloud in VRF

Requires VASI

NAT outside in VASIleft

Requires VASI


Requires VASI


Routing over VASI iBGP iBGP eBGP

the most

common

AS: Autonomous System

8


• HCS service in global routing table

• Inter-AS Option A

• VRF/VLAN sub-interface as VRF aware NAT inside Interface

• Global interface as NAT outside interface

Connectivity Model 1a

N x eBGP

VRFR VRFR

C_NetworkR

PE GW

AS577

Global

AS65004

C_NetworkB

C_NetworkG

VRFB VRFB

VRFG VRFG

HCS

ip nat inside ip nat outside

PE – Provider Edge Router; GW – Cloud Gateway Router; SR – Service Router

S_Network SR

9


• HCS service in VRF

• Inter-AS Option A

• VRF/VLAN sub-interface as VRF aware NAT inside Interface

Connectivity Model 1b

N x eBGP

VRFR VRFR

C_NetworkR

PE GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

VRFB VRFB

VRFG VRFG

HCS

ip nat inside ip nat outside VASILeftx VASIRightx

• VASI to facilitate Inter-VRF communication

• VASIleft VRF interface as NAT outside interface

S_Network SR

10


• HCS service in global route table

• Inter-AS Option B

• MPLS as VRF Aware of NAT inside interface

• Global Interface as NAT outside interface

Connectivity Model 2a

1 x eBGP

C_NetworkR

PE GW

AS577

Global

AS65004

C_NetworkB

C_NetworkG

MPLS MPLS HCS

ip nat inside ip nat outside

S_Network SR

11



• Inter-AS Option B


Connectivity Model 2b


• VASILeft VRF interface as NAT outside interface

1 x eBGP

C_NetworkR

PE GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

MPLS MPLS HCS

ip nat inside ip nat outside VASILeftx VASIRightx

S_Network SR

12



• MP-iBGP


Connectivity Model 3


• VASILeft VRF interface as NAT outside interface

MP-iBGP

C_NetworkR

PE GW

AS65004

MPLS

C_NetworkB

C_NetworkG

MPLS/VPN

ip nat outside VASILeftx VASIRightx

S_Network SR

MPLS

ip nat inside

MP-iBGP

13


• Inter-AS Option A is the most secure and easiest to provision

• Inter-AS Option A may face manageability challenge as #s of VRFs grow

Connectivity Model 1 – Control Plane

N x eBGP

VRFR VRFR

C_NetworkR

PE GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

VRFB VRFB

VRFG VRFG

HCS

• GW and SR can run Static/IGP/BGP to exchange routes, though BGP scales and seamless

• iBGP can run in the VASI pairs to exchange routes between VRFs

S_Network SR

eBGP

AS223

iBGP

202.255.254.1 202.255.254.1 202.255.254.1

202.255.254.1

10.254.254.4 10.254.254.4

201.255.254.1 201.255.254.1

Cloud Service Network Advertised to the Customers

NAT Pool Advertised to the Cloud

14


• Customer initiate the connection to the cloud

• Routing lookup performed before VRF-Aware NAT translation

Connectivity Model 1 – Data Plane

N x eBGP

VRFR VRFR

C_NetworkR

PE GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

VRFB VRFB

VRFG VRFG

HCS

• VASI allows customer VRF traffic flow to the Cloud Service VRF and Vice Versa

• For the return traffic, NAT performed in customer VRF first, then make the routing lookup.

S_Network SR

eBGP

AS223

iBGP 10.254.254.4(S)|202.255.254.1(D) 201.254.254.1(S)|202.255.254.1(D)

(D)201.254.254.1|(S)202.255.254.1 (D)10.254.254.4|(S)202.255.254.1

15


Connectivity Model 1 – Configuration interface GigabitEthernet0/0/0.2 description PE facing interface for VRFR encapsulation dot1Q 2 vrf forwarding VRFR ip address 192.255.254.2 255.255.255.252 ip nat inside ! interface vasileft1 vrf forwarding VRFR ip address 100.4.4.1 255.255.255.252 ip nat outside ! interface vasiright1 vrf forwarding VRFS ip address 100.4.4.2 255.255.255.252 ! interface GigabitEthernet0/0/1 description Cloud facing interface for Service VRF vrf forwarding VRFS ip address 99.99.70.2 255.255.255.0 ! access-list 4 permit 10.254.254.0 0.0.0.255 ! ip nat pool pool-PC 201.255.254.1 201.255.254.1 prefix-length 30 ip nat inside source list 4 pool pool-PC vrf VRFR overload

router bgp 577 address-family ipv4 vrf VRFS bgp router-id 1.1.1.1 bgp log-neighbor-changes neighbor 99.99.70.1 remote-as 223 neighbor 99.99.70.1 description PEERING to SR neighbor 99.99.70.1 active neighbor 100.4.4.1 remote-as 577 neighbor 100.4.4.1 next-hop-self neighbor 100.4.4.1 description PEERING to VASI VRFR interface neighbor 100.4.4.1 active ! address-family ipv4 vrf VRFR bgp router-id 4.4.4.4 redistribute static neighbor 100.4.4.2 remote-as 577 neighbor 100.4.4.2 description PEERING to VASI VRFS interface neighbor 100.4.4.2 activate neighbor 100.4.4.2 prefix-list VRF_Pool out neighbor 192.255.254.1 remote-as 65004 neighbor 192.255.254.1 description PEERING to PE neighbor 192.255.254.1 activate ! ip prefix-list VRF_Pool seq 5 permit 201.255.254.1/32 ! Ip route vrf VRFR 201.255.254.1 255.255.255.255 null0

16


• Inter-AS Option B - single eBGP session to exchange VPN routes and labels

• Label spoofing could be concern


1 x eBGP

C_NetworkR

PE GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

MPLS MPLS HCS

• GW and SR can run Static/IGP/BGP to exchange routes, though BGP scales and seamless

• iBGP can run in the VASI pairs to exchange routes between VRFs

S_Network SR

eBGP

AS223

iBGP

202.255.254.1 202.255.254.1 L2|202.255.254.1

202.255.254.1

10.254.254.4 10.254.254.4|L6

201.255.254.1 201.255.254.1



17



• Label disposition, followed by routing lookup, then performed VRF-Aware NAT translation


1 x eBGP

C_NetworkR

PE GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

MPLS MPLS HCS


• For the return traffic, NAT performed in customer VRF first, then make the routing lookup, then impose the label

S_Network SR

eBGP

AS223

iBGP 10.254.254.4(S)|202.255.254.1(D)|L2 201.254.254.1(S)|202.255.254.1(D)

(D)201.254.254.1|(S)202.255.254.1 L6|(D)10.254.254.4|(S)202.255.254.1

18


Connectivity Model 2 – Configuration interface GigabitEthernet0/0/0 description PE facing interface ip address 192.255.254.2 255.255.255.252 ip nat inside mpls bgp forwarding ! interface vasileft1 vrf forwarding VRFR ip address 100.4.4.1 255.255.255.252 ip nat outside ! interface vasiright1 vrf forwarding VRFS ip address 100.4.4.2 255.255.255.252 ! interface GigabitEthernet0/0/1 description Cloud facing interface for Service VRF vrf forwarding VRFS ip address 99.99.70.2 255.255.255.0 ! access-list 4 permit 10.254.254.0 0.0.0.255 ! ip nat pool pool-PC 201.255.254.1 201.255.254.1 prefix-length 30 ip nat inside source list 4 pool pool-PC vrf VRFR overload

router bgp 577 neighbor 192.255.254.1 remote-as 65004 neighbor 192.255.254.1 description PEERING to PE ! address-family ipv4 vrf VRFS bgp router-id 1.1.1.1 bgp log-neighbor-changes neighbor 99.99.70.1 remote-as 223 neighbor 99.99.70.1 description PEERING to SR neighbor 99.99.70.1 active neighbor 100.4.4.1 remote-as 577 neighbor 100.4.4.1 next-hop-self neighbor 100.4.4.1 active neighbor 100.4.4.1 description PEERING to VASI VRFR interface ! address-family ipv4 vrf VRFR bgp router-id 4.4.4.4 redistribute static neighbor 100.4.4.2 remote-as 577 neighbor 100.4.4.2 description PEERING to VASI VRFS interface neighbor 100.4.4.2 activate neighbor 100.4.4.2 prefix-list VRF_Pool out ! address-family vpnv4 neighbor 192.255.254.1 active neighbor 192.255.254.1 send-community both ! ip prefix-list VRF_Pool seq 5 permit 201.255.254.1/32 ! Ip route vrf VRFR 201.255.254.1 255.255.255.255 null0

19


• VASI becomes VRF termination point in the GW, an ideal place to apply per VRF Security and QoS policy

Connectivity Model 2 – Configuration (cont’d)

interface vasileft1

vrf forwarding VRFR

ip address 100.4.4.1 255.255.255.252

ip access-group VASI-1-LEFT-IN in

ip access-group VASI-1-LEFT-OUT out

ip nat outside

service-policy output Police_Cloud_ACCESS_VRFR_10meg*

interface vasiright1

vrf forwarding VRFS

ip address 100.4.4.2 255.255.255.252

ip access-group VASI-1-RIGHT-IN in

ip access-group VASI-1-RIGHT-OUT out

*Queuing Policy is not supported, only policing and marking

20


• Cloud service is part of business VPN network

• MP-iBGP – full mesh with all other PEs/RR/SR to exchange VPN routes and labels


MP-iBGP

C_NetworkR

PE GW

AS65004

MPLS

C_NetworkB

C_NetworkG

MPLS MPLS/VPN

• eBGP can run in the VASI pairs to exchange routes between VRFs

S_Network SR

MP-iBGP

eBGP

202.255.254.1|L1 202.255.254.1 L2|202.255.254.1

202.255.254.1

10.254.254.4 10.254.254.4|L6

201.255.254.1 201.255.254.1|L3



21



• Label disposition, followed by routing lookup, then performed VRF-Aware NAT translation


MP-iBGP

C_NetworkR

PE GW

AS65004

MPLS

C_NetworkB

C_NetworkG

MPLS MPLS/VPN


• For the return traffic, NAT performed in customer VRF first, then make the routing lookup, then impose the label

S_Network SR

MP-iBGP

eBGP 10.254.254.4(S)|202.255.254.1(D)|L2 201.254.254.1(S)|202.255.254.1(D)|L1

L3|(D)201.254.254.1|(S)202.255.254.1 L6|(D)10.254.254.4|(S)202.255.254.1

22


Connectivity Model 3 – Configuration interface GigabitEthernet0/0/0 description MPLS VPN facing interface ip address 192.255.254.2 255.255.255.252 ip nat inside mpls ip ! interface vasileft1 vrf forwarding VRFR ip address 100.4.4.1 255.255.255.252 ip nat outside ! interface vasiright1 vrf forwarding VRFS ip address 100.4.4.2 255.255.255.252 ip policy route-map PBR_FW ! interface GigabitEthernet0/0/1 description Service facing interface ip address 99.99.70.2 255.255.255.0 mpls ip ! route-map PBR_FW permit 10 match ip address PBR_FW set ip nexthop recursive vrf FW_VRF 202.255.254.1 ! access-list 4 permit 10.254.254.0 0.0.0.255 ! ip nat pool pool-PC 201.255.254.1 201.255.254.1 prefix-length 30 ip nat inside source list 4 pool pool-PC vrf VRFR overload

router bgp 65004 neighbor 192.255.254.100 remote-as 65004 neighbor 192.255.254.100 description PEERING to RR neighbor 192.255.254.100 update-source loopback0 ! address-family ipv4 vrf VRFS redistribute connected neighbor 100.4.4.1 remote-as 65534 neighbor 100.4.4.1 local-as 65535 neighbor 100.4.4.1 update-source vasiright1 neighbor 100.4.4.1 active ! address-family ipv4 vrf VRFR redistribute static neighbor 100.4.4.2 remote-as 65535 neighbor 100.4.4.2 local-as 65534 neighbor 100.4.4.2 update-source vasileft1 neighbor 100.4.4.2 activate neighbor 100.4.4.2 prefix-list VRF_Pool out default-information originate ! address-family vpnv4 neighbor 192.255.254.100 active neighbor 192.255.254.100 send-community both ! ip prefix-list VRF_Pool seq 5 permit 201.255.254.1/32 ! Ip route vrf VRFR 201.255.254.1 255.255.255.255 null0

23


Design of NAT Pool

Pool per VRF Shared Pool by all VRFs

1. Easy of maintenance

2. Easy of debugging

3. Add/Remove customers without service

disruption

1. Efficient use of addresses

2. Less configuration

3. Removing of one customer cause interruption

of all other customers NAT

ip nat pool customer1-nat-pool 15.1.1.1 15.1.1.255 prefix-length 24

!

ip access-list extended customer1-acl

deny ip <router-generated-ip>

permit ip 10.0.0.0 0.255.255.255

!

ip nat inside source list customer1-acl pool customer1-nat-pool overload

vrf customer1-vrf

!

ip nat pool customer2-nat-pool 16.1.1.1 16.1.1.255 prefix-length 24

!

ip access-list extended customer2-acl


permit ip 10.0.0.0 0.255.255.255

!

ip nat inside source list customer2-acl pool customer2-nat-pool overload

vrf customer2-vrf

ip nat pool shared-nat-pool 15.1.1.1 15.1.255.255 prefix-length 16

!

ip access-list extended shared-cust-acl


permit ip 10.0.0.0 0.255.255.255

!

ip nat inside source list shared-cust-acl pool shared-nat-pool overload

vrf customer1-vrf

ip nat inside source list shared-cust-acl pool shared-nat-pool overload

vrf customer2-vrf

24


What Mode of NAT to Run - NAT vs. CGN

Traditional NAT Carrier Grade NAT (CGN)

Session Entry full 5 tuples – {protocol, source address, source

port, destination address, destination port}

3 tuples - {protocol, source address,

source port}

Default timeout 24 hrs for TCP 15 mins for TCP

Outside mapping rule

(ip nat outside source)

Supported Not supported

EIM/EIF Not Supported Supported

High Speed Logging

(HSL)

Log full tuples No destination info in the logging

record

Bulk logging and Port

Block Allocation

Not Supported Supported

Salability - Double than traditional NAT

License No license required Require license

25


• Traditional NAT

Pro Inside global Inside local Outside local Outside global

tcp 26.1.1.6:1024 27.1.1.10:29439 26.1.1.2:23 26.1.1.2:23

• CGN


tcp 26.1.1.6:1024 27.1.1.10:11806 --- ---

NAT vs. CGN – Session Entry

26


• Endpoint-Independent Mapping (EIM) provides a stable, long-term binding where internal hosts may connect by utilizing the same NAT binding for multiple external hosts (as long as the internal port does not change)

• Endpoint-Independent Filtering (EIF) is closely related to EIM, and controls which external servers may access a host using an established binding

• This is typical for peer-to-peer applications and some Internet messenger protocols.

NAT vs. CGN – EIM/EIF

inside outside

SrcIP:Port DstIP:Port

X:x Y1:y1


X1:x1 Y1:y1


X:x Y2:y2


X1:x1 Y2:y2

EIM implies X1:x1 = X2:x2 for all Y:y (Y1:y1 and Y2:y2)


tcp 26.1.1.6:1024 27.1.1.10:11806 --- ---

CGN

27


• High speed NAT device generate NAT transaction events (creation/deletion) in the rate of >100k events/sec, syslog is not able to support it.

• HSL enables NAT datapath directly export the transaction records (NetFlowv9-like) to an external collector.

NAT vs. CGN – High Speed Logging (HSL)

Field Format

Source IP address IPv4 address

Translated source IP address IPv4 address

Destination IP address IPv4 address

Translated destination IP address IPv4 address

Original source port 16-bit port

Translated source port 16-bit port

Original destination port 16-bit port

Translated destination port 16-bit port

VRF ID 32-bit ID

Protocol 8-bit value

Event 0-Invalid

1-Adds event

2-Deletes event

Unix timestamp in milliseconds 64-bit value

Destination Info not available in CGN

Mode

Destination Info not available in CGN

Mode

28


• Problem: High setup/teardown rates on NAT devices cause customers to have to store Terabits of data a day for NAT HSL. Customer want to see this volume of logging significantly reduced.

• Solution: Provide each end user with a block of ports. Only log when the block get (dis)associated with a user.

NAT vs. CGN – Bulk Logging and Port Block Allocation (BPA)

Field Format

Source IP address IPv4 address

Translated source IP address IPv4 address

VRF ID 32-bit ID

Protocol 8-bit value

Event 0-Invalid

1-Adds event

2-Deletes event

Unix timestamp in milliseconds 64-bit value

Port block start 16-bit port

Port block step size 16-bit step size

Number of ports in the block 16-bit number

For example: a BPA configuration with set size 8 and step size of 4.

Set 0 = {1024, 1028, 1032, 1036, 1040, 1044, 1048, 1052}

Set 1 = {1025, 1029, 1033, 1037, 1041, 1045, 1049, 1053}

Set 2 = {1026, 1030, 1034, 1038, 1042, 1046, 1050, 1054}

Set 3 = {1027, 1031, 1035, 1039, 1043, 1045, 1051, 1055}

…

29

Cloud Gateway HA Design


• Dual-GWs; Dual-PEs; Dual-SRs

• Fast Failure Detection: BFD (sub-second) – may not all platforms support BFD

• Common Failure Detection: BGP (~tens of sec)

High Availability Design

• BGP determines the active path, symmetric routing and convergence time

• GWs are in (stateless) active/standby from NAT perspective

N x eBGP/BFD

VRFR VRFR

C_NetworkR

PE1 GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

HCS S_Network SR1

eBGP/BFD

AS223

GW2

VRFR VRFR

GW1

SR2 Service VRF

iBGP

PE2

31


• PE1-GW1-SR1 active path; PE2-GW2-SR2 standby path.

• GW1-SR1 BGP session down

• GW1 withdraw S_Network from PE1

• PE2-GW2-SR2 become the best, GW2 will begin to set up NAT translations

Failover Scenario – GW1-SR1 BGP session down

N x eBGP/BFD

VRFR VRFR

C_NetworkR

PE1 GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

HCS S_Network SR1

eBGP/BFD

AS223

GW2

VRFR VRFR

GW1

SR2 Service VRF

iBGP

PE2

202.255.254.1

202.255.254.1

202.255.254.1

32


• PE1-GW1-SR1 active path; PE2-GW2-SR2 standby path.

• PE1-GW1 BGP session down

• GW1 is still advertising the NAT_Pool to SR1, which cause SR1 to blackhole customer traffic to GW1!

Failover Scenario – PE1-GW1 BGP session down

N x eBGP/BFD

VRFR VRFR

C_NetworkR

PE1 GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

HCS S_Network SR1

eBGP/BFD

AS223

GW2

VRFR VRFR

GW1

SR2 Service VRF

iBGP

PE2

201.255.254.1

201.255.254.1

33


• Solution: BGP VRF Aware Conditional Advertisement

• The condition is that C_networkR exist in BGP VRFR table in GW1, then GW1 can advertise NAT_Pool to VASIRight, otherwise withdraw NAT_Pool back from VASIRight

Failover Scenario – PE1-GW1 BGP session down (cont’d)

N x eBGP/BFD

VRFR VRFR

C_NetworkR

PE1 GW

AS577

Service VRF

AS65004

C_NetworkB

C_NetworkG

HCS S_Network SR1

eBGP/BFD

AS223

GW2

VRFR VRFR

GW1

SR2 Service VRF

iBGP

PE2

201.255.254.1

201.255.254.1

10.254.254.4

10.254.254.4

34


Failover Scenario – PE1-GW1 BGP session down (cont’d)

interface GigabitEthernet0/0/0.2 description PE facing interface for VRFR encapsulation dot1Q 2 vrf forwarding VRFR ip address 192.255.254.2 255.255.255.252 ip nat inside bfd interval 50 min_rx 50 multiplier 3 ! interface vasileft1 vrf forwarding VRFR ip address 100.4.4.1 255.255.255.252 ip nat outside ! interface vasiright1 vrf forwarding VRFS ip address 100.4.4.2 255.255.255.252 ! interface GigabitEthernet0/0/1 description Cloud facing interface for Service VRF vrf forwarding VRFS ip address 99.99.70.2 255.255.255.0 ! access-list 4 permit 10.254.254.0 0.0.0.255 ! ip nat pool pool-PC 201.255.254.1 201.255.254.1 prefix-length 30 ip nat inside source list 4 pool pool-PC vrf VRFR overload ! ip prefix-list VRF_Pool seq 5 permit 201.255.254.1/32 ip prefix-list p1-adv-1 seq 5 permit 201.255.254.1/32 ip prefix-list p1-exist-1 seq 5 permit 10.254.254.4/32

router bgp 577 address-family ipv4 vrf VRFS bgp router-id 1.1.1.1 bgp log-neighbor-changes neighbor 99.99.70.1 remote-as 223 neighbor 99.99.70.1 description PEERING to SR neighbor 99.99.70.1 active neighbor 100.4.4.1 remote-as 577 neighbor 100.4.4.1 next-hop-self neighbor 100.4.4.1 description PEERING to VASI VRFR interface neighbor 100.4.4.1 active ! address-family ipv4 vrf VRFR bgp router-id 4.4.4.4 redistribute static neighbor 100.4.4.2 remote-as 577 neighbor 100.4.4.2 description PEERING to VASI VRFS interface neighbor 100.4.4.2 activate neighbor 100.4.4.2 advertise-map ADV-1 exist-map EXIST-1 neighbor 100.4.4.2 prefix-list VRF_Pool out neighbor 192.255.254.1 remote-as 65004 neighbor 192.255.254.1 description PEERING to PE neighbor 192.255.254.1 activate ! route-map ADV-1 permit 10 match ip address prefix-list p1-adv-1 ! route-map EXIST-1 permit 10 match ip address prefix-list p1-exist-1 ! Ip route vrf VRFR 201.255.254.1 255.255.255.255 null0

35


• Redundant ESP / RP on ASR 1006 and ASR 1013

• Zero packet loss on RP fail-over

• < 50ms loss for ESP fail-over

• Intra-chassis Stateful Switchover (SSO) Support for NAT

• IOS XE also provides full support for Network Resiliency

– NSR/GR for BGP

– BFD SSO

• Support for ISSU

GW Intra-Chassis Redundancy - ASR 1000 built for Carrier-Grade HA

RP

CPU RP

CPU

ES

P

QFP Crypto

Assist. PPE BQS

FECP

Crypto ES

P

QFP Crypto

Assist. PPE BQS

FECP

Crypto

RP

CPU

Crypto

Assist.

RP

CPU

ES

P

QFP PPE BQS

FECP

Crypto ES

P

QFP Crypto

Assist. PPE BQS

FECP

Crypto

SIP

SPA SPA

IOCP SPA

Aggreg.

SIP

SPA SPA

IOCP SPA

Aggreg.

SIP

SPA SPA

IOCP SPA

Aggreg.

36

Performance, Scalability & Operation Best Practice


Midplane

ASR1000 Building Blocks

ES

P FECP

QFP Crypto

Assist.

interconnect

PPE BQS

ES

P

FECP

QFP Crypto

Assist.

interconn.

PPE BQS

FECP

Crypto

Assist.

interconnect

RP

CPU

interconn. GE switch S

IP

SPA SPA

IOCP SPA

Aggreg.

interconnect

RP

CPU

interconn. GE switch

SIP

SPA SPA

IOCP SPA

Aggreg.

interconnect

SIP

SPA SPA

IOCP SPA

Aggreg.

interconnect

Route Processor

Handles control plane

Manages system Embedded Service Processor

Handles forwarding plane traffic

SPA Interface Processor

Houses SPA’s

Buffer packets in & out

• Route Processor (RP) • Handles control plane traffic • Manages system

• Embedded Service Processor (ESP) • Handles forwarding plane traffic

• SPA Interface Processor (SIP) • Shared Port Adapters provide interface

connectivity

• Centralized Forwarding Architecture • All traffic flows through the active ESP,

standby is synchronized with all flow state with a dedicated 10-Gbps link

• Distributed Control Architecture • All major system components have a

powerful control processor dedicated for control and management planes

38


NAT <> ESP Resources Dependency

QFP complex

Crypto

(Nitrox-II

CN2430)

FECP GE, 1Gbps

I2C

SPA Control

SPA Bus

ESI, 11.2Gbps

SPA-SPI, 11.2Gbps

Hypertransport, 10Gbps

Other

RPs RPs RPs ESP SIPs

E-RP* PCI*

E-CSR

TCAM Resource

DRAM

Packet Buffer

DRAM

SA table

DRAM

Dispatcher Packet Buffer

DDRAM

Boot Flash

(OBFL,…)

JTAG Ctrl

Reset / Pwr Ctrl

…

Packet Processor Engines

PPE1 PPE2 PPE3 PPE4 PPE5

PPE6 PPE7 PPE8 PPE40

BQS

Reset / Pwr Ctrl

Interconnect

SPI Mux

Interconnect

EEPROM

Temp Sensor

• NAT sessions

• Memory for FECP

• QFP client / driver

• Statistics

• ACL ACEs copy

• NAT config objects

• NAT VFR re-assembly • ACL/ACE, Route-map

39


ASR1000 NAT Scalability (uni-dimensional)

ASR 1001 ASR 1002-X ESP5

ESP10 ESP20 ESP40 ESP100 ESP200

NAT

Sessions

(classic)

250k 2M 250k 1M 2M 2M 4M 4M

NAT

Sessions

(CGN)

500k 4M 500k 1.75M 4M 4M 12M 12M

NAT Pools - 1200 - - 1200 1200 1200 1200

VRFs for

VRF-Aware

NAT

4k 4k 1k 1k 4k 4k 4k 4k

Route-maps

w/ NAT 1024 1024 1024 1024 1024 1024 1024 1024

40


ASR1000 NAT Performance (uni-dimensional)

g ASR 1001 ASR 1002-X ESP5

ESP10 ESP20 ESP40 ESP100 ESP200

NAT Session

Setup Rate 50cps 230cps(pat) 50kcps 100kcps

200kcps(dyn)

139kcps(pat)

200kcps(dyn)

95kcps(pat) 250kcps(pat) 300kcps(pat)

NAT (classic)

Performance 3Mpps 10Mpps 3Mpps 6Mpps 8Mpps 9Mpps 23Mpps 45Mpps

NAT (CGN)

Performance 2.2Mpps - 2.2Mpps 5Mpps 7Mpps 7Mpps 18Mpps 34Mpps

NAT (classic)

Throughput 5Gbps 36Gbps 5Gbps 10Gbps 20Gbps 40Gbps 100Gbps 200Gbps

41


Application Layer Gateway (ALG)

ALG VFR vTCP L4 VRF HA

FTP Yes No tco Yes Yes

H323 No Yes tcp,udp Yes Yes

RTSP Yes Yes tcp Yes Yes

SCCP No No tcp Yes Yes

SIP Yes Yes tcp,udp Yes Yes

TFTP No N/A udp Yes Yes

NETBIOS No No tcp,udp Yes Yes

RCMD No No tcp Yes Yes

LDAP No No tcp Yes Yes

DNS Yes Yes tcp,udp Yes Yes

SUNPRC Yes No tcp Yes Yes

MSRPC Yes No tcp Yes Yes

PPTP No tcp Yes Yes

• ASR 1000 support comprehensive ALGs

• With ALG traffic, "any any" ACL is not supported. This could lead to undesired payload translations, causing unexpected application behavior

42


• Isarflow

• Lancope

• ActionPacked

ASR 1000 HSL Supported Collector

43


Key System Resources to Monitor

IOS

Forwarding

Manager

Forwarding

Manager

QFP Client

Driver

Datapath

SIP

show proc cpu sort show mem stat

RP memory RP CPU

TCAM

resource DRAM

pkt memory

crypto assist QFP

ESP memory

show plat

software status

control-processor

brief

show plat

software status

control-processor

brief

show plat

software status

control-processor

brief

FECP CPU

show plat software

status control-

processor brief

show plat hardware

qfp active infra

exmem statistics

show plat hardware

qfp active datapath

util summary

show plat hardware

qfp active tcam

resource-manager

-usage

75%

85%

44


ASR 1000 Cloud Gateway Monitoring Guide (1)

• It is general best practice that ASR 1000 in live deployment RP/IOS/ESP CPU and Memory utilization do not exceed 75% in steady state

• It is general best practice that ASR 1000 in live deployment QFP DRAM utilization do not exceed 85% in steady state

45


ASR 1000 Cloud Gateway Monitoring Guide (2)

• For TCAM monitoring, keep an eye on syslog:

%QFPTCAMRM-6-TCAM_RSRC_ERR: F0: QFP_sp: Allocation failed because of insufficient TCAM resources in the system

• Recommendations

1. Test out TCAM utilization before making changes

2. Always there should be unused TCAM entries which are = or > the size of biggest ACL on the router.

• Be aware of the TCAM deny jump issue

46


SET the Limit

• Set NAT max-entries per system to no more than platform scale: ip nat translation max-entries <number of entries>

Be aware of that

1. NAT sessions scaling numbers are based on a few pools

2. PAT session scaling numbers are expected to be reduced while the number of overload pools are rising

3. One data point we have is ESP20 support 500k sessions w/ 1200 overload pools vs. 2M session w/ a few pools

• Set NAT max-entries per VRF to prevent single customer starving entire system translation limit:

ip nat translation max-entries vrf <vrf_name> <number of entries>

47


Features Interaction

• This architecture is proven with following features on Cloud Gateway, do not enable more features unless been tested prior to deployment.

– VRF Aware NAT + VASI + MP-BGP

– On VASI: ACL, Policing/Marking MQC, PBR, eBGP or iBGP

48


Common Issues - TCAM Deny-Jump (1)

• Problem Description:

In ASR 1000 IPsec/FW/NAT deployment, user may see following message:

“%CPP_FM-3-CPP_FM_TCAM_ERROR: F0: cpp_sp: TCAM limit exceeded…”

• Error Message Explanation:

This is an protection mechanism prevents system from crashing with WATCH-DOG timeout error or malloc failure.

• Root Cause Analysis:

1. Classification engine in the TCAM can only represent permit.

2. System convertes the DENY entries into PERMIT ones using cross product

3. This recursive nature cause the required number of entries to “explode”.

49


Common Issues - TCAM Deny-Jump (2) • Workaround:

1. Before deploying the platform in production, apply the configuration in lab

2. Modify the ACLs to use multiple specific permit statement, and try to reduce or eliminate the explicit use of deny statement

3. Use PBR to bypass NAT

4. Static NAT

• Solutions:

1. IOS XE3.10 introduced the SW classification engine to handle deny-jump like classification

2. System still use TCAM as long as it has room, in case TCAM does not fit, it will switch to SW classification engine.

Original NAT Config VASI & PBR to bypass NAT

ip nat inside source list NAT-ACL pool NAT-POOL overload

!

ip access-list extended NAT-ACL

deny ip any 129.25.0.0 0.0.255.255

permit ip 172.19.0.0 0.0.0.255 any


!

interface GigabitEthernet0/0/1

description nat inside interface

ip address 6.1.1.1 255.255.255.0

ip nat inside

ip policy route-map no-NAT-rmap

interface vasileft1

ip address 13.1.1.1

!

interface vasiright1

ip address 13.1.2.1 255.255.255.0

!


permit ip 172.19.0.0 0.0.0.255 any

ip access-list extended bypass-NAT

permit ip any 129.25.0.0 0.0.255.255

!

route-map no-NAT-rmap permit 10

match ip address bypass-nat

set interface vasileft1

Original NAT Config Identity NAT


!


deny ip host 172.19.1.1 any

permit ip 172.19.0.0 0.0.0.255 any

ip nat inside source static 172.19.1.1 172.19.1.1 no-alias

ip nat inside source list NAT-ACL pool NAT-POOL overload !


permit ip 172.19.0.0 0.0.0.255 any

50


Common Issues - NAT ADDR ALLOC FAILURE (1) • Problem Description:

In ASR 1000 PAT/Overload configuration, system get error message:

"%NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 1 may be exhausted”

• Debug Information that should be gathered: show platform hardware qfp active feature nat data pool

show platform hardware qfp active feature nat data port

show platform hardware qfp active feature nat data stat

show platform hardware qfp active feature nat data base

show ip nat translation | inc <global address of interest>

• Common Reason for Failure:

1. Customer has a small pool which is being consumed by non-PATTAble binds.

2. A non-PATtable bind will show in 'sh ip nat trans' as a single local associated with a single global IP address.

3. It consumes an entire address in the pool.

--- 213.252.7.132 172.16.254.242 ---

51


Common Issues - NAT ADDR ALLOC FAILURE (2) • Solution 1

1. A non-PAttable bind could be created by packet with a non-PATTable protocol.

2. The best way to prevent this is to tighten the ACL to exclude non-PAttable protocols.

• Solution 2

1. A non-PAttable bind could be created by ALG like DNS which does not have ports in its L7 header has requested a global NAT address.

2. Often customers do not need the DNS ALG so the solution is to turn it off.

3. Below shows the most common ALGs which produce non-PAttable binds being turned off.

access-list 100 permit udp 13.1.0.0 0.0.255.255 any

access-list 100 permit tcp 13.1.0.0 0.0.255.255 any

access-list 100 permit icmp 13.1.0.0 0.0.255.255 any

no ip nat service dns udp

no ip nat service dns tcp

no ip nat service netbios-ns tcp

no ip nat service netbios-ns udp

no ip nat service netbios-ssn

no ip nat service netbios-dgm

no ip nat service ldap

52

Summary and Take Away


NAT Deployment in Cloud Networks Summary and Take Away …

• Follow proven connectivity models

• Stateless failover with BGP/BFD

• High scale, high performance NAT on ASR 1000

• Monitor key system resources proactively

200Gbps

Cloud Gateway

HA BGP VASI 12M Sess ALG HSL Connectivity NAT/CGN

54


Relevant Sessions at Cisco Live 2014

Breakout Sessions

• BRKSPG-2602 - IPv4 Exhaustion: NAT and Transition to IPv6 for Service Providers

• BRKARC-2019 – Operating an ASR 1000

• BRKARC-2021 - IOS XE Advanced Troubleshooting (NAT, VPN, FW packet forwarding)

55


Complete Your Online Session Evaluation

• Give us your feedback and you could win fabulous prizes. Winners announced daily.

• Complete your session evaluation through the Cisco Live mobile app or visit one of the interactive kiosks located throughout the convention center.

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

56

https://www.ciscolive.com/online


Continue Your Education

• Demos in the Cisco Campus (ASR1001-X Live Demo)

• Walk-in Self-Paced Labs

• Table Topics

• Meet the Engineer 1:1 meetings

57

Thank you.

nat deployment in cloud networksd2zmdbbm9feqrf.cloudfront.net/2014/usa/pdf/brkdct-2448.pdf · how...

Documents