double redundancy with aws direct connect - pop-up loft tel aviv

Double Redundancy with AWS Direct Connect

Steve SeymourSpecialist Solutions Architect

Agenda

• Building network foundations in AWS• Connecting your onsite deployment to AWS• Adding some redundancy into the mix• Demo: Taking our environment live and

introducing some failures!

Foundations: Amazon VPCYour own private, isolated section of the AWS cloud

VPC CIDR 10.1.0.0/16

Availability Zone A Availability Zone B

Public Subnet Public Subnet

Private Subnet Private Subnet

Instance A10.1.1.11 /24

Instance B10.1.2.22 /24

Instance C10.1.3.33 /24

Instance D10.1.4.44 /24

10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

Only 1 IGW and 1 VGW per VPC

Foundations: Other ServicesLets add some AWS services outside of VPC

AWS Region - eg: US-WEST1

Our VPC from Earlier

AWS Region

AWS Region Level Services (plus many more)

AWS VPC Internal Services (e.g. Amazon EMR, Elastic Load Balancing, Amazon RDS)

IGW, gateway between AWS region level services and internal VPC services

Instance A10.1.1.11 /24

Availability Zone A Availability Zone B

Public Subnet Public Subnet

Private Subnet Private Subnet

Instance B10.1.2.22 /24

Instance C10.1.3.33 /24

Instance D10.1.4.44 /24

10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

Amazon SNS

Amazon SQS

Amazon SWF

Amazon SES

Amazon S3

Amazon Glacier

Amazon DynamoDB

AWS Lambda

Connectivity: AWS to On-PremisesUsing AWS Direct Connect

10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

Customer DCColocation Facility - e.g. Equinix SV1

VPC CIDR 10.1.0.0/16

Service ProviderNetwork

Customer Subnet

192.168.0.0/16

Direct Connect POP

Colocation Facility

Customer or Partner Device

AWS Direct ConnectPoint of Presence Customer Gateway

Cross Connect

Customer Data Center

Service Provider Backhaul

Anatomy of AWS Direct Connect

Private VIF

Private Virtual Interface

Configure Customer Gateway

VLAN / Sub Int

VPC VGW

Standard Interface & BGP Configuration…

interface GigabitEthernet0/1

no ip address

interface GigabitEthernet0/1.807

description "Direct Connect to your Amazon VPC or AWS Cloud"

encapsulation dot1Q 807

ip address 172.16.7.5 255.255.255.252

router bgp 65001

neighbor 172.16.7.6 remote-as 7224

neighbor 172.16.7.6 password 7 $1$zVOvlUSp$UrqWP2awtiG8ZbXo9BwcB

network 0.0.0.0

exit

Physical Interface that fiber is plugged into

Sub-interface (Generally matches VLAN)

VLAN Association

/30 Private P2P addressBGP ASN

Route Advertisement to AWS

Just a description

BGP MD5 PasswordNeighbor Peer Address

10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

VPC CIDR 10.1.0.0/16



Customer Subnet

192.168.0.0/16


Private VIF

Customer Gateway

VLAN / Sub Int

BGP Comes up, prefixes are advertised.%BGP-5-ADJCHANGE: neighbor 172.16.6.6 Up

AWS Direct ConnectPoint of Presence

Anatomy of AWS Direct Connect continued...

10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

VPC CIDR 10.1.0.0/16



Customer Subnet

172.160.0.0/16


Private VIF

Customer Gateway

VLAN / Sub Int


My Private Virtual Interface is up, now what?What about my S3 bucket or DynamoDB? – in comes Public Virtual Interfaces!

10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

VPC CIDR 10.1.0.0/16

Amazon SNS

Amazon SQS

Amazon SWF

Amazon SES

Amazon S3 Amazon DynamoDB


AWS LambdaAmazon Glacier



Customer Subnet

172.160.0.0/16Private VIF

Customer Gateway

VLAN / Sub Int

AWS Regions much larger than just what’s inside a VPC

Create Public Virtual Interface


BGP Comes up, prefixes are advertised (Public only!).%BGP-5-ADJCHANGE: neighbor 203.50.24.5 Up



VLAN / Sub Int

Public VIF

Adding Redundancy“Everything fails, all the time.” – Werner Vogels

Anatomy of a redundant AWS Direct Connect

Customer DC



Colocation Facility - e.g. Equinix SV1

Public VIF

Private VIFVLAN / Sub Int

VLAN / Sub Int



VLAN / Sub Int

VLAN / Sub Int

Public VIF

Private VIF Customer Subnet

172.160.0.0/16

Double connectivity

The standard connectivity we built earlierVPC VGW

Redundant DX POP LocationOther AWS Services

10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

VPC CIDR 10.1.0.0/16

Amazon SNS

Amazon SQS

Amazon SWF

Amazon SES




Amazon SNS

Amazon SQS

Amazon SWF

Amazon SES





Customer DC




Public VIF


VLAN / Sub Int



VLAN / Sub Int

VLAN / Sub Int

Public VIF


172.160.0.0/16

How do we configure redundant BGP?

And here too!

10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

VPC CIDR 10.1.0.0/16


#Active Passive deployment:

router bgp 65001neighbor 10.1.0.2 remote-as 65200neighbor 10.1.0.2 description Backupneighbor 10.1.0.2 route-map prepend out

route-map prepend permit 10set as-path prepend 65001 65001 65001

Using one link as the primary, and the other “Prepended” as the secondary and less preferred route

Autonomous System (AS) Path Prepending?

ASN65001

ASN65001

ASN65001

ASN65001

ASN65001

Origin NetworkPrepended ASNPrepended ASNPrepended ASN

Verses.

Origin Network

Metric 4

Metric 1

Less Preferred

More Preferred

0%

100%


#Active Active deployment:

router bgp 1maximum-paths 4 Usually reserved for a single customer router scenario,

can be configured at the service provider level as well.

Note: By default we “Multi-path” outbound from VGW over equal cost paths unless you set a metric such as AS PATH on one route.

Autonomous System (AS) Equal Paths

ASN65001

ASN65001

Origin Network

Vs.

Origin Network

Metric 1

Metric 1

Both Preferred

Both Preferred

50%

50%

Did I hear Double Redundancy?You can use VPN as your backup of backups

Amazon SNS

Amazon SQS

Amazon SWF

Amazon SES




10.1.1.0/16

10.1.2.0/16

10.1.3.0/16

VPC CIDR 10.1.0.0/16


Customer DC




Public VIF


VLAN / Sub Int



VLAN / Sub Int

VLAN / Sub Int

Public VIF


172.160.0.0/16

Most MPLS Providers can “trunk” you an internet circuitOur VGW’s are also used as VPN

connection points remember!Dual VPN tunnels providing connectivity and encryption.

VPN & BGP Redundancy Configuration…

#Direct Connect Interface:

interface GigabitEthernet0/0/0.259

description "Direct Connect to your Amazon VPC or AWS Cloud"

encapsulation dot1Q 259

ip address 169.254.254.2 255.255.255.252

bfd interval 300 min_rx 300 multiplier 3

!

Subinterface

VLAN IDLocal IP AddressBFD Configuration


#Inter Router Interface:

interface GigabitEthernet0/1

description ** Internal Interface - SW2 Gi2/0/1 **

ip address 192.168.51.253 255.255.255.0

ip virtual-reassembly in

standby 1 ip 192.168.51.254

standby 1 timers msec 300 msec 900

standby 1 priority 110

standby 1 preempt

duplex auto

speed auto

!

Local LAN IP

HSRP ConfigurationHSRP sub second hello

This router is primaryPreempt primary if not active


BGP Configuration:

router bgp 65501

bgp log-neighbor-changes


neighbor 169.254.254.1 password 7 124B36F51

neighbor 169.254.254.1 fall-over bfd


!

Direct Connect neighbor

BFD ConfigurationInter router neighbor


Secondary router BGP and route-map assignment:

router bgp 65501

bgp log-neighbor-changes


neighbor 169.254.254.37 route-map LOCAL-PREF in

neighbor 169.254.254.37 route-map AS-PREPEND out

Secondary Direct Connect neighbor

Inbound route-map

Outbound route-map


Secondary router route-map:

ip prefix-list LOCAL-ROUTES seq 10 permit 192.168.0.0/16 le 32

route-map AS-PREPEND permit 10

match ip address prefix-list LOCAL-ROUTES

set as-path prepend 65501 65501

!

route-map LOCAL-PREF permit 10

set local-preference 90

!

Match local routes for AS prepending

Match above prefix list

Add ASN x 2 to AS Path

Set local preference to 90 (for secondary)

Now adding VPN….

VPN Tunnel interface (Straight forward):

interface Tunnel1

ip address 169.254.20.62 255.255.255.252


ip tcp adjust-mss 1387

tunnel source 62.216.229.132

tunnel mode ipsec ipv4

tunnel destination 52.17.141.73

tunnel protection ipsec profile ipsec-vpn-946e19df-0

!

Now adding VPN….

VPN Tunnel interface (Straight forward):

interface Tunnel2

ip address 169.254.20.162 255.255.255.252


ip tcp adjust-mss 1387

tunnel source 62.216.229.132

tunnel mode ipsec ipv4

tunnel destination 52.18.219.193

tunnel protection ipsec profile ipsec-vpn-946e19df-1

!

Plus your other VPN goodness like crypto-maps…

Now adding VPN….

VPN BGP Configuration (Still standard..)

Router BGP 65501


neighbor 169.254.20.61 timers 10 30 30

!

Address-family ipv4

network 192.168.51.0

neighbor 169.254.20.61 activate

neighbor 169.254.20.61 route-map LOCAL-PREF-VPN in

neighbor 169.254.20.61 route-map AS-PREPEND-VPN out

!

Standard BGP Configuration

Where it gets interesting…

Now adding VPN….

#Where we add our metrics:

route-map AS-PREPEND-VPN permit 10

match ip address prefix-list LOCAL-ROUTES

set as-path prepend 65501 65501 65501

!

route-map LOCAL-PREF-VPN permit 10

set local-preference 80

!

An additional ASN beyond our backup direct connect link

Local Preference is 10 lower than our backup Direct Connect link

Our real life environment

Test Node172.16.0.15 /24

Availability Zone A

Public Subnet

AWS Region, e.g. EU-West1

AWS Direct Connect

POP

TelecityGroup, London Docklands

Private VIF

Host

192.168.51.10

The Interwebs

VLAN / Sub Int

CustomerRouter

VPC CIDR 172.16.0.0/16

“Customer” Rack in Telecity

VLAN / Sub Int

CustomerRouterPrivate VIF

The Interwebs

DemoLet’s see how our use case was built on AWS

Our real life environment

Test Node172.16.0.15 /24

Availability Zone A

Public Subnet

AWS Region, e.g. EU-West1

AWS Direct Connect

POP

TelecityGroup, London Docklands

Private VIF

Host

192.168.51.10

The Interwebs

VLAN / Sub Int

CustomerRouter

VPC CIDR 172.16.0.0/16

“Customer” Rack in Telecity

VLAN / Sub Int

CustomerRouterPrivate VIF

The Interwebs

RT1

RT2

In summary

• Built our network foundations in AWS• Connected your onsite deployment to AWS• Added some redundancy into the mix• Demo: Took our environment live and

introduced some failures!

Steve SeymourSpecialist Solutions [email protected]

@sseymour

double redundancy with aws direct connect - pop-up loft tel aviv

Technology