scalable ra-vpn in aws · how an aws instance looks like inside of the cloud. please look at the...

Doc type / Publish Date

Cisco Public / 8.4.2020

© 2020 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.

Cisco Reference Architecture

Scalable RA-VPN in AWS

Version 1.1

Security Business Group Network Security Technical Marketing Engineering

05 August 2020

© 2020 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. of 31

Contents PURPOSE OF THIS DOCUMENT 3

PROBLEM STATEMENT 3

PREREQUISITES 3

SOLUTION COMPONENTS 4

AWS PUBLIC CLOUD COMPONENTS 4 AWS Regions 4 AWS VPC 4 AWS Subnets 4 AWS Route tables 4 AWS Gateways 4 AWS EC2 Instances 4 Cisco Firewall Offerings in AWS 5

SOLUTION DESIGN 6

AWS FIREWALL BASICS 6 (THIS IS AN OPTIONAL READ FOR THOSE WHO ARE NOT VERY FAMILIAR WITH AWS) 6 Generic Firewall Deployment 6 Cisco NGFWv and ASAv Deployment 7 AWS Networking and Firewalls 8

SCALABLE REMOTE ACCESS VPN SOLUTION IN AWS 10 VPC and Networking Design 11 Load-Balancing layer 12 AWS Route 53 services 12 AWS NLB and Global Accelerator 13 Firewall Deployment 13 End-to-End Traffic Flow 14 Recommendations 16

TOOLS AND RESOURCES 17

DEPLOYMENT GUIDE 19

INSTALLING TERRAFORM AND AWS-CLI 19 DEPLOYING THE PUBLIC-KEY 19 DEPLOYING THE VPC 22 DEPLOYING THE LOAD BALANCER ELEMENTS (IN CASE OF AWS R53 - SKIP THIS SECTION) 24 GENERATE ASAV CONFIGURATIONS (ONLY FOR THE AWS R53 DEPLOYMENT) 24 GENERATE ASAV CONFIGURATIONS FOR NLB (IN CASE OF AWS R53 PLEASE SKIP THIS SECTION) 26 DEPLOY AWS R53 HOSTED ZONE (ONLY FOR THE AWS R53 DEPLOYMENT) 26 DEPLOY THE ASAV INSTANCES 28 DEPLOY THE S2S VPN CONNECTIONS (ONLY FOR THE AWS R53 DEPLOYMENT) 29 DESTROY OR EXTEND DEPLOYMENT 30 GENERATING EXAMPLE CERTS USING AWS PKI 31


Purpose of this Document

Is to give guidance on how to set up a scalable Remote Access Virtual Private Network (RA-VPN) solution in Amazon Web Services (AWS). The solution, as part of this document, is using Cisco Adaptive Security Appliance Virtual (ASAv) Firewalls and Terraform infrastructure as a code framework to orchestrate the Cloud infrastructure and ASA firewalls’ initial deployment. All Terraform templates are shared for re-use.

Problem Statement

One of the main challenges for companies in today’s business environment is to provide a stable/scalable remote access virtual private network (RA-VPN) solution to their workforce. Since many business's current network capacity cannot support 100% of the employees work from home needs, it becomes a real challenge to expand it, especially when it's based on hardware appliances that have limited resources. Furthermore, it is important to note that these usually are temporary issues. So, investing in additional hardware appliances to scale up might not be a financially attractive option - as the resources would be required only for a brief period. This is what a cloud-based solution easily addresses, as the resources in Cloud can be easily instantiated or torn down at any given time. Hence, the investment for a temporary situation is 100% utilized.

Prerequisites

Please note that this document assumes a basic knowledge of the following topics:

- Cisco ASA software with Anyconnect VPN services

- Amazon Web Services Building Blocks

- Basic knowledge of DNS infrastructure

- Basic understanding of Infrastructure as a Code concept

- Basic knowledge of Terraform

As part of the solution, we have also assumed that your company has an existing AWS connectivity through AWS Direct Connect or other means such as S2S VPN through AWS's Transit Gateway, Virtual Private Gateway, or an AWS hosted Cisco CSR.


Solution Components

Since the term "Cloud" is relatively new, your company might not be prepared with any resources which would be very familiar with it. This section tries to give a high-level overview of the AWS building blocks to ease that unfamiliarity since our currently recommended VPN solution is based on ASA software. Therefore all the use-cases and supporting materials will use ASAv.

AWS Public Cloud Components

AWS Regions The AWS regions are different geographical locations around the world where AWS has a physical presence. As a customer, you choose a Region closest to you. All the Cloud resources are typically tied to a region. The region is divided into different availability-zones – which are the representation of different sub-geographical locations per region.

AWS VPC VPC (Virtual Private Cloud) in AWS is one of the most important building blocks. Consider it as a private/isolated space in the Cloud for the deployment of Cloud resources. When a VPC is created – AWS is asking for a custom CIDR to be provided, which typically has to be between /16 and /28. A best practice is to use a bigger block such as /16, which can then be broken down into smaller subnets and distributed across different Availability Zones. An AWS Availability Zone is a logical group of Cloud resources spread across isolated locations – Data Centers – within a Region. As a customer, you prefer to distribute your production applications between many Availability Zones for high availability, resiliency and scalability.

AWS Subnets Subnets represent normal network subnets. The only specialty about them in AWS is that when a subnet is created – it has to be tied to a particular Availability Zone. A good example would be a redundant server-farm where the Site-1 servers have IPs from the subnet of 192.168.100.0/24, which is assigned to us-east-1a, and Site-2 servers have IPs from a subnet of 192.168.200.0/24 which is assigned to us-east-1b.

AWS Route tables A route table contains a set of rules, called routes, that are used to determine where network traffic from your subnet or gateway is directed. The default route table is automatically created together with the VPC. It is possible to create custom route tables and associate subnets to them. Route tables are useful to define custom routes, overwrite default routes, and implement segmentation in the Cloud.

AWS Gateways Gateways in AWS are defined for specific networking activities in the Cloud. The two main gateway types are Internet Gateways and Transit Gateways. There are plenty of other options, but we will consider these two as part of the document. The Internet gateway (IGW) is essentially a distributed Internet breakout function that is attached to a VPC. Once an IGW is associated with a VPC, it can be utilized in a route table to provide internet access to any specific subnet. It is also enabling the subnet to have publicly available hosts in it. Essentially, a default route can be created in the routing table to point towards the attached IGW. On the other hand, Transit Gateway (TGW) is the other important gateway function in AWS. This is created to enable massive scale in the Cloud. TGW enables the communication between VPCs, which will open up new use cases such as centralized Security VPC, or dedicated VPC functions for multiple In/Outbound connections. TGW can also be used to extend the connectivity to your on-premises (on-prem) resources by utilizing dedicated connections like Site-to-Site (S2S) VPN or AWS Direct Connect service.

AWS EC2 Instances This component is another important element in AWS, as this represents the Cloud workload. Any workload in the Cloud – could be an application, database, network appliance, security appliance - is running as an EC2 Instance. AWS offers a wide variety of options when it comes to creating an EC2 instance. However, the type of instance will highly influence the monthly Cloud costs. In the following table, we will show what are the current Cisco Firewall (ASAv), Next-Generation Firewall (NGFWv) – also known as Firepower Threat Defene (FTDv), and the NGFW Firepower Management Console (FMCv) instances available and supported in AWS.


Cisco Firewall Offerings in AWS Cisco’s Firewall offerings in AWS give you the flexibility to choose between different types of virtual Firewall Appliances. In the following table, we list the available Cisco NGFWv and ASAv instances in AWS in detail (instances, management options, performance comparison, licensing, etc..). Table 1 –Cisco Firewall instance types for AWS

AWS Instance Type vCPU Memory: GiB Storage Max NICs** ASAv, NGFWv, FMCv

m4.large 2 8 EBS-Only 2 ASAv

m4.xlarge 4 16 EBS-Only 4 ASAv

c4.large 2 3,75 EBS-Only 3 ASAv

c4.xlarge 4 7,5 EBS-Only 4 ASAv, NGFWv, FMCv

c3.large* 2 3,75 EBS-Only 3 ASAv

c3.xlarge* 4 7,5 EBS-Only 4 ASAv, NGFWv, FMCv

c4.2xlarge 8 15 EBS-Only 4 FMCv

c3.2xlarge* 8 15 EBS-Only 4 FMCv

c5.large*** 2 4 EBS-Only 3 FMCv

c5.xlarge*** 4 8 EBS-Only 4 NGFWv,FMCv,ASAv

c5.2xlarge*** 8 16 EBS-Only 4 NGFWv,FMCv,ASAv

c5.4xlarge*** 16 32 EBS-Only 8 NGFWv,FMCv,ASAv

*Previous generation AWS instances **Maximum amount of network interfaces are based on AWS limitations per instance type.

***From the release of NGFWv 6.6 and ASAv 9.13.1


Solution Design

Users utilizing a Remote Access VPN service want to access the Information Technology (I.T) resources available in your Data Center (DC) or Head Quarters (HQ). In this document, we focus on how you can leverage the flexibility of the cloud to instantiate at speed and at scale your Remote Access VPN service. The solution can be taken apart for two layers. The frontend, and the backend. The frontend will be based on Cisco’s Anyconnect running on ASAvs with a DNS load-balancing in front. From the backend aspect, we will rely on the most popular use case, which is to utilize an existing cloud-interconnect service Site-to-Site (S2S) VPN or AWS Direct Connect through Transit-Gateway. The TGW acts as a central chokepoint in AWS, which provides inter-connect between VPCs, S2S VPNs, and AWS Direct Connect services. The Remote Access VPN traffic coming from the frontend will be backhauled through the TGW towards the on-prem resources. NOTE: In this document, we assume that your company already has some sort of Cloud presence in AWS and that you have AWS Direct Connect service or S2S VPN capabilities available, which gives you direct connectivity towards your on-prem resources.

AWS Firewall basics

(THIS IS AN OPTIONAL READ FOR THOSE WHO ARE NOT VERY FAMILIAR WITH AWS)

Generic Firewall Deployment Before going into too much detail on the scalable design, we need to clarify the basics when it comes to deploying a generic Firewall Instance into AWS. The most important point to highlight is when deploying to the Cloud, we need to consider some of the cloud-specific behavior and limitations, which depend on the Cloud provider, in this case, AWS. To give a basic understanding of those specifics, we need to break them down to a very granular level on how an AWS instance looks like inside of the Cloud. Please look at the following diagram (Figure 1), which represents an AWS C5 instance inside of an availability-zone. The instance has four network interfaces, which are considered as separate AWS building blocks, and they are associated with an instance through an attachment. Another association happens between the subnet and the Network Interfaces. AWS has an interesting behavior when it comes to networking, which is related to a subnet. Each AWS subnet has a reserved number of IPs that AWS reserves for special purposes. These are the reservations to keep in mind (A.B.C is the subnet CIDR):

A.B.C.0: Network address.

A.B.C.1: Reserved by AWS for the VPC router. (First address of the subnet)

A.B.C.2: Reserved by AWS. The IP address of the DNS server is the base of the VPC network range

plus two. For VPCs with multiple CIDR blocks, the IP address of the DNS server is located in the primary

CIDR. We also reserve the base of each subnet range plus two for all CIDR blocks in the VPC.

A.B.C.3: Reserved by AWS for future use. (Third IP address of the subnet)

A.B.C.255: Network broadcast address. AWS does not support broadcast in a VPC. Therefore, AWS

reserves this address.


Figure 1 - AWS Zoom-In Instance

The main message here is that the first four IP addresses and the last IP address in each subnet CIDR block are not available to be used, and are unassignable to an instance.

Cisco NGFWv and ASAv Deployment When we select the instance to deploy our Cisco Firewall – we need to consider the type of instance which will define some of the core capabilities of the firewall, such as the number of physical interfaces (see Table 1).

In the following diagram (Figure 2), it's shown how a simple NGFWv instance looks like in a single VPC inside of an availability-zone. This particular firewall is running on a C5.xlarge instance, which has a hard limit on four network interfaces to be used by the firewall. Because NGFWv requires one interface for Management and one interface for the Diagnostics interface, the leftover are two interfaces. These remaining two interfaces can be used to attach the firewall to an AWS subnet (inside of the same Availability-Zone).

In an example scenario the first interface would be attached to a Public subnet. The second interface would be internally-facing, connected to a Private Subnet, and forwarding the traffic towards the Direct-Connect or VPN gateway. The Management (interface 0) and Diagnostic interfaces are attached to a Management subnet, where the FMC, and maybe a jump box (virtual management station) would be deployed.

Typically, this design is replicated across availability-zones to ensure high-availability and high-resiliency. In case of the C5.4xlarge type instance, the firewall can connect to more than two or three data subnets, because it can scale up to 8 interfaces (2 Mgmt/Diag (for FTD) + 6 Data networks or 7 for ASAv).

Figure 2 - NGFWv on C5.xlarge example

The same design works with ASAv. In ASAv’s case, the Diag interface is not required for Firewall management and can be attached to another private network instead. A typical design for an ASAv is shown below (Figure 3).


Figure 3 - AWS - ASAv simple design

AWS Networking and Firewalls In AWS, each instance by default does "Source and Destination check." This means the EC2 instance must be the source or destination of all the network traffic it sends or receives. However, this will not work for a firewall instance because the Firewall acts as a Gateway. Therefore, this feature must be turned off, as well as for the associated interfaces. Once the firewall has been deployed in the AWS environment – we need to consider how we will route the traffic towards our firewall. The answer to that is the AWS Routing Table building block.

There are multiple options on how the routing table could look like in AWS as it has a very flexible way of working. For example, we could point a route and set the next-hop to be an instance. However, in the case of firewall, our instance has multiple interfaces and IPs associated with it, and the result is that AWS infrastructure does not “resolve,” which is the correct path. To overcome this issue, we need to always point the route towards the firewall interface. On top of route table definition, we also need to associate the AWS Subnet to a routing table, meaning that we instruct AWS to apply the routing table only for the associated subnets.

NOTE: That there is a default route table when you create the VPC and all subnets initially associated with this route table. Since this default route table can't provide good enough control on how we route the traffic in each of the subnets, it's a good practice to create a new routing table per function of the subnet or per subnet.

Our hosts and the instances are tied to a separate availability-zone and can't communicate with each other. First, a Firewall is needed if we want to inspect the N/S or the E/W traffic between subnets inside of the availability-zones. Second, we need to create a custom route table for each private subnet(s) pointing the default route to the firewall interfaces - in each availability-zone, to force the traffic through the local firewall. Please see the following example explaining the route tables setup for the topology below (Figure 4).


Figure 4 - Example AWS setup with load balancers

To achieve this topology, we need to have the following route tables defined:

Table 2 - RT for Management Traffic

Route Table for MGMT

Destination Target Description

100.0.0.0/16 local VPC routes

0.0.0.0 IGW Default route to IGW

Table 3 - RT for Webserver traffic in AZ-A

Route Table for WWW-A



0.0.0.0 Firewall interface (eni) in AZ-A Default route the FW ENI

Table 4 - RT for Webserver traffic in AZ-B

Route Table for WWW-B



0.0.0.0 Firewall interface in AZ-B Default route to the FW ENI

Table 5 - RT for Public/Internet Facing subnets

Route Table for Public Subnets



0.0.0.0 IGW Default route to IGW

This is it in a nutshell about the firewall deployments in AWS. Let's now move on to the use case.


Scalable Remote Access VPN Solution in AWS

The Scalable Remote Access Virtual Private Network (RA-VPN) solution has been designed to help companies that wish to scale up their VPN concentrators at a fast pace by leveraging the cloud-native scaling mechanisms. As part of this document, we refer to two different types of scaling mechanisms. The preferred and recommended method is to use AWS Route 53 or any other DNS based load-balancing. The second and alternative approach could be to use AWS Network Load-balancer enhanced with AWS’s Global Accelerator Services. For the authentication part in the provided sample, you will see Local Authentication is used. However, the solution could be easily extended to leverage existing on-prem or cloud hosted AAA/MFA services such as ISE, AD, and DUO. As part of this solution the following AWS building blocks will be used:

- AWS VPC

- AWS Subnet

- AWS Routing Table

- AWS Elastic IP

- AWS EC2 Instance

- AWS Network Interfaces

- AWS Transit Gateway

- AWS Route 53 Cloud-based DNS Service

- AWS Network Load Balancer

- AWS Target Groups

Figure 5 - Solution Reference Topology


VPC and Networking Design

In this design, we reference a single VPC that is connected to a Transit Gateway and has subnets defined in at least two availability-zones for high-availability. The subnets defined in this VPC are the followings:

- Public Subnets (Internet Facing)

- Private Subnets (Internally Facing)

- Management Subnets (For management needs)

- Transit Gateway Subnets (Only required for TGW peering)

The same design would work in an environment where you may have multiple VPCs. (Transit-Gateway with multiple VPCs) In that case, the VPN front-end should be hosted in your Inbound Security VPC. For the Public, Management and Transit Gateway subnets, it is recommended to define a single routing table. However, for the Private subnets, it is recommended to define a separate routing table per availability-zone because there is no inter-availability-zone communication possible in AWS. Therefore, we need to point the default route towards the internally-facing interface of the "local" ASAv. (There is only one default route per subnet is allowed, but the provided terraform framework takes care of this) NOTE: This design will also help you achieve the optimal routing of the VPN traffic through AWS. At the edge of the VPC, we have deployed an Internet Gateway, which provides Inbound/Outbound Internet connectivity for the components deployed in our VPC's Public Subnets. In a Transit Gateway design with multiple VPCs, your Inbound and Outbound connectivity will most likely be handled by different VPCs.

Figure 6 - Stage 1 Deployment of the Network


Load-Balancing layer

Load-Balancing Layer is one of the most important parts of the scalable solution. Therefore, we have two options documented. The first is the AWS DNS based load-balancing, and the second is a Global Load Balancing solution using NLB and Global Accelerator. The first option is available by AWS Route 53 cloud-native DNS service and is the recommended by AWS. This option supports both TLS and DTLS protocols for Anyconnect. Alternatively, if DTLS support is not important for your company, you could use the NLB based solution.

AWS Route 53 services

The following three major components of the service are playing the part of AWS Route 53 DNS based load-balancing.

Hosted Zone:

o The AWS Route 53 Hosted zone is essentially the domain that is delegated to be managed by the service.

For example, if you want to have the VPN service running on vpn.example.com, you need to create a hosted

zone for “example.com” and when you create the policy record, it will create the “vpn.example.com” sub-

domain underneath.

Traffic Policy:

o The traffic policy is the definition of the load-balancing policy on AWS Route 53. It could be many different

types of rules created by utilizing the Traffic Flow Editor of AWS or Terraform automation. However, in

this case, we recommend you use the “Weighted rule” where you could add a custom amount of “weights”

which will be pointing to the Elastic/Public IP addresses of the ASAvs. The weight defines the load

distribution towards the endpoint. The weight calculation is simple – it sums up the weight you assigned

to the endpoints and divided by the weight for a specific record. For example, if you want to send a tiny

portion of your traffic to ASAv1 and the rest to another ASAv, you might specify weights of 1 and 255. The

resource with a weight of 1 gets 1/256th of the traffic (1/1+255), and the other resource gets 255/256ths

(255/1+255). You can gradually change the balance by changing the weights. If you want to stop sending

traffic to a resource, you can change the weight for that record to 0. If you want to distribute the load

equally, then you could change all weights to 0 in your ruleset.

Policy Record:

o The policy record is where you can define the domain you want to apply the policy you have created. In

this case, it will be “vpn.example.com”

Figure 7 - Reference Topology for the Route53 based Load-balancing

If you have a regional setup with multiple geographical locations, then you could also consider using rules in your traffic policy, which are based on latency or geographical proximity.

Please also keep in mind that if you want your existing domain to be working together with AWS Route 53 services, you would be required to add or replace the current NS (nameservers) with the AWS Route 53 nameservers which you get when you create the hosted zone.


You could read more about the AWS Route53 traffic policies on the following link: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/traffic-policies.html

AWS NLB and Global Accelerator

We wanted to provide you an alternative option based on AWS’s Network Load-Balancer enhanced with AWS’s Global Accelerator services. In this case, behind the Internet Gateway, there is an Internet-Facing "AWS Network Load-Balancer" with Cross-Zone Load-Balancing enabled. Cross-Zone Load-Balancing is disabled by default. However, if you have resources in multiple availability-zones, it is necessary to enable it to leverage the benefits of different availability-zones (the provided terraform template considers this). The Load-Balancer has a listener enabled on TCP/443 port, which makes available the Cisco Anyconnect service running on any of the ASAvs and acts as the front-end of the Remote VPN service. On this listener, you define a Target Group that groups together your ASAv instances with a default action to load balance the incoming traffic between the ASAv group members. Since we leverage automation for the setup, the targets are based on IP addressing. The provided terraform template dynamically takes care of when it instantiates the ASAv.

Figure 8 - Stage 2 - Deployment of the LB layer

IMPORTANT: This Load-balancing layer is representative to show you an example of a scalable design. This solution with the NLB will not support DTLS.

In case if you require a regional design where you have multiple Load-Balancers in different regions, you need to

attach AWS’s Global Accelerator Service. The AWS’s Global Accelerator Service acts like a Geographical load-balancer and enables you to have a multi-regional design. You could also have different pools in the region with different network Load-Balancers. Hence, the traffic distribution could be controlled within the region or across the regions too.

For more information visit: https://aws.amazon.com/global-accelerator/

Firewall Deployment

Going towards the Internal part of the AWS network, the next in the path are the ASAv devices. All ASAv are deployed in at least two different availability-zones with Public IPs associated to the outside interfaces. In the case of a network Load-Balancing design, they are also registered to the Target Groups as targets, and the load-balancer distributes the VPN traffic evenly. On the ASAv, there is a basic Anyconnect Zero-Day configuration, which can be customized according to your company's requirements. The Zero-Day configuration is provided along with the terraform templates.

https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/traffic-policies.html

https://aws.amazon.com/global-accelerator/


Figure 9 - Stage 3 - ASAv Config generation

A couple of important points here regarding the ASAv configuration:

- VPN pool configuration. All ASAvs run as individual appliances without any high availability between the

nodes. That allows us to potentially have all ASAvs sharing the same VPN pools and define an inbound

NAT/PAT for all the VPN traffic. Alternatively, we could also define unique VPN pools per ASAv. (This is a

configurable option in the Terraform framework)

- Certificates. This is critical for having the Anyconnect part of the solution working as expected. You must

setup all proper certificates on the ASAv by having in mind that the ASAs are not synchronized. Therefore,

the same Certificate needs to be imported and used on each ASAv. In this example setup, we've used AWS's

Certificate Authority to generate a private Certificate for the service DNS names. (The procedure will be

described in the deployment guide section)

Figure 10 - Stage 4 - ASAv deployment

End-to-End Traffic Flow

When Remote Access VPN user traffic hits the ASAvs through Route53 or the Load-Balancers, the VPN tunnel is established through one of the ASAv in the pool. The route which must be defined on the ASAv (or included in the zero-day configuration) must point towards the On-Prem subnets via the AWS VPC router of the Private subnet (.1 IP address of the subnet in each AZ). In the following example, the “192.168.128.0/24” is the Company’s On-Prem subnet, and the 100.100.11.1 is the VPC router of the Private Subnet in the “A” availability-zone.

route inside 192.168.128.0 255.255.255.0 100.100.11.1 1


There is another step that is required here to define the routes towards the on-prem networks in the Private Subnets Routing table to point to TGW or VPGW. After the network traffic leaves ASAv, the VPC will be able to forward the traffic in the right direction towards the Transit Gateway or Virtual Private Gateway. To achieve the optimal end-to-end reachability, there are some more steps required depends on which way the VPN-pools are defined on the ASAvs and whether NAT/PAT is implemented for the VPN traffic or not:

1. NAT or PAT: Network or Port Address Translation (NAT/ PAT) of the VPN traffic (outside, inside)

on the ASAvs. In this solution, AWS is not aware of the VPN pools hence NAT/PAT is required to

let AWS know where to send the return traffic. If you prefer, the VPN pool can be the same on all

the ASAvs since they will be NAT-ed behind the individual ASAv inside interface. You must also

configure a route entry on the Private Subnet Route Table (per availability-zone), which points to

the On-Prem subnets through the Transit Gateway.

2. “Ghost Pool”: In this case, NAT is not required on the ASAvs. You must allocate a VPN-Pool per

ASAv from a CIDR and must utilize an IP range outside of your VPC CIDR (i.e., 172.16.100.0/24 is

outside of 10.0.0.0/16 range). Allocate, means theoretical allocation as you need to keep in mind

those subnets. As an example, your VPN pool is 172.16.100.0/24, and it will be used by ASAv1 in

the “A” availability-zone. To make AWS understand the routing, you need to define two more entries

on top of the already mentioned route at the beginning of this section. One of the routes must be

created in the Transit Gateway Routing Table, routing the traffic of 172.16.100.0/24 via the VPC

attachment towards the VPC. At this point, the traffic reaches the VPC, so here comes our second

route that must be created in the VPC’s Transit Gateway specific Routing table this time. The VPC

routing table must be the one that has been created for Transit Gateway peering. Under the VPC,

there is a special routing table for TGW peering with only local routes defined. You need to create

an extra route in this route table, which defines the “ghost” VPN pool pointing towards the “inside”

interface of the ASAv in that availability-zone. The reason it’s called “Ghost Pool” because AWS

doesn’t have this network defined – it only exists in the routing tables. AWS is supporting this option

too.

The following is an example to show what is required to achieve the “Ghost Pool” option for one availability-zone:

ASAv Route Table in AZ-A


0.0.0.0/0 Default route receive on “outside” int via DHCP

Default

192.168.128.0/24 100.100.11.1 (route defined on the inside int) On-Prem subnet

VPC Private Subnet Route Table in AZ-A


100.100.0.0/16 local Required

0.0.0.0/0 Inside Network Interface of the ASAv (eni) Default for E/W

192.168.128.0/24 Transit Gateway (sending traffic to On-Prem via TGW)

On-Prem

VPC Route Table for TGW peering


100.100.0.0/16 local Required


172.16.100.0/24 Inside Network Interface of the ASAv (eni) – VPN pool

Static

Transit Gateway Security Route Table


100.100.0.0/16 VPC Attachment (points to VPC) Propagated

172.16.100.0/24 VPC Attachment (this is required for every VPN pool)

Static

192.168.128.0/24 VPN or Direct Connect GW (On-Prem resources) Static

Recommendations In this design, we recommend choosing the ASA 9.13.1 version from the AWS marketplace. This version supports the new C5 instance type with 10Gbit interface capability that gives you equivalent feature tier as the ASAv50 in a Private Cloud.

Regarding the ASAv running ASA 9.13.1 version, the following information applies from throughput and licensing aspect: https://www.cisco.com/c/en/us/td/docs/security/asa/asa913/asav/getting-started/asav-913-gsg/asav_intro.html

An important highlight is when you use the provided terraform framework, it is enough to define your smart licensing token in the meta-data. The ASAvs will register automatically after boot as long as you have the right entitlements under your Cisco Smart Account.

There are some AWS specific limitations applicable for the Network Load-Balancer on AWS. Please see the following link for more information: https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-limits.html

https://www.cisco.com/c/en/us/td/docs/security/asa/asa913/asav/getting-started/asav-913-gsg/asav_intro.html

https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-limits.html


Tools and resources

The provided Terraform template will help you to deploy in AWS the whole solution, as is explained above. It is important to highlight that the solution design and a successful implementation is not relying on Terraform. It could be done by other automation frameworks or tools, as well as native AWS Cloud Formation templates. To ease the deployment, we have created terraform templates, which are capable of automating the following activities:

- AWS SSH Key instantiation

- AWS VPC Creation

- AWS Subnet Creation

- AWS Route Table Creation

- AWS Route Table Association

- AWS Default Route Creation

- AWS Transit Gateway Creation

- AWS Transit Gateway route association and propagation with VPN attachments

- AWS Route 53 Hosted Zone Creation

- AWS 53 Dynamic Record Creation

- AWS 53 Health-check Creation

- AWS Network Load balancer Creation

- AWS Target Group Creation

- AWS Target Registration to Target Group (Target Group Attachment)

- AWS EC2 Instance Creation

- Cisco ASAv dynamic zero-day configuration generation

- Cisco ASAv instantiation in the Cloud (custom number of instances with auto-target group registration)

All Terraform modules are implemented according to Infrastructure as a Code methodology, and the deployment will happen in stages. These stages are the same as the solution design steps described already in this document depends on the solution you choose to implement:

- AWS Route 53 based solution deployment steps:

o Stage 0: Deploying an SSH Key which will be your management key to access the devices

o Stage 1: Deploying the VPC and all the related network basics

o Stage 2: Generate the ASAv 0-day configurations

o Stage 3: Deploy the AWS Route 53 frontend

o Stage 4: Deploy the ASAv Instances

- AWS NLB based solution deployment steps:

o Stage 0: Deploying an SSH Key which will be your management key to access the devices

o Stage 1: Deploying the VPC and all the related network basics

o Stage 2: Deploying the Load-balancer elements

o Stage 3: Generate the ASAv 0-day configurations

o Stage 4: Deploy the ASAv Instances

The following figure (Figure 11) represents the structure of the Terraform framework provided. The top-line "Providers" are part of the Terraform framework. The small cubes called Modules are written by us, and the bottom "EnvX" sections are representative environments – in this case, the Scalable Remote Access VPN with the two different load-balancing options. “EnvX” uses the above Modules to implement the solution. Furthermore, there is the "Metadata" section, which is essentially the inputs such as network CIDRs, subnet names, etc.…


Figure 11 - Terraform Modular Diagram

Link to the Git Repository: https://github.com/netsec-design/TerraformBluePrints

https://github.com/netsec-design/TerraformBluePrints


Deployment Guide

DISCLAIMER: Please do not run this in your production environment. First, test out the solution, familiarize yourself with the framework, customize it to your Company needs, and after successful testing, deploy it in production.

Installing Terraform and AWS-CLI

There are a lot of resources out there to help get involved with Terraform. You can download it for free and learn more about how to install using the following link: https://learn.hashicorp.com/terraform/getting-started/install.html

You must install AWS-CLI and create an API access for your AWS account - Terraform requires this for the deployments. The following link will explain how to setup aws-cli and API user account:

https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html

Deploying the Public-Key

Once the above tools are installed, the next step will be to prepare the public-private key-pair, as it will be used later on for management access purposes. When the firewalls or other EC2 Instances will be deployed, they will use public-key authentication to authenticate the administrators. You only need to deploy the public-key to AWS as the private key will be somewhere safe on your computer. To prepare a key-pair, you can use the "ssh-keygen" tool. The following example key-pair has been generated under macOS:

After the key-pair has been generated we need to copy/paste the public-key into the metadata.

$ cat aws-test.pub

https://learn.hashicorp.com/terraform/getting-started/install.html

https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html


Take the output of this and open “prod/scalable_ravpn/variables/main.tfvars” file and fill out the following variables:

#global aws-region = "us-east-1" # SSH Key Variables ssh-key-name = "aws-test" ssh-key = "ssh-rsa

AAAAB3NzaC1yc2EAAAADAQABAAABgQDf3QBWev86il5bJUk8PtdwVAPoRhasULcC4s+nT8wXfzICZvTBxYodtfaDX1d/nmIDnRSXqhSBusAUonc5PwCUJPUZaHSTvzI++Tzy2XefArJeUPRBsZ0pIyf8HFzOUiEOLayUgKYdhWd9c7PYS8PEr84VKaNvcgFrskt5QsMLJzupu3Jhm8C5pfzgdc9T0MUVQR3WdR5UnlR1RWBK1YiiSNL4k9ARU1akBhKEjvnJ9QuDVdlN23mi9OKYAuyzpG9t0KM5uDi2F2pKUSJqInLMssdbiDgaDjR8iFv7C/G5bQSuqaeL9vf4qckS+GrMG8FpC/vdUb/vsLGVjxz85fHkoxjENqghby3gEKyGGPDLpyVXC1XuRY/uNkuHkw56G6fGJJj6zOQUFSZRcqWyJNymqOExBMVnF8zhB1u4J4VwcUjfbWvwtkNQrz2ZF3m53hu4a6owtEjtuBz5F6baSlVLXXwl608iCUpfjPyQGYJwVpVn58Snaoj5otfFGcc2C3c= [email protected]"

Keep in mind that the region is a global variable, so all the following instances will be deployed under that region. If you require resources across multiple regions, we recommend copying the prod folder per region – so you will have an environment per region. After filling out the parameters, the key is ready to be deployed in AWS. There is a deploy.sh in each stage folder, which made it easy to not have to type in the terraform plan/deploy/destroy commands every time. The scripts consist of the following command:

terraform apply -var-file=../variables/main.tfvars

To deploy the public-key, navigate to the "prod/scalable_ravpn/0_aws_ssh_key" or "prod/scalable_ravpn_r53/0_aws_ssh_key" folder and run the “terraform init” command and then "deploy.sh." Once the deployment script has finished, Terraform will suggest the resources which will be deployed, in this case, the AWS Key. If you are happy with the plan, type "yes" to accept and hit enter. After that, Terraform deploys it in AWS.


Figure 12 - Deploying AWS Key with Terraform

You can read more on AWS's public key guidance: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#how-to-generate-your-own-key-and-import-it-to-aws If everything went well in this section, that's a good indication that your environment is set up correctly.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#how-to-generate-your-own-key-and-import-it-to-aws

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#how-to-generate-your-own-key-and-import-it-to-aws


Deploying the VPC

When you run the terraform to deploy your networking on AWS, these are the important components from the VPC aspect:

The module that deploys the VPC components is located under: "modules/aws/vpc"

The module typically doesn't need to be modified as it defines the creation logic on an abstract level. When you instantiate it, you need to give the custom values such as VPC CIDR, Subnet names, Internet GW name, etc.

Our example VPC environment which instantiates the module is located under: "prod/scalable_ravpn/1_aws_network" or "prod/scalable_ravpn_r53/1_aws_network" and “prod/scalable_ravpn_r53/2_tgw_associations”.

If you deploy the example environment, you will have a VPC ready with all the basic components. If you want to modify or just simply check the details of the network configurations, open the "prod/scalable_ravpn/variables/main.tfvars" file and check the following variables:

#VPC Parameters main-cidr = "10.0.0.0/8" #required for transit-gw (in case if there are multiple vpcs the main /8 subnet will be routed through TGW) vpc1-name = "CloudEdge_Terraform" #Name of the VPC vpc1-cidr = "100.100.0.0/16" #CIDR of the VPC vpc1-type = "sec" #Type of subnet in the TGW scenario (it can be sec=Security or Spoke – in this case this is a Security – for Inbound Security) public-subnet-a = "CloudEdge-Pub-A" #Name of the Public subnet in AZ-A public-subnet-b = "CloudEdge-Pub-B" #Name of the Public subnet in AZ-B private-subnet-a = "CloudEdge-Priv-A" #Name of the internal subnet in AZ-A private-subnet-b = "CloudEdge-Priv-B" #Name of the internal subnet in AZ-B mgmt-subnet-a = "CloudEdge-MGMT-A" #Name of the management subnet in AZ-A mgmt-subnet-b = "CloudEdge-MGMT-B" #Name of the management subnet in AZ-B tgw-subnet-a = "RAVPN-TGW-A" #name of the TGW peering subnet in AZ-A tgw-subnet-b = "RAVPN-TGW-B" #name of the TGW peering subnet in AZ-B public-cidr-a = "100.100.110.0/24" #CIDR of the Public Subnet in AZ-A public-cidr-b = "100.100.120.0/24" #CIDR of the Public Subnet in AZ-B private-cidr-a = "100.100.11.0/24" #CIDR of the Private (Internal) Subnet in AZ-A private-cidr-b = "100.100.12.0/24" #CIDR of the Private (Internal) Subnet in AZ-B mgmt-cidr-a = "100.100.1.0/24" #CIDR of the Management subnet in AZ-A mgmt-cidr-b = "100.100.2.0/24" #CIDR of the Management subnet in AZ-B tgw-cidr-a = "10.10.111.0/24" #CIDR of the TGW peering subnet in AZ-A tgw-cidr-b = "10.10.222.0/24" #CIDR of the TGW peering subnet in AZ-B igw-name = "CloudEdge-IGW" #Name of the Internet Gateway rt1-postfix = "RT1-PUB" #NamePrefix for the Public Route Table (naming conv. is important!) rt2-postfix = "RT2-PRIVA" #Name Prefix for the Private Route Table for AZ-A rt3-postfix = "RT2-PRIVB" #Name Prefix for the Private Route Table for AZ-B rt4-postfix = "RT3-MGMT" #Name Prefix for the Management Route Table rt5-postfix = "RAVPN-RT4-TGW" #Name Prefix for the TGW Route Table #TGW Parameters tgw-name = "RAVPN-TGW" #Name of the Transit Gateway tgw-rt1-name = "RAVPN-TGW-Security". #Name of the TGW Security Route Table tgw-rt1-type = "sec" #Type of the TGW Security Route Table tgw-rt2-name = "RAVPN-TGW-Spoke" #Name of the TGW Spoke Route Table tgw-rt2-type = "spoke" #Type of the TGW Spoke Route Table #ON-Prem CIDR definitions for the S2S VPN Setup on-prem-cidr = "192.168.128.0/24" #definition of the on-prem subnet in a CIDR format on-prem-pool = "192.168.128.0" #definition of the on-prem pool the same as above on-prem-netmask = "255.255.255.0" #definition of the on-prem netmask same as above

In this example, we have defined a single VPC with an Internet Gateway and a Transit Gateway attached

to it. The Transit Gateway has two route tables, one for the Security VPCs and the second for the Spoke

VPCs. If you require multiple VPC, subnets, route table, transit gateway route table, or any customization,

it is required to modify the "prod/scalable_ravpn_r53/1_aws_network/main.tf” or the

"prod/scalable_ravpn/1_aws_network/main.tf" file. Just a simple example – if you would like to create a

new VPC, then copy the whole "module "vpc1" {…} "section and replace "vpc1" to "vpc2" than define all

the variables required and create a new section in the “main.tfvars” file for the metadata of the new VPC.

To get this example deployment to work well, we recommend not to modify the basic setup if you don't


have deep knowledge of Terraform or AWS. The simplest things to modify are the CIDRs and names as a

start (the variables above), which can be done without any additional changes.

IMPORTANT: If you deployed your infrastructure with Terraform – it will keep a "state" file of what has

been deployed. If you make any changes in the configuration of an existing deployment, it will result in

destroying or changing some existing resources. If you wish to customize certain things, it is recommended

to create a new "environment" and test it there.

To deploy the VPC networking, navigate to the "prod/scalable_ravpn_r53/1_aws_network" or "prod/scalable_ravpn/1_aws_network" folder and run the “terraform init” then execute the "deploy.sh" in the library in a similar way to the ssh-key. It will create a plan for you about the resources to be deployed and needs to be accepted by typing "yes" (as long as you agree with it). In case if you deploy the R53 design it consists of a Transit Gateway as well where you need to deploy some of the associations – so please also navigate to the “prod/scalable_ravpn_r53/2_tgw_associations” ( in case of R53 design) and do the same – execute the “deploy.sh” to have the VPC and the route tables associated with the TGW.


Deploying the Load Balancer elements (in case of AWS R53 - skip this section)

Once the VPC networking part has been successfully deployed, the next step is to deploy the Load-balancers. In this example, we will deploy two different load-balancers – one internal and one external. The definition of the Load-Balancers are in the same "variables/main.tfvars" file as the sections before.

Note that in the implementation, the internal Load-Balancer is using a different module (“modules/aws/nlb”) than the external Load-Balancer (“modules/aws/nlb_enhanced”). It is because the workloads module (“modules/aws/ec2/instance”) does not yet support the enhanced Load-Balancing layer module, as is written for the firewalls part. Therefore, some of the variable definitions are slightly different between the two types of load-balancers. (It is an improvement point for the framework).

The module location for the external Load-Balancer is under "modules/aws/nlb_enhanced/main.tf" and

it supports AWS Network Load-Balancer creation with a dynamic configuration for the listeners and target

groups coming from the metadata. So, if you want to add extra ports to your Load-Balancer, you must

extend the "nlb1-fw-config = {…}" section in the "variables/main.tfvars" with a "key = value" pair where

your key will be the port number and the value will be the protocol. Since in this setup, we use "IP" based

target groups only "TCP, TLS, HTTP, HTTPS" protocols are supported at this point (as per AWS). Please

keep in mind the "name" parameter of the "nlb1-tg-config" - which is the target group configuration, will

be used to create the target group attachments in the firewall deployment section, and this parameter acts

as a prefix. The final target group name will be defined based on the forwarding configuration: as many

ports as you define – the same amount of target groups will be created with the name as a prefix + protocol

+ port. (for example: TG01ASAvTCP80 – TG01ASAv = name, TCP = protocol, 80 = port)

#NLB Parameters #params: #Name - LB name #Internal - boolean - to say whether the LB should be internal or external facing #Cross-Zone - enabled? = Bool to enable cross-zone loadbalancing #tg-config: target group configuration (multiple target groups are being created based on forwarding configuration this is a dynamic section) #name inside of tg-config: prefix to create the target group - the final name will look like: ${prefix}+${protocol}+${port} - for example TG01ASAvTCP80 #nlb1-fw-config: forwarding configuration with given amount of ports #External Load-balancer definiton nlb1-name = "NLB01External" nlb1-internal = "false" nlb1-cross-zone = true nlb1-tg-config = { name = "TG01ASAv" target_type = "ip" health_check_protocol = "TCP" } nlb1-fw-config = { 80 = "TCP" 443 = "TCP" } #Params for the internal load-balancer are slightly different as for the workloads module does not support my enhanced LB layed module yet. #Not a big deal, but it's a bit more static config in this case - doesn't have impact on the RAVPN front-end #NLB2-Params nlb2-name = "NLB02Internal" nlb2-internal = "true" nlb2-tg-group = "TG02WWWInternal" nlb2-cross-zone = true

The parameters for the second Load-Balancer are not important for this document. You may take this as an optional step to represent a hosted solution such as web servers deployed in AWS, which could also be protected by the ASAv. To deploy the Load-Balancer layer, please navigate into the "prod/scalable_ravpn/2_aws_lb_layer" folder and run the “terraform init” and then execute the "deploy.sh" script which will run terraform deploy and will come up with a plan. If you agree with the plan – type "yes" and your load-balancing layer will be deployed as per your metadata.

Generate ASAv configurations (Only for the AWS R53 Deployment)

The ASAv configuration happens straight after you have deployed the VPC networking. The ASAv

configuration module will iterate through on the list of ASAvs you defined in the "asa-instances = {…}"

section of your "variables/main.tfvars" file. This section represents a list of ASAvs with the following

parameters:

ASAv01={ #Please make sure that the name of the ASA will be unique as it is used as a key value for the set availability-zone = "a" #availability-zone where to deploy the ASAv template-file = "../2_asav_config/ASAv01.txt" # and the name of the config for your ASA will be the same as the name of the instance which you've defined in

the first line


token = "" #smart licensing token from your Cisco Smart Account (copy paste) default-to-private = true #important parameter – you can designate 1 ASAv per availability-zone to be the default gateway for the traffic coming from the Private

subnet towards outside. It will define a default route into the Private Subnet route table for the selected ASA's internal interface. So this parameter will be true only for one ASAv per AZ.

vpn-pool-from = "192.168.6.1" #VPN Pool from param vpn-pool-to = "192.168.6.254" #VPN Pool to param vpn-pool-mask = "255.255.255.0" #VPN pool subnet param attach-to-dns = yes #It is required if you want to create an A Record for the ASAv in the DNS zone so it will be part of R53 Load-balancing weight = 10 #weight is required so you could able define how much percentage of the traffic you would want to send towards each ASAv (equal weight = equal

load-balancing) },

You can define as many ASAvs as your company requires, and they will be dynamically created out of this

list. Please make sure that you read the comments of the parameters. You have the option to use the same

or a unique token parameter to register your ASAv with Smart Licensing. Once the ASAvs have been

defined in the “main.tfvars” file, navigate to the "prod/scalable_ravpn_r53/3_asav_config" folder and

run the “terraform init”. Then execute the "deploy.sh" to generate the zero-day configuration for the

ASAvs.

NOTE: If you want to deploy more than 6-8 ASAvs you need to increase Terraform's parallelism option in

the "deploy.sh" on the following way:

terraform apply -parallelism=20 -var-file=../variables/main.tfvars

The above will increase the amount of threads, and Terraform will start to generate the configurations. It could also happen that you need to execute the "deploy.sh" a couple of times as the configs are generated from the "terraform output" - sometimes it takes multiple runs to fill up all the configurations in case of ten or more ASAvs.

Another tip if you want to tweak the zero-day configuration or are simply curious, please check the template under "prod/scalable_ravpn_r53/3_asav_config/asav_config_template.txt". This is the basic configuration template that will be deployed on each of the ASAvs. Because this is an ASA zero-day config, you must be careful of the configuration commands ordering for successful application and make sure the order of the commands are not modified.

In terms of content or extension of the configuration, that can be done easily. If you want to define new parameters, you can see in the configuration template that all the variables are between ${}, enabling the Terraform framework to be easily extended to support more variables:

hostname ${hostname} ! license smart feature tier standard throughput level ${license_throughput} call-home

If you wish to run ASAv utilizing an older than 9.13.1 version on an older EC2 instance, such as C4 - please check the extra configuration template called asav_config_template_C4.txt. The main reason for having a different template is the gigabit interface support. The name of the interface on the ASAv is changing from TenGigabitEthernet to GigabitEthernet when you change from C5 instance to an older C4 instance. Hence, it's required to have a different template file for those.


Generate ASAv configurations for NLB (in case of AWS R53 please skip this section)

The ASAv configuration only happens after the Load-Balancers have been deployed. As the module

queries the IP of the Internal and External Load-Balancers in the given availability-zone, it will create the

objects and relevant NAT configurations for the firewalls. The ASAv configuration module will iterate

through on the list of ASAvs you defined in the "asa-instances = {…}" section of your

"variables/main.tfvars" file. This section represents a list of ASAvs with the following parameters:

ASAv01={ #Please make sure that the name of the ASA will be unique as it is used as a key value for the set availability-zone = "a" #availability-zone where to deploy the ASAv template-file = "../3_asav_config/ASAv01.txt" #the name of the config for your ASA will be the same as the name of the instance which you've defined in the first

line token = "" #smart licensing token from your Cisco Smart Account (copy paste) default-to-private = true #important parameter – you can designate 1 ASAv per availability-zone to be the default gateway for the traffic coming from the Private

subnet towards outside. It will define a default route into the Private Subnet route table for the selected ASA's internal interface. So this parameter will be true only for one ASAv per AZ.

vpn-pool-cidr = “192.168.6.0/24” # VPN Pool in CIDR format – required for AWS VPN routing vpn-pool-from = "192.168.6.1" #VPN Pool from param vpn-pool-to = "192.168.6.254" #VPN Pool to param vpn-pool-mask = "255.255.255.0" #VPN pool subnet param attach-to-dns = false #please leave it as it is – not required for this – not required with NLB weight = null #please leave as it is not required with NLB

},

You can define as many ASAvs as your company requires, and they will be dynamically created out of this

list. Please make sure that you read the comments of the parameters. You have the option to use the same

or a unique token parameter to register your ASAv with Smart Licensing. Once the ASAvs have been

defined in the “main.tfvars” file, navigate to the " prod/scalable_ravpn/3_asav_config" folder and run

the “terraform init”. Then execute the "deploy.sh" to generate the zero-day configuration for the ASAvs.

NOTE: If you want to deploy more than 6-8 ASAvs you need to increase Terraform's parallelism option in

the "deploy.sh" on the following way:

terraform apply -parallelism=20 -var-file=../variables/main.tfvars

The above will increase the amount of threads, and Terraform will start to generate the configurations. It could also happen that you need to execute the "deploy.sh" a couple of times as the configs are generated from the "terraform output" - sometimes it takes multiple runs to fill up all the configurations in case of ten or more ASAvs.

Another tip if you want to tweak the zero-day configuration or are simply curious, please check the template under " prod/scalable_ravpn/3_asav_config/asav_config_template.txt". This is the basic configuration template that will be deployed on each of the ASAvs. Because this is an ASA zero-day config, you must be careful of the configuration commands ordering for successful application and make sure the commands ordering is not be modified.

In terms of content or extension of the configuration, that can be done easily. If you want to define new parameters, you can see in the configuration template that all the variables are between ${}, enabling the Terraform to be easily extended to support more variables:

hostname ${hostname} !

license smart feature tier standard throughput level ${license_throughput} call-home

If you wish to run ASAv utilizing an older than 9.13.1 version on an older EC2 instance, such as C4 - please check the extra configuration template called asav_config_template_C4.txt. The main reason for having a different template is the gigabit interface support. The name of the interface on the ASAv is changing from TenGigabitEthernet to GigabitEthernet when you change from C5 instance to an older C4 instance. Hence, it's required to have a different template file for those.

Deploy AWS R53 Hosted Zone (Only for the AWS R53 Deployment)

Before deploying the ASAvs, we need to deploy the hosted zone, which creates the domain for our VPN front-end. For this purpose there are some specific variables under the “prod/scalable_ravpn_r53/variables/main.tfvars”. #R53 Parameters


dns-name = "example.com" #main domain for the VPN Front-end such as “example.com” vpn-sub-domain = "vpn" #this will define the sub-domain for the VPN prefix is enough – so if you define the value as “vpn” as a result of that it

will be “vpn.example.com” when it will be deployed r53-health-check-port = "443" #healthcheck port for the DNS service – it has to be the same as your Anyconnect service

Once you define these parameters, please navigate to the “prod/scalable_ravpn_r53/4_aws_r53_frontend” folder and run the “terraform init” and then run the “deploy.sh”. It will deploy the AWS R53 Hosted zone based on the parameters you have defined. Please make sure that after the deployment, you will configure your Provider’s DNS settings to point to the name-servers which AWS will generate for you. You can find it under the “AWS -> Route53 -> Hosted Zones -> (Select the domain you’ve created with terraform “example.com”) -> Under the NS record you will see something similar:

Figure 13 - AWS R53 NS Record

You will need to copy these addresses and setup these on your provider DNS settings where the domain itself is hosted. (or you can host your domain with AWS directly) You can find more details under the following link (Step 6): https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/migrate-dns-domain-in-use.html

https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/migrate-dns-domain-in-use.html


Deploy the ASAv Instances

The last step is to deploy the ASAv instances. Most of the variables have been explained in the config generator section except these:

#ASAv General Parameters ami-owner = "679593333241" #AWS image owner ID asa-ami-id = "asav9-13-1-7-ENA-6836725a-4399-455a-bf58-01255e5213b8-ami-056e4d25f7577b998.4" #AWS image ID asav-instance-size = "c5.xlarge" #AWS instance size for the ASAvs asa-tg-name = { #Load Balancer Target groups where you want to attach your ASAvs – only important for the NLB scenario 80 = "TG01ASAvTCP80" 443 = "TG01ASAvTCP443" } asa-license-throughput = "10G" #License throughput for smart licensing asa-unique-vpn-pools = true #In case if you go with the Unique VPN Pools this needs to be true – make sure your VPN pools will be distinct per

ASAv. It will implement the “Ghost-pool” option described in the design guide

The above parameters are going to be applied to all the ASAvs you will deploy. The ami-owner = and the asa-ami-id are the AWS Marketplace image IDs. If you wish to test your deployment with different ASA versions, you need to change these parameters. If not, these parameters will point to the BYOL ASA 9.13.1 version. The Instance-size defines the AWS instance size for the ASAvs – you can easily replace it with "c5.2xlarge" for example, or any other AWS instance size, which is supported by a specific ASAv version. In this example, we consider the ASAvs as a Load-Balanced pool of devices, and the instance size should be the same. If instance size varies, the framework could be easily modified to support that. NOTE: In case that you deploy the Load-Balancer scenario, one of the important configuration section is the " asa-tg-name = {…}" which defines the Load-Balancer target groups of the ASAvs that will be automatically registered. There is a particular naming convention that needs to be followed, and it has been explained in the Load-Balancing layer deployment section. Once you've successfully run the config generator module, the final configurations will be created for the ASAvs. You can deploy the ASAvs by navigating, in case of R53 under the "prod/scalable_ravpn_r53/5_asav_deploment" or in case of NLB based deployment the "prod/scalable_ravpn/4_asav_deploment" folder and run the “terraform init” following up by executing the "deploy.sh". The result will be the creation of the deployment plan for the AWS resources, including the DNS records and health-checks for AWS Route 53 or the target group registrations for the Load-Balancers. If you are happy with this, type "yes" to initiate ASAvs deployment in AWSs (It takes around two and a half minutes for the ASAv to boot up and become functional). NOTE: For the AWS R53 setup, you can wait a couple of minutes for all services initialization before trying to connect to the domain such as “vpn.example.com”. NOTE: In the case of the Load-Balancer, you still need the domain where to connect with the Anyconnect. After the script finalizes the whole deployment, please navigate into your AWS account under the "Services -> EC2 -> Load Balancers" and choose the "NLB01External" which in this example is your external load-balancer and check the "DNS name". This is the DNS name where you need to connect with the Anyconnect client to test your deployment.


Deploy the S2S VPN Connections (Only for the AWS R53 Deployment)

As part of the solution, we have also included an example S2S VPN setup. It could be an on-prem ASA, FTD, Meraki, or any AWS supported device. The parameters are the following inside of the “prod/scalable_ravpn_r53/variables/main.tfvars” file: #S2SVPN Parameters s2s-gw-name = "Home GW" #Name of the GW which you want to setup S2S tunnel with s2svpn-bgp-asn = 65534 #BGP AS number – in case if you don’t have use from a private pool (64512-65534) s2svpn-endpoint-ip = "A.B.C.D" #Public IP address of the S2S VPN endpoint

Once the ASAvs and all the other elements are deployed – you can deploy the S2S VPN by navigating into “prod/scalable_ravpn_r53/6_s2s_vpn” folder and executing the “deploy.sh” command. It will deploy an S2S VPN connection attached to the Transit Gateway. The only step you are required to do manually is to setup the S2S VPN on your gateway. The easiest way to do this is to navigate on the AWS website under the “Services -> VPC -> Site-To-Site VPN Connections”, select the deployed connection, and click on “Download Configuration”. It will offer you a list of devices that are supported by AWS so you can download the AWS prepared configuration and implement it on your on-prem device. Don’t forget to include the “VPN Pools” in the definition of the interesting traffic on your on-prem devices. Once the S2S VPN gets established, just initiate traffic from your VPN towards the On-prem networks, and the connections should start to work through the VPN.


Destroy or Extend deployment

One of the biggest benefits of using Terraform is that it keeps track of each resource it's created in the Cloud Environment. If you want to add an extra ASAv, you just need to add an extra section like this in the "main.tfvars": ASAvXYZ availability-zone = "a" #availability zone where you wish to deploy the ASAv template-file = "../3_asav_config/ASAvXYZ.txt" #Configuration template file token = "" #Smart token licensing to be inserted from the smart-licensing website – same could be used for all ASAv as long as you have

enough entitlements. default-to-private = false #Default route to be injected towards th ASA network interface in the private networks for EW inspection. Only one

ASAv can be nominated for this per availability-zone. vpn-pool-cidr = "192.168.6.0/24" #VPN Pool required in CDR format vpn-pool-from = "192.168.6.1" #VPN Pool range start vpn-pool-to = "192.168.6.254" #VPN Pool range finish vpn-pool-mask = "255.255.255.0" #VPN Pool range netmask attach-to-dns = true #only for AWS R53 weight = 10 #only for AWS R53 },

Once you've changed the config, you need to re-run the ASAv Configuration stage and the ASAv Deployment stage to generate the additional configuration and deploy the delta between the existing Cloud environment and what's defined in the metadata. If it finds there is an extra ASAv included in the "main.tfvars" it will offer you to create a new ASAv in the Cloud and not touch anything else. The same applies to other parts of the environment. If you require to change anything in the networking or load-balancing layers, you need to modify the configuration and re-run the stages accordingly. The same happens when you want to destroy the whole environment you've just deployed. You need to start to destroy resources backward by executing the "destroy.sh" in each of the folders. So you need to start with navigating to “prod/scalable_ravpn_r53/6_s2s_vpn” and execute “destroy.sh”. After that "prod/scalable_ravpn_r53/5_asav_deployment" (R53) or "prod/scalable_ravpn/4_asav_deployment" (NLB) and execute the “destroy.sh”. It will call terraform to destroy all the resources it has deployed in this step. After this, do the same with “4_aws_r53_frontend” (R53), "3_asav_config", "2_aws_lb_layer" (in case of Load-Balancer), “2_tgw_associations” (R53), “1_aws_network” and “0_aws_ssh_key” folders. In the end, you will have a clean Cloud environment. NOTE: The successful destroy of resources are subject to the fact that you don't create additional items or change things manually in the Cloud as those become "untracked" changes, Terraform is not able to resolve dependencies for those items. It is also suggested to keep your infrastructure definition in some form of version control such as "git" so you can easily track the changes in your environment.


Generating example certs using AWS PKI

For this example, we use AWS's PKI environment to set up a private CA and generate certificates for the ASAs. The domain names used to generate the certificates is the DNS name of the Load-Balancers or the Global Accelerator services. Once you have successfully generated the certificates, those will consist of the actual Certificate, the Certificate-chain, and the private-key. First, download them from AWS and then use the following commands to import them on the ASAvs and use them for the Remote Access VPN service: #Run these commans on the system where you've downloaded the files: #These 2 commands required to convert 3 different files into PKCS12 format then to encode it with Base64 openssl pkcs12 -export -inkey private_key.txt -in Certificate.txt -certfile Certificate_chain.txt -out test_final.pfx openssl base64 -in test_final.pfx > test_final.base64 cat test_final.base64 #ASAv commands crypto ca import aws-cert pkcs12 cisco #cisco is the passphrase you gave in the previous steps on your machine …. ASAv will ask you to copy paste the output of "cat test_final.base64" …. You will need to type quit …. ASAv will acknowledge that it's been successfully imported the Certificate. …. ssl trust-point aws-cert outside #set the aws-cert trustpoint to be used for SSL VPN #you may also require to execute the following commands: webvpn no enable webvpn outside enable webvpn outside You can read more about the Private CA and Certificate manager here: https://aws.amazon.com/certificate-manager/

https://aws.amazon.com/certificate-manager/

scalable ra-vpn in aws · how an aws instance looks like inside of the cloud. please look at the...

Documents