the great azure networking tour - copenhagen | april 2019€¦ · • expressroute ipv6 for o365...

The Great Azure Networking Tour

Morgan Simonsen

Innofactor

About Your Speaker: Morgan Simonsen

• Cloud Evangelist@Innofactor

• P-TSP@Microsoft

• MCSE, MCSA, MCT

• MVP

• Twitter: @msimonsen

• Email: [email protected]

• Blog: morgansimonsen.com

Agenda

• Microsoft Hyper Scale Network

• Network Topology Architecture Patterns

• Network Performance

• Network Virtual Appliances

• Connectivity

• Availability

• Network Security

• Virtual Data Center (vDC)

Users

Internet

The Big (Network) Picture

Backend Connectivity

ExpressRouteVPN Gateways

Azure

Virtual Network

Front-End Access

Reserved Public IPs

ACLs for security

Load balancing

DNS services

DDoS protection (Basic/Standard)

Virtual Network

“Bring Your Own Network”

Segmentation with Subnets

Full control with Routes

and Security groups

vNet Service Endpoints

Backend Connectivity

Point-to-site for dev/test for

Mac/Windows

VPN Gateways for secure

site-to-site connectivity

ExpressRoute for private

enterprise grade connectivity

Infrastructure Services

Platform Services

• 42 Azure regions

• 8,000+ ISP sessions

• 100+ edge sites

• 44 ExpressRoute locations

• 53,000+ km of lit fiber

• 2 per DC (metro)

• 3 per Region (terrestrial)

• 3+ per Continent (subsea)

• Low cost 100G QSPF

• Optimized for metro networks

• Intelligent traffic management

• Automatic failure recovery

Microsoft’s Global Network

Connecting to Azure Services

Regional Networks

Azure Availability Zones

• Protects applications and data from datacenter failures

• Unique physical locations within an Azure region

• Zone: 1 or more datacenters with independent cooling, power and network

• Each region with AZs have at least 3 zones

• Zone-redundant services replicate applications and data across Availability Zones

• AZ give 99,99% SLA on VM uptime

• 2 categories of services that support AZ:

• Zonal service: pinned to a region (VM, managed disk, IP address)

• Zone-redundant service: platform replicates automatically across zones (zone redundant storage, SQL database, Standard LB)

• For best results: combine AZ with Azure Regions

Azure Availability ZonesRegions that support AZ:

• Central US

• France Central

• East US 2 (Preview)

• West Europe (Preview)

• Southeast Asia (Preview)

Services that support AZ:• Linux Virtual Machines• Windows Virtual Machines• Virtual Machine Scale Sets• Managed Disks• Load Balancer• Public IP address• Zone-redundant storage• SQL Database

Pricing

• No additional cost when service deployed across AZs

• AZ-to-AZ data charges

Your Network in Azure

Your Network in Azure

New VPN Gateway SKUs – 6x faster!

SKU Workload Throughput* S2S/V2V P2S SLA

VpnGw1 Production 650 Mbps Max. 30 128 99.95%

VpnGw2 Production 1 Gbps Max. 30 128 99.95%

VpnGw3 Production 1.25 Gbps Max. 30 128 99.95%

Basic Dev/Test 100 Mbps Max. 10 128 99.9%

Behind the Scenes

Management

Control

Data

Proprietary

applianceAzure Resource

Manager

Controller

Switch (Host)

Management

plane

Data plane

SDN

Control

plane

Key to flexibility and scale is Host SDN

Network Performance – Virtual Network

• SND: Network operations implemented in SW are much more variable than those implemented in HW

• Data center architecture is not homogenous and even varies within a DC; this will impact performance

• For some VMs encapsulation (NVGRE) is handled in SW by the host; for other VMs it can be offloaded to HW

• Latency within vNet: 1-2 ms

• Latency between peered vNets: 1-2 ms (except cross-region peerings)

• Occasional peaks of 3-4 ms due to network fabric reconfigs

• First ping might take 3-4 ms because SND routes need to be built

• Anything above 6 ms would warrant investigation• Unless otherwise explained

Global DDoS Protection

• 2016/12/17: R.I.U. Star Patrol boasts they will take down Xbox and PSN onChristmas Day: https://www.youtube.com/watch?v=L1R3rPdVl1Q

• Feb 2018: Github attacked via a reflection exploit in Memcached generating 1.35terabits of attack traffic, the largest DDoS attack ever recorded.

https://www.youtube.com/watch?v=L1R3rPdVl1Q

Azure DDoS Protection offerings

DDoS Protection Standard and WAF

Azure DNS - Private Zone Support

Connectivity – Cross-Premises

Cross-Premises Connectivity

ExpressRoute


ExpressRoute

S2S VPN


New Features – ExpressRoute & VPN

• 6x VPN throughput

• Improved UltraPerf ExpressRoute gateways (latency)

• Azure Monitor for ExpressRoute & VPN

• OMS Network Performance Monitoring for ExpressRoute

• Network Watcher for VPN

• Resource Health Check for ExpressRoute & VPN

Security & compliance• Custom IPsec/IKE policy for VPN

• Merging Microsoft & Azure Public Peering

• macOS support for P2S

• AD (Radius)-based authentication

• ExpressRoute IPv6 for O365 & Azure public services

• ExpressRoute route filter for Microsoft peering – selected services

• ExpressRoute connection weight

• Multi-site for policy-based firewalls

• VPN BGP routes & peering status

ExpressRoute

Customer’s Network

Azure Public Peering for Azure public IPs

Azure Private Peering for Virtual Networks

Microsoft Peering for Office 365 and Dynamics 365

Partner

Edge

✔

✔

✔

✔

Microsoft

Edge

ExpressRoute

Customer’s Network

Azure Private Peering for Virtual Networks

Microsoft Peering for Office 365 and Dynamics 365

Partner

Edge

✔

✔

✔

✔

Microsoft

Edge

Global VNet Peering

Network Virtual Appliances

• Integrated with Azure Marketplace

• Consistency between on premises & Azure

• Audit/filter traffic to/from VNet

• Re-direct traffic flow with custom routes

Shared services such as NVA can be placed in Central/Hub VNet

UDR routes to resources in peered VNet

10.0.0.0/16

10.2.0.0/16

10.1.0.0/16

10.0.0.1

NVA Transit with vNet Peering Internet



Destination Next hop Next hop IP

0.0.0.0/0 NVA 10.0.0.1


0.0.0.0/0 NVA 10.0.0.1

10.0.0.0/16

10.2.0.0/16

10.1.0.0/16

10.0.0.1UDR

UDR




VNet Peering is non transitive

Spoke can connect to spoke directly or through Forwarder in the Hub


0.0.0.0/0 NVA 10.0.0.1


0.0.0.0/0 NVA 10.0.0.1

10.0.0.0/16

10.2.0.0/16

10.1.0.0/16

10.0.0.1UDR

UDR


FWD

10.1.0.0/16 NVA FWD

On-premises sites can be connected to VNets using shared Gateways

Gateway Transit is easy to setup and cost effective

10.0.0.0/16

10.2.0.0/16

10.1.0.0/16

Gateway Transit with vNet Peering

Gateway

On-premises sites can be connected to VNets using shared Gateways

Gateway Transit is easy to setup and cost effective

10.0.0.0/16

10.2.0.0/16

10.1.0.0/16

Gateway Transit with vNet Peering

GatewayAllowGWTransit

AllowGWTransit

UseRemoteGW

UseRemoteGW

Security for access to Azure services

Securing Access to PaaS Services

Securing Access to PaaS Services

✔

✔

X

VNet Service Endpoints

• Services in your VNet, managed by Azure!

• Directly extends your VNet to the service

• Secure your critical Azure resources to only your VNet

• Traffic remains on the Microsoft backbone

GA : Azure Storage, Azure SQL Database & Azure Cosmos DB

Allow VNet A

AccountA

Azure Storage

Vnet ARoute

Service Endpoints : Configuration

• Simple-click setup on an Azure subnet

• No NAT or GW devices!

• Network admins can set independently

Securing Azure resources to VNets

Integrated with service portal dashboards

Set for existing or new resources

Secure to VNets by name: Service endpoints is a pre-requisite

Configure once. Automatically applies to all access to child resources.

VNet Service Endpoints : Deep-Dive

Private IP: 10.0.0.6

Public IP1

Subnet 1

Azure Storage

Source: VM private IP (10.0.0.6)

Access is from VNet1: Subnet1

Before:

Source: Public IP1

Storage Diagnostics

SrcIP: 10.0.0.6, AcctA:GetBlob

SrcIP: 10.0.0.6, AcctB:PutBlob

Endpoints:

• Carry VNet identity to the service

• Source IPs switch to private VNet IPs

Service IPs and DNS entries remain as-is today

Enable service endpoint once. Secure resources as and when you want.

VNet1

Access from on-premises

• Secure Azure resources to on-premises via NAT IPs

• Add NAT IPs using IP firewall for services

• With endpoints, service IPs and DNS entries remain as is today

SERVICE ENDPOINT

On-premises

Allow VNet A

Allow Onprem:

NATIPs

AccountAVNetA

ExpressRoute Public Peering

or the Internet

ExpressRoute Private Peering

or Site-to-Site VPN

Internet

Service Tags in NSGs

• Restrict network access to just the Azure services you use.

• Maintenance of IP addresses for each tag provided by Azure

• Support for global and regional tags (varies by service)

Network Security Group (NSG)

Action Name Source Destination Port

Allow AllowStorage VirtualNetwork Storage Any

Allow AllowSQL VirtualNetwork Sql.EastUS Any

Deny DenyAllOutBound Any Any Any

Azure Services

Internet

Allow only Azure service traffic

Deny Internet outbound

Preview: Azure Storage, SQL, TrafficManager

Service Endpoints: Filter service traffic with appliances

Subnet 1 NVA Subnet

Allow NVA Subnet

SERVICE ENDPOINT

Route: 0/0->NVA

Filter: Allow myacct*.blob.core.windows.net

• Service endpoints bypass NVAs for service traffic, if registered on originating subnet

• Optionally, continue using NVAs for auditing/filtering service traffic

• Register endpoint on NVA subnet instead and secure resouces to that subnet

Stitching Azure services together

DEPLOYHDI Subnet

Allow HDI subnet InternetHDI Service

• Secure Azure resources to managed service subnets with endpoints

• Applies to all services directly deployed into VNet

Allow only Azure storage traffic

Services in a Virtual Network

Azure Storage

Azure SQL Database

Azure Cosmos DB

SQL DB Managed Instance

Azure Active Directory Domain Services for ARM

Azure Batch for ARM

Azure App Service V2

Azure API Management

Azure Batch for ASM VNets

HDInsight

Azure App Service V1

RedisCache

More services on the roadmap!

Deploy into Virtual Network VNet Service Endpoints

vNet Security

Simplifying the NSG Management

• Denying Outbound Internet access is common practice

• Automating NSG rules to whitelist Azure Datacenter IP ranges is tedious and error prone

• Introducing Augmented rules, comma separated list of IP address ranges

• Service Tags are named monikers for Azure Services that are usable in NSG



Allow AllowStorage VirtualNetwork IP1, IP2 … IP100 Any

Allow AllowSQL VirtualNetwork IP101, IP102 .. Any

Deny DenyAllOutBound Any Internet Any

Azure Services

Internet

Allow only Azure service traffic

Deny Internet outbound

Storage, SQL, AzureTrafficManager



Allow AllowStorage VirtualNetwork Storage Any

Allow AllowSQL VirtualNetwork Sql.EastUS Any


Application Security Groups

Network security as natural extension of application architecture.

Security policies for dynamic workloads

Universal NSG:

Single policy for your virtual network



Deny BlockQuarantineVMs Any QuarantineVMs Any

Allow RemoteRDP VirtualNetwork WindowsVMs 3389 (RDP)

Allow RemoteSSH SysAdminVMs LinuxVMs 22 (SSH)

Allow AllowInternetToWeb Internet Web 80,8080 (HTTP)

Allow AllowWebToApp Web App 443 (HTTPS)

Allow AllowAppToDb App Db 3306 (MySQL)

Allow AllowMyOnPremises10.0.1.0/24, 192.168.2.1/25

13.68.1.64/28, 137.116.1.0/25,…

80,8080, 443


Application Security Groups

Effortless scaling

Network security management simplified

Easy to manage, update and enforce new policies



Deny BlockQuarantineVMs Any QuarantineVMs Any

Allow AllowInternetToWeb Internet Web 80,8080 (HTTP)

Allow AllowWebToApp Web App 443 (HTTPS)

Allow AllowAppToDb App Db 3306 (MySQL)


Web App Db

QuarantineVMs

Enhanced Networking Snapshot for VMs

Improved Auditing & Metrics

✓ NSG Data plane logs

✓ NSG Data plane analytics

✓ WAF Diagnostics logs

✓ DDoS metrics and logs

Availability

Outbound Connections in Azure

Scenario Method IP protocols Description

1. VM with an Instance Level Public IP address (with or without Load Balancer)

SNAT, port masquerading not used

TCP, UDP, ICMP, ESP Azure uses the public IP assigned to the IP configuration of the instance's NIC. The instance has all ephemeral ports available.

2. Public Load Balancer associated with a VM (no Instance Level Public IP address on the instance)

SNAT with port masquerading (PAT) using the Load Balancer frontends

TCP, UDP Azure shares the public IP address of the public Load Balancer frontends with multiple private IP addresses. Azure uses ephemeral ports of the frontends to PAT.

3. Standalone VM (no Load Balancer, no Instance Level Public IP address)

SNAT with port masquerading (PAT)

TCP, UDP Azure automatically designates a public IP address for SNAT, shares this public IP address with multiple private IP addresses of the availability set, and uses ephemeral ports of this public IP address. This is a fallback scenario for the preceding scenarios. We don't recommend it if you need visibility and control.

https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-outbound-connections#ilpip

https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-outbound-connections#lb

https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-outbound-connections#defaultsnat

Why Load Balancing?

Load Balancer Application Gateway

aka.ms/lbpreview aka.ms/appgw

Traffic Manager

aka.ms/trafficmanager

Load Balancer

Public IP

Address

Private IP

Address

Outbound

connections

Public

Load Balancer

Internal

Load Balancer

Public IP

Address

Part of the Azure SDN stack

High performance & low latency for

all TCP & UDP applications

Flow-based Load Balancing

with Health Probing

Inbound NAT rules

Outbound Connections

Load balancing scenarios

DB DB

Scalable

Redundant

DB DB

Scalable

Redundant

+ Protection

DB DB DB DB

REGION A REGION B

Load Balancer SKU Choices

Metric Load Balancer Basic Load Balancer Standard

Scale Up to 100 Backend instances Up to 1000 backend instances

LB Scope Non-zonal Frontend IPs Zone redundant and Zonal Frontend IPs

Fault Tolerance Works in a Availability set Works in Availability set and Availability zones

Diagnostics Basic NAT and Probe health status Integrated Front end and Backend health metrics

NVA - Supports HA Ports

Cost Free Charged at GA

Http://aka.ms/lbpreview

http://aka.ms/lbpreview

Load BalancerNew Load Balancer, SKUs

Pool up to 1000 instances in VNet, incl. 1000 instance VMSS

Multiple VMSS

Zone-redundant data path with single IP

Zonal frontends

Cross-zone load balancing

Cross-zone VMSS

HA Ports for NVAs and more

Advanced analytics (Traffic Counters, Per Endpoint health probe status, Continuous in-band data plane health, Inbound connection attempts, Outbound connections)

Application GatewayWhat’s New?

Traffic Manager

Expands to entire VNet

• Standalone VMs without Availability Sets

• Standalone VMs with Availability Sets

• Virtual machine scale sets with up to 1000 instances

• Cross-zone VMSS https://aka.ms/xzonevmss

• Multiple VMSS

• Blending VMs and virtual machine scale sets

Load Balancer no longer uses Availability Set as a boundary for the backend pool

Network Security Groups are required for Public IP and all backend instances

Load Balancer Standard

VNet

VM scale set VM scale set VM scale set

Zone 1 Zone 2 Zone 3


VNet

VM Scale Set (coming soon)

Zone 1 Zone 2 Zone 3

https://aka.ms/xzonevmss

Reference Architecture : Web App with AZs

Zone 1

Web

Ap

pSQ

L

Fro

nt

En

d

Su

bn

et

Zone 2

SQ

L

Zone 3

SQ

L

DB

Su

bn

et

VN

etW

eb

Ap

p

Web

Ap

p

Load Balancer Standard • Zone-redundant Load Balancer Standard balances across web frontends with single frontend IP

• Web frontend across 3 AZs

• Data layer across 3 AZs

• SQL on IaaS

• SQL Azure or CosmosDB

• NoSQL (Cassandra, MongoDB)

ARM Template—Cross-Zone VMSS and LB

ARM Template—Zonal VM

Takeaways

Standard

Up to 1000 backend instances

Zone-redundant frontendZonal frontend

Availability Sets not required and Availability Zones

Integrated Frontend and Backend health metrics

Supports HA Ports

NSG required

Charged at GA

Basic

Up to 100 backend instances

Non-zonal frontend

Availability Set (single)

Basic NAT and Probe health status

-

NSG optional

Free

vs.

You can use


for TCP & UDP scenarios with:

• Larger scale

• Greater flexibility

• HA Ports

• New metrics

• Availability zones

Preview GA

Network Topology Network

Architecture Pattern – Virtual

Datacenter

• Best practices vs. Best ideas• Scenario dependent designs

Thank you!

Please do not forget to evaluate the session before you leave!

the great azure networking tour - copenhagen | april 2019€¦ · • expressroute ipv6 for o365...

Documents