Real World How-To

Dennis Bray, ENS-Inc.Chris Binger, California Department of Water ResourcesReza Namin, California Department of Water Resources

Software Defined ______ Overview of SDDC and Network Virtualization

Requirements

Deployment Considerations

Case Study and Demo

Questions

Questions

4

Traditional Networking is Hard!

Physical Networking Configuration Tasks

5

L3

L2

Initial configuration• Multichassis LAG

• Routing configuration

• Switch virtual

interfaces (SVIs)/

Router virtual

interfaces (RVIs)

• Virtual Router

Redundancy

Protocol (VRRP)/

Hot Standby Router

Protocol (HSRP)

• Spanning Tree

Protocol (STP)‾ Instances/mappings

‾ Priorities

‾ Safeguards

• Link Aggregation

Control Protocol

(LACP)

• VLANs‾ Infra networks on

uplinks and downlinks

‾ STP

Recurring configuration

• SVIs/RVIs

• VRRP/HSRP

• Advertise new subnets

• Access lists (ACLs)

• VLANs

• Adjust VLANs on trunks

• VLANs STP/Multiple

Spanning Tree (MST)

protocol mapping

• VLANs STP/MST mapping

• Add VLANs on uplinks

• Add VLANs to server ports

Anim

ate

d S

lide

Configuration consistency!

Networking Before and After Server Virtualization

• Before

• 100s of physical servers

• Change the VLAN on a switch port to control server connectivity

• Features are dependent on hardware functionality (ASICs)

• Complexity with configuring network services

• Traffic flow is mostly North-South

• After

• 1,000s of VMs

• VLAN trunking configurations

• Different teams manage different network components

• Features are still dependent on hardware functionality

• Complexity of network services (firewalls, and so forth) increased because of the number of servers

• Data center traffic flow now predominately East-West, which the network is not designed for

• Reduced visibility of network endpoints (policy enforcement, monitoring, and so forth)

6

Data Center Virtualization Layer

Intelligence in SoftwareOperational Model of VM for Data CenterAutomated Configuration & Management

What is a Software Defined Data Center (SDDC)?

Intelligence in HardwareDedicated, Vendor Specific InfrastructureManual Configuration & Management

Software

Hardware Compute, Network and Storage CapacityPooled, Vendor Independent, Best Price/Performance InfrastructureSimplified Configuration & Management

Taking what we have learned….

Software

Hardware

VirtualMachines

ComputeCapacity Network Storage

Applications

Server Virtualization

• Intelligence in the virtualization layer

• Vendor independent x86 capacity

• Transformative operational model

• Automated configuration & management

Intelligence in hardware

Dedicated, vendor specific infrastructure

Manual configuration & management

Manual Operational Model

Automated Operational Model

Programmatically Create,Snapshot,

Store,Move,

Delete,Restore

To deliver a Software Defined Data Center approach

Software

Hardware

VirtualMachines

VirtualNetworks

VirtualStorage

ComputeCapacity

NetworkCapacity

StorageCapacity

Applications

Location Independence

Data Center Virtualization

Pooled compute, network and storage capacity

Vendor independent, best price/performance

Simplified configuration & management

Automated Operational Model

Programmatically Create,Snapshot,

Store,Move,

Delete,Restore

What it is. How it works.

14

Network Capacity…

Internet

Compute Capacity….

Internet

Data Center Virtualization Layer…

Internet

A “Network Hypervisor”

Internet

The Operational Model of a VM for the Networking

Internet

Software Defined Data Center Deployed

Web Tier

App Tier

DB Tier

L3 Subnet

L3 Subnet

L3 Subnet

All

So

ftw

are

Co

nst

ruct

Physical Network

NAT

Internet

A Virtual Network?

A Virtual Network?

Non-Disruptive Deployment

Programmatically Provisioned

Services Distributed to the Virtual Switch

Services Distributed to the Virtual Switch

Encap Header

Physical Network

DSCP Tagging

Native Isolation

192.168.2.10

192.168.2.10

192.168.2.11

192.168.2.11

Leveraging the Power of SDDC Network & Security Services Distribution for Data Center Micro-Segmentation

CONFIDENTIAL 33

Problem: Data Center Network SecurityPerimeter-centric network security has proven insufficient, and micro-segmentation is operationally infeasible

Little or no

lateral controls

inside perimeter

Internet Internet

Insufficient OperationallyInfeasible

35

Internet

Security Policy

Perimeter Firewalls

CloudManagementPlatform

Solution: Leverage SDDC Approach for Micro-Segmentation

• Hypervisor-based, in kernel distributed firewalling

• Platform-based automated provisioning and workload adds/moves/changes

There is a BIG difference…

36

NSX Distributed Firewalling Performance

37

20Gbps Per Host of Firewall Performancewith Negligible CPU Impact

NSX Distributed Firewalling Performance

CONFIDENTIAL 38

80K CPS with 100+ Rules per Host

A Typical Virtual Appliance does ~6K CPS per VMA Physical Appliance performs 300K – 400K CPS per appliance

SDDC Platform – Native Security Capabilities

39

Hypervisor-based, in kernel distributed firewalling

• High throughput rates on a per hypervisor basis

• Every hypervisor adds additional east-west firewalling capacity

• Native feature of the VMware NSX platform

Platform-based automation

• Automated provisioning and workload adds/moves/changes

• Accurate firewall policies follow workloads as they move

20 Gbps Firewallingthroughput per host

Data center micro-segmentationbecomes operationally feasible

Dev

Test

Production

Isolation

Web

App

DB

NoCommunication Path

ControlledCommunication Path

Web

App

DB

Advanced Services ControlledCommunication Path

Segmentation Service Insertion

40

Automated Security in a Software-Defined Data CenterData Center Micro-Segmentation

CONFIDENTIAL 41

Automated Security in a Software-Defined Data CenterData Center Micro-Segmentation

CONFIDENTIAL 42

NSX for vSphere Components

NSX Manager NSX for vSphere centralized management plane

1:1 mapping between an NSX Manager and vCenter Server

Provides the management UI and API for NSX for vSphere

VMware vSphere Web Client Plug-In

Deploys NSX Controller and NSX Edge Virtual Appliances (OVF)

Installs VXLAN, distributed routing and firewall kernel modules plus UW Agent on ESXi hosts

NSX Controllers Provides control plane to distribute VXLAN and logical routing

network information to ESXi hosts

NSX Controllers are clustered for scale out and high availability

Network information is sliced across nodes in an NSX Controller cluster

Enables dependency on multicast routing/PIM in the physical network to be removed

Provides suppression of ARP broadcast traffic in VXLAN networks

NSX Controllers

NSX User World Agent User World Agent is a TCP (SSL) client that communicates with the NSX

Controller using the control plane protocol

May connect to multiple NSX Controllers

Mediator between the ESXi Hypervisor Kernel Modules and NSX Controllers

Also communicates with message bus agent to retrieve information from NSX Manager

Runs as a service daemon on ESXi: netcpa

Logs to: /var/log/netcpa.log

NSX Distributed Firewall has a separate service daemon: vsfwd

NSX User World AgentNSX Controller Cluster

NSX Controller NSX Controller NSX Controller

ESXi Host

Kernel Modules

Client Client

User World Agent

LR

NSXMGR

Client

VXLAN

NSX Virtual Switch and NSX Edge

• NSX Virtual Switch

• VMkernel modules

VXLAN

Distributed routing

Distributed firewall

Switch security

Message bus

vSphere NSX Edge Services GatewayNSX Edge Logical Router

ESXi

vSphereDistributedSwitch

Hypervisor Kernel Modules (vSphere VIBs)

FirewallLogical RouterVXLAN

NSX VirtualSwitch

NSX Edge Services Gateway

NSX Logical Router Control

VM

• Control functions only

• Dynamic routing and updates to NSX Controller

• Determines active ESXi host for Layer 2 bridging

• Layer 3 – Layer 7 services:

• NAT, DHCP, LB, VPN, Interface-based firewall

• Dynamic routing

• VM form factor

• High availability

Building the NSX for vSphere Platform

NSX for vSphere Requirements Well maintained, reliable physical infrastructure

Data Center Network

For Greenfield – consider Spine/Leaf architecture with IP/Layer 3 connections

For VXLAN – set all network devices MTU to greater than 1600

vSphere environment

Distributed Virtual Switch

Deployment considerations A vSphere environment with separate Management, Edge and Compute

clusters

Management Cluster – In addition to vCenter and vSphere management components, an NSX manager and a 3 node NSX controller cluster are deployed. It is recommended that the NSX controller VMs run on separate ESXi hosts

Compute Cluster – you can use new or existing ESXi clusters to deploy VMs connected to logical switches

NSX/Edge Cluster – It is recommended to deploy a new ESXi cluster to host Edge Services Gateways and Distributed Logical Router Control VMs.

Deployment Considerations

Integrations

Case Study

California Natural Resources AgencyThe California Natural Resources Agency consists of 33 different organization entities whose mission is to restore, protect and manage the state's natural, historical and cultural resources for current and future generations using creative approaches and solutions based on technology, science, engineering, and collaboration, and respect for all the communities and interests involved.

The Journey The last 6 years CNRA has been transforming information technology

services and operations through Virtualization, Cloud, and Software Defined technologies and employment of a Shared-Services and Service Provider model to become an effective Business Enabling IT organization

Goals Ability to effectively balance the need to maintain sustainable operations

Continue to be innovative

Engineer for improvements and future capabilities

Provide timely services.

Software-Defined X is a Paradigm ShiftBrings New Challenges

Network VirtualizationSoftware-Hardware Interaction,

Visibility & Operations

Operational Readiness

Toolset, Skillset, Training

Micro-Segmentation SecurityUnderstanding East-West,

Designing & Planning, Getting it Right

CNRA Next Generation Infrastructure

CNRA Next Generation Infrastructure

Key Software-Defined Technologies & VendorsPalo Alto

PAN-VM-1000-HV

Panorama 7.0.5

VMware

vSphere 6.0u1b

NSX 6.2.2

Arkin

Security & Operations Platform 2.0

And Hardware Too ..

Palo Alto 5060 Firewalls

Brocade VDX 6740 & DCX

HP c7000 chassis with BL 460 gen8 blades

PURE Storage Flash Array M50 series

F5 Viprion 2400 chassis with 2250 blades

Yes, A Software-Defined Data Center Does Require Hardware!

Operations Model and Considerations

People, Roles, Responsibilities

Multi-Vendor Environment. Cross Vendor/Technology Interaction

Information Distributed Across Multiple Planes. Consistency and Sync

Tools and Training

Application Tiers and Micro-Segmentation

Demo

Questions