network virtualization and data center networks … virtualization and data center networks...

Network Virtualization and Data Center Networks

263-3825-00

SDN – Software Stack Qin Yin

Fall Semester 2013

1

Network Restructuring Taking Place

2

Custom Hardware

Custom Hardware

Custom Hardware

Custom Hardware

Custom Hardware

OS

OS

OS

OS

OS

Control Control

Control Control

Control Control

Control Control

Control Control


3

Custom Hardware

Custom Hardware

Custom Hardware

Custom Hardware

Custom Hardware

Control Control

Control Control

Control Control

Control Control

Control Control

Network OS


4

Custom Hardware

Custom Hardware

Custom Hardware

Custom Hardware

Custom Hardware

Network OS

Control Control

Software Defined Network

5

Feature Feature

Network OS

Packet Forwarding

Packet Forwarding

Packet Forwarding

Packet Forwarding

Packet Forwarding


6

Feature Feature

Network OS

1. Open interface to packet forwarding

Packet Forwarding

Packet Forwarding

Packet Forwarding

Packet Forwarding

Packet Forwarding


7

Feature Feature

Network OS

1. Open interface to packet forwarding

2. Well-defined open API 2. At least one Network OS probably many.

Open- and closed-source

Packet Forwarding

Packet Forwarding

Packet Forwarding

Packet Forwarding

Packet Forwarding

Network OS

• Network OS: distributed system that creates a consistent, up-to-date network view – Runs on servers (controllers) in the network

– NOX, ONIX, Trema, Beacon, Maestro, … + more

• Uses forwarding abstraction to: – Get state information from forwarding elements

– Give control directives to forwarding elements

8

Control Program

• Control program operates on view of network – Input: global network view (graph/database)

– Output: configuration of each network device

• Control program is not a distributed system – Abstraction hides details of distributed state

9

Forwarding Abstraction

• Purpose: Abstract away forwarding hardware

• Flexible – Behavior specified by control plane – Built from basic set of forwarding primitives

• Minimal – Streamlined for speed and low-power – Control program not vendor-specific

• OpenFlow is an example of such an abstraction

10

Outline

• OpenFlow: Enabling Innovation in Campus Networks

• NOX: towards an operating system for networks

• Onix: a distributed control platform for large-scale production networks

11

OpenFlow: Enabling Innovation in Campus Networks

Nick McKeown, Tom Anderson, Hari Balakrishnan, Guru Parulkar, Larry Peterson,

Jennifer Rexford, Scott Shenker, Jonathan Turner

SIGCOMM Comput. Commun. Rev. 2008

12 12

Motivation – Experiments

• To experiment with new network protocols – (e.g., new routing protocols, or alternatives to IP)

• in sufficiently realistic settings – (e.g., at scale carrying real traffic)

• to gain the confidence needed for their widespread deployment

13

Motivation – GENI Example

• Vision – Nationwide research facility for experimenting with

new network architectures and distributed systems

• Virtualized programmable networks – Programmable switches and routers that (using

virtualization) can process packets for multiple isolated experimental networks simultaneously.

• Each experimental network: Slice – Allocated resources: a portion of network links, packet

processing elements (e.g. routers) and end-hosts

14

Experimenter’s Dream (Vendor’s Nightmare)

15

Standard Network

Processing

hw sw Experimenter writes

experimental code on switch/router

User- defined

Processing

OpenFlow (Or: “Why can’t I innovate in my wiring closet?”) - Nick McKeown

No Obvious Way

• Commercial vendor won’t open software and hardware development environment – Complexity of support – Market protection and barrier to entry

• Open software platforms

– E.g.:PC with several network interfaces and an operating system • Insufficient fanout or packet processing performance

• Specialized hardware/software for line-rate processing

– ATCA-based programmable router: too expensive – NetFPGA: low-cost, limited 4 network interface (need >100

ports for wiring closet)

16

Furthermore

• Isolation – Regular production traffic untouched

• Virtualized and programmable – Supporting a broad range of experiments – Different flows processed in different ways

• Hardware equipment – High-performance in packet processing

• Open development environment for all researchers – e.g. Linux, Verilog, etc.

• Flexible definitions of a flow – Individual application traffic – Aggregated flows – Alternatives to IP running side-by-side – Etc.

17

OpenFlow Switching

• A “pragmatic” compromise – Allow researchers to run experiments on heterogeneous

switches and routers in a uniform way … – … without requiring vendors to expose internal workings – … or researchers to write vendor-specific control software

• Basics – An Ethernet switch (e.g. 128-ports of 1GE) – An open protocol to remotely add/remove flow entries

18

OpenFlow Switching

19

Controller

OpenFlow Switch

Flow Table

Secure Channel

PC

hw

sw

OpenFlow Switch specification

OpenFlow (Or: “Why can’t I innovate in my wiring closet?”) - Nick McKeown

OpenFlow Switch Components

• Flow table – An action associated with each flow entry

• Secure channel – Connecting the switch to a remote controller – SSL Connection, site-specific key – Transmitting commands and packets

• Encapsulate packets for controller • Controller discovery protocol • Send link/port state to controller

• OpenFlow protocol – An open and standard way for switch/controller

communication

20

Flow Table Entry - “Type 0”

21

Switch Port

MAC src

MAC dst

Eth type

VLAN ID

IP Src

IP Dst

IP Prot

TCP sport

TCP dport

Rule Action Stats

1. Forward packet to port(s) 2. Encapsulate and forward to controller 3. Drop packet 4. Send to normal processing pipeline

+ mask

Packet + byte counters

Flow Table Entry Examples (OpenFlow is Backward Compatible)

• Ethernet Switching

• IP Routing

• Application Firewall

22/17

Switch Port

MAC src

MAC dst

Eth type

VLAN ID

IP Src

IP Dst

IP Prot

TCP sport

TCP dport

Action

* * 00:1F:. * * * * * * * port6

Switch Port

MAC src

MAC dst

Eth type

VLAN ID

IP Src

IP Dst

IP Prot

TCP sport

TCP dport

Action

* * * * * * 5.6.7.8 * * * port6

Switch Port

MAC src

MAC dst

Eth type

VLAN ID

IP Src

IP Dst

IP Prot

TCP sport

TCP dport

Action

* * * * * * * * * 22 drop

Flow Table Entry Examples (OpenFlow allows layers to be combined)

• Flow Switching

• VLAN + App

• Port + Ethernet + IP

23/17

Switch Port

MAC src

MAC dst

Eth type

VLAN ID

IP Src

IP Dst

IP Prot

TCP sport

TCP dport

Action

port3 00:2E:.. 00:1F:. 0800 vlan1 1.2.3.4 5.6.7.8 4 17264 80 port6

Switch Port

MAC src

MAC dst

Eth type

VLAN ID

IP Src

IP Dst

IP Prot

TCP sport

TCP dport

Action

* * * * vlan1 * * * * 80 port6

Switch Port

MAC src

MAC dst

Eth type

VLAN ID

IP Src

IP Dst

IP Prot

TCP sport

TCP dport

Action

port3 00:2E:.. * 0800 * * 5.6.7.8 4 * 22 drop

OpenFlow “Type 1”

• Additional actions – Rewrite packet headers – Map to queue/class – Encrypt

• More flexible header – Allow matching on arbitrary fields in headers – Enable experiments with non-IP protocols

• Support multiple controllers – Load-balancing and reliability

24

OpenFlow-Enabled Commercial Switches and Access Points

25

Controller

PC

OpenFlow Access Point

Server room

OpenFlow

OpenFlow

OpenFlow

OpenFlow-enabled Commercial Switch

Normal Software

Normal Data Path

Secure Channel

Secure Channel

Flow Table

OpenFlow Usage Models • Experiments at the flow level

– User-defined routing protocols – Admission control – Network access control – Network management – Energy management – VOIP mobility and handoff – …

• Experiments at the packet level

– Slow: Controller handles packet processing – Fast: Redirect flows through programmable hardware – Modified routers, firewalls, NAT, congestion control…

• Alternatives to IP

26

– Experiment-specific controllers – Static or dynamic flow-entries

Example Experiment at Flow Level

• Mobility

27

Lots of interesting questions • Management of flows • Control of switches • Access control of users and devices • Tracking user location and motion

OpenFlow Usage Models • Experiments at the flow level

– User-defined routing protocols – Admission control – Network access control – Network management – Energy management – VOIP mobility and handoff – …

• Experiments at the packet level

– Slow: Controller handles packet processing – Fast: Redirect flows through programmable hardware – Modified routers, firewalls, NAT, congestion control…

• Alternatives to IP

28

– Experiment-specific controllers – Static or dynamic flow-entries

Experiments at the packet level

29

Controller

PC OpenFlow-enabled Commercial Switch

Normal Software

Secure Channel

Secure Channel

NetFPGA

Laboratory

Normal Data Path

Flow Table

NOX: towards an operating system for networks

Natasha Gude, Teemu Koponen, Justin Pettit, Ben Pfaff, Martin Casado, Nick McKeown, and

Scott Shenker

SIGCOMM Comput. Commun. Rev. 2008

30 30

Network Operating Systems

• Provide a programmatic interface to observe and control the entire network.

• Applications (controllers) perform the actual management by making system calls.

• Two paradigm shifts – Logical Centralization: Applications are written as if

network were present on single machine. – Abstraction: Applications are written in terms of abstract

entities, e.g. users and hosts.

31

NOX OpenFlow Controller

• Centralized intelligential agency for entire OpenFlow network

• NOX is an open-source OpenFlow Controller • Researchers can insert their software code into

NOX controller for testing their idea

32

Nox Controller

OpenFlow Switch OpenFlow Switch OpenFlow Switch

NOX Components

• OpenFlow Switches

• PC servers running – NOX controller processes

– Management applications

– A single network view

33

Application I/O

• Observation granularity: – Switch-level topology – Locations of users, hosts, middleboxes – Services offered, e.g. HTTP or NFS – Bindings between names and addresses – NOT the entire packet state or flow state

• Control granularity: flows

– Decisions about one packet are applied to all subsequent packets in the flow.

• Tradeoff: scalability and flexibility

34

Scalability

• Events (per second) – Packet arrivals (106): handled by switches – Flow initiations (105) : handled by controller – View change (10) for networks of thousands of hosts:

handled by controller

• Controller

– Can be replicated – Only global network state - network view: maintained

centrally

35

NOX’s Programmatic Interface

• NOX exposes network events to applications and applications consist of code fragments that respond to these events. – Events generated by OpenFlow messages

• Switch join, switch leave, packet received, switch statistic received – Events generated by NOX application

• User authenticated • Flow initiated

• Network view and namespace – High-level names <-> low-level addresses

• High-level services – Routing module, policy-based network filtering module …

36

Example: Access Control

def handle_flow_initialize(packet):

usersrc = nox.resolve_user_src(packet)

hostsrc = nox.resolve_host_src(packet)

usertgt = nox.resolve_user_tgt(packet)

hosttgt = nox.resolve_host_tgt(packet)

prot = nox.resolve_ap_prot(packet)

if deny(usersrc,hostsrc,usertgt,hosttgt,prot):

nox.drop(packet)

else:

nox.installpath(p, nox.computepath(p))

def deny(usersrc, hostsrc, usertgt, hosttgt, prot): …

37

Onix: a distributed control platform for large-scale production networks

Teemu Koponen, Martin Casado, Natasha Gude, Jeremy Stribling, Leon Poutievski, Min Zhu, Rajiv

Ramanathan, Yuichiro Iwata, Hiroaki Inoue, Takayuki Hama, and Scott Shenker.

OSDI 2010

38 38

SDN Paradigm

• A single control platform to implement – A range of control functions

• E.g., routing , traffic engineering, access control, VM migration

– … over a spectrum of control granularities • From individual flows to large traffic aggregates

– … in a variety of contexts • Enterprise data center, public cloud, WAN

39

Onix

• To implement centralized control logic as distributed system without re-inventing distribution mechanisms

• Useful tools: – Simple, general API for control plane

implementation – Flexible state distribution primitives – Flexible scaling and reliability mechanisms

40

Challenges

• Generality: – Support a wide range of network management applications

• Scalability: – Limitations should be duet to the problems of state

management or the implementation of the control platform • Reliability:

– Graceful failure handling • Simplicity:

– Simplify the task of building network management applications • Performance:

– Adequate control-place performance

41

Onix Architecture

42

Onix Components

• Physical infrastructure: – Switches, routers, and other network elements

• Connectivity infrastructure: – Channels for control messages.

• Onix: – A distributed system running the controller.

• Control logic: – Network management applications on top of Onix

43

Onix API

• Developers program against a network graph

• Nodes represent physical network entities

44

Onix API

• Developers program against a network graph

• Nodes represent physical network entities

45

ForwardingEngine: • Write flow entry • List port • Register for updates

Network Information Base

• Graph of network elements – NIB • The NIB is the focal point of the system

– State for applications to access – External state changes imported into it – Local state changes exported from it

46




47




48




49

State Distribution

• Across (and within) controllers, different policies for different data: – Strong consistency for more critical, stable state – Eventual consistency for dynamic, inconsistency-

tolerant state • Onix provides two built-in storage options:

– Replicated transactions (SQL) storage – One-hop memory-based DHT

• Example: – Switch topology is hard state, relatively static – IP->MAC mapping are soft state, change rapidly

50

Data Storage and Dissemination

• Applications initially configure the distribution mechanisms

• But interact at run-time only with the NIB

51

Scalability/Reliability Requirements

• Boils down to distributed state management – To/from managed switches – Across Onix instances

• Each control application will want different tradeoffs

• Example tradeoffs – Every switch connects to every Onix instance

• Fast, reliable failover

– Switches connect to a subset of Onix instances • More scalable to larger networks

52

Scalability

• Partitioning for scale – Multiple dimensions available to applications:

• Onix instances with different computations tasks

• Onix instances have only subsets of the NIB

• Switches connect to a subset of Onix instances

• Aggregating for scale – Reduce fidelity of information before

disseminating within the cluster

53

Reliability

• Network Element & Link Failures: – Applications' responsibility

• Connectivity Infrastructure Failures:

– Assumed reliable management connectivity

• Onix Failures:

– Onix provides distributed coordination facilities provided for app failover

54

References • OpenFlow: enabling innovation in campus networks. Nick McKeown, Tom

Anderson, Hari Balakrishnan, Guru Parulkar, Larry Peterson, Jennifer Rexford, Scott Shenker, and Jonathan Turner. 2008. SIGCOMM Comput. Commun. Rev. 38, 2 (March 2008), 69-74.

• NOX: towards an operating system for networks. Natasha Gude, Teemu Koponen, Justin Pettit, Ben Pfaff, Martin Casado, Nick McKeown, and Scott Shenker. 2008. SIGCOMM Comput. Commun. Rev. 38, 3 (July 2008), 105-110.

• Onix: a distributed control platform for large-scale production networks. Teemu Koponen, Martin Casado, Natasha Gude, Jeremy Stribling, Leon Poutievski, Min Zhu, Rajiv Ramanathan, Yuichiro Iwata, Hiroaki Inoue, Takayuki Hama, and Scott Shenker. 2010. In Proceedings of OSDI'10. USENIX Association, Berkeley, CA, USA, 1-6.

• Nick’s talks: http://yuba.stanford.edu/~nickm/talks.html • Onix OSDI talk: https://www.usenix.org/conference/osdi10/onix-

distributed-control-platform-large-scale-production-networks

55

http://yuba.stanford.edu/~nickm/talks.html

https://www.usenix.org/conference/osdi10/onix-distributed-control-platform-large-scale-production-networks

https://www.usenix.org/conference/osdi10/onix-distributed-control-platform-large-scale-production-networks

network virtualization and data center networks … virtualization and data center networks...

Documents