POSTECH 1/62CSED702Y: Software Defined Networking
Software Defined Networking:OpenFlow Switches & Controllers
James Won-Ki Hong
Department of Computer Science and Engineering
POSTECH, Korea
POSTECH 2/62CSED702Y: Software Defined Networking
Outline
OpenFlow Switches OpenFlow Controllers
NOX & POX Floodlight Ryu Open Daylight (ODL) Open Network Operating System (ONOS)
Conclusions
POSTECH 3/62CSED702Y: Software Defined Networking
Introduction to OpenFlow Switches
Hardware-based OpenFlow Switches Commercial hardware switches with OpenFlow capability
• Network abstraction is realized by firmware upgrading
Show high processing speed Have space limitation on saving the flow table entries
• Approximately store 1500 flow entries (due to expensive CAM)
Not easy to upgrade• Most switches only support OpenFlow up to version 1.0
Software-based OpenFlow Switches OpenFlow enabled software switch (runs on x86 commodity computer) Performance is relatively low Store large amount of flow entries with bound (theoretically) Under active development, support most recent OpenFlow spec.
Hybrid OpenFlow Switch A virtual switch with specialized hardware device Much faster than software-based switches
POSTECH 4/62CSED702Y: Software Defined Networking
Hardware-basd OpenFlow Switches
Ciena Coredirector
NEC IP8800
More coming soon...
Juniper MX-series
HP Procurve 5400
Pronto 3240/3290
WiMax (NEC)
PC EnginesNetgear 7324
POSTECH 5/62CSED702Y: Software Defined Networking
Software-based OpenFlow Switches (1/3)
OpenvSwitch (OVS) Overview
• A virtual switch or Virtual Ethernet Bridge (VEB)• User-space: configuration, control• Kernel-space: datapath (included in main Linux kernel from v3.3)
Features• Support OpenFlow protocol• Support multiple tunneling protocols
• VxLAN, Ethernet over GRE, IPsec, GRE over IPsec
• Fine-grained QoS
Main components• ovs-vswitchd: a daemon that implements the switch• ovsdb-server: lightweight database server that ovs-vswitch queries to• ovs-vsctl: a utility for querying and updating the config. of ovs-vswitchd• ovs-dpctl: a tool for configuring and monitoring the switch kernel module• ovs-ofctl: a tool for monitoring and administering OpenFlow switches• ovs-controller: a simple OpenFlow controller reference implementation• openvswitch.ko: OpenvSwitch switching datapath
POSTECH 6/62CSED702Y: Software Defined Networking
Software-based OpenFlow Switches (2/3)
OpenvSwitch Architecture
POSTECH 7/62CSED702Y: Software Defined Networking
Software-based OpenFlow Switches (3/3)
OpenFlow Software Switch An OpenFlow compatible user-space software switch implementa-
tion The original code is based on the Stanford 1.0 reference switch Initially developed by Ericsson Research Lab The implementation is feature-complete, and passes the available of-
test 1.1 test case• Compatible with most of OpenFlow controllers which support OF 1.1
CPqD version supports OpenFlow up to v1.3 Components
• ofdatapath: the switch implementation• ofprotocol: a secure channel for connecting the switch to the controller• oflib: a library for converting to/from 1.1 wire format• dpctl: a tool for configuring the switch from the console
POSTECH 8/62CSED702Y: Software Defined Networking
Hybrid OpenFlow Switch (1/3)
Problems of Software-based Switch Cannot fully utilize the hardware resources
• E.g., OVS only exploits single CPU core
Tightly coupled with the OS kernel• Increase the management complexity
Low performance• Massive RX interrupts handling for NIC device• Shared data access between threads competition makes bottleneck
Hybrid OpenFlow Switch Separate the roles of virtual switch into two parties
• Hardware: pure packet processing• Software: switch abstraction (e.g., flow table)
Data Plane Development Kit (DPDK)• A set of libraries and drivers for fast packet processing• Incorporate with x86 CPU• Fast network I/O in user space
POSTECH 9/62CSED702Y: Software Defined Networking
Hybrid OpenFlow Switch (2/3)
Features of Hybrid OpenFlow Switch Polling-based packet handling Core assignment
• Assign the polling task to a dedicated CPU core less context switching
Reduction of # of access in I/O and memory Lockless-queue, batch processing
NIC
skb_buf
Ethernet Driver API
Socket API
vswitch
packetbuffer
memory
Pure Software-based Switch
1. Interrupt & DMA
2. system call (read)
User space
Kernel space
Driver
4. DMA
3. system call (write)
NIC
Ethernet Driver API
User-mode I/O & HAL
vswitch
packetbuffer
Hybrid Switch with Intel DPDK
1. Read Write
DPDK Library
Polling-base packet handling
packet handling
POSTECH 10/62CSED702Y: Software Defined Networking
Hybrid OpenFlow Switch (3/3)
Packet Processing using Multi Core CPUs Exploit many core CPUs Decouple I/O processing and flow processing Explicit thread assignment to CPU core
NIC 3RX
NIC 4 RX
NIC 3 TX
NIC 4 TX
NIC 1 RX
NIC 2 RX
I/O RX CPU0
I/O RX CPU1
NIC 1 TX
NIC 2 TX
I/O TX CPU6
I/O TXCPU7
Flow lookuppacket process-
ing CPU2
Flow lookup packet
processing CPU4
Flow lookup packet
processing CPU3
Flow lookup packet
processing CPU5
NIC RX buf-fer
Ring buf-fer
NIC TX buffer
Ring buf-fer
POSTECH 11/62CSED702Y: Software Defined Networking
Introduction to OpenFlow Controller (1/2)
Problem Statement New functions require new hardware
• E.g., VNTag, TrustSec No support for network-wide control or high-level abstractions Distributed control reduces the controllability
…… Control
Datapath
Distributed Control: Per-box control Config/Mgmt ≠
fine-grained control
HW Functions: Inflexible Proprietary Expensive
Fn…F1
Network Management System (NMS)
Switch 1 Switch n
Management, No Controllability Monitor: collect network-wide statistics using CLI,
SNMP, NetConf interfaces Control: No real control of packet/flow forwarding No much thing to do with monitored data…
POSTECH 12/62CSED702Y: Software Defined Networking
Introduction to OpenFlow Controller (2/2)
Solution Need a Network Operating System (NOS), provide a uniform and
centralized programmatic interface to the entire network NOS does not manage the network itself, instead it provides a pro-
grammatic interface
…… Control
Datapath
F1 F2 Fn…F3
External Controller
Switch 1 Switch n
Controllability: Fully remove to ex-
ternal controller Centralized control
Network Operating System (NOS)
Application Programming Interface (API)
POSTECH 13/62CSED702Y: Software Defined Networking
Pedigree Chart of OpenFlow Controllers
TremaFull-stack OpenFlow
Framework in Ruby and C
(Proprietary) C/C++Python
JavaBig NetworkController
(Proprietary)
ETRIController
NOX Classic: C++/PythonNOX: C++
CiscoController
(Proprietary)
ScalabilityComparison
POSTECH 14/62CSED702Y: Software Defined Networking
NOX & POX
POSTECH 15/62CSED702Y: Software Defined Networking
History of NOX History
Started from SANE project and applied to Ethane in 2006 Ethane project was announced via SIGCOMM in 2007 Announced the OpenFlow spec 0.1.0 in Nov. 2007, spec 0.8.0 in May, 2008 NOX was initially developed by Nicira side-by-side with OpenFlow, and open
sourced as the first OpenFlow controller in 2008 NOX was further developed by CPqD to support OpenFlow 1.3 in Nov. 2012
OpenFlow v -2.0
Topology
Routing
Host Tracking
Network As a File System
OpenFlow v -1.0
Topology
Routing
Host Tracking
PolicyCompiler
SANE Ethane NOX-Classic/NOX
OpenFlow v 1.0
Topology
Routing
Host Tracking
SANE Ethane …
NOX-Classic (CPqD)
OpenFlow v 1.3
Topology
Routing
Host Tracking
SANE Ethane …
POSTECH 16/62CSED702Y: Software Defined Networking
NOX Classic vs NOX (1/2) NOX Versions
NOX-Classic Original NOX (now, officially deprecated) C++ based SDN controller, applications can be developed using Python
• Use SWIG to integrate Python with C++
Version Name Branch Name Version No. Release Date OF Spec GUI
NOX Clas-sic
Zaku 0.9.0 2010-09-15 v1.0 Support
Destiny 0.9.1 2012-07-20 v1.0 Support
CPqD ver. N/A 2012-11-10 v1.3 Not support
(new) NOX Verity 0.9.2 2012-05-11 v1.0 Not support
Sample apps written in C++
Sample apps written in Python
Sample apps written in Python and C++ using SWIG lib
POSTECH 17/62CSED702Y: Software Defined Networking
NOX Classic vs NOX (2/2) NOX
New NOX, separated from NOX-classic branch Only support C++ for application development Fewer default applications than NOX-classic Enhanced performance and better source readability
• Written on top of Boost library
No graphic user interface
NOX Code SnippetNOX-Classic Code Snippet
POSTECH 18/62CSED702Y: Software Defined Networking
POX Versions
POX NOX’s younger sibling implemented using
Python For the rapid development and prototyping
of network control software Support all OS (e.g., Linux, Mac, Windows) Can bundle with install-free PyPy runtime Performs well compared to NOX-classic
Python• Note that NOX-classic does not support PyPy run-
time
Used for research and education purpose
POX Overview
Controller Name Branch NameVersion No.
Release Date OF Spec GUI
POX
Angler 0.0.0 2012 Fall v1.0 Support
Betta 0.1.0 2013 Spring v1.0 Support
Carp 0.2.0 2013 Fall v1.0 Support
Dart 0.3.02014 Sum-mer
v1.0 Support
POSTECH 19/62CSED702Y: Software Defined Networking
NOX/POX Overview (1/4) Components
NOX Controller – Network OS Provide system-wide abstractions Turn networking into a software problem
Network Application Services New functions as software services
OpenFlow Enabled Switches New functions as software services
Northbound API Provide interface to network applications Not yet standardized
Southbound API Standardized OpenFlow protocol Controller, switch interoperability
Controller
…
TopologyDiscovery
VLANTagging
ScanDetection…
Northbound API
…
POSTECH 20/62CSED702Y: Software Defined Networking
Granularity Choosing granularity involves trading off scalability against flexibility Observation: provides adequate information and changes slowly
• Includes switch-level topology, does not include the current state of network traffic
Control: scale to large networks, while providing flexible control
Switch Abstraction Switch instructions should be independent of the particular HW and
support the flow-level control granularity Solution: adopt OpenFlow switch abstraction
NOX/POX Overview (2/4)
Finer grained controlLowest performance
Coarser grained controlHighest performance
Per-packet control Per-flow control Prefix-based routing control
POSTECH 21/62CSED702Y: Software Defined Networking
NOX/POX Overview (3/4) Operation
Observation: construct the network view• Use DNS, DHCP, LLDP and flow-initiation
Control: determine whether to forward traffic, and to which route• Access control, routing and failover applications
Network view generation
LLDP
PACKET_OUT with LLDP
PACKET_OUT with LLDP
PACKET_INwith LLDP
NOX Controller
OpenFlowSwitch B
OpenFlowSwitch A
Reactive routingIDX SRC DST SRC P DST P
153 sw. A sw. B p2 p1
… … … … …
357 sw. B sw. A P1 p2
Host A Host B
OpenFlowSwitch
PACKET_IN
FLOW_MOD
SRC DST ACT …SRC DST ACT …
h. A h. B p1
Host C
p1
p2
NOX Controller
Packet headerSRC: Host ADST: Host B
Flow Table
p1p2 p1
p2
POSTECH 22/62CSED702Y: Software Defined Networking
NOX/POX Overview (4/4) Scaling
Packet level parallelization• Millions of arrival packets per second• Packets are handled by individual switches
Flow level parallelization• Flow initialization rate is high but lower than
packet arrival rate• Flow initialization is handled by individual con-
troller
Periodic synchronization• Network view changes slowly enough that can
be maintained centrally
Public Release A PoC of OpenFlow controller Follow GPL license Many network applications
Source: Comparing OpenFlow Controller Paradigms Scalability: Reactive and Proactive. In-ternational Conference on Advanced Information Networking and Applications
POSTECH 23/62CSED702Y: Software Defined Networking
NOX/POX Architecture
Component API
CooperativeThreading
EventHarness
OpenFlowAPI
Asynchronous I/O
Socket I/O File I/O
OpenRoadsL2_MultiL3_learning
Components
Core
Common
Packet_dump
MAC_blocker Authenticator
L2_learning
HostTracking
Routing
Spanning_tree
WebServices
TopologyDiscovery
POSTECH 24/62CSED702Y: Software Defined Networking
Floodlight
POSTECH 25/62CSED702Y: Software Defined Networking
Introduction to Floodlight
Floodlight A completely open, free, OpenFlow controller developed by Big
Switch Network Currently supports OpenFlow up to v1.0
Research and commercial friendly Easy to build, run, and develop
Rich set of build and debugging tools
Toolchain
Community of OpenFlow experts, ac-cess to commercial upgrades, and fre-quent testing
POSTECH 26/62CSED702Y: Software Defined Networking
Floodlight Overview (1/2)
Floodlight Architecture
Floodlight Controller
ModuleManager
ThreadPool
PacketStreamer
JythonSever
WebUI
UnitTests
DeviceManager
TopologyService
LinkDiscovery
FlowCache
Storage
OpenFlow Services
CounterStore
SwitchesControllerMemory
PerfMon Trace
Java
AP
I
Memory
NoSQL
Module Applications
VNF
Firewall
StaticFlowEntryPusher
PortDownReconciliation
Forwarding
HubLearningSwitch
REST ApplicationsCircuitPusher (python)
OpenStackQuantumPlugin (python)
Your Applications…
REST API (Implement Restlet Routable Interface)
POSTECH 27/62CSED702Y: Software Defined Networking
Floodlight Overview (2/2)
Application Modules Forwarding: default reactive packet forwarding application Static Flow Entry Pusher
• Install specific flow entry (match + action) to a specific switch
Firewall• An application to apply ACL rules to allow/deny traffic based on specified match
Port Down Reconciliation: reconcile flows across a network Virtual Network Filter (VNF)
• Simple MAC-based network isolation application
Core REST APIs Static Flow Pusher REST API
• Allow the user to proactively insert/delete/list the flows to OpenFlow switch
Firewall REST API• Allow the user to insert/delete/list rules for firewall
POSTECH 28/62CSED702Y: Software Defined Networking
Floodlight Programming Model
IFloodlightModule Java module that runs as part of Floodlight Consumes services and events exported by other modules
• OpenFlow (e.g., Packet-in, Packet-out…)• Switch add / remove• Device add / remove / move• Link discovery
External Application Communicates with Floodlight via REST
• Static Flow Pusher• Add flow, delete flow,
list flows, removeAll flows
• Normalized network state• List hosts, list links, list switches,
getStats, getCounters
• Maybe your applications?
vSwitch
vSwitch
vSwitch
vSwitch
IfloodlightModule
External Application
REST
Floodlight Controller
Northbound APIs
POSTECH 29/62CSED702Y: Software Defined Networking
Module Description
TopologyManager
Floodligh
tProvider
LinkDiscovery
Forwarding
DeviceManager
StorageSource
RestServer
StaticFlowPusher
Maintains the topology information for the controllerReceives information from LinkDiscovery module
Maintains state of links in networkUses LLDP message
Basic reactive packet forwarding module
Manage the end-host (device) location information(mac, IP …)
DB style storage for Topology and LinkDiscovery data
Implements via Restlets (restlet.org)REST API modules must implement RestletRoutable
Supports the insertion and removal of static flows
POSTECH 30/62CSED702Y: Software Defined Networking
Ryu
POSTECH 31/62CSED702Y: Software Defined Networking
Introduction to Ryu
流Flow
龍Oriental Dragion,
A god of water
POSTECH 32/62CSED702Y: Software Defined Networking
Introduction to Ryu
Ryu A platform for building OpenFlow applications Manage “flow” control to enable intelligent networking Features
• Generality: vendor-neutral, support open interface• Agile: not the all-purpose big monolithic “controller”, but
the framework for SDN app dev. Supportability
• OpenFlow protocol– OF 1.0, OF 1.2, OF 1.3 and OF-CONFIG 1.1
• Other protocol: NetCONF, SNMP, OVSDB• Apps/libs
– topology view, firewall, OpenFlow Restful, etc.
• Integration with other project– OpenStack, HA with Zookeeper, IDS with snort
License: Apache 2.0
Developed &Maintained By
POSTECH 33/62CSED702Y: Software Defined Networking
Ryu Architecture
Follow Standard SDN Architecture
OpenFlow switch OpenFlow switch
SDN appsWell defined API
(REST, RPC...)
Open protocols (OpenFlow, OF-config, NETConfig, OVSDB)
SDN apps SDN apps
Ryu SDN framework
OpenFlow Parser/serializer
Event dispatcher
Ryu built-in app (Tenant Isolation,
Topology Discovery, Firewall …)
Ryu App
libraries
Protocol support (OVSDB, VRRP, ...)
Ryu App
...Control layer
Application layer
Network device
POSTECH 34/62CSED702Y: Software Defined Networking
Ryu Implementation (1/2)
Event Driven Event source/dispatcher/sink
• Source• Call methods of the event dispatcher to generate events
• Sink• Subclass of class RyuApp• Avoid race condition
• Dispatcher• Decouples event sources/sinks• Dispatches events based on class of event• Knows which methods are interested in which event
by methods attributes
RyuAppRyuApp
queue
Event sink
RyuAppRyuApp
queue
RyuAppRyuApp
queueEvent dispatcher
dispatchEvent
Event
Event source
Event source
Events are read only because it is shared with many RyuApps
Determine which RyuApp to de-liver based on class of event
POSTECH 35/62CSED702Y: Software Defined Networking
Ryu Implementation (2/2)
Connection to OpenFlow Switch Receiving loop and sending loop
OpenFlow switch
Receiving thread:Generates OFPEvents
Sending thread: Serialize and send OF packets
Send queueEventOFP
message
RyuAppRyuApp
queue
Event sink
Ryu
POSTECH 36/62CSED702Y: Software Defined Networking
ODL: Open Daylight
POSTECH 37/62CSED702Y: Software Defined Networking
OpenDaylight Scope and Projects
OpenDaylight Scope
POSTECH 38/62CSED702Y: Software Defined Networking
OpenDaylight Scope and Projects
OpenDaylight Projects 15 projects currently in bootstrap or incubation
Project Description Originator
Controller Modular, extensible, scalable, and multi-protocol SDN controller based on OSGi Cisco
YANG Tools Java-based NETCONF and YANG tooling for OpenDaylight projects Cisco
OpenFlow Protocol Library OF 1.3 protocol library implementation Pantheon
OpenFlow Plugin Integration of OpenFlow protocol library in controller SAL Ericsson, IBM, Cisco
Defense4All DDoS detection and mitigation framework Radware
OVSDB OVSDB configuration and management protocol support Univ. of Ken-tucky
LISP Flow Mapping LISP plugin, LISP mapping service ConteX-
tream
POSTECH 39/62CSED702Y: Software Defined Networking
OpenDaylight Scope and Projects
Project Framework
POSTECH 40/62CSED702Y: Software Defined Networking
OpenDaylight Controller
Features of OpenDaylight Controller Built using Java OSGi Framework – Equinox
• Provides Modularity & Extensibility• Bundle Life-cycle management• In-Service-Software Upgrade (ISSU) & multi-version support
Service Abstraction Layer (SAL) • Provides Multi-Protocol Southbound support• Abstracts/hides southbound protocol specifics from the applications
High availability & horizontal scaling using clustering
Releases Hydrogen
• Initial release, support OpenFlow 1.0 and 1.3
Helium• Current stable release, plan to support OpenFlow 1.5
Lithium• The next release (6/25/2015 SR1 8/13 9/24)• New project IoT Data Management (IoTDM)
POSTECH 41/62CSED702Y: Software Defined Networking
Hydrogen Architecture
41
Base Network Service Functions
Management GUI/CLI
Controller Platform
Southbound Interfaces& Protocol Plugins
OpenDaylight APIs (REST)
DOVE Mgr
Data Plane Elements(Virtual Switches,Physical Device
Interfaces)
Service Abstraction Layer (SAL)(plug-in mgr., capability abstractions, flow programming, inventory, …)
OpenFlow
1.0 1.3LISP
Topology Mgr
Stats Mgr
Switch Mgr
Host Tracker
Shortest Path
Forwarding
VTN Coordinator
Affinity Service
Network Applications Orchestration & ServicesOpenStack
Neutron
VTN Manager
VTN: Virtual Tenant NetworkDOVE: Distributed Overlay Virtual EthernetDDoS: Distributed Denial Of ServiceLISP: Locator/Identifier Separation ProtocolOVSDB: Open vSwitch DataBase ProtocolBGP: Border Gateway ProtocolPCEP: Path Computation Element Communication ProtocolSNMP: Simple Network Management Protocol
LISP Service
NETCONF BGP-LSSNMP
DDoS Protection
OVSDB PCEP
OpenStack Service
NetworkConfig
Main difference from other OpenFlow-centric controller platforms
OpenFlow Enabled Devices
Devices with Proprietary control plane
OVSDB enabled devices
POSTECH 42/62CSED702Y: Software Defined Networking
Helium Architecture
42
Base Network Service Functions
Dlux UI
Controller Platform
Southbound Interfaces
& Protocol Plugins
OpenDaylight APIs (REST)
Data Plane Elements(Virtual Switches,Physical Device
Interfaces)
Service Abstraction Layer (SAL)(plug-in mgr., capability abstractions, flow programming, inventory, …)
OpenFlow
1.0 1.3LISP
Topology Mgr
Stats Mgr
Switch Mgr
Host Tracker Flow Rules
Mgr
VTN Coordinator
AAA
Network Applications Orchestration &
Services
OpenStackNeutron
OpenFlow Enabled Devices
VTN: Virtual Tenant NetworkDDoS: Distributed Denial Of ServiceLISP: Locator/Identifier Separation ProtocolOVSDB: Open vSwitch DataBase ProtocolBGP: Border Gateway ProtocolPCEP: Path Computation Element Communication ProtocolPCMM: Packet Cable MultiMediaSNMP: Simple Network Management Protocol
LISP Service
NETCONF BGP-LS
Devices with Proprietary control plane
SNMP
DDoS Protection
OVSDB enabled devices
OVSDB PCEP
Main difference from other OpenFlow-centric controller platforms
SDNi Wrapper
GBP Service
SFC
TTPSNBIPCMM/
COPSOPEN
CONTRAIL
GBPrenderer
OVSDB Neutron
VTN Mgr
OpenStack Service
OpenContrail
L2switch
SNBIservic
e
DOCSIS
SDNI aggreg
AAA – AuthN filter
POSTECH 43/62CSED702Y: Software Defined Networking
OpenDaylight Controller
Plugin Build Process
POSTECH 44/62CSED702Y: Software Defined Networking
OpenDaylight Controller
Hydrogen OpenFlow Plugin Architecture
POSTECH 45/62CSED702Y: Software Defined Networking
Model-Driven SAL (1/2)
Model-Driven Service Abstraction Layer (SAL)
POSTECH 46/62CSED702Y: Software Defined Networking
Model-Driven SAL (2/2)
Model-Driven Service Abstraction Layer (SAL) Yang tools – supporting for model-driven SAL Provides tooling to build Java bindings in yang from
yang models
API-Driven SAL Model-Driven SAL
POSTECH 47/62CSED702Y: Software Defined Networking
ONOS: Open Network Operating System
POSTECH 48/62CSED702Y: Software Defined Networking
SDN Evolution and ON.LAB
Platform Development
2007 – Ethane2008 – OpenFlow2009 – FlowVisor, Mininet, NOX 2010 – Beacon
2009 – Stanford2010 – GENI started and grew to 20 universities2013 – 20 more cam-puses to be added
Deployments
Demonstrations
2008-2011 – SIGCOMM2011 – Open Networking Summit, Interop
2012 –Define SDN research agenda for the coming years
And Beyond
Invention
2007 – Creation of SDN Concept
Non-profit, carrier and vendor neutral
Provide technical shepherding, core team
Build community Many organizations supports
POSTECH 49/62CSED702Y: Software Defined Networking
Introduction
ONOS: Open Network Operating System SDN OS for service provider networks Key features
• Scalability, high availability & performance• Northbound & southbound abstraction• Modularity
• Various usage purposes, customization and development
History
Founded – 2012
ONOS Prototype 1 – 2013
(scalability, high availabil-ity)
ONOS Prototype 2 – 2013
(performance)
ONOS VERSION 1 – Open sourced on Dec 5th, 2014
POSTECH 50/62CSED702Y: Software Defined Networking
Key Performance Requirements Requirements for Supporting Service Provider Networks
High throughput• 500K ~ 1M paths setups / second, 3-6M network state operations / second
High volume• 500GB ~ 1TB of network state data
Need to choose distribution approach!
POSTECH 51/62CSED702Y: Software Defined Networking
ONOS Tiers and Distributed Architecture
Distributed Architecture Six tiered architecture Each ONOS instance is equipped with the same software stack
Northbound Abstraction• Network graph• Application intents
Core• Distributed• Protocol independent
Southbound Abstraction• Generalized OpenFlow• Pluggable & extensive
Adapters• Multiple southbound
protocol enabling layer Protocols
• Self-defined protocols usinggeneralized SDN functions
instance 1 instance 2 instance 3 instance 4
POSTECH 52/62CSED702Y: Software Defined Networking
ONOS Architecture (Prototype 2)
Cassandra Distributed Key-Value Store
Titan Graph DB
Network Graph (Eventually consistent)In-memory Network Graph (eventually consistent)
Host
Host
Host
Instance 1 Instance 2 Instance 3
OpenFlow Manager+
OpenFlow Manager+
OpenFlow Manager+
+ Floodlight Drivers
Control Application
Dis
trib
ute
d
Reg
istr
y(S
tro
ng
ly
Co
ns
iste
nt)
Zo
oke
eper Control Application
Scale-out
Co
ord
inat
ion
Distributed Network
Graph/State
Applications ONOS Graph API
IndexingONOS Graph Abstraction
RAMCloudUltra-low latency distributed data store in DRAM
Eve
nt
No
tifi
cati
on
s
Haz
elca
st
POSTECH 53/62CSED702Y: Software Defined Networking
ONOS Scale-Out
Distributed Network OS
Instance 2 Instance 3
Instance 1
Network GraphGlobal network view
Data plane
An instance is responsible for maintaining a part of network graph
Control capacity can grow with network size or application need
POSTECH 54/62CSED702Y: Software Defined Networking
ONOS Control Plane Failover
Master Switch A = ONOS 1
Candidates = ONOS 2, ONOS 3
Master Switch A = ONOS 1Candidates = ONOS
2, ONOS 3
Master Switch A = ONOS 1
Candidates = ONOS 2, ONOS 3
Distributed Network OS
Instance 2 Instance 3Instance 1
Distributed Registry
Host
Host
Host
A
B
C
D
E
F
Master Switch A = NONE
Candidates = ONOS 2, ONOS 3
Master Switch A = NONE
Candidates = ONOS 2, ONOS 3
Master Switch A = NONE
Candidates = ONOS 2, ONOS 3
Master Switch A = ONOS 2
Candidates = ONOS 3
Master Switch A = ONOS 2
Candidates = ONOS 3
Master Switch A = ONOS 2
Candidates = ONOS 3
POSTECH 55/62CSED702Y: Software Defined Networking
ONOS Distributed Core
Distributed Core Responsible for all state management concerns Organized as a collection of “STORES”
• E.g., topology, links, link resources and etc. State management choices (ACID vs. BASE)
• ACID (Atomicity, Consistency, Isolation, Durability)• BASE (Basically Available, Soft state, Eventually consistency)
State and Properties
State Properties
Network Topology Eventually consistent, low latency access
Flow Rules, Flow Stats Eventually consistent, shardable, soft state
Switch – Controller MappingDistributed Locks
Strongly consistent, slow changing
Application IntentsResource Allocations
Strongly consistent, durable
POSTECH 56/62CSED702Y: Software Defined Networking
Network Topology State Global Network View for NB Applications
State inventory of devices, hosts and links Goal: closely track the state of physical network Fully replicated fro scale and low latency access Causal consistency using logical timestamps (logical clock systems)
<mastership term #, seq #>
Logical clock system(logical local time,logical global time)
logical local time↓
event sequence #
logical global time↓
mastership term
<1,1> <2,1>
<4,1> <2,2>
<3,1>
POSTECH 57/62CSED702Y: Software Defined Networking
Other States States of Flow Rules and Flow Stats
Switch specific state• Flow rules switch specific match-action pairs• Flow stats flow specific traffic stats
Partitioned with current switch master serving as the authoritative copy• Can be fully replicated for quick failover
Soft state• Inconsistencies are reconciled by refreshing from source
States of Other Elements Switch to controller mapping, application intents, resource allocations Require strong consistency Possible solutions
• Hazelcast based solution• Used in ONOS version 1.0
• Has some shortcoming on supporting for durability
• Use of Raft consensus algorithm• Provides all the desired consistency and durability properties
POSTECH 58/62CSED702Y: Software Defined Networking
Application and Intent Framework
Intent Framework Programming abstraction
• Intents• Compilers• Installers
Execution framework• Intent service• Intent store
Intents Provide a high-level interface that focuses on what should be done
rather than how it is specifically programmed Abstract network complexity from applications Extend easily to produce more complex functionality through com-
binations of other intents
POSTECH 59/62CSED702Y: Software Defined Networking
Intent Example
Host to Host Intent
Path IntentPath Intent
Flow Rule Batch
Flow Rule Batch
Flow Rule Batch
Flow Rule Batch
Compilation
Installation
Compiler & Installer Compiler: produce more specific Intents given the environment
• E.g., find the corresponding paths between two hosts
Installer: transform Intents into device commands
POSTECH 60/62CSED702Y: Software Defined Networking
Intent Framework Design Intent Framework
Translates intents into device instructions (state, policy) Reacts to changing network conditions Extends dynamically to add, modify functionality (compilers, installers)
POSTECH 61/62CSED702Y: Software Defined Networking
Representing Networks Network Graph
Graph has basic network objects like switch, port, device and links Application computes path by traversing the links from SRC to DST Application writes each flow entry for the path
port
switch port
host
Flow pathFlow entry
port
onport
port
port
linkswitch
inport
on
Flow entry
host
outportswitchswitch
flowflow
POSTECH 62/62CSED702Y: Software Defined Networking
Adapter Layer Design Consideration of Adapter
ONOS supports multiple southbound protocols Adapters provide descriptions of data plane elements to the core Adapters hide protocol complexity from ONOS
Device Subsystem Responsible for discovering and
tracking the devices Enable operators and apps to
control the devices Core model relies on the Device
and Port model objects
DeviceManager OFDeviceProvider
Device OpenFlowSwitch
DeviceId/ElementId Dpid
Port OFPortDesc
MastershipRole RoleState
POSTECH 63/62CSED702Y: Software Defined Networking
Q&A
POSTECH 64/62CSED702Y: Software Defined Networking
Comparisons on OpenFlow Controllers Remarks
Python controllers do not support multi-threading, no scalability Beacon shows the best scalability Scalability discrepancy due to following two reasons:
• Algorithm of distributing incoming messages between threads• Mechanism or the libraries used for network interaction
Back
Source: Shalimov, Alexander, et al. "Advanced study of SDN/OpenFlow controllers." Proceedings of the 9th Central & Eastern European Software Engineering Conference in Russia. ACM, 2013.