midonet deep dive

Taku Fukushima

Agenda1. MidoNet components (revisited)

2. Midolman deep dive

3. Getting started with coding

4. Wrap-up

MidoNet components

NSDB NSDB

NSDB

Open vSwitch Datapath

IF IFInterfaces on the host

IF

VM VM VM Midolman(MidoNet

agent)

Network

Flow Table

MidoNet APINova API

Horizon MidoNet CLI

Watch/modify

Add/remove flows

Neutron API

MidoNet Plugin

Host

Cache+

local state

Store virtual topology

information

Clients / Users

Nova compute

Midolman (MidoNet agent)

NSDB NSDB

NSDB

Open vSwitch Datapath

IF IFInterfaces on the host

IF

VM VM VM Midolman(MidoNet

agent)

Network

Flow Table

MidoNet APINova API

Horizon MidoNet CLI

Watch/modify

Add/remove flows

Neutron API

MidoNet Plugin

Host

Cache+

local state

Store virtual topology

information

Clients / Users

Nova compute

Midolman deep dive

Foundations• Java and Scala

• Java 8 migration is coming

• Guava and Guice

• Akka Actor

• Future and RxJava

• Jackson → Protocol Buffer

• Disruptor

Upcall Lifecycle1. Input stage

• Get upcalls with packets from the datapath

2. Packet processing stage

1. Deduplicate and queue packets

2. Simulate packets on the virtual topology

3. Deal with the wildcard flows

4. Determine the egress physical port

3. Output stage

• Emit packets and install flows based on the sims

Module diagrams

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

Midolman.java

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

1. Input stage

Select Loop

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

2. Packet processing stage

PacketContext

PacketContextPacketContextPacketContext

PacketContext

PacketContext

Routing by hashing with FlowKey

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

2. Packet processing stage

Retrieve virtual resources

Virtual Resource

Virtual Resource

Virtual Resource

Future • Completed • NotYetException

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

2. Packet processing stage

Manage virtual to local and physical mapping• Interface name to UUID • UUID to local port number • Which host has the interface • …

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

2. Packet processing stage

PacketContextPacketContextPacketContext

PacketContext

PacketContext

PacketContext

Simulation

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

2. Packet processing stage

Local datapath management

• Create local datapath ports • Track UUID to port # mapping • Manage overlay tunnels

PacketContext

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

2. Packet processing stage

Flow

Flow

Flow

Flow

Query statistics

Invalidate flows

PacketContext

MidolmanActorsService

Datapath Controller

Flow Controller

PacketsEntryPoint

RoutingManagerActor

DatapathReady

VirtualTopologyActor

VirtualToPhysicalMapper

HealthMonitor

NetlinkCallbackDispatcher

MtuIncreaser

DeduplicationActor

PacketWorkflow

UpcallDatapathConnectionManager

NSDB NSDB

NSDB

One-to-ManyOne-to-One

HTB

DisruptorRing Buffer

Supended Packets

WaitingRoom

(NetlinkInputChannel)

NetlinkChannel

(NetlinkOutputChannel)

DatapathChannel

Fast Path

State Management

Open vSwitch Datapath

Flow Table

Open vSwitch Datapath

Flow Table

DatapathReady

WildcardFlow

Upcall

Flow Invalidationby Tag

Packet

Virtual TopologyState data / Messages

Packet

Flow

Wildcard Flows

DatapathReady

Datapath portoperations

FlowManagement

3. Output stage

Select Loop

https://github.com/midonet/midonet/blob/master/docs/

“Read the docs.”

Getting started with coding

Prerequisites• JDK7

• JDK8 is coming

• protoc (Protocol Buffer >= 2.6.0)

!

• build-essential (for packaging deb packages)

• rpm (for packaging RPM packages)

• fpm (for packaging)

Dev environment (recommended)• JDK 7 (required)

• JDK8 is coming

• Protocol Buffers >= 2.6.0 (required)

• IntelliJ IDEA 14 community edition (OSS!) (optional)

• Scala plugin + protoc plugin

• Nice displays and keyboards

Building MidoNet

Building IDEA project

and modules

Importing MidoNet project

Wrap-up

Midolman deep dive• Upcall lifecycle

1. Input stage

2. Packet processing stage

3. Output stage

• Fast path and state management

• Read the docs

• Getting started with coding

The end of slides. Any Questions?