openstackネットワーク入門 – openstack最新情報セミナー 2015年4月

OpenStackネットワーク入門

Midokura JapanTakaaki Suzuki

Agenda

- History of Neutron- Nova Network- Quantum- Neutron- Packet Flow- Neutron and SDN

History of Neutron...


● In 2010, OpenStack debut with its monolithic networking model → Nova Network

2010

Nova

Network


● Nova Networkの機能o L2o DHCP

2010

Nova

Network


● In 2011, pluggable networking model was proposed to the community → Quantum

Nova

NetworkQuantum

OVS MidoNet…...OVS plugin and Vendor plugins

2010 2011

History of Neutron...● Quantum

o L3/L2o DHCPo FloatingIPo Secutiry Group

2011

Quantum


● In 2013, OpenStack Foundation received a notice that Quantum was a trademark and they had to change the name → Neutron

Nova

NetworkQuantum

OVS MidoNet…...

OVS plugin and Vendor plugins

Neutron

OVS MidoNet…...

20112010 2013


● Neutron (Juno)o L3/L2o DHCPo FloatingIPo Secutiry Group

Neutron

o LBaaS

o FWaaS

o VPNaaS

o DVR(分散ルーティング)

o L3HA

Server

eth0

LinuxBridge

VM01

192.168.10.2/24

Nova Network (Flat DHCP Manager)

L2 Switch Vlan1

nova-network

nova-compute

VM02

192.168.10.3/24

dnsmasq

Server

eth0

LinuxBridge

VM01

192.168.10.2/24

Nova Network (VLAN Manager)

L2 Switch

Trunk

nova-network

nova-compute

VM02

192.168.10.3/24

dnsmasqLinuxBridge

dnsmasq

VM01

172.16.0.2/24

VM02

172.16.0.2324

VLAN100 VLAN200

VLAN100

VLAN200

Server

iptables PRE-POST Routing

(DNAT,SNAT)

eth0

VM01

192.168.10.2/24

L2 Switch

nova-network

nova-compute

VM02

192.168.10.3/24

Vlan1

Nova Network (Routing NAT)

DNAT 119.15.120.4 : 172.16.0.2

SNAT any: 119.15.120.3

eth1

LinuxBridge

dnsmasq

Server

iptables PRE-POST Routing

(DNAT,SNAT)

Server

eth0

VM01

192.168.10.2/24

L2 Switch

nova-compute

VM02

192.168.10.3/24

VLAN1

Nova Network (Routing NAT)

DNAT 119.15.120.4 : 172.16.0.2

SNAT any: 119.15.120.3

eth1

LinuxBridge

dnsmasq

eth0

nova-network

GW: 192.168.10.1

Nova Network

● Linux Bridgeやiptablesを使う.

● ネットワークを分ける為に

VLANを利用している.

● L3ルーティングは外部機器におまかせ.

● 枯れた技術を好む人もいる.

Compute

ovs-agent

Quantum

Controller

Quantum-server

Compute

ovs-agent

Compute

quantum

openvswitch

plugin-agent

Network

quantum-l3-agent

quantum-dchp-agent

quantum-metadata-agent

quantum-openvswitch-plugin-

agent

OpenvSwitch

Plugin

Quantum Server (API)

Controller

Quantum-server

Quantum Server (API)● Plugin形式で様々なPluginを１つ選べる

Controller

Quantum-server

OpenvSwitch

Plugin

MidoNet

PluginLinuxBridge

Plugin

Quantum L2 agent

Compute

quantum

openvswitch

plugin-agent

Quantum L2 agent

● VM用の仮想L2設定自動的に設定してくれる.

Compute

quantum

openvswitch

plugin-agent

VM

VM

VM

VM

VM

VM

Quantum L3/DHCP/Metadata agent

Network

quantum-l3-agent

quantum-dchp-agent



agent

Quantum L3/DHCP/Metadata agent

● DHCP、インターネット外部接続

● VM用メタデータの提供

(hostname, SSH鍵等)

Network

quantum-l3-agent

quantum-dchp-agent



agent

Compute

ovs-agent

Quantum

Controller

Quantum-server

Compute

ovs-agent

Compute

quantum

openvswitch

plugin-agent

Network

quantum-l3-agent

quantum-dchp-agent



agent

OpenvSwitch

Plugin

Quantum

・Agentを使って様々な機能を提供する.・Plugin形式により様々な技術を利用できる.・NetoworkノードはL3機能(Routing, NAT)

ComputeノードではL2機能が利用できる.

Quantum -> Neutron

Compute

ovs-agent

Neutron

Controller

neutron-server

Compute

ovs-agent

Compute

neutron

openvswitch

agent

Network

neutron-l3-agent

neutron-dchp-agent

neutron-metadata-agent

neutron-openvswitch-agent

OpenvSwitch

Plugin

Compute

ovs-agent

Neutron (Juno)

Controller

neutron-server

Compute

ovs-agent

Compute

neutron

openvswitch

agent

Network

neutron-l3-agent (FWaaS)

neutron-dchp-agent



neutron-lbaas-agent

neutron-vpn-agent ML2

Plugin

Neutron Multi Layer 2 Plugin

Controller

neutron-server

ML2

Plugin


Controller

neutron-server

ML2

Plugin

Type Driver

GRE

VXLAN

FLAT

Mechanism

Driver

OpenvSwitch

Cisco

Arista


Controller

neutron-server

ML2

Plugin

Type Driver

GRE

VXLAN

FLAT

Mechanism

Driver

OpenvSwitch

LinuxBridge

Cisco

Compute

neutron

openvswitch

agent

Compute

neutron

cisco

agent


Controller

neutron-server

ML2

Plugin

Type Driver

GRE

VXLAN

FLAT

Mechanism

Driver

OpenvSwitch

LinuxBridge

Cisco

Compute

neutron

openvswitch

agent

Compute

neutron

cisco

agent

VM

VM

VM

VM

VM

VM

VM

VM

様々なネットワーク技術を複数組合せて利用できる.

Network

neutron-l3-agent (FWaaS)

neutron-dchp-agent



neutron-lbaas-agent

neutron-vpn-agent

Neutron (Juno)

Neutronでユーザーができること


1. 仮想ネットワーク(L2)の作成

① ① ①


2. 仮想L3ルータの作成

neutron router-create

① ① ①

② ②

3. ネットワークとルータの接続(router-interface-add)


① ① ①

② ②

③ ③ ③


4. ルータと外部ネットワークの接続(router-gateway-set)

① ① ①

② ②

③ ③ ③

④ ④


5. FloatingIP(グローバルIP DNAT/SNAT)をVMに付与できる.① ① ①

② ②

③ ③ ③

④ ④

⑤⑤ ⑤ ⑤


6. VMのポートに対してSecurityGroupを作成しSSHやHTTPの通信を許可等を適用することができる.

① ① ①

② ②

③ ③ ③

④ ④

⑤⑤ ⑤ ⑤⑥ ⑥ ⑥ ⑥ ⑥ ⑥


① ① ①

② ②

③ ③ ③

④ ④

⑤⑤ ⑤ ⑤⑥ ⑥ ⑥ ⑥ ⑥ ⑥

*外部ネットワーク作成は管理者(admin権限)のみ

Neutron環境のパケットフロー


Compute

neutron

openvswitch

agent

VMVM VM VM

eth0

VM VM

qbr43d***qbr5db***qbr5db*** qbr43d***

VM VM

br-tun

br-int

br-tun

br-int

Compute

neutron

openvswitch

agent

eth0VXLAN


Compute

neutron

openvswitch

agent

VMVM VM VM

eth0

VM VM

qbr43d***qbr5db***qbr5db*** qbr43d***

VM VM

br-tun

br-int

br-tun

br-int

Compute

neutron

openvswitch

agent

eth0VXLAN

tap interface

veth interface

LinuxBridge

ovs bridge

ovs bridge

vxlan-port

patch

tap interface

LinuxBridge

veth interface

ovs bridge

ovs bridge

patch

vxlan-port


Compute

neutron

openvswitch

agent

VMVM

eth0

VM VM

qbr43d***qbr5db***

br-tun

br-int

br-tun

br-int

VXLAN

DHCP

namespace

br-ex

router

namespace

Networknode

eth1eth0

neutron-l3-agent



Compute

neutron

openvswitch

agent

VMVM

eth0

VM VM

qbr43d***qbr5db***

br-tun

br-int

br-tun

br-int

VXLAN

DHCP

namespace

br-ex

router

namespace

Networknode

eth1eth0

neutron-l3-agnet


Network Namespace

Network Namespace

ip netnsip netns exec qrouter-*** ip linkip netns exec qrouter-*** netstat -nr


br-tun

br-int

br-ex

router

namespace

Networknode

eth1

neutron-l3-agnet



br-tun

br-int

br-ex

router

namespace

Networknode

eth1

GigabitEthenet0/0

119.15.120.129neutron-l3-agnet


router namespaceインターフェースqr-4894a511-cf: 192.168.73.1

qr-5ad834a4-df: 192.168.83.1


br-tun

br-int

br-ex

router

namespace

Networknode


qr-5ad834a4-df: 192.168.83.1

router namespaceルーティングテーブル192.168.73.0 255.255.255.0 qr-4894a511-cf CONNECTED

192.168.83.0 255.255.255.0 qr-5ad834a4-df CONNECTED

eth1

neutron-l3-agnet



br-tun

br-int

br-ex

router

namespace

Networknode

router namespaceルーティングテーブル192.168.73.0 255.255.255.0 qr-4894a511-cf CONNECTED

192.168.83.0 255.255.255.0 qr-5ad834a4-df CONNECTED

119.15.120.130 255.255.255.128 qg-ad4adf-1e CONNECTED

0.0.0.0 0.0.0.0 119.15.120.129 qg-ad4adf-1e STATIC

eth1

GigabitEthenet0/0

119.15.120.129


qr-5ad834a4-df: 192.168.83.1

qg-ad4adf22-1e: 119.120.130


br-tun

br-int

br-ex

router

namespace

Networknode

eth1

neutron-l3-agnet


router

namespace

router

namespace

router

namespace

ルーターnamespaceがどんどん作成される


br-tun

br-int

br-ex

router

namespace

Networknode

eth1

neutron-l3-agnet


router

namespace

router

namespace

router

namespace

router

namespacerouter

namespace

router

namespace router

namespace

router

namespacerouter

namespace

router

namespace

router

namespace router

namespacerouter

namespacerouter

namespace router

namespacerouter

namespace

router

namespace

router

namespacerouter

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespace

router

namespacerouter

namespace

router

namespace

ユーザーからルーターを定義すればするほどネットワークノードがボトルネックに...

Distributed Virtual Router

Neutron環境のパケットフロー (DVR)

Compute


neutron-l3-agent

VMVM

eth0

qbr5db***qbr43d***

VM VM

br-tun

br-int

br-tun

br-int

Compute


neutron-l3-agenteth0VXLAN

router

namespace

router

namespace


Compute


neutron-l3-agent

VMVM

eth0

qbr5db***qbr43d***

VM VM

br-tun

br-int

br-tun

br-int

Compute



router

namespace

router

namespace


qr-5ad834a4-df: 192.168.83.1


qr-5ad834a4-df: 192.168.83.1


Compute


neutron-l3-agent

VMVM

eth0

qbr5db***qbr43d***

VM VM

br-tun

br-int

br-tun

br-int

Compute



router

namespace

eth1 eth1

router

namespace


qr-5ad834a4-df: 192.168.83.1


qr-5ad834a4-df: 192.168.83.1


Compute


neutron-l3-agent

VMVM

eth0

qbr5db***qbr43d***

VM VM

br-tun

br-int

br-tun

br-int

Compute



router

namespace

router

namespace


qr-5ad834a4-df: 192.168.83.1


qr-5ad834a4-df: 192.168.83.1


Compute


neutron-l3-agent

VMVM

eth0

qbr5db***qbr43d***

VM VM

br-tun

br-int

br-tun

br-int

Compute


neutron-l3-agent

eth0VXLAN

router

namespace

FloatingIP

namespace

br-ex

eth1 eth1

router

namespace

br-ex

FloatingIP

namespace


Compute


neutron-l3-agent

VMVM

eth0

qbr5db***

br-tun

router

namespace

VXLAN

br-tunbr-ex

SNAT

namespace

Networknode

eth1

neutron-l3-agnet


eth0

eth1

br-ex

FloatingIP

namespacebr-int

br-int

L3HA OVS Network Node

● Active/Standby deployment with VRRP● Up to two Network Nodes

Network Node

(Active)

Network Node

(Standby)

L3 agentL3 agent

Ext Switch Ext Switch

VRRP

DHCP and MetaData


Compute

neutron

openvswitch

agent

VMVM

eth0

VM VM

qbr43d***qbr5db***

br-tun

br-int

br-tun

br-int

VXLAN

DHCP

namespace

Networknode eth2eth0

neutron-l3-agnet


neutron-dhcp-agent



Compute

neutron

openvswitch

agent

VMVM

eth0

VM VM

qbr43d***qbr5db***

br-tun

br-int

br-tun

VXLAN

DHCP

namespace


neutron-l3-agnet


neutron-dhcp-agent


dnsmasq

br-int


Compute

neutron

openvswitch

agent

VMVM

eth0

VM VM

qbr43d***qbr5db***

br-tun

br-int

br-tun

VXLAN

DHCP

namespace


neutron-l3-agnet


neutron-dhcp-agent


dnsmasq

br-int

metadataproxy

metadataagent

Novaapi

Networknode


Compute

neutron

openvswitch

agent

VMVM

eth0

VM VM

qbr43d***qbr5db***

br-tun

br-int

br-tun

br-int

VXLAN

br-ex

router

namespace

eth0

neutron-l3-agent


DHCP

namespace metadataproxy

metadataagent

Novaapi

eth2

NeutronとSDN製品

OpenStack Kilo (Neutron)

● 4/30リリース

Brocade ML2 driver for MLX and ICX switches

Brocade L3 routing plugin for MLX switch

Brocade Vyatta vRouter L3 Plugin

Brocade Vyatta vRouter Firewall Driver

Brocade Vyatta vRouter VPN Driver

Cisco Nexus

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/openstack-at-cisco/data_sheet_c78-727737.html

Contrail

http://www.slideshare.net/natiueno/contrail-overview-open-stack-days-tokyofeb2015

IBM SDN-VE

http://www-06.ibm.com/systems/jp/networking/software/sdnve/

MidoNet

http://midonet.org/midonet-tv.php

MidoNet and Cumulus

Janさんお願いします！

openstackネットワーク入門 – openstack最新情報セミナー 2015年4月

Technology