midonet quick start guide for ubuntu 16.04 / newton being used with the default openstack networking...

docs.midonet.org

http://docs.midonet.org

MidoNet Quick Start Guide forUbuntu 16.04 / Newton

2017-03-23 14:49 UTC 5.4

ii

MidoNet Quick Start Guide for Ubuntu 16.04 / Newton5.4 (2017-03-23 14:49 UTC)Copyright © 2017 Midokura SARL All rights reserved.

MidoNet is a network virtualization software for Infrastructure-as-a-Service (IaaS) clouds.

It decouples your IaaS cloud from your network hardware, creating an intelligent software abstractionlayer between your end hosts and your physical network.

This guide walks through the minimum installation and configuration steps neccessary to use MidoNetwith OpenStack.

Note

Please consult the MidoNet Mailing Lists or Chat if you need assistance.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS"BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific lan-guage governing permissions and limitations under the License.

https://github.com/midonet/midonet/wiki/Communication

http://www.apache.org/licenses/LICENSE-2.0


2017-03-23 14:49 UTC 5.4

iii

Table of ContentsPreface ...................................................................................................................... iv

Conventions ...................................................................................................... iv1. Architecture ........................................................................................................... 1

Hosts and Services ............................................................................................. 22. Basic Environment Configuration ........................................................................... 4

Networking Configuration ................................................................................. 4Repository Configuration ................................................................................... 4

3. OpenStack Installation ........................................................................................... 6Identity Service (Keystone) ................................................................................ 6Compute Services (Nova) ................................................................................... 6Networking Services (Neutron) .......................................................................... 7

4. MidoNet Installation ............................................................................................ 13NSDB Nodes .................................................................................................... 13Controller Node ............................................................................................... 16Midolman Installation ...................................................................................... 17MidoNet Host Registration .............................................................................. 19

5. Initial Network Configuration .............................................................................. 21External Network ............................................................................................. 21Tenant Network .............................................................................................. 23

6. Edge Router Setup .............................................................................................. 257. BGP Uplink Configuration .................................................................................... 278. Further Steps ....................................................................................................... 29


2017-03-23 14:49 UTC 5.4

iv

Preface

ConventionsThe MidoNet documentation uses several typesetting conventions.

Notices

Notices take these forms:

Note

A handy tip or reminder.

Important

Something you must be aware of before proceeding.

Warning

Critical information about the risk of data loss or security issues.

Command prompts

$ prompt

Any user, including the root user, can run commands that are prefixed with the $prompt.

# prompt

The root user must run commands that are prefixed with the # prompt. You can alsoprefix these commands with the sudo command, if available, to run them.


2017-03-23 14:49 UTC 5.4

1

1. Architecture

Table of ContentsHosts and Services ..................................................................................................... 2

This guide assumes the following example system architecture.

OpenStack Controller Node:

• Controller Node (controller)

Compute Node:

• Compute Node (compute1)

Since MidoNet is a distributed system, it does not have the concept of a Network Nodeas being used with the default OpenStack networking plugin. Instead it uses two ormore Gateway Nodes that utilize Quagga to provide connectivity to external networksvia the Border Gateway Protocol (BGP).

• Gateway Node 1 (gateway1)

• Gateway Node 2 (gateway2)

Three or more hosts are being used for the MidoNet Network State Database (NSDB)cluster which utilizes ZooKeeper and Cassandra to store virtual network topology andconnection state information:

• NSDB Node 1 (nsdb1)



Important

Ideally, both the ZooKeeper transaction log and Cassandra data files needtheir own dedicated disks, with additional disks for other services on thehost. However, for small POCs and small deployments, it is ok to share theCassandra disk with other services and just leave the ZooKeeper transactionlog on its own.

The MidoNet Agent (Midolman) has to be installed on all nodes where traffic enters orleaves the virtual topology. In this guide this are the gateway1, gateway2 and com-pute1 hosts.

The Midonet Cluster can be installed on a separate host, but this guide assumes it to beinstalled on the controller host.

The Midonet Command Line Interface (CLI) can be installed on any host that has con-nectivity to the MidoNet Cluster. This guide assumes it to be installed on the controllerhost.

http://www.quagga.net/

https://zookeeper.apache.org/

http://docs.datastax.com/en/cassandra/2.2/cassandra/cassandraAbout.html


2017-03-23 14:49 UTC 5.4

2

The Midonet Neutron Plugin replaces the ML2 Plugin and has to be installed on the con-troller.

Hosts and ServicesController Node (controller)

• General

• Database (MariaDB)

• Message Broker (RabbitMQ)

• OpenStack

• Identity Service (Keystone)

• Image Service (Glance)

• Compute (Nova)

• Networking (Neutron)

• Neutron Server

• Dashboard (Horizon)

• MidoNet

• Cluster

• CLI

• Neutron Plugin

Compute Node (compute1)

• OpenStack

• Compute (Nova)

• Networking (Neutron)

• MidoNet

• Agent (Midolman)

NSDB Nodes (nsdb1, nsdb2, nsdb3)

• Network State Database (NSDB)

• Network Topology (ZooKeeper)

• Network State Information (Cassandra)

Gateway Nodes (gateway1, gateway2)

• BGP Daemon (Quagga)


2017-03-23 14:49 UTC 5.4

3

• MidoNet

• Agent (Midolman)


2017-03-23 14:49 UTC 5.4

4

2. Basic Environment Configuration

Table of ContentsNetworking Configuration ......................................................................................... 4Repository Configuration ........................................................................................... 4

Networking ConfigurationImportant

All hostnames must be resolvable, either via DNS or locally.

This guide assumes that you follow the instructions in Host networking of the OpenS-tack Documentation.

Repository ConfigurationConfigure necessary software repositories and update installed packages.

1. Configure Ubuntu repositories

Edit the /etc/apt/sources.list file to contain the following:

# Ubuntu Main Archivedeb http://archive.ubuntu.com/ubuntu/ xenial maindeb http://security.ubuntu.com/ubuntu xenial-updates maindeb http://security.ubuntu.com/ubuntu xenial-security main

# Ubuntu Universe Archivedeb http://archive.ubuntu.com/ubuntu/ xenial universedeb http://security.ubuntu.com/ubuntu xenial-updates universedeb http://security.ubuntu.com/ubuntu xenial-security universe

1. Configure DataStax repository

Create the /etc/apt/sources.list.d/datastax.list file and edit it to con-tain the following:

# DataStax (Apache Cassandra)deb http://debian.datastax.com/community 2.2 main

Download and install the repository’s key:

# curl -L https://debian.datastax.com/debian/repo_key | apt-key add -

1. Configure MidoNet repositories

Create the /etc/apt/sources.list.d/midonet.list file and edit it to con-tain the following:

# MidoNetdeb http://builds.midonet.org/midonet-5.4 stable main

# MidoNet OpenStack Integration

http://docs.openstack.org/mitaka/install-guide-ubuntu/environment-networking.html


2017-03-23 14:49 UTC 5.4

5

deb http://builds.midonet.org/openstack-mitaka stable main

# MidoNet 3rd Party Tools and Librariesdeb http://builds.midonet.org/misc stable main

Download and install the repositories' key:

# curl -L https://builds.midonet.org/midorepo.key | apt-key add -

2. Install available updates

# apt-get update# apt-get dist-upgrade

3. If necessary, reboot the system

# reboot


2017-03-23 14:49 UTC 5.4

6

3. OpenStack Installation

Table of ContentsIdentity Service (Keystone) ........................................................................................ 6Compute Services (Nova) ........................................................................................... 6Networking Services (Neutron) .................................................................................. 7

Important

Follow the OpenStack newton Installation Guide for Ubuntu , but note thefollowing differences.

Identity Service (Keystone)Important

Follow the OpenStack documentation’s Add the Identity service instruc-tions, but note the following additions.

1. Verify operation

Do not apply step '1. For security reasons, disable the temporary authentication to-ken mechanism' from the 'Verify operation' section.

2. Create MidoNet API Service

As Keystone admin, execute the following command:

$ openstack service create --name midonet --description "MidoNet API Service" midonet

3. Create MidoNet Administrative User

As Keystone admin, execute the following commands:

$ openstack user create --domain default --password-prompt midonet$ openstack role add --project service --user midonet admin

Compute Services (Nova)Important

Follow the OpenStack documentation’s Compute service instructions, butnote the following differences.

Controller Node

Note

Follow the OpenStack documentation’s Install and configure controllernode instructions as is.

http://docs.openstack.org/newton/install-guide-ubuntu/index.html

http://docs.openstack.org/newton/install-guide-ubuntu/keystone.html

http://docs.openstack.org/newton/install-guide-ubuntu/nova.html

http://docs.openstack.org/newton/install-guide-ubuntu/nova-controller-install.html

http://docs.openstack.org/newton/install-guide-ubuntu/nova-controller-install.html


2017-03-23 14:49 UTC 5.4

7

Compute Node

Important

Follow the OpenStack documentation’s Install and configure a computenode instructions, but note the following additions.

1. Configure libvirt

Edit the /etc/libvirt/qemu.conf file to contain the following:

user = "root"group = "root"

cgroup_device_acl = [ "/dev/null", "/dev/full", "/dev/zero", "/dev/random", "/dev/urandom", "/dev/ptmx", "/dev/kvm", "/dev/kqemu", "/dev/rtc","/dev/hpet", "/dev/vfio/vfio", "/dev/net/tun"]

2. Restart the libvirt service

# service libvirt-bin restart

Networking Services (Neutron)Important

Follow the OpenStack documentation’s Networking service instructions, butnote the following differences.

Controller Node

Important

Follow the OpenStack documentation’s Install and configure controllernode instructions, but note the following differences and additions.

1. Prerequisites

Apply as is.

2. Configure networking options

Do not apply.

a. Instead, install the following packages:

# apt-get install neutron-server python-networking-midonet-ext python-neutronclient# apt-get purge neutron-plugin-ml2

b. Configure the server component:

Edit the /etc/neutron/neutron.conf file and configure the following keys:

[DEFAULT]...

http://docs.openstack.org/newton/install-guide-ubuntu/nova-compute-install.html

http://docs.openstack.org/newton/install-guide-ubuntu/nova-compute-install.html

http://docs.openstack.org/newton/install-guide-ubuntu/neutron.html

http://docs.openstack.org/newton/install-guide-ubuntu/neutron-controller-install.html

http://docs.openstack.org/newton/install-guide-ubuntu/neutron-controller-install.html


2017-03-23 14:49 UTC 5.4

8

core_plugin = midonet_v2_extservice_plugins = midonet_l3_ext...dhcp_agent_notification = False...allow_overlapping_ips = True...rpc_backend = rabbit...auth_strategy = keystone...notify_nova_on_port_status_changes = Truenotify_nova_on_port_data_changes = Truenova_url = http://controller:8774/v2.1...# In order to enable 'fip64' extension feature, the API extension path must# be specified. The path depends on the directory location in which# python-networking-midonet-ext is installed. For example, if the installation# path is /usr/lib/python2.7/dist-packages/midonet-ext, add the following:api_extensions_path = /usr/lib/python2.7/dist-packages/midonet-ext/neutron/extensions

[database]...connection = mysql+pymysql://neutron:NEUTRON_DBPASS@controller/neutron

[oslo_messaging_rabbit]...rabbit_host = controllerrabbit_userid = openstackrabbit_password = RABBIT_PASS

[keystone_authtoken]...auth_uri = http://controller:5000auth_url = http://controller:35357memcached_servers = controller:11211auth_plugin = passwordproject_domain_id = defaultuser_domain_id = defaultproject_name = serviceusername = neutronpassword = NEUTRON_PASS

[nova]...auth_url = http://controller:35357auth_plugin = passwordproject_domain_id = defaultuser_domain_id = defaultregion_name = RegionOneproject_name = serviceusername = novapassword = NOVA_PASS

Note

When using multiple service plugins, separate them with commas:

[DEFAULT]


2017-03-23 14:49 UTC 5.4

9

service_plugins = foo,bar,midonet.neutron.services.l3.l3_midonet.MidonetL3ServicePlugin

3. Configure the MidoNet plug-in

a. Create the directory for the MidoNet plugin:

mkdir /etc/neutron/plugins/midonet

b. Create the /etc/neutron/plugins/midonet/midonet.ini file and edit itto contain the following:

[MIDONET]# MidoNet API URLmidonet_uri = http://controller:8181/midonet-api# MidoNet administrative user in Keystoneusername = midonetpassword = MIDONET_PASS# MidoNet administrative user's tenantproject_id = service# MidoNet API clientclient = midonet_ext.neutron.client.api.MidonetApiClient

c. Edit the /etc/default/neutron-server file to contain the following:

NEUTRON_PLUGIN_CONFIG="/etc/neutron/plugins/midonet/midonet.ini"

4. Configure the metadata agent

Do not apply.

5. Configure Compute to use Networking*

Apply as is.

6. Configure Load-Balancer-as-a-Service (LBaaS)

Additionally to the OpenStack Installation Guide, configure Load-Balancer-as-a-Ser-vice (LBaaS) as described in the section called “Configure Load-Balancer-as-a-Service(LBaaS)” [10].

7. Configure FireWall-as-a-Service (FWaaS)

Additionally to the OpenStack Installation Guide, configure FireWall-as-a-Ser-vice (FWaaS) as described in the section called “Configure FireWall-as-a-Service(FWaaS)” [10].

8. To finalize installation

Do not apply.

Instead, perform the following steps.

a. Populate the database:

# su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/midonet/midonet.ini upgrade head" neutron# su -s /bin/sh -c "neutron-db-manage --subproject networking-midonet upgrade head" neutron

b. Restart the Compute service:


2017-03-23 14:49 UTC 5.4

10

# service nova-api restart

c. Restart the Networking service:

# service neutron-server restart

Configure Load-Balancer-as-a-Service (LBaaS)

1. Install Neutron Load-Balancing-as-a-Service

# apt-get install python-neutron-lbaas

2. Enable the MidoNet driver

Enable the MidoNet driver by using the service_provider option in the /etc/neutron/neutron.conf file:

[service_providers]service_provider = LOADBALANCERV2:Midonet:midonet_ext.neutron.services.loadbalancer.v2_driver.MidonetLoadBalancerDriver:default

3. Enable the LBaaS plug-in

Enable the LBaaS plug-in by using the service_plugins option in the [DEFAULT]section of the /etc/neutron/neutron.conf file:

[DEFAULT]service_plugins = neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2

Note


[DEFAULT]service_plugins = foo,bar,neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2

4. Enable load balancing in the dashboard

Follow the instructions in LBaaS Newton documentation.


Finalize the installation as described in Neutron Controller Node Installation.

Configure FireWall-as-a-Service (FWaaS)

1. Install Neutron FireWall-as-a-Service

# apt-get install python-neutron-fwaas

2. Enable the MidoNet FWaaS plug-in

Enable the MidoNet FWaaS plug-in by using the service_plugins option in the /etc/neutron/neutron.conf file:

service_plugins = midonet.neutron.services.firewall.plugin.MidonetFirewallPlugin

http://docs.openstack.org/newton/networking-guide/config-lbaas.html


2017-03-23 14:49 UTC 5.4

11

Note


[DEFAULT]service_plugins = foo,bar,midonet.neutron.services.firewall.plugin.MidonetFirewallPlugin

3. Enable firewall in the dashboard

Change the enable_firewall option to True in the /etc/openstack-dash-board/local_settings.py file:

OPENSTACK_NEUTRON_NETWORK = { 'enable_firewall': True, ...}



Configure Quality of Service (QoS)

1. Enable the QoS plug-in

Enable the QoS plug-in by using the service_plugins option in the [DEFAULT]section of the /etc/neutron/neutron.conf file:

[DEFAULT]service_plugins = qos

Note


[DEFAULT]service_plugins = foo,bar,qos

2. Enable the MidoNet QoS notification driver

Enable the MidoNet QoS notification driver by using the notification_driversoption in the qos section of /etc/neutron/neutron.conf file:

[qos]notification_drivers = midonet



Compute Node

Important

Follow the OpenStack documentation’s Install and configure compute nodeinstructions, but note the following differences.

1. Install the components

http://docs.openstack.org/newton/install-guide-ubuntu/neutron-compute-install.html


2017-03-23 14:49 UTC 5.4

12

Do not apply.

2. Configure the common component

Do not apply.

3. Configure networking options

Do not apply.

4. Configure Compute to use Networking

Apply as is.

5. Finalize installation

Do not apply.

a. Instead, restart the following service:

# service nova-compute restart


2017-03-23 14:49 UTC 5.4

13

4. MidoNet Installation

Table of ContentsNSDB Nodes ............................................................................................................ 13Controller Node ....................................................................................................... 16Midolman Installation .............................................................................................. 17MidoNet Host Registration ...................................................................................... 19

NSDB Nodes

ZooKeeper Installation

1. Install ZooKeeper packages

# apt-get install openjdk-8-jre-headless# apt-get install zookeeper zookeeperd zkdump

2. Configure ZooKeeper

a. Common Configuration

Edit the /etc/zookeeper/conf/zoo.cfg file to contain the following:

server.1=nsdb1:2888:3888server.2=nsdb2:2888:3888server.3=nsdb3:2888:3888autopurge.snapRetainCount=10autopurge.purgeInterval =12

Important

For production deployments it is recommended to configure the stor-age of snapshots in a different disk than the commit log, this is doneby setting the parameters dataDir and dataLogDir in zoo.cfg. Inaddition we advice to use an SSD drive for the commit log.

b. Node-specific Configuration

i. NSDB Node 1

Create the /var/lib/zookeeper/myid file and edit it to contain the host’sID:

# echo 1 > /var/lib/zookeeper/myid

ii. NSDB Node 2



iii. NSDB Node 3


2017-03-23 14:49 UTC 5.4

14



3. Restart ZooKeeper

# service zookeeper restart

4. Verify ZooKeeper Operation

After installation of all nodes has been completed, verify that ZooKeeper is operatingproperly.

A basic check can be done by executing the ruok (Are you ok?) command on allnodes. This will reply with imok (I am ok.) if the server is running in a non-error state:

$ echo ruok | nc 127.0.0.1 2181imok

More detailed information can be requested with the stat command, which listsstatistics about performance and connected clients:

$ echo stat | nc 127.0.0.1 2181Zookeeper version: 3.4.5--1, built on 06/10/2013 17:26 GMTClients: /127.0.0.1:34768[0](queued=0,recved=1,sent=0) /192.0.2.1:49703[1](queued=0,recved=1053,sent=1053)

Latency min/avg/max: 0/4/255Received: 1055Sent: 1054Connections: 2Outstanding: 0Zxid: 0x260000013dMode: followerNode count: 3647

Cassandra Installation

1. Install Cassandra packages

# apt-get install openjdk-8-jre-headless# apt-get install dsc22

2. Configure Cassandra

a. Common Configuration

Edit the /etc/cassandra/cassandra.yaml file to contain the following:

# The name of the cluster.cluster_name: 'midonet'

...

# Addresses of hosts that are deemed contact points.seed_provider: - class_name: org.apache.cassandra.locator.SimpleSeedProvider parameters: - seeds: "nsdb1,nsdb2,nsdb3"


2017-03-23 14:49 UTC 5.4

15

b. Node-specific Configuration

i. NSDB Node 1


# Address to bind to and tell other Cassandra nodes to connect to.listen_address: nsdb1

...

# The address to bind the Thrift RPC service.rpc_address: nsdb1

ii. NSDB Node 2



...


iii. NSDB Node 3



...


3. Clean existing data and restart Cassandra

# service cassandra stop# rm -rf /var/lib/cassandra/*# service cassandra start

4. Verify Cassandra Operation

After installation of all nodes has been completed, verify that Cassandra is operatingproperly.

Important

If Cassandra fails to start and prints a "buffer overflow" error messagein its log file, you may try associating 127.0.0.1 with the hostname inetc/hosts (so that hostname -i will show 127.0.0.1). This maysolve the Cassandra start problem.

A basic check can be done by executing the nodetool status command. This willreply with UN (Up / Normal) in the first column if the servers are running in a non-er-ror state:

$ nodetool --host 127.0.0.1 status


2017-03-23 14:49 UTC 5.4

16

[...]Status=Up/Down|/ State=Normal/Leaving/Joining/Moving-- Address Load Tokens Owns Host ID RackUN 192.0.2.1 123.45 KB 256 33.3% 11111111-2222-3333-4444-555555555555 rack1UN 192.0.2.2 234.56 KB 256 33.3% 22222222-3333-4444-5555-666666666666 rack1UN 192.0.2.3 345.67 KB 256 33.4% 33333333-4444-5555-6666-777777777777 rack1

Controller Node

MidoNet Cluster Installation

1. Install MidoNet Cluster package

# apt-get install midonet-cluster

2. Set up mn-conf

Edit /etc/midonet/midonet.conf to point mn-conf to the ZooKeeper cluster:

[zookeeper]zookeeper_hosts = nsdb1:2181,nsdb2:2181,nsdb3:2181

3. Configure access to the NSDB

This step needs to happen only once, it will set up access to the NSDB for the Mi-doNet Cluster and Agent nodes.

Run the following command to set the cloud-wide values for the ZooKeeper and Cas-sandra server addresses:

$ cat << EOF | mn-conf set -t defaultzookeeper { zookeeper_hosts = "nsdb1:2181,nsdb2:2181,nsdb3:2181"}

cassandra { servers = "nsdb1,nsdb2,nsdb3"}EOF

Run the following command to set the Cassandra replication factor:

$ echo "cassandra.replication_factor : 3" | mn-conf set -t default

4. Configure Keystone access

This step needs to happen only once, it will set up access to Keystone for the Mi-doNet Cluster node(s).

Determine domain_name and domain_id to be used for Keystone authentication:

# openstack domain list+---------+---------+---------+----------------------------------------------------------------------+| ID | Name | Enabled | Description |


2017-03-23 14:49 UTC 5.4

17

+---------+---------+---------+----------------------------------------------------------------------+| default | Default | True | Owns users and tenants (i.e. projects) available on Identity API v2. |+---------+---------+---------+----------------------------------------------------------------------+

For tenant_name, use the project/tenant that the midonet user belongs to, asconfigured during user creation.

Configure the authentication parameters for MidoNet Cluster via mn-conf:

$ cat << EOF | mn-conf set -t defaultcluster.auth { admin_role = "admin" provider_class = "org.midonet.cluster.auth.keystone.KeystoneService" keystone { admin_token = "" protocol = "http" host = "controller" port = 35357 domain_name = "Default" domain_id = "default" tenant_name = "$MIDONET_TENANT" user_name = "midonet" user_password = "$MIDONET_PASS" version = 3 }}EOF

5. Start the MidoNet Cluster

# service midonet-cluster start

MidoNet CLI Installation

1. Install MidoNet CLI package

# apt-get install python-midonetclient

2. Configure MidoNet CLI

Create the ~/.midonetrc file and edit it to contain the following:

[cli]api_url = http://controller:8181/midonet-apiusername = adminpassword = ADMIN_PASSproject_id = admin

Midolman InstallationThe MidoNet Agent (Midolman) has to be installed on all nodes where traffic enters orleaves the virtual topology, in this guide this are the gateway1, gateway2 and com-pute1 nodes.

1. Install Midolman package

# apt-get install openjdk-8-jre-headless# apt-get install midolman


2017-03-23 14:49 UTC 5.4

18

2. Set up mn-conf

Edit /etc/midolman/midolman.conf to point mn-conf to the ZooKeeper cluster:

[zookeeper]zookeeper_hosts = nsdb1:2181,nsdb2:2181,nsdb3:2181

3. Configure resource usage

Run these steps on each agent host in order to configure resource usage.

Important

For production environments the large templates are strongly recom-mended.

a. Midolman resource template

Run the following command to configure the Midolman resource template:

$ mn-conf template-set -h local -t TEMPLATE_NAME

Replace TEMPLATE_NAME with one of the following templates:

agent-compute-largeagent-compute-mediumagent-gateway-largeagent-gateway-mediumdefault

b. Java Virtual Machine (JVM) resource template

Replace the default /etc/midolman/midolman-env.sh file with one of thebelow to configure the JVM resource template:

/etc/midolman/midolman-env.sh.compute.large/etc/midolman/midolman-env.sh.compute.medium/etc/midolman/midolman-env.sh.gateway.large/etc/midolman/midolman-env.sh.gateway.medium

4. Configure MidoNet Metadata Proxy for all agents

This step needs to happen only once, it will set up MidoNet Metadata Proxy for allMidoNet Agent nodes.

Run the following commands to set the cloud-wide values for the MidoNet MetadataProxy:

$ echo "agent.openstack.metadata.nova_metadata_url : \"http://controller:8775\"" | mn-conf set -t default$ echo "agent.openstack.metadata.shared_secret : shared_secret" | mn-conf set -t default$ echo "agent.openstack.metadata.enabled : true" | mn-conf set -t default

controller, 8775, and shared_secret should be replaced with appropriate values.They need to match with the corresponding Nova Metadata API configuration.

controller and 8775 specify the address on which Nova accepts Meta-data API requests. shared_secret has to be the same as specified by the"metadata_proxy_shared_secret" field in the "neutron" section of nova.conf.


2017-03-23 14:49 UTC 5.4

19

The Nova side of the configuration for the metadata service is same as when usingNeutron Metadata Proxy. See the OpenStack documentation for details:

Cloud Administrator Guide: Configure Metadata

Important

The Metadata Proxy creates an interface on the hypervisor hosts, named"metadata".

When using iptables it may be necessary to add a rule to accept trafficon that interface:

iptables -I INPUT 1 -i metadata -j ACCEPT

1. Start Midolman

# service midolman start

MidoNet Host Registration1. Launch MidoNet CLI

$ midonet-climidonet>

2. Create tunnel zone

MidoNet supports the Virtual Extensible LAN (VXLAN) and Generic Routing Encapsu-lation (GRE) protocols to communicate to other hosts within a tunnel zone.

To use the VXLAN protocol, create the tunnel zone with type 'vxlan':

midonet> tunnel-zone create name tz type vxlantzone0

To use the GRE protocol, create the tunnel zone with type 'gre':

midonet> tunnel-zone create name tz type gretzone0

Important

Make sure to allow GRE/VXLAN traffic for all hosts that belong to the tun-nel zone. For VXLAN MidoNet uses UDP port 6677 as default.

1. Add hosts to tunnel zone

midonet> list tunnel-zonetzone tzone0 name tz type vxlan

midonet> list hosthost host0 name controller alive truehost host1 name gateway1 alive truehost host2 name gateway2 alive truehost host3 name compute1 alive true

midonet> tunnel-zone tzone0 add member host host0 address ip_address_of_host0

http://docs.openstack.org/admin-guide-cloud/networking_config-identity.html#configure-metadata


2017-03-23 14:49 UTC 5.4

20

zone tzone0 host host0 address ip_address_of_host0

midonet> tunnel-zone tzone0 add member host host1 address ip_address_of_host1zone tzone0 host host1 address ip_address_of_host1




2017-03-23 14:49 UTC 5.4

21

5. Initial Network Configuration

Table of ContentsExternal Network ..................................................................................................... 21Tenant Network ...................................................................................................... 23

Before launching your first instance, you must create the necessary virtual network in-frastructure to which the instances connect, including the external network and tenantnetwork.

External NetworkThe external network typically provides Internet access for your instances. By default,this network only allows Internet access from instances using Network Address Transla-tion (NAT). You can enable Internet access to individual instances using a floating IP ad-dress and suitable security group rules. The admin tenant owns this network because itprovides external network access for multiple tenants.

Note

Perform these commands on the controller node.

1. Create an external network

Source the admin credentials to gain access to admin-only CLI commands:

$ source admin-openrc.sh

Create the network:

$ neutron net-create ext-net --router:externalCreated a new network:+---------------------------+--------------------------------------+| Field | Value |+---------------------------+--------------------------------------+| admin_state_up | True || id | 893aebb9-1c1e-48be-8908-6b947f3237b3 || name | ext-net || provider:network_type | flat || provider:physical_network | external || provider:segmentation_id | || router:external | True || shared | False || status | ACTIVE || subnets | || tenant_id | 54cd044c64d5408b83f843d63624e0d8 |+---------------------------+--------------------------------------+

Like a physical network, a virtual network requires a subnet assigned to it. The ex-ternal network shares the same subnet and gateway associated with the physicalnetwork connected to the external interface on the network node. You should speci-fy an exclusive slice of this subnet for router and floating IP addresses to prevent in-terference with other devices on the external network.

2. Create a subnet on the external network


2017-03-23 14:49 UTC 5.4

22

Create the subnet:

$ neutron subnet-create ext-net EXTERNAL_NETWORK_CIDR --name ext-subnet \ --allocation-pool start=FLOATING_IP_START,end=FLOATING_IP_END \ --disable-dhcp --gateway EXTERNAL_NETWORK_GATEWAY

Replace FLOATING_IP_START and FLOATING_IP_END with the first and last IPaddresses of the range that you want to allocate for floating IP addresses. ReplaceEXTERNAL_NETWORK_CIDR with the subnet associated with the physical network.Replace EXTERNAL_NETWORK_GATEWAY with the gateway associated with the phys-ical network, typically the ".1" IP address. You should disable DHCP on this subnet be-cause instances do not connect directly to the external network and floating IP ad-dresses require manual assignment.

For example, using 203.0.113.0/24 with floating IP address range203.0.113.101 to 203.0.113.200:

$ neutron subnet-create ext-net 203.0.113.0/24 --name ext-subnet \ --allocation-pool start=203.0.113.101,end=203.0.113.200 \ --disable-dhcp --gateway 203.0.113.1Created a new subnet:+-------------------+------------------------------------------------------+| Field | Value |+-------------------+------------------------------------------------------+| allocation_pools | {"start": "203.0.113.101", "end": "203.0.113.200"} || cidr | 203.0.113.0/24 || dns_nameservers | || enable_dhcp | False || gateway_ip | 203.0.113.1 || host_routes | || id | 9159f0dc-2b63-41cf-bd7a-289309da1391 || ip_version | 4 || ipv6_address_mode | || ipv6_ra_mode | || name | ext-subnet || network_id | 893aebb9-1c1e-48be-8908-6b947f3237b3 || tenant_id | 54cd044c64d5408b83f843d63624e0d8 |+-------------------+------------------------------------------------------+


2017-03-23 14:49 UTC 5.4

23

Tenant NetworkThe tenant network provides internal network access for instances. The architecture iso-lates this type of network from other tenants. The demo tenant owns this network be-cause it only provides network access for instances within it.

Note

Perform these commands on the controller node.

1. Create the tenant network

Source the demo credentials to gain access to user-only CLI commands:

$ source demo-openrc.sh

Create the network:

$ neutron net-create demo-netCreated a new network:+-----------------+--------------------------------------+| Field | Value |+-----------------+--------------------------------------+| admin_state_up | True || id | ac108952-6096-4243-adf4-bb6615b3de28 || name | demo-net || router:external | False || shared | False || status | ACTIVE || subnets | || tenant_id | cdef0071a0194d19ac6bb63802dc9bae |+-----------------+--------------------------------------+

Like the external network, your tenant network also requires a subnet attached to it.You can specify any valid subnet because the architecture isolates tenant networks.By default, this subnet uses DHCP so your instances can obtain IP addresses.

2. Create a subnet on the tenant network

Create the subnet:

$ neutron subnet-create demo-net TENANT_NETWORK_CIDR \ --name demo-subnet --dns-nameserver DNS_RESOLVER \ --gateway TENANT_NETWORK_GATEWAY

Replace TENANT_NETWORK_CIDR with the subnet you want to associate withthe tenant network, DNS_RESOLVER with the IP address of a DNS resolver, andTENANT_NETWORK_GATEWAY with the gateway you want to associate with the ten-ant network, typically the ".1" IP address.

Example using 192.168.1.0/24 with DNS resolver 8.8.4.4 and gateway192.168.1.1:

$ neutron subnet-create demo-net 192.168.1.0/24 \ --name demo-subnet --dns-nameserver 8.8.4.4 --gateway 192.168.1.1Created a new subnet:+-------------------+------------------------------------------------------+| Field | Value |


2017-03-23 14:49 UTC 5.4

24

+-------------------+------------------------------------------------------+| allocation_pools | {"start": "192.168.1.2", "end": "192.168.1.254"} || cidr | 192.168.1.0/24 || dns_nameservers | 8.8.4.4 || enable_dhcp | True || gateway_ip | 192.168.1.1 || host_routes | || id | 69d38773-794a-4e49-b887-6de6734e792d || ip_version | 4 || ipv6_address_mode | || ipv6_ra_mode | || name | demo-subnet || network_id | ac108952-6096-4243-adf4-bb6615b3de28 || tenant_id | cdef0071a0194d19ac6bb63802dc9bae |+-------------------+------------------------------------------------------+

3. Create a router on the tenant network and attach the external and tenant net-works to it

$ neutron router-create demo-routerCreated a new router:+-----------------------+--------------------------------------+| Field | Value |+-----------------------+--------------------------------------+| admin_state_up | True || external_gateway_info | || id | 635660ae-a254-4feb-8993-295aa9ec6418 || name | demo-router || routes | || status | ACTIVE || tenant_id | cdef0071a0194d19ac6bb63802dc9bae |+-----------------------+--------------------------------------+

Attach the router to the demo tenant subnet:

$ neutron router-interface-add demo-router demo-subnetAdded interface b1a894fd-aee8-475c-9262-4342afdc1b58 to router demo-router.

Attach the router to the external network by setting it as the gateway:

$ neutron router-gateway-set demo-router ext-netSet gateway for router demo-router


2017-03-23 14:49 UTC 5.4

25

6. Edge Router SetupPrior to v5.0, with Neutron, you could set up the gateway in only one way, which wasto have a special singleton gateway router called the Provider Router created implic-itly when an external network was created in Neutron. The provider router sits at theedge of the cloud and interfaces with the uplink router. The Provider Router is whereBGP was typically configured. The biggest limitation of this approach was that it tookaway the scenario in which you wanted to have an L2 network at the edge instead of arouter. Another limitation was that only one such router could exist for the entire cloud.

These limitations are removed in v5.0, where you could design your gateway to be ei-ther L2 network or router with as many routers as you wish, all using the Neutron API.

There are two main changes:

Edge Router

The Provider Router is no longer implicitly created upon the external network creation.Instead, the edge gateway routers, called the Edge Routers, are created explicitly usingstandard Neutron API. With this approach, multiple Edge Routers can be created, andthey are optional.

Gateway Virtual Topology

In the previous model, the Provider Router was connected directly to the tenantrouters, with the external networks hanging off of the Provider Router.

In the new model, the external networks exist between the edge and the tenantrouters.

To create the gateway topology issue the following Neutron commands.

Create a standard neutron router:

neutron router-create <EDGE_ROUTER_NAME>

Attach the edge router to an external network:

neutron router-interface-add <EDGE_ROUTER_ID> <EXT_SUBNET_ID>

Create a special network called uplink network, representing the physical networkoutside of the cloud:

neutron net-create <UPLINK_NET_NAME> --tenant_id admin --provider:network_type uplink

Create a subnet for the uplink network matching the CIDR used in the uplink network(could just be /30 if linked directly to another router):

neutron subnet-create --tenant_id admin --disable-dhcp --name <UPLINK_SUBNET_NAME> <UPLINK_NET_NAME> <CIDR>

Create a port on the uplink network with a specific IP that you want to use and thebinding details so that this virtual port gets bound to a specific NIC on the gatewayhost:

neutron port-create <UPLINK_NET_ID> --binding:host_id <HOST_NAME> --binding:profile type=dict interface_name=<INTERFACE_NAME> --fixed-ip ip_address=<IP_ADDR>


2017-03-23 14:49 UTC 5.4

26

Attach the uplink port to the Edge Router:

neutron router-interface-add <EDGE_ROUTER_ID> port=<UPLINK_PORT_ID>


2017-03-23 14:49 UTC 5.4

27

7. BGP Uplink ConfigurationMidoNet utilizes the Border Gateway Protocol (BGP) for external connectivity.

For production deployments it is strongly recommended to use BGP due to it’s scalabili-ty and redundancy.

For demo or POC environments, alternatively static routing can be used.

The following instructions assume below sample environment:

• One floating IP network

• 192.0.2.0/24

• Two MidoNet gateway nodes

• gateway1, connecting to bgp1 via eth1

• gateway2, connecting to bgp2 via eth1

• Two remote BGP peers

• bgp1, 198.51.100.1, AS 64513

• bgp2, 203.0.113.1, AS 64514

• Corresponding MidoNet BGP peers

• 198.51.100.2, AS 64512

• 203.0.113.2, AS 64512

Follow these steps to configure the BGP uplinks.

1. Launch the MidoNet CLI and find the Edge Router

midonet-cli> router listrouter router0 name Edge Router state uprouter router1 name Tenant Router state up infilter chain0 outfilter chain1

In this example the Edge Router is router0.

2. Create and bind virtual ports for the BGP sessions

Refer to Chapter 6, “Edge Router Setup” [25] for instructions on how to create thenecessary ports and bind them to the Gateway hosts' physical network interfaces.

You can confirm the port configuration within MidoNet CLI by listing the EdgeRouter’s ports:

midonet> router router0 port listport port0 device router0 state up mac fa:16:3e:11:11:11 addresses 198.51.100.2/30port port1 device router0 state up mac fa:16:3e:22:22:22 addresses 203.0.113.2/30[...]

3. Configure basic BGP settings


2017-03-23 14:49 UTC 5.4

28

midonet> router router0 set asn 64512

midonet> router router0 add bgp-peer asn 64513 address 198.51.100.1router0:peer0

midonet> router router0 add bgp-peer asn 64514 address 203.0.113.1router0:peer1

midonet> router router0 list bgp-peerpeer peer0 asn 64513 address 198.51.100.1peer peer1 asn 64514 address 203.0.113.1

4. If needed, configure MD5 authentication:

midonet> router router0 bgp-peer peer0 set password BGP_PASSWORDmidonet> router router0 bgp-peer peer1 set password BGP_PASSWORD

5. If needed, configure custom timers that will take precedence over the default onesdefined in the MidoNet configuration:

midonet> router router0 bgp-peer peer0 set connect-retry 10midonet> router router0 bgp-peer peer0 set hold-time 5midonet> router router0 bgp-peer peer0 set keep-alive 5midonet> router router0 bgp-peer peer1 set connect-retry 10midonet> router router0 bgp-peer peer1 set hold-time 5midonet> router router0 bgp-peer peer1 set keep-alive 5midonet> router router0 list bgp-peerpeer peer0 asn 64513 address 198.51.100.1 keep-alive 5 hold-time 5 connect-retry 10peer peer1 asn 64514 address 203.0.113.1 keep-alive 5 hold-time 5 connect-retry 10

6. Add routes to the remote BGP peers

In order to be able to establish connections to the remote BGP peers, correspondingroutes have to be added.

midonet> router router0 route add src 0.0.0.0/0 dst 198.51.100.0/30 port router0:port0 type normalrouter0:route0

midonet> router router0 route add src 0.0.0.0/0 dst 203.0.113.0/30 port router0:port1 type normalrouter0:route1

7. Advertise BGP routes

In order to provide external connectivity for hosted virtual machines, the floating IPnetwork has to be advertised to the BGP peers.

midonet> router router0 add bgp-network net 192.0.2.0/24router0:net0

midonet> router router0 list bgp-networknet net0 net 192.0.2.0/24


2017-03-23 14:49 UTC 5.4

29

8. Further StepsMidoNet installation and integration into OpenStack is completed.

Note

Consult the Operations Guide for further instructions on operating Mi-doNet.

midonet quick start guide for ubuntu 16.04 / newton being used with the default openstack networking...

Documents