kuryr - zhaw blogs · • midonet agent sees the binding and updates the ovs datapath • service...
TRANSCRIPT
![Page 1: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/1.jpg)
Alexander Gabert, Senior System Engineer
Kuryr
![Page 2: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/2.jpg)
Container Networking: give me your lunch money!● Reinventing networking abstractions
○ CNM/CNI
● Changing and technology specific solutions○ Weave○ Flannel
● Each COE has its abstractions● Lack of isolation and policy level constructs● Overlay of VM nested containers
○ Performance, latency, SLA, management penalties
● No unified network (virtualisation) infrastructure○ Hard to connect VMs, Bare Metal and container workloads
![Page 3: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/3.jpg)
Neutron vs. Libnetwork. Different words. Same meaning.
![Page 4: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/4.jpg)
Kuryr: bring network packets from one world to another
● Kuryr is Czech word for "courier"● Open Source project● Part of OpenStack Big-Tent● Translates calls {from,to} containers {from,to} Neutron network model● Aims to support different Container Runtimes
○ Docker, rkt
● Aims to support different Container Orchestration Engines○ Docker Swarm○ Kubernetes○ Mesos
● Working together with OpenStack community○ Neutron, Magnum, Kolla, Higgins (NEW!)
![Page 5: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/5.jpg)
Kuryr Project Overview
● Key contributors from OpenStack hemisphere
● High level of collaboration on Specs and code
![Page 6: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/6.jpg)
One api to bind them all. One api to rule them all.
● Neutron as a proven production ready containers networking abstraction● Brings Network to COE the neutron way
○ Map container networking abstractions to the Neutron API○ Allow free of vendor lock-in choice
● Implements all the common code for vendors, allowing them to provide advanced container networking by just providing a simple binding script
● https://github.com/openstack/kuryr/blob/master/usr/libexec/kuryr/iovisor● https://github.com/openstack/kuryr/blob/master/usr/libexec/kuryr/midonet
![Page 7: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/7.jpg)
Kuryr- Supported Features
● Utilizes Neutron and Keystone● Supports Docker Networking
○ Network Plugin○ IPAM Plugin
● Supports neutron API driven security groups● Supports use of existing Neutron resources
○ Networks, Subnets, Load Balancers
● Supports Docker Swarm
![Page 8: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/8.jpg)
Kuryr opens Neutron services to container networking
● Security Groups● Subnet Pools● NAT (SNAT / DNAT – Floating IP)● Port Security (ARP Spoofing)● QoS● Quota Management● Neutron pluggable IPAM● Translate COE Load balancing services into Neutron LbaaS setups ● FWaaS for Containers
![Page 9: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/9.jpg)
How does all of this work?
![Page 10: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/10.jpg)
Kuryr Generic VIF Binding● Binds the container networking namespace to the networking infra● Generic OS VIF Binding Library (Nova)
Common part (container side)● IPAM● vEth creation
Vendor-specific part○ Based on Neutron port type○ Free implementation language○ Root context
![Page 11: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/11.jpg)
Kuryr vs. Docker Libnetwork Remote Driver
● Maps Docker's CNM operations into a Neutron API usage● Any Neutron plugin can use it (for example OVS)● Keep Up to date with the changing libnetwork remote driver API
![Page 12: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/12.jpg)
Docker Native API Usage
$ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet=10.10.0.0/24 \ -- gateway=10.10.0.1 kuryr-netec37650c5480f19a0d9f00c948ed5d916f8b2dd1f43701a66b11724ed523ad42
$ docker run -it --net=kuryr-net centos
![Page 13: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/13.jpg)
Neutron Network View
Kuryr uses network tags to associate Neutron and Docker Networks
$ neutron net-list+--------------------------------------------------------+---------------------------+--------------------------------------------------------------------------+| id | name | subnets |+--------------------------------------------------------+---------------------------+---------------------------------------------------------------------------+| 8e57483c-867c-4fb8-aa4c-98d477a1c087 | kuryr-net-ec37650c | 924ab81e-ad5c-448c-88f7-7edbd8f39346 10.10.0.0/24 |
$ neutron net-show kuryr-net-ec37650c+------------------+-------------------------------------------------------------------------+| Field | Value |+------------------+-------------------------------------------------------------------------+| id | 8e57483c-867c-4fb8-aa4c-98d477a1c087 | name | kuryr-net-ec37650c | subnets | 924ab81e-ad5c-448c-88f7-7edbd8f39346 | tags | kuryr.net.uuid.lh:ec37650c5480f19a0d9f00c948ed5d91 | | kuryr.net.uuid.uh:6f8b2dd1f43701a66b11724ed523ad42
![Page 14: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/14.jpg)
Spawn Container On Existing Neutron Network
$ neutron net-create kuryr-net$ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet=10.10.0.0/24 \ --gateway=10.10.0.1 \ -o neutron.net.name=kuryr-net kuryr_net_d
$ID=$(neutron net-list | grep kuryr-net | awk ‘{print $2}’)$ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet=10.10.0.0/24 \ --gateway=10.10.0.1 \ -o neutron.net.uuid=$ID kuryr_net_d
![Page 15: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/15.jpg)
Container On Existing Neutron Network
● Neutron tags added to network● Existing subnet is used, created otherwise● Neutron network is not deleted when docker network deleted.● Pre Mitaka versions have limitations
$ neutron network-show kuryr-net+------------------+-------------------------------------------------------------------------+| Field | Value |+------------------+-------------------------------------------------------------------------+| id | 8e57483c-867c-4fb8-aa4c-98d477a1c087 | name | kuryr-net-ec37650c | subnets | 924ab81e-ad5c-448c-88f7-7edbd8f39346 | tags | kuryr.net.uuid.lh:a8440885777818570682b257a84152be| | kuryr.net.uuid.uh:05c39028beaa250756a5df7ea8640d4f| | kuryr.net.existing
![Page 16: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/16.jpg)
Docker Swarm with Kuryr
![Page 17: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/17.jpg)
Kubernetes and Kuryr
● Neutron Network per k8s namespace
● Pods on same worker node can be on different networks
● CNI Driver uses Kuryr port binding lib
![Page 18: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/18.jpg)
Kubernetes LB Service● Kubernetes LB service is implemented by Neutron LBaaS
(pool and members)● Cluster service IP/VIP is defined by k8s
![Page 19: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/19.jpg)
Kubernetes Services with MidoNet• Service creation on API server triggers Raven• Raven creates the VIP and Pool in neutron.• Pod creation on API server triggers
• Raven• Kubelet
• Raven creates the port in Neutron• Kubelet asks CNI to add the pod• CNI driver asks API server for neutron port data
in pod definition• CNI driver goes to the MidoNet cluster to bind
the port to the Pod interface• MidoNet agent sees the binding and updates the
OVS datapath• Service Endpoint creation on the API server
triggers Raven• Raven adds endpoint IP as the service pool
member
![Page 20: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/20.jpg)
VM Nested Containers (Overlay in Overlay) - Native
![Page 21: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/21.jpg)
Nested Containers Networking - Kuryr
● VLAN Trunk API○ Neutron API to
define nested containers ports
![Page 22: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/22.jpg)
VMs and Containers Networking
● Leverage the same Neutron solution for tenant containers networking
○ Neutron features○ Easier management○ Same “implementation” ○ Support containers
networks and VM network isolation
○ Neutron plugins already support this: OVN, Midonet, Dragonflow
● Magnum● Backend Implementations
interoperability
![Page 23: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/23.jpg)
Packaging & Deployment● Automated container builds at https://hub.docker.com/r/kuryr/
● Libnetwork● Raven● Kubelet
● Kolla Integration● Under review
● Distribution packaging with systemd unit files● Debian● RDO
● Heat Templates● Magnum Integration
![Page 24: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/24.jpg)
Kuryr Roadmap - Newton Release● Kubernetes integration
○ CNI and watcher parts upstreaming○ Policy support using security groups
● Nested containers and Magnum integration● Neutron advanced services (FWaaS, VPNaaS)● DNS integration and Port Forwarding● Docker exposed ports● Packaging and Deployment● Mesos● Storage
○ Persistent Storage ○ Backup and application data protection for storage
![Page 25: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/25.jpg)
Join the project!
● Project Launchpad● https://launchpad.net/kuryr
● Project Git Repository● https://github.com/openstack/kuryr
● Weekly IRC Meeting● http://eavesdrop.openstack.org/#Kuryr_Project_Meeting
● IRC● #openstack-kuryr @ Freenode
● Mailing List● [email protected] ([Kuryr])
![Page 26: Kuryr - ZHAW Blogs · • MidoNet agent sees the binding and updates the OVS datapath • Service Endpoint creation on the API server triggers Raven • Raven adds endpoint IP as](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98c21677e3c7a13593111/html5/thumbnails/26.jpg)