vbrownbag container networking for real workloads
TRANSCRIPT
Matt Johnson #vBrownBag at Cisco LIVE! 2015 9th June 2015
Container networking for real workloads. Why Default isn’t Best
[email protected] @mattdashj
Presentation recording available at: https://www.youtube.com/watch?v=z-GD-ctjCRc
2 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hard to say ‘container’ nowadays without…
So we’ll start our story here.
3 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Quick Overview: Docker Networking
host docker0 172.17.42.1/16
veth049e89e
eth0
172.17.0.21/16
veth049e88f veth049e87af
eth0
192.168.1.10
eth0
192.168.1.10
eth0
192.168.1.10
host eth0
docker run --net=bridge (default) docker run --net=host
Container shares the host’s IP address and MAC address. vEth pair linked into container from docker0 bridge. DNAT provides inbound ports to containerized apps.
• Default IP address is dynamically allocated from unused range (ie 172.17.0.0/16.)
can be set with –fixed-cidr on docker daemon • Default GW for containers is the IP addr of the host docker0 interface
can be set with –bip on docker daemon
eth0
172.17.0.21/16
eth0
172.17.0.21/16 containers
3
iptables (NAT) iptables (NAT)
4 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Why NAT? - Works out of the box on a host with a single IP
Dockers’ rise to prominence due to User Experience. Not new tech.
- LXC, Existed - Containers, Nothing new (Solaris Zones Anyone?)
The difference is it had never been as easy to use/explain/build/share.
Not without it’s limits: - Container IP tied to Host.
- Only one container can have a ‘common’ port per host. - Careful manual port assignment.
- Dynamic assignment with support in the app to support it. - App’s need to be told to read ‘real’ network info from $ENV’s
6 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Extending Networking: Kubernetes Pod - a co-located group of Docker containers with shared volumes. They're the smallest deployable units that can be created, scheduled, and managed with Kubernetes.
Service - provide a single, stable name and address for a set of pods. They act as basic load balancers.
Label - are used to organize and select groups of objects based on key/value pairs.
Replication Controller - are used to organize and select groups of objects based on key/value pairs.
master server apiserver
etcd
controller manager
scheduler
skydns
pod C11
C12
C13
minion
pod C21
C22
C23
pod C41
C42
C43
minion
pod
C14
C15
pod
C44
minion
pod
C24
pod
C44
C45
minion
pod
C24
pod C11
C12
C13
minion
pod C21
C22
C23
service(IP addr/DN) replication Controller service(IP addr/DN)
cluster
6
7 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Extending Networking: Kubernetes
7
Tunnels are great… Right?
9 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Tunnels
We don’t have to deal with whatever the underlying
network is.
10 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Tunnels Although we do have to deal
with…
Ramps! On/Off the overlay. Potential bottleneck / scalability issues
An Interesting L2 Design.
Subject to implementation, may lack multicast support, treat broadcast strangely etc.
Debug overhead.
I only have 50 VM’s, but which one of my 1,225 tunnels (full mesh) is causing the issue.
11 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Tunnels Although we do have to deal
with…
MTU and Frag. avoidance. It is a tunnel after all!
Security.
Subject to implementation, spanning ARP really isn’t ideal. Lots of containers running
privileged.
12 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Future Docker Networking: libnetwork The Container Network Model (CMN)
12
Docker Container Docker Container Docker Container
Network Sandbox Network Sandbox Network Sandbox
Endpoint Endpoint Endpoint Endpoint
Backend Network Frontend Network
• Network Sandbox -- isolated environment where container network configuration lives • Endpoint – network interface tied to a specific network
• Network – a uniquely identifiable collection of Endpoints that are able to communicate with each other
A pluggable interface. Expected to first ship in Docker 1.7. Distributed bridge plugin under development.
13 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Future Docker Networking: Overlay/L2 Centric The Container Network Model (CMN)
13
14 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
So wouldn’t it be great if…
We could network containers without NAT,
Without requiring a ‘tunnels everywhere’ approach, Without on or off ramps, With full IPv4 and IPv6 support,
And at Internet-scale?
15 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
We Can! .. If we don’t treat containers as Unique/Special
16 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Scalable DataCenter topologies have existed for quite a while…
And no, this isn’t about to turn into a Cisco sales pitch… promise!
18 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Standards Use Networking standards
tried and tested by the rest of the industry. Already used
every day at unprecedented Scale.
Don’t reinvent the wheel.
Integrate and benefit from
existing network team skillsets.
19 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Proposed solution with Docker / Container Networking
20 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Containers as a first class citizen of the real network • Devolve routing down into each host. • BGP as the saleable, proven control plane. • Standard, proven linux networking stack. • Containers have IP’s..
• No NAT. • DNS works properly. • No tunnels necessary. • Support for legacy applications & clustering. • Support for multicast. • IPv4 & IPv6 Support.
• Standard network protocols that the rest of the industry uses • Service providers and Enterprises can easily integrate.
21 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Buy Today…
And we’ll throw in container mobility for free!
IP’s no longer bound to the IP of the host, If a workload moves, the IP can go with it.
Containerized data services will benefit
massively from this.
22 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
So where is this work going? Validated ideas through other container Industry supporters. Not the only people thinking this way. Implementation is happening now! Which brings me onto a side note…
Matt Johnson #vBrownBag at Cisco LIVE! 2015 9th June 2015
This isn’t a real title.
The ‘Nothing really to do with networking’ part of the talk
@mattdashj
Snowflakes!
25 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
GLUEFlakes!
Snowflakes!
27 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Avoid GlueFlakes!
There’s something to this whole ‘user experience’
thing!
MicroServices Infrastructure release 0.3 TODAY!
we’re changing the name to something less sucky soon too!
28 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Cisco Microservices Environment based on Mesos. • Fully open source.
• Framework support in Mesos leveraged to deploy scalable data, monitoring and assurance services all-in-one.
• We’ve designed, written and tested the glue.
• ‘Dogfooding’ internally. We’ve built it to solve real problems.
• Contribute vs Creating a new GlueFlake.
• ‘Out of the Box’ environment for real-life workloads.
Enable the automatic deployment of development dependencies!
29 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
So back to that network thing…
You guessed it… It’s going into MicroServices Infrastructure. Preconfigured to work with data and marathon workloads out of the box.
/ciscocloud/microservices-infrastructure
Roadmap updates and official announcements soon…
Thanks for Listening!
Questions?