Virtual Infrastructure:

kvm, uvb + libvirt

Bryan McLellanhttp://loftninjas.org

btm@loftninjas.org

Senior Systems Administrator

Widemile

Widemile is awesome

I work there, and thus the world is a better place for all

of humanity.

What I got for “free”

•Identical Blades

•Running non identical Debian installs (etch/sid, i386/amd64)

•Running different versions of vmware-server•Running different versions of vmware-server

• OS, software, installed by hand

• Configure using notes in an outlook public folder

•Host database tracked with DNS

vm04-something.test.widemile.com

Problems

• Lack of Homogeneity

• Monkey configuration sucks

• Documentation offensive

http://www.flickr.com/photos/annebuzz/2666654754/

What I did

• Lack of Homogeneity

– debian pxe install with preseeds

• Monkey configuration sucks

– puppify (configuration management)– puppify (configuration management)

– capistrano (ruby scripting over ssh)

– iclassify (node classification)

• Documentation offensive

– Configuration management is self documenting

– Wikify the notes

We can always do better

http://www.flickr.com/photos/jpasden/67513019/

What upset me (nobody else cares)

• PXE installs take soooooo long

• Backporting too many packages to debian

• Interacting with VMware programmatically

too much work.too much work.

• VMware UIs cost $$

• Dreams: cpu/ram hotplug, live migration

How I Learned to Stop Worrying and

Love the Bomb• PXE installs take soooooo long

– deb-bootstrap with ubuntu-vm-builder (2.5min)

• Backporting too many packages to debian– switch to Ubuntu!

• Interacting with VMware programmatically too much • Interacting with VMware programmatically too much work.– libvirt is much easier and cleaner

• VMware UIs cost $$– Libvirt is free

• Dreams: cpu/ram hotplug, live migration– Word on the street is that kvm + libvirt support these

– I don’t think they’re there yet though.

libvirt – virtualization API

Started at Redhat ‘s Emerging Technology Group

Was Xen focused, but Redhat recently bought Qumranet, makers of KVM

Ubuntu/Canonical picked KVM as virtualization platform of choice

libvirt doesn’t really support all that

Provides CLI management (virsh)

Provides GUI management (virt-manager)

virsh

virt-manager

ubuntu-vm-builder

Bash script wrapper for deb-bootstrap + qemu

apt-get install ubuntu-vm-builder

capistrano script:

vmware: 102 lines

uvb: 12 lines

ubuntu-vm-builder kvm hardy ubuntu-vm-builder kvm hardy

--addpkg openssh-server

-d /srv/kvm/#{hostname}

--domain #{domain}

--hostname #{hostname}

--mem #{memory}

--mirror http://ubuntu.widemile.com/ubuntu

--libvirt qemu:///system

--bridge #{eth0}

ubuntu-vm-builder

Bash script wrapper for deb-bootstrap + qemu

apt-get install ubuntu-vm-builder

capistrano script:

vmware: 102 lines

uvb: 12 lines

ubuntu-vm-builder kvm hardy

Not a lot of enterprise features yet

Bridging vs NATubuntu-vm-builder kvm hardy

--addpkg openssh-server

-d /srv/kvm/#{hostname}

--domain #{domain}

--hostname #{hostname}

--mem #{memory}

--mirror http://ubuntu.widemile.com/ubuntu

--libvirt qemu:///system

--bridge #{eth0}

Bridging vs NAT

Root password / creates user

kvm

• The cool hypervisor on the block (ubuntu, redhat)

• Kernel modifications are mainlined (unlike xen)

• Super fast / lightweight (feel the wind in your hair)

• Growing fast

class kvm {

case $lsbdistid {

"Ubuntu": {"Ubuntu": {

# Only Ubuntu releases are new enough for this

package {

"kvm":

ensure => present;

"libvirt-bin":

ensure => present;

"ubuntu-vm-builder":

ensure => latest;

}

file { "/srv/kvm":

ensure => directory,

group => admins,

mode => 0775,

}

# Permissions required for remote access

# Puppet doesn't support managing the libvirtd group members through the

group typegroup type

file {

"/var/run/libvirt/libvirt-sock":

group => admins,

require => Service["libvirt-bin"];

"/var/run/libvirt/libvirt-sock-ro":

group => admins,

require => Service["libvirt-bin"];

}

service { "libvirt-bin":

ensure => running,

hasstatus => true,

require => Package["libvirt-bin"],

}

# required by u-v-b

exec { "devmapper-autoload":

command => "/bin/echo dm_mod >> /etc/modules",

onlyif => "/usr/bin/test `grep -c '^dm_mod' /etc/modules` -lt 1",onlyif => "/usr/bin/test `grep -c '^dm_mod' /etc/modules` -lt 1",

}

exec { "devmapper-load":

command => "/sbin/modprobe dm_mod",

onlyif => "/usr/bin/test `/bin/lsmod | grep -c '^dm_mod'` -lt 1",

}

realize Group["libvirtd"]

}

}

}

http://loftninjas.org