all virtual all the time

MotivationImplementation

Status and future workSummary

All Virtual All the Time

Implementing x86 Virtualization at the BIOS level

Cristi M gheru³an

1Technical University of Cluj-Napoca, Romania

2008-2009

Cristi M gheru³an All Virtual All the Time



Outline

1 Motivation

Virtualization - a hot topic these days

Virtualization inside the BIOS - Why not?

2 Implementation

AVATT components

Size constraints

Development Toolkit

3 Status and future work

Current Status

Future work

Di�culties




Virtualization - a hot topicVirtualization inside the BIOS - Why not?

About Virtualization

Modern hardware is getting faster and faster each year

Multiple OSes can now symultaneously run on the same

machine

Most people use it for server consolidation or testing

There are quite a few virtualization types

Full (QEMU)- emulates a full computer, maybe even anotherarchitecture (usually slow)Hardware-assisted full virtualization (KVM)- The CPUprovides some special features to make it faster (decent speed)Paravirtualization (XEN)- the guest OS is aware it's beingvirtualized and provides some optimizations (decent speed)OS virtualization (OpenVZ) - containers, di�erent instances ofthe same kernel (fastest)





Getting closer to the bare hardware I

There is a tendency to move the virtualization software closer

to the bare machine

At �rst, they were simple user applications (qemu)

Then, kernel components were added to enhance speed(KQemu, VMWare, VirtualBox)

Hypervisors replaced the OS that ran on the bare machine(Xen, VMWare ESX)

Some mainstream OSes got hypervisor capabilities (LinuxKVM)

Most of the Unices are capable of running chroot-ed containers(chroot, jail, OpenVZ, zones)





Getting closer to the bare hardware II

If so, why not move it even closer to the machine, and put it

inside the BIOS? Yeah, it's possible!

The BIOS would start

Make all the needed initializations

Then automatically run the VMs stored on the disk instead ofa single OS

Ideal for server consolidation workloads




AVATT componentsSize constraintsDevelopment Toolkit

The coreboot BIOS makes it possible I

Ordinary BIOSes are fully written in assembly language, prone

to bugs, poorly-designed and quite slow

�No keyboard detected, press F1 to continue� on 1000 cluster

nodes is not funny ;-)

LinuxBIOS was started back in 1999 by Ron Minnich (then

working on the LANL GRID cluster)

It aims to provide a fully open source BIOS replacement, and

started by using Linux as a BIOS

Had a tiny C code used to initialize the hardware, then ran

kexec() to start the kernel stored on the disk

It's very fast (3 seconds to the Linux payload's prompt)

Written mostly in C, not such bug-prone as ASM





The coreboot BIOS makes it possible II

Ever since it was re-designed, no longer needs to incorporate

Linux and was renamed coreboot soon after

Nowadays it initializes the hardware, and runs an executable

called payload

There are lots of them (even Tetris is available!)

Linux can still be run as a payload

Currently it supports a few dozens of motherboards, but it's

very hard to keep up with the hardware makers

Hardware-compatible with most modern CPUs and chipsets,

but mainboard makers often induce their own modi�cations

Currently supported by some major hardware makers (AMD,

VIA, Sun, Acer, SiS), which often help a lot





So what's inside AVATT?

The coreboot BIOS, ofcourse

A payload that contains the following pieces of software:

A Linux kernel compiled with KVM support, providinghardware-assisted virtualizationA small Linux userland containing the followingembedded-optimized components:

uClibc - tiny C library made speci�cally for embeddedapplicationsthe busybox binary - provides a basic Linux userland (shell,coreutils, and so on)the KVM tools that can be used to create and start virtualmachinesaditional libraries needed by the KVM tools (currently ncurses)





Size constraints

All these, including the kernel, had to �t in a 2MB �ash image.

I achieved this goal, with 1.8MB used, if compressed with

LZMA(7zip)

Heavily stripped Linux kernel, built with KVM support - 900KBuClibc - 150KBbusybox - 150KBKVM tools - 500KBthe aditional libraries - remaining 100KB





Development Toolkit I

For building the AVATT I had to modify the buildrom utility

quite heavily

buildrom is a tool provided by the coreboot project

It can be used to create ROM images ready to be burned onthe mainboard's �ash

I modi�ed buildrom by adding a few scripts that compiled thetools I needed

I created patches that �xed the incompatibilities between thesetools

They were stripped down to minimum size and spartan features

The GCC compiler suite was used, version 4.3





Development Toolkit II

For testing I used QEMU, an emulator that can provide the

hardware-assistance CPU instructions

It can use a BIOS image created by buildrom

The developement machine was my laptop running Linux

It was tested and should compile under most modern Linux

distributions like Gentoo, Ubuntu or Debian Lenny




Current StatusFuture workDi�culties

Current Status

All the tools can be built into the same ROM image

The system starts and provides a Linux shell

The disks can be manually mounted

The user can create a VM disk image using a special tool

A Virtual machine can be started from a CDROM image but it

crashes soon during the boot process

The crash is caused by missing Thread Local Storage(TLS)support from uClibc's threading implementationsNone of the three!!! threading implementations provided byuClibc support TLS on x86 machinesThe uClibc team is currently working on this issue, and I mayhelp them in my spare time





Future Work

Keep the components up to date as they evolve at upstream

After the blocker TLS issue is implemented in uClibc, make it

work at its full potential

Automate as much as possible

User-friendly tool for creating VMs

Until the uClibc thing is getting �xed, we could switch to

OpenVZ instead of KVM, providing container-based OS-level

virtualization

OpenVZ should be a lot easier to get compiled than the KVMtoolsI started working on it, but the progress stalled lately due tolack of spare time





Di�culties

The buildrom build system is strange, not a true toolchain

I had to use lots of GCC CFLAGS tricks to get the stu� built

A few patches had to be created and applied by buildrom to

most of the programs in order to �x compilation errors

Especially the KVM tools were quite hard to make work on top

of uClibc, but the other ones were also problematic




Summary

The virtualization is getting closer to the hardware

Putting it in the BIOS would be great for server consolidation

Outlook

We need the TLS support from uClibcOpenVZ worth being considered as a viable alternative to KVM


AppendixAcknowledgementsFor Further Reading

Acknowledgements

Thanks Google for o�ering me the opportunity to work on this

as a Summer of Code project

Thanks IBM Romania for appreciation and support

Many thanks to the coreboot team and especially Ron Minnich

for creating coreboot and mentoring me during this great

summer


AppendixAcknowledgementsFor Further Reading

For Further Reading

Coreboot website http://coreboot.org

AVATT wiki page http://www.coreboot.org/AVATT

Google Summer of Code http://code.google.com/soc


http://coreboot.org

http://www.coreboot.org/AVATT

http://code.google.com/soc

all virtual all the time

Technology