Setting up KVM on Ubuntu 10.04 (Lucid Lynx)

After doing a KVM install on Debian Squeeze and trying to get a VM up and running, the hassle convinced me to go back to Ubuntu and their vm-builder package, which allow ones to create VMs relatively easy once the setup is complete. There is a vm-builder port for Debian, though that only works for building older versions of Ubuntu and I want to run the latest, Ubuntu 11.10 (Oneiric Ocelot).

Starting with a bare-metal Ubuntu 10.04 LTS (Lucid Lynx) 64-bit, below is the list of commands and instructions to install and set up the KVM. Details on these instructions can be read in the Ubuntu community documentation, KVM Installation and KVM Networking:

• sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils


• virsh -c qemu:///system list (To verify installation, should have no errors)


• sudo apt-get install libcap2-bin


• sudo setcap cap_net_admin=ei /usr/bin/qemu-system-x86_64


• sudo vi /etc/network/interfaces
  
  • Original file:
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
address 10.10.3.140
broadcast 10.10.3.143
netmask 255.255.255.248
gateway 10.10.3.137

# default route to access subnet
up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 eth0
  
  
  • Modified file:
# The loopback network interface
auto lo
iface lo inet loopback

# device: eth0
auto eth0
iface eth0 inet manual

# The primary network interface
auto br0
iface br0 inet static
address 10.10.3.140
broadcast 10.10.3.143
netmask 255.255.255.248
gateway 10.10.3.137
bridge_ports eth0
bridge_stp off
bridge_fd 9
bridge_hello 2
bridge_maxage 12

# default route to access subnet
up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 eth0
up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 br0


• sudo /etc/init.d/networking restart


• Running ifconfig lists the following interfaces br0, eth0, lo, virbr0

This completes the KVM installation and creation of a bridge for the VMs. Up next is replacement of the vm-builder. The one in the Ubuntu packages is faulty and also will not allow you to install Ubuntu 11.10 (Oneiric Ocelot). So I updated to the latest, downloading the source, building and installing it. The steps below can be found in this accepted answer:

• sudo apt-get install bzr


• sudo apt-get install epydoc (big install here, ~400mb)


• bzr branch lp:ubuntu/vm-builder ubzr-vm-builder


• cd ubzr-vm-builder


• fakeroot debian/rules binary


• sudo dpkg -i ../*vm-builder*.deb

With that, everything is installed and vm-builder is ready to run. The easiest way is to use a script so that vm creation can be set once and repeated as desired. The only changes required being hostname, ip and maybe memory. Obtain the Ubuntu 11.10 64-bit server iso and put it in the same place as the script. The directory I used is ~/vm/basekvm:

• cd ~/vm/basekvm


• sudo vi create_vm.sh
  
  • File:
#!/bin/bash

# Configure this before running the command
HOSTNAME=myhostname
MEMORY=2048
IP=192.168.122.10
# -- End of configuration

vmbuilder kvm ubuntu \
--destdir=/var/lib/libvirt/images/$HOSTNAME \
--ip=$IP \
--hostname=$HOSTNAME \
--mem=$MEMORY \
--suite=oneiric \
--flavour=virtual \
--arch=amd64 \
--iso=/root/vm/basekvm/ubuntu-11.10-server-amd64.iso \
--mirror=http://de.archive.ubuntu.com/ubuntu \
--libvirt=qemu:///system \
--domain=localdomain \
--part=/root/vm/basekvm/vmbuilder.partition \
--bridge=virbr0 \
--gw=192.168.122.1 \
--mask=255.255.255.0 \
--user=myusername \
--name=myname \
--pass=mypassword \
--tmpfs=- \
--addpkg=vim-nox \
--addpkg=acpid \
--addpkg=unattended-upgrades \
--addpkg=openssh-server \
--firstboot=/root/vm/basekvm/fboot.sh \
-o


• sudo chmod 700 create_vm.sh


• sudo vi fboot.sh (Optional)
  
  • File:
# This script will run the first time the virtual machine boots
# It is ran as root.

# Expire the user account
passwd -e administrator

# Install openssh-server
apt-get update
apt-get install -qqy --force-yes openssh-server


• sudo chmod 777 fboot.sh


• sudo vi vmbuilder.partition
  
  • File:
root 8000
swap 4000
---
/var 8000


• cd ~/vm


• ln -s /var/lib/libvirt/images/ images

The create_vm.sh is basically a template script. You can modify it to accept console input so that you don't need to go and edit the file values, that is left for another time. The symbolic link shows the directory where the VM disk images are located once created. Below is how you would use it to create a VM:

• sudo cp basekvm/create_vm.sh create_vm_myvmname.sh


• sudo vi create_vm_myvmname.sh. Edit the HOSTNAME, IP and MEMORY as desired


• sudo ./create_vm_myvmname.sh


• virsh start myvmname

And that's it! A VM has been successfully created and started up. Give it a few minutes and then you can log in through ssh using the information in the script. If the ssh is slow to connect, try this.

Notes about Delayed Job and Upstart

Upstart?

If you've been working with Linux systems for a while might have heard about upstart. As the upstart project’s home page describes, it's an event-based replacement for the System V init daemon.

Basically upstart takes care of starting and stopping services and ad hoc tasks when events happen such as when your system boots up or shuts down. And more!

Why Upstart?

As the saying goes: “don’t fix things it they aren’t broken,” so why would we bother with upstart in the first place when SysV init has been working fine for decades? Well, for one thing Ubuntu, Fedora, and others are moving to it! And upstart is event-based so you can easily write scripts to handle specific events. For example, it's trivial to run a script after the network services started. With init we have to rely on the static priorities of the scripts, which can get pretty messy and inflexible over time. Upstart is much more than what's described here and has a many more features as described in the project's feature highlights.

Delayed Job, RVM, and Upstart

Recently I needed to write a basic script to make sure that delayed_job gets started when the server is rebooted. Since I was using rvm to manage all the ruby processes on the server, I needed to make sure that rvm is properly loaded before actually running the delayed_job process.

There are enough articles out there about getting your scripts to run with RVM. However I keep running into problems with this very thing all the time!

Finally this is the script I came up with. It's a fairly simple script to get the delayed_job process started.





Notes

• I decided to keep the /etc/init/file.conf simple and put most of the code in my external script file


• the ‘task’ parameter can be used when you just want to do something simple like launch the daemon (and not creating a deamon for upstart to watch and restart...etc)


• ‘start on runlevel [2345]’ tells upstart to start your task on runlevels 2,3,4, and 5


• you can write your script inside the script…end script block. Here I'm just calling my external script


• if you're running a Ruby script and have your Ruby set up via rvm be sure to include the two lines:
  
  
  
  • rvm_path=/home/deployer/.rvm
  
  
  • source "/home/deployer/.rvm/scripts/rvm"
  
  
  • this needs to be changed according to your rvm installation path
  
  
• I'm also keeping a custom log that will have an entry for each time the task is run

KVM host with gateway guest using port-forwarding

Using the 3 rules listed here and below, a KVM host can forward all http and ssh traffic to a specified gateway guest VM:

iptables -t nat -I PREROUTING -p tcp --dport 80 -j DNAT --to-destination 10.0.0.1:80
iptables -t nat -I PREROUTING -p tcp --dport 22 -j DNAT --to-destination 10.0.0.2:22
iptables -I FORWARD -m state -d 10.0.0.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT

To make it permanent, requires one to go through this page and use the following commands:

sudo sh -c "iptables-save -c > /etc/iptables.rules" (after applying the 3 commands above)
sudo vi /etc/network/if-pre-up.d/iptablesload

The /etc/network/if-pre-up.d/iptablesload file will have the following text:

#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0

The KVM host will now have VM creation as its sole focus. Ensure the host's ssh port have been changed to make it accessible from outside, otherwise it can be accessed from the gateway guest.

All redirection and VM access are transferred to the the gateway guest. The guest will need to install nginx so it can act as a http proxy for other VMs. All ssh access use the gateway guest as a stepping stone to the other VMs.

Following the 3 rules, it is found the traffic essentially loops back to the gateway guest. This makes it incapable of reaching the other VMs. Applying 1 more rule after the 3 above solves this. The rule accepts all packets from the VM ip range and does not do any forwarding:

iptables -t nat -I PREROUTING -p tcp --source 10.0.0.0/24 -j ACCEPT