Setting up KVM on Ubuntu 10.04 (Lucid Lynx)

After doing a KVM install on Debian Squeeze and trying to get a VM up and running, the hassle convinced me to go back to Ubuntu and their vm-builder package, which allow ones to create VMs relatively easy once the setup is complete. There is a vm-builder port for Debian, though that only works for building older versions of Ubuntu and I want to run the latest, Ubuntu 11.10 (Oneiric Ocelot).

Starting with a bare-metal Ubuntu 10.04 LTS (Lucid Lynx) 64-bit, below is the list of commands and instructions to install and set up the KVM. Details on these instructions can be read in the Ubuntu community documentation, KVM Installation and KVM Networking:

• sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils


• virsh -c qemu:///system list (To verify installation, should have no errors)


• sudo apt-get install libcap2-bin


• sudo setcap cap_net_admin=ei /usr/bin/qemu-system-x86_64


• sudo vi /etc/network/interfaces
  
  • Original file:
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
address 10.10.3.140
broadcast 10.10.3.143
netmask 255.255.255.248
gateway 10.10.3.137

# default route to access subnet
up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 eth0
  
  
  • Modified file:
# The loopback network interface
auto lo
iface lo inet loopback

# device: eth0
auto eth0
iface eth0 inet manual

# The primary network interface
auto br0
iface br0 inet static
address 10.10.3.140
broadcast 10.10.3.143
netmask 255.255.255.248
gateway 10.10.3.137
bridge_ports eth0
bridge_stp off
bridge_fd 9
bridge_hello 2
bridge_maxage 12

# default route to access subnet
up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 eth0
up route add -net 10.10.3.136 netmask 255.255.255.248 gw 10.10.3.137 br0


• sudo /etc/init.d/networking restart


• Running ifconfig lists the following interfaces br0, eth0, lo, virbr0

This completes the KVM installation and creation of a bridge for the VMs. Up next is replacement of the vm-builder. The one in the Ubuntu packages is faulty and also will not allow you to install Ubuntu 11.10 (Oneiric Ocelot). So I updated to the latest, downloading the source, building and installing it. The steps below can be found in this accepted answer:

• sudo apt-get install bzr


• sudo apt-get install epydoc (big install here, ~400mb)


• bzr branch lp:ubuntu/vm-builder ubzr-vm-builder


• cd ubzr-vm-builder


• fakeroot debian/rules binary


• sudo dpkg -i ../*vm-builder*.deb

With that, everything is installed and vm-builder is ready to run. The easiest way is to use a script so that vm creation can be set once and repeated as desired. The only changes required being hostname, ip and maybe memory. Obtain the Ubuntu 11.10 64-bit server iso and put it in the same place as the script. The directory I used is ~/vm/basekvm:

• cd ~/vm/basekvm


• sudo vi create_vm.sh
  
  • File:
#!/bin/bash

# Configure this before running the command
HOSTNAME=myhostname
MEMORY=2048
IP=192.168.122.10
# -- End of configuration

vmbuilder kvm ubuntu \
--destdir=/var/lib/libvirt/images/$HOSTNAME \
--ip=$IP \
--hostname=$HOSTNAME \
--mem=$MEMORY \
--suite=oneiric \
--flavour=virtual \
--arch=amd64 \
--iso=/root/vm/basekvm/ubuntu-11.10-server-amd64.iso \
--mirror=http://de.archive.ubuntu.com/ubuntu \
--libvirt=qemu:///system \
--domain=localdomain \
--part=/root/vm/basekvm/vmbuilder.partition \
--bridge=virbr0 \
--gw=192.168.122.1 \
--mask=255.255.255.0 \
--user=myusername \
--name=myname \
--pass=mypassword \
--tmpfs=- \
--addpkg=vim-nox \
--addpkg=acpid \
--addpkg=unattended-upgrades \
--addpkg=openssh-server \
--firstboot=/root/vm/basekvm/fboot.sh \
-o


• sudo chmod 700 create_vm.sh


• sudo vi fboot.sh (Optional)
  
  • File:
# This script will run the first time the virtual machine boots
# It is ran as root.

# Expire the user account
passwd -e administrator

# Install openssh-server
apt-get update
apt-get install -qqy --force-yes openssh-server


• sudo chmod 777 fboot.sh


• sudo vi vmbuilder.partition
  
  • File:
root 8000
swap 4000
---
/var 8000


• cd ~/vm


• ln -s /var/lib/libvirt/images/ images

The create_vm.sh is basically a template script. You can modify it to accept console input so that you don't need to go and edit the file values, that is left for another time. The symbolic link shows the directory where the VM disk images are located once created. Below is how you would use it to create a VM:

• sudo cp basekvm/create_vm.sh create_vm_myvmname.sh


• sudo vi create_vm_myvmname.sh. Edit the HOSTNAME, IP and MEMORY as desired


• sudo ./create_vm_myvmname.sh


• virsh start myvmname

And that's it! A VM has been successfully created and started up. Give it a few minutes and then you can log in through ssh using the information in the script. If the ssh is slow to connect, try this.

KVM host with gateway guest using port-forwarding

Using the 3 rules listed here and below, a KVM host can forward all http and ssh traffic to a specified gateway guest VM:

iptables -t nat -I PREROUTING -p tcp --dport 80 -j DNAT --to-destination 10.0.0.1:80
iptables -t nat -I PREROUTING -p tcp --dport 22 -j DNAT --to-destination 10.0.0.2:22
iptables -I FORWARD -m state -d 10.0.0.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT

To make it permanent, requires one to go through this page and use the following commands:

sudo sh -c "iptables-save -c > /etc/iptables.rules" (after applying the 3 commands above)
sudo vi /etc/network/if-pre-up.d/iptablesload

The /etc/network/if-pre-up.d/iptablesload file will have the following text:

#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0

The KVM host will now have VM creation as its sole focus. Ensure the host's ssh port have been changed to make it accessible from outside, otherwise it can be accessed from the gateway guest.

All redirection and VM access are transferred to the the gateway guest. The guest will need to install nginx so it can act as a http proxy for other VMs. All ssh access use the gateway guest as a stepping stone to the other VMs.

Following the 3 rules, it is found the traffic essentially loops back to the gateway guest. This makes it incapable of reaching the other VMs. Applying 1 more rule after the 3 above solves this. The rule accepts all packets from the VM ip range and does not do any forwarding:

iptables -t nat -I PREROUTING -p tcp --source 10.0.0.0/24 -j ACCEPT


Setting up mail for your Ubuntu server

Need your bare-bones Ubuntu server to send out email notification and the like? Well, first you need to check what your hostname is and modify accordingly:

hostname -f

So my server's hostname is microdude, you might also get it as microdude.localdomain. You'll want to convert it into an FQDN (Fully Qualified Domain Name). That means I need to change my microdude to microdude.favoritemedium.com.

sudo vi /etc/hostname
sudo vi /etc/hosts

The output of hosts:
127.0.0.1 localhost
127.0.1.1 microdude.localdomain microdude

Would now be:
127.0.0.1 localhost
127.0.1.1 microdude.favoritemedium.com microdude

Reboot and check that the hostname has been changed to microdude.favoritemedium.com:

sudo reboot
hostname -f

After that, you're supposed to set up RDNS (Reverse Domain Name System), this helps avoid your mails falling prey to the all-powerful spam filter. You can check on this RDNS with dig which can be installed from this package:

sudo apt-get install dnsutils

This is optional, though if your mails start going into the spam folder, you'd best revisit this in the links provided.

Now to install the actual mail agent onto the server with a one-liner operation:

sudo tasksel install mail-server

The process stops at two points for user input. The first is to pick the postfix configuration for the server, select the default Internet site. The second is to set the main domain name, this should be already be filled with microdude.favoritemedium.com and all you need to do is accept.

Once finished, the postfix daemon should be running now. You need to know about two files, the main configuration file and the mail log. You don't need to mess around with the config file, since that should be set up correctly. The mail log is useful to check on whether mails have been sent or other information. Their locations:

/etc/postfix/main.cf
/var/log/mail.log

Now you'll want to test whether mails can be sent out. An email can be typed out in the terminal using the mail command:

mail testing@mailaddress.com
Subject: Do not panic, this is a test
Panic panic panic panic panic
.
Cc:

Check the email account you sent the mail to and you should see that it is present.

Congratulations, you can now spam to your heart's content =^_^=.

From:

• http://articles.slicehost.com/email
• http://articles.slicehost.com/2008/7/28/email-preparing-the-slice
• http://articles.slicehost.com/2008/7/29/postfix-installation
• http://articles.slicehost.com/2010/3/1/barebones-postfix-install-for-ubuntu

Moving Drupal from one host to another

You've got a Drupal site running and now want to move it to another server which is bare-bones. This server is luckily running Ubuntu, which makes installing the rest of the LAMP (Linux, Apache, MySQL and PHP) stack a one-line operation:

sudo tasksel install lamp-server

The install process will stop at one point to ask for the password of the root user for MySQL. Once the install completes, use the MySQL command-line client to add in the same database (an empty one) and user that Drupal uses.

Use mysqldump to dump out the Drupal database data into an sql file. Load it in the new database, populating it. You'll want to tar up the whole directory where the Drupal files reside (not forgetting the .htaccess) and expand it out in the same location within the new machine. You may (not) need to edit sites/default/settings.php or .htaccess for any host information changes. settings.php is also where you edit the database login information, if you decide to use different ones.

With Apache running, you should be able to hit the new Drupal site. It looks to be running but there are a few gotchas. If you are (most likely) running Clean URLs, the links don't work anymore. You'll need to disable it to get navigation working at http://hostname/?q=admin/settings/clean-urls, maybe going through http://hostname/?q=user to log in first. To set up Clean URLs again, you need to run this line:

sudo a2enmod rewrite

And edit /etc/apache2/sites-available/default, changing the AllowOverride None to AllowOverride All inside the directory with the path /var/www, or where the Drupal files live. Restart the server and you find that you can enable Clean URLs now.

The other gotcha is that the status report is complaining it can't find the PHP GD library. Apparently PHP library is installed but not configured so you run these lines:

sudo apt-get install php5 (optional?)
sudo apt-get install php5-gd

Lastly, cron needs to be set up to update the Drupal site. I use curl so the lines are:

sudo apt-get install curl
crontab -e
0 * * * * curl --silent --compressed http://localhost/cron.php

The last line is done inside the vi editor so after that, save and quit.

Congratulations! Your Drupal site is now running fine on the new machine.

Install Apache-Passenger (mod_rails) on Ubuntu 9.04 in 10 steps

A very quick guide on installing Passenger (mod_rails) on a clean install of Ubuntu jaunty 9.04.

1. apt-get install build-essentials
2. apt-get ruby apt-get ruby-dev
3. gem install rubygems-update
4. cd /var/lib/gems/1.8/rubygems-update-1.3.x/
5. ruby setup.rb or bin/update_rubygems
6. gem update --system
7. gem update
8. gem install passenger
9. apt-get install libopenssl-ruby1.8 apache2-prefork-dev libaprutil1 libaprutil1-dev
10. passenger-install-apache2-module

At the end of the installation, you will be prompted to add a few lines to your apache conf -- copy and paste the lines to /etc/apache2/apache2.conf and you are ready to deploy Rails apps immediately.

For the rest of the setup and configuration see here. Hope this helps someone out there.

Installing MongoDB on Ubuntu 9.04 Jaunty Jackalope

For the installation, I decided to build MongoDB (http://www.mongodb.org/) from source. The documentation regarding this is thorough and getting it running was a cinch. A Javascript engine called Spider Monkey is used and you need to build it with UTF8 support.

The documents referred to:
http://www.mongodb.org/display/DOCS/Building
http://www.mongodb.org/display/DOCS/Building+Spider+Monkey
http://www.mongodb.org/display/DOCS/Building+for+Linux

The all-in-one command line version:
Dependencies:
sudo apt-get install curl tcsh git-core scons g++
sudo apt-get install libpcre++-dev libboost-dev libmozjs-dev

Spider Monkey installation:
curl -O ftp://ftp.mozilla.org/pub/mozilla.org/js/js-1.7.0.tar.gz
tar zxvf js-1.7.0.tar.gz
cd js/src
export CFLAGS="-DJS_C_STRINGS_ARE_UTF8"
make -f Makefile.ref
sudo JS_DIST=/usr make -f Makefile.ref export

MongoDB installation:
git clone git://github.com/mongodb/mongo.git
scons all
sudo scons --prefix=/opt/mongo install

Finishing up:
add /opt/mongo/bin to your path
create a /data/db directory structure and chown it to your user or add read-write permission.
http://www.howtogeek.com/howto/ubuntu/how-to-add-a-program-to-the-ubuntu-startup-list-after-login/
System -> Preferences -> Startup Applications -> Startup Programs
click add, click browse and select /opt/mongo/bin/mongod
add run to the end of the line so that it reads '/opt/mongo/bin/mongod run'
enter name and comment then save

And you're done. mongodb will start up with no authentication as default at localhost:27017, the web information interface is at localhost:28017. You can use mongo to enter the interactive shell, similar to mysql's shell client.

Some links to get you started:
http://www.mongodb.org/display/DOCS/Overview+-+The+MongoDB+Interactive+Shell
http://www.mongodb.org/display/DOCS/File+Based+Configuration
http://www.mongodb.org/display/DOCS/Command+Line+Parameters
http://www.mongodb.org/display/DOCS/Security+and+Authentication
http://www.mongodb.org/display/DOCS/Manual