KVM

From Noah.org
Jump to: navigation, search


KVM

What is the command to start KVM?

The command used to launch KVM seems to keep changing between different versions and distributions. Often you launch it with the qemu command. I don't know what there is all this confusion when it seems like just calling the command kvm would be simple and avoid confusing people. The arguments also change a lot.

kvm
qemu-system-x86_64

KVM disk

Disk info

kvm-img info disk.qcow2

Create a QCOW2 disk image

This will generate a "sparse" disk image. That means the initial real physical size will only be about 256KB. The disk image file will grow as needed.

kvm-img create -f qcow2 disk.qcow2 8G

Compress a QCOW2 image

After you do the initial installation of an OS onto a QCOW2 image you can compress the image to save space. Note that after you start using the compressed image that new data will not be compressed. Only the data in the image at the time of compression will be compressed. You can periodically recompress an image to compress new data.

kvm-img convert -c -f qcow2 -O qcow2 -o cluster_size=2M old_disk.qcow2 new_disk.qcow2

raw disk image to compressed qcow2

This shows how to convert a raw disk image (as made from a dd dump) to a qcow2 image. This creates a complete copy with no backing file. It enables compression. Note that this compression is a hybrid, one-time compression of the blocks in use. After mounting, any new data written to the image is not compressed. This is useful for compressing the initial operating system files that will not get updated very often.

kvm-img convert -c -f raw -O qcow2 -o cluster_size=4096 www.example.org.img www.example.org.qcow2

This shows how to convert an LVM disk to qcow2.

qemu-img convert -c -f raw -O qcow2 -o cluster_size=4096 /dev/vg-example.org/example.org-lvm-disk /backup/example.org.qcow2

As near as I can tell qem-img is the same as kvm-img despite the fact that the binaries are different sizes. Their built-in help and version numbers are even the same.

VMWare split image to raw

This shows how to convert a VMWare flat split image disk set to raw. Just cat the files together.

cat linux-server-f001.vmdk linux-server-f002.vmdk linux-server-f003.vmdk > linux-server.img

KVM script to start guests

Here is a simple KVM start script I've been working on. Mostly all it does it pull out all the key parameters into one place at the top of the script and it also handles setup of the bridge and tap network devices. You may hardcode the boot disk image and boot device or you may pass the image file on the command line. The script will figure out if it is CDROM or drive image.

#!/bin/bash
#
# KVM guest launcher.
#
# Depends on bridge-utils uml-utilities.
#
# Copyright (c) 2010, Noah Spurrier <noah@noah.org>
# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
#
# Version 1
#

# KVM_GUEST_ID should be unique on your subnet.
KVM_GUEST_ID=3 
HOST_PHYSICAL_IF="em1"
CDROM_IMAGE=""
DRIVE_IMAGE="03-www.example.com.img"
BOOT_DEVICE=${DRIVE_IMAGE}
GUEST_MEMORY=512
GUEST_CPU_CORES=1

#               BRIDGE_IF
#                 /   \
#                /     \
#   HOST_PHYSICAL_IF  TAP_IF

# Cleanup the guest ID and generate other values based on it.
KVM_GUEST_ID=$(printf "%02x" ${KVM_GUEST_ID})
KVM_GUEST_NAME=`basename ${DRIVE_IMAGE} .img`
# KVM_GUEST_MAC_ADDR="52:54:00:00:00:${KVM_GUEST_ID}"
KVM_GUEST_MAC_ADDR="4e:4f:41:48:00:${KVM_GUEST_ID}"
# cleanup the MAC address
#KVM_GUEST_MAC_ADDR=$(printf "%02x:%02x:%02x:%02x:%02x:%02x" ${KVM_GUEST_ID})
#0x${KVM_GUEST_MAC_ADDR//:/ 0x}
TAP_IF="kvmtapif${KVM_GUEST_ID}"
BRIDGE_IF="kvmbrif${KVM_GUEST_ID}"
KVM_HOST_VNC_PORT_TO_GUEST=$(( 5900 + ${KVM_GUEST_ID} ))
SERIAL_TCP_PORT="44${KVM_GUEST_ID}"
AVAHI_PUBLISH_PID=0
HOSTNAME=$(hostname)
# If a required binary is missing then it will be added to this list.
MISSING_LIST=""

# Cleanup the BOOT_DEVICE. Setting it to the image name is just a convenience.
if [ ${BOOT_DEVICE} = ${DRIVE_IMAGE} ]; then
        BOOT_DEVICE="c"
elif [ ${BOOT_DEVICE} = ${CDROM_IMAGE} ]; then
        BOOT_DEVICE="d"
fi

###########################################################
##
## Parse command-line arguments
##
###########################################################

if [ $# -eq 1 ]; then
        if [ -f "$1" ]; then
                if file $1 | grep -q "ISO 9660"
                then
                        CDROM_IMAGE=$1
                        BOOT_DEVICE="d"
                else
                        # Also look for "Qemu Image" or "boot sector".
                        DRIVE_IMAGE=$1
                        BOOT_DEVICE="c"
                fi  
        else
                echo "ERROR: File not found: $1"
                echo "Expected a filename to a CDROM or drive image, or nothing."
                exit 1
        fi
fi

###########################################################
##
## Functions
##
###########################################################

print_section ()
{
        message=$1
        echo
        echo -e "\033[31m== ${message} ==\033[0m"
}

on_exit_cleanup_kvm ()
{
        print_section "on_exit_cleanup_kvm"
        destroy_tap
        destroy_bridge
        if [ ${AVAHI_PUBLISH_PID} != 0 ]; then
                /bin/kill -int ${AVAHI_PUBLISH_PID} 2>/dev/null
                sleep 1
                /bin/kill -9 ${AVAHI_PUBLISH_PID} 2>/dev/null
        fi
        on_exit_trap_cleanup
}

# This is the exit trap handler for the 'tee' logger.
on_exit_trap_cleanup ()
{
        # Close stdin and stdout which closes our end of the pipe
        # and tells `tee` we are done.
        exec 1>&- 2>&-
        # Wait for `tee` process to finish. If we exited here then the `tee`
        # process might get killed before it hand finished flushing its buffers
        # to the logfile.
        wait ${TEEPID}
        rm ${PIPEFILE}
}

tee_log_output ()
{
        LOGFILE=$1
        PIPEFILE=$(mktemp -u $(basename $0)-pipe-pid$$-XXX)
        mkfifo ${PIPEFILE}
        tee ${LOGFILE} < ${PIPEFILE} &
        TEEPID=$!
        # Redirect subsequent stdout and stderr output to named pipe.
        exec > ${PIPEFILE} 2>&1
        trap on_exit_trap_cleanup EXIT
}

exit_error ()
{
        echo "$@" >&2
        exit 1
}

create_tap ()
{
        #tunctl -u kvm -t ${TAP_IF}
        if ! tunctl -t ${TAP_IF}; then exit_error "create_tap() failed calling tunctl."; fi
        if ! ifconfig ${TAP_IF} up; then exit_error "create_tap() failed calling ifconfig."; fi
}

destroy_tap ()
{
        print_section "Destroy tap ${TAP_IF}."
        ifconfig ${TAP_IF} down
        sleep 1
        tunctl -d ${TAP_IF}
        sleep 1
        return 0
}

create_bridge ()
{
        if brctl show | grep -q ${BRIDGE_IF}; then
                echo "ERROR: Bridge interface, ${BRIDGE_IF}, already exists."
                exit 1
        fi
        echo "Bridge interface, ${BRIDGE_IF}, does not exit."
        echo "Create bridge ${BRIDGE_IF}."
        brctl addbr ${BRIDGE_IF}
        #    brctl stp ${BRIDGE_IF} on
        ifconfig ${HOST_PHYSICAL_IF} up
        echo "Bridge ${BRIDGE_IF} to ${HOST_PHYSICAL_IF}."
        brctl addif ${BRIDGE_IF} ${HOST_PHYSICAL_IF}
        brctl addif ${BRIDGE_IF} ${TAP_IF}
        ifconfig ${BRIDGE_IF} up
        sleep 1
}

destroy_bridge ()
{
        print_section "Destroy bridge ${BRIDGE_IF} to ${HOST_PHYSICAL_IF}."
        if ! brctl show | grep -q ${BRIDGE_IF}; then
                echo "WARNING: Bridge interface, ${BRIDGE_IF}, does not exit."
                echo "...will not attempt to destroy it."
                return 0
        fi
        ifconfig ${BRIDGE_IF} down
        brctl delif ${BRIDGE_IF} ${TAP_IF}
        brctl delif ${BRIDGE_IF} ${HOST_PHYSICAL_IF}
        brctl delbr ${BRIDGE_IF}
        sleep 1
        return 0
}

# This function checks if a given program is available.
# It if is not found then it is added to a global called MISSING_LIST.
check_missing ()
{
        check_program=$1
        if ! which ${check_program} >/dev/null; then
                if [ "${MISSING_LIST}" != "" ] ; then
                        MISSING_LIST="${check_program} ${MISSING_LIST}"
                else
                        MISSING_LIST="${check_program}"
                fi
        fi
}

preflight_check ()
{
        if [ ! -r "/dev/kvm" ]; then
                echo "========================================================="
                echo "ERROR: Current user is not authorized to run KVM"
                echo "---------------------------------------------------------"
                echo "You must have read/write access to /dev/kvm."
                echo "Perhaps you forgot to run this with 'sudo'."
                echo "Also check the group ownership pf /dev/kvm."
                echo "========================================================="
                exit 1
        fi

        # tunctl is in 'uml-utilities'
        check_missing tunctl
        # brctl is in 'bridge-utils'
        check_missing brctl
        check_missing kvm
        if [ "${MISSING_LIST}" != "" ] ; then
                echo "========================================================="
                echo "ERROR: One or more required programs are not installed."
                echo "---------------------------------------------------------"
                echo ${MISSING_LIST}
                echo "========================================================="
                exit 1
        fi
}

###########################################################
##
## MAIN
##
###########################################################
main ()
{
        print_section "Opening logfile: ${KVM_GUEST_NAME}.$$.log"
        tee_log_output ${KVM_GUEST_NAME}.$$.log
        date --rfc-3339=seconds

        # SAVED_OLD_TRAPS=$(trap)
        trap on_exit_cleanup_kvm EXIT
#on_exit_trap_cleanup

        print_section "Booting ${KVM_GUEST_NAME}"

        print_section "Preflight check to be sure we have everything we need."
        preflight_check

        print_section "Creating TAP device"
        create_tap

        print_section "Creating bridge"
        create_bridge

        ## if ifconfig -s -a | grep -q ${TAP_IF}; then
        ##     echo "Old TAP interface found, ${TAP_IF}. Attempting to destroy it."
        ##     destroy_tap
        ## fi

        print_section "VNC Display"
        echo "Guest VNC available on host port: ${KVM_HOST_VNC_PORT_TO_GUEST}"
        echo "Connect to this guest via SSH+RFB with the following command:"
        echo "    xvncviewer -via ${USER}@${HOSTNAME} AutoSelect=0 FullColor=1 localhost:${KVM_GUEST_ID}"
        echo " # or try the following if -AutoSelect does not work:"
        echo "    xvncviewer -via ${USER}@${HOSTNAME} localhost:${KVM_GUEST_ID}"
        echo "Note that ${HOSTNAME} may not resolve, so you should replace it with the IP address of the host."
        echo

        if which avahi-publish >/dev/null; then
                print_section "Service announcement on network broadcast via Zeroconf."
                avahi-publish -s "KVM guest VNC for ${KVM_GUEST_NAME}" _rfb._tcp ${KVM_HOST_VNC_PORT_TO_GUEST} "KVM guest with VGA screen via RFB (VNC)." >/dev/null 2>&1 &
                AVAHI_PUBLISH_PID=$!
                sleep 1
        fi

        kvm_cmd="kvm \
                -name ${KVM_GUEST_NAME} \
                -boot ${BOOT_DEVICE} \
                -drive file=${DRIVE_IMAGE},if=ide,index=0,cache=none \
                -m ${GUEST_MEMORY} \
                -smp ${GUEST_CPU_CORES} \
                -vnc :${KVM_GUEST_ID} \
                -net nic,macaddr=${KVM_GUEST_MAC_ADDR} \
                -net tap,ifname=${TAP_IF},script=no \
                -usbdevice tablet \
                -serial stdio \
                -vga std \
        "
        kvm_cmd=$(echo ${kvm_cmd})
        #       -net dump \
        #       -vga std \
        ## This forces RAW format, but this is usually not necessary.
        #       -drive file=${DRIVE_IMAGE},if=ide,index=0,cache=none,format=raw \
        #       -cdrom ${CDROM_IMAGE} \
        #       -daemonize \
        #       -serial tcp:127.0.0.1:${SERIAL_TCP_PORT},server,nowait \

        print_section "KVM command-line being used:"
        echo "    ${kvm_cmd}"

        ${kvm_cmd}
        #on_exit_cleanup_kvm 
}

# Start here.
main

#
#  __________________            _-_
#  \___=NCC-1701= __))  ____.---'---`---.____
#              \_ \     \----._________.----'
#                \ \     / /    `-_-'
#            __,--`-`---'-'-.
#           /}___           )(-
#                `--.____,-'
#
# vim:set sr et ts=4 sw=4 ft=sh:

KVM Networking

Edit the file /etc/network/interfaces:

auto eth0
iface eth0 inet static
# Don't need to set eth0 since it will get bridged.
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_fd 9
bridge_stp off
bridge_maxage 12
bridge_hello 2

edit libvirt guest XML definition

Edit the file for your given VM. In this example the guest name is model1. In my system the full path is /etc/libvirt/qemu/model1.xml. Look for the interface section and change it to a bridge something like the following:

    <interface type='bridge'>
        <mac address='54:52:00:1e:67:94'/>
        <source bridge='br0'/>
    </interface>

Restart the host's networking

You will notice a momentary interruption of the connection to the machine.

/etc/init.d/networking restart

Restart libvirtd

/etc/init.d/libvirt-bin restart

Start your libvirt client and test the connection

I use `virt-manager` or `virsh`.


KVM network console port

Add the following to /etc/rc.local, where 00:E0:81:2B:0C:C1 is the MAC address of the destination nc listener and 12345 is the listening port:

/sbin/modprobe netconsole netconsole=@/eth0,12345@10.0.0.1/00:E0:81:2B:0C:C1

The log client is just the NetCat command:

nc -dul 12345

It helps to disable sync logging. Edit /etc/syslog.conf:

 /var/log/messages

to

 -/var/log/messages

Add the following to /etc/rc.local:

echo 9 > /proc/sysrq-trigger

libvirtd

Libvirt does not do the actual virtualization itself. It is just a client-server interface to virtual machines. This allows you to manage the guests remotely through a simple GUI interface. I have not decided if this tool is worth the trouble or not. It's simple to just SSH into a server and manage it from the command-line. Libvirt is still a little rough. It's very fragile. You will find that you have to restart it often:

/etc/init.d/libvirt-bin restart

virsh

The XML config files are stored here: /etc/libvirt/qemu/*.xml.

virsh -c qemu:///system

Common commands. Note that domain is the guest name. Also note that list should really be called list_only_running; you want the --all option to make it actually list all the domains no matter what state they are in.

help
list --all
domstate [domain]
start [domain]
destroy [domain]

KVM Remote via VNC

Set VNC password via the interactive monitor

Some VNC clients require a password (OS X built-in). You can set a VNC password on the KVM server. This requires that you start KVM with both he -vnc :0" option and set the -monitor stdio option, which will start an interactive monitor to control the guest. From the monitor command-line you can set the monitor password with the change vmc password command.

# qemu-system-x86_64 -m 1024 -smp 1 -hdb /dev/sdc -vga std -k en-us -boot menu=on -vnc :0 -monitor stdio
QEMU 1.4.0 monitor - type 'help' for more information
(qemu) change vnc password
Password: ********
(qemu)

Set VNC password via script by talking to the monitor through a FIFO named pipe

You can plug the monitor into a standard Linux FIFO pipe. This makes it easy to change the VNC password from a script. You don't have to fool around with UNIX or network sockets. Easy!

PASSWORD="password"
mkfifo /tmp/guest.monitor.{in,out}
qemu-system-x86_64 -m 1024 -smp 1 -hda /dev/sdb -vga std -vnc :0 -chardev pipe,id=monitor,path=/tmp/guest.monitor -monitor chardev:monitor –daemonize
echo "change vnc password ${PASSWORD}" >> /tmp/guest.monitor.in
rm /tmp/guest.monitor.{in,out}

localhost

Multiple VNC screen sessions can run on the same machine. By default each session starts at port 5900+session_number. So if you are running just one VNC screem it will usually be mapped to port 5900; the next one will be mapped to 5901; and the next one 5902 and so on.

ssh -N -f -L 5900:192.168.1.13:5900 username@gateway.example.com
vncviewer AutoSelect=0 FullColor=1 localhost

Or like this to view VNC session 0 on the machine 192.168.1.13.

vncviewer AutoSelect=0 FullColor=1 -via username@gateway.example.com 192.168.1.13:0

session 1 on the machine 192.168.1.13:

vncviewer AutoSelect=0 FullColor=1 -via username@gateway.example.com 192.168.1.13:1

Note that when you use via the names and addresses are from the point of view of the gateway machine. So machine 192.168.1.13 on the LAN visible to the gateway machine. You could also just say localhost if you want to connect to a VNC session running on the gateway machine itself.

vncviewer AutoSelect=0 FullColor=1 -via username@gateway.example.com localhost

So why use via at all in that case? Why not just do this?

vncviewer AutoSelect=0 FullColor=1 username@gateway.example.com

Remember that we may still be going through a firewall. Firewalls rarely allow the VNC ports 5900 through. The via option says to go through SSH (port 22), which most firewalls will allow -- plus this adds SSH encryption to the session.

remote KVM server

sudo virt-install --connect qemu:///system --name ubuntu910 --force --ram 512 --file ubuntu910.qcow2 --file-size 12 --cdrom ubuntu-9.10-desktop-i386.iso --vnc --noautoconsole --accelerate --os-type linux --os-variant generic26

Error: Rect too big

This is a bug in the vncviewer. Add these options to the command-line:
AutoSelect=0 FullColor=1
vncviewer AutoSelect=0 FullColor=1 -via username@gateway.example.com localhost:0

Boot from CD-ROM image

Testing a CD-ROM image is as simple as the following command.

kvm -m 1024 -cdrom ubuntu-12.10-desktop-i386.iso

If you want to boot a CD-ROM to install onto a drive image, use the following command.

kvm -m 1024 -drive file=disk.img -cdrom ubuntu-12.10-desktop-i386.iso -boot d

It's important to use the -m 1024 because the default amount of RAM is only 128 MB, which is far too small to run a GUI installer.

Boot from USB drive

This is useful for testing Linux distros on USB flash drives.

This example assumes that your physical USB drive shows up on your host as /dev/sdc. This is a "full" example that also shows creating a virtual disk to install Linux onto, and creating a named pip fifo to set the VNC password.

qemu-img create -f qcow disk.qcow 4G
mkfifo /tmp/guest.monitor.{in,out}
qemu-system-x86_64 -m 1024 -smp 1 -drive file=/root/disk.qcow,index=0,media=disk -drive file=/dev/sdc,index=1,media=cdrom -vga std -vnc :0 -chardev pipe,id=monitor,path=/tmp/guest.monitor -monitor chardev:monitor -daemonize
rm /tmp/guest.monitor.{in,out}
xvnc4viewer localhost:0

Here are some other basic examples.

Determine where your USB drive device node file is located. In this example the device node is /dev/sdb1. I think this is wrong. It should be /dev/sdb. You might also want to enable VNC. See the example after this one for an alternative.

kvm -m 1024 -smp 1 -hdb /dev/sdb1 -boot menu=on

This is a better alternative (and at the qemu command-line run command, change vmc password):

qemu-system-x86_64 -m 1024 -smp 1 -hdb /dev/sdb -vga std -usb -usbdevice tablet -vnc :0 -monitor stdio

Or this:

qemu-system-x86_64  -m 1024 -smp 1  -hdb /dev/sdb -curses

Sometimes this does not work. The following has been suggested; however, I have not been able to get this to work yet:

kvm -m 1024 -smp 1 -usb -usbdevice host:0781:5406 -boot menu=on

Error: "Warning: KVM is not available."

The virt-manager program may give the following warning when you try to create a new virtual machine.

Warning: KVM is not available. This may mean the KVM package is not installed, or the KVM kernel modules are not loaded.

The most likely cause is that virtualization has been disabled in your BIOS. Look for problems in dmesg.

dmesg | grep kvm

You may see messages like the following:

[  146.864343] kvm: disabled by bios
[  146.897394] init: qemu-kvm pre-start process (1656) terminated with status 1

If you see these messages then you will need to reboot and update your BIOS settings. On my Lenovo Thinkpad T430s it's under Security | Virtualization. The virtualization settings should all be enabled.

Virtualization
Intel (R) Virtualization Technology   [Enabled]
Intel (R) VT-d Feature                [Enabled]

Check to make sure that your CPU actually supports virtualization. This is rarely a problem since almost all modern CPUs support Intel vmx or AMD svm. Atom processors are the notable exception.

grep -e 'vmx' -e 'svm' /proc/cpuinfo