KVM
Contents
KVM
What is the command to start KVM?
The command used to launch KVM seems to keep changing between different versions and distributions. Often you launch it with the qemu command. I don't know what there is all this confusion when it seems like just calling the command kvm would be simple and avoid confusing people. The arguments also change a lot.
kvm qemu-system-x86_64
KVM disk
Disk info
kvm-img info disk.qcow2
Create a QCOW2 disk image
This will generate a "sparse" disk image. That means the initial real physical size will only be about 256KB. The disk image file will grow as needed.
kvm-img create -f qcow2 disk.qcow2 8G
Compress a QCOW2 image
After you do the initial installation of an OS onto a QCOW2 image you can compress the image to save space. Note that after you start using the compressed image that new data will not be compressed. Only the data in the image at the time of compression will be compressed. You can periodically recompress an image to compress new data.
kvm-img convert -c -f qcow2 -O qcow2 -o cluster_size=2M old_disk.qcow2 new_disk.qcow2
raw disk image to compressed qcow2
This shows how to convert a raw disk image (as made from a dd dump) to a qcow2 image. This creates a complete copy with no backing file. It enables compression. Note that this compression is a hybrid, one-time compression of the blocks in use. After mounting, any new data written to the image is not compressed. This is useful for compressing the initial operating system files that will not get updated very often.
kvm-img convert -c -f raw -O qcow2 -o cluster_size=4096 www.example.org.img www.example.org.qcow2
This shows how to convert an LVM disk to qcow2.
qemu-img convert -c -f raw -O qcow2 -o cluster_size=4096 /dev/vg-example.org/example.org-lvm-disk /backup/example.org.qcow2
As near as I can tell qem-img is the same as kvm-img despite the fact that the binaries are different sizes. Their built-in help and version numbers are even the same.
VMWare split image to raw
This shows how to convert a VMWare flat split image disk set to raw. Just cat the files together.
cat linux-server-f001.vmdk linux-server-f002.vmdk linux-server-f003.vmdk > linux-server.img
KVM script to start guests
Here is a simple KVM start script I've been working on. Mostly all it does it pull out all the key parameters into one place at the top of the script and it also handles setup of the bridge and tap network devices. You may hardcode the boot disk image and boot device or you may pass the image file on the command line. The script will figure out if it is CDROM or drive image.
#!/bin/bash # # KVM guest launcher. # # Depends on bridge-utils uml-utilities. # # Copyright (c) 2010, Noah Spurrier <noah@noah.org> # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # # THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES # WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF # MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR # ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES # WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN # ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF # OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. # # Version 1 # # KVM_GUEST_ID should be unique on your subnet. KVM_GUEST_ID=3 HOST_PHYSICAL_IF="em1" CDROM_IMAGE="" DRIVE_IMAGE="03-www.example.com.img" BOOT_DEVICE=${DRIVE_IMAGE} GUEST_MEMORY=512 GUEST_CPU_CORES=1 # BRIDGE_IF # / \ # / \ # HOST_PHYSICAL_IF TAP_IF # Cleanup the guest ID and generate other values based on it. KVM_GUEST_ID=$(printf "%02x" ${KVM_GUEST_ID}) KVM_GUEST_NAME=`basename ${DRIVE_IMAGE} .img` # KVM_GUEST_MAC_ADDR="52:54:00:00:00:${KVM_GUEST_ID}" KVM_GUEST_MAC_ADDR="4e:4f:41:48:00:${KVM_GUEST_ID}" # cleanup the MAC address #KVM_GUEST_MAC_ADDR=$(printf "%02x:%02x:%02x:%02x:%02x:%02x" ${KVM_GUEST_ID}) #0x${KVM_GUEST_MAC_ADDR//:/ 0x} TAP_IF="kvmtapif${KVM_GUEST_ID}" BRIDGE_IF="kvmbrif${KVM_GUEST_ID}" KVM_HOST_VNC_PORT_TO_GUEST=$(( 5900 + ${KVM_GUEST_ID} )) SERIAL_TCP_PORT="44${KVM_GUEST_ID}" AVAHI_PUBLISH_PID=0 HOSTNAME=$(hostname) # If a required binary is missing then it will be added to this list. MISSING_LIST="" # Cleanup the BOOT_DEVICE. Setting it to the image name is just a convenience. if [ ${BOOT_DEVICE} = ${DRIVE_IMAGE} ]; then BOOT_DEVICE="c" elif [ ${BOOT_DEVICE} = ${CDROM_IMAGE} ]; then BOOT_DEVICE="d" fi ########################################################### ## ## Parse command-line arguments ## ########################################################### if [ $# -eq 1 ]; then if [ -f "$1" ]; then if file $1 | grep -q "ISO 9660" then CDROM_IMAGE=$1 BOOT_DEVICE="d" else # Also look for "Qemu Image" or "boot sector". DRIVE_IMAGE=$1 BOOT_DEVICE="c" fi else echo "ERROR: File not found: $1" echo "Expected a filename to a CDROM or drive image, or nothing." exit 1 fi fi ########################################################### ## ## Functions ## ########################################################### print_section () { message=$1 echo echo -e "\033[31m== ${message} ==\033[0m" } on_exit_cleanup_kvm () { print_section "on_exit_cleanup_kvm" destroy_tap destroy_bridge if [ ${AVAHI_PUBLISH_PID} != 0 ]; then /bin/kill -int ${AVAHI_PUBLISH_PID} 2>/dev/null sleep 1 /bin/kill -9 ${AVAHI_PUBLISH_PID} 2>/dev/null fi on_exit_trap_cleanup } # This is the exit trap handler for the 'tee' logger. on_exit_trap_cleanup () { # Close stdin and stdout which closes our end of the pipe # and tells `tee` we are done. exec 1>&- 2>&- # Wait for `tee` process to finish. If we exited here then the `tee` # process might get killed before it hand finished flushing its buffers # to the logfile. wait ${TEEPID} rm ${PIPEFILE} } tee_log_output () { LOGFILE=$1 PIPEFILE=$(mktemp -u $(basename $0)-pipe-pid$$-XXX) mkfifo ${PIPEFILE} tee ${LOGFILE} < ${PIPEFILE} & TEEPID=$! # Redirect subsequent stdout and stderr output to named pipe. exec > ${PIPEFILE} 2>&1 trap on_exit_trap_cleanup EXIT } exit_error () { echo "$@" >&2 exit 1 } create_tap () { #tunctl -u kvm -t ${TAP_IF} if ! tunctl -t ${TAP_IF}; then exit_error "create_tap() failed calling tunctl."; fi if ! ifconfig ${TAP_IF} up; then exit_error "create_tap() failed calling ifconfig."; fi } destroy_tap () { print_section "Destroy tap ${TAP_IF}." ifconfig ${TAP_IF} down sleep 1 tunctl -d ${TAP_IF} sleep 1 return 0 } create_bridge () { if brctl show | grep -q ${BRIDGE_IF}; then echo "ERROR: Bridge interface, ${BRIDGE_IF}, already exists." exit 1 fi echo "Bridge interface, ${BRIDGE_IF}, does not exit." echo "Create bridge ${BRIDGE_IF}." brctl addbr ${BRIDGE_IF} # brctl stp ${BRIDGE_IF} on ifconfig ${HOST_PHYSICAL_IF} up echo "Bridge ${BRIDGE_IF} to ${HOST_PHYSICAL_IF}." brctl addif ${BRIDGE_IF} ${HOST_PHYSICAL_IF} brctl addif ${BRIDGE_IF} ${TAP_IF} ifconfig ${BRIDGE_IF} up sleep 1 } destroy_bridge () { print_section "Destroy bridge ${BRIDGE_IF} to ${HOST_PHYSICAL_IF}." if ! brctl show | grep -q ${BRIDGE_IF}; then echo "WARNING: Bridge interface, ${BRIDGE_IF}, does not exit." echo "...will not attempt to destroy it." return 0 fi ifconfig ${BRIDGE_IF} down brctl delif ${BRIDGE_IF} ${TAP_IF} brctl delif ${BRIDGE_IF} ${HOST_PHYSICAL_IF} brctl delbr ${BRIDGE_IF} sleep 1 return 0 } # This function checks if a given program is available. # It if is not found then it is added to a global called MISSING_LIST. check_missing () { check_program=$1 if ! which ${check_program} >/dev/null; then if [ "${MISSING_LIST}" != "" ] ; then MISSING_LIST="${check_program} ${MISSING_LIST}" else MISSING_LIST="${check_program}" fi fi } preflight_check () { if [ ! -r "/dev/kvm" ]; then echo "=========================================================" echo "ERROR: Current user is not authorized to run KVM" echo "---------------------------------------------------------" echo "You must have read/write access to /dev/kvm." echo "Perhaps you forgot to run this with 'sudo'." echo "Also check the group ownership pf /dev/kvm." echo "=========================================================" exit 1 fi # tunctl is in 'uml-utilities' check_missing tunctl # brctl is in 'bridge-utils' check_missing brctl check_missing kvm if [ "${MISSING_LIST}" != "" ] ; then echo "=========================================================" echo "ERROR: One or more required programs are not installed." echo "---------------------------------------------------------" echo ${MISSING_LIST} echo "=========================================================" exit 1 fi } ########################################################### ## ## MAIN ## ########################################################### main () { print_section "Opening logfile: ${KVM_GUEST_NAME}.$$.log" tee_log_output ${KVM_GUEST_NAME}.$$.log date --rfc-3339=seconds # SAVED_OLD_TRAPS=$(trap) trap on_exit_cleanup_kvm EXIT #on_exit_trap_cleanup print_section "Booting ${KVM_GUEST_NAME}" print_section "Preflight check to be sure we have everything we need." preflight_check print_section "Creating TAP device" create_tap print_section "Creating bridge" create_bridge ## if ifconfig -s -a | grep -q ${TAP_IF}; then ## echo "Old TAP interface found, ${TAP_IF}. Attempting to destroy it." ## destroy_tap ## fi print_section "VNC Display" echo "Guest VNC available on host port: ${KVM_HOST_VNC_PORT_TO_GUEST}" echo "Connect to this guest via SSH+RFB with the following command:" echo " xvncviewer -via ${USER}@${HOSTNAME} AutoSelect=0 FullColor=1 localhost:${KVM_GUEST_ID}" echo " # or try the following if -AutoSelect does not work:" echo " xvncviewer -via ${USER}@${HOSTNAME} localhost:${KVM_GUEST_ID}" echo "Note that ${HOSTNAME} may not resolve, so you should replace it with the IP address of the host." echo if which avahi-publish >/dev/null; then print_section "Service announcement on network broadcast via Zeroconf." avahi-publish -s "KVM guest VNC for ${KVM_GUEST_NAME}" _rfb._tcp ${KVM_HOST_VNC_PORT_TO_GUEST} "KVM guest with VGA screen via RFB (VNC)." >/dev/null 2>&1 & AVAHI_PUBLISH_PID=$! sleep 1 fi kvm_cmd="kvm \ -name ${KVM_GUEST_NAME} \ -boot ${BOOT_DEVICE} \ -drive file=${DRIVE_IMAGE},if=ide,index=0,cache=none \ -m ${GUEST_MEMORY} \ -smp ${GUEST_CPU_CORES} \ -vnc :${KVM_GUEST_ID} \ -net nic,macaddr=${KVM_GUEST_MAC_ADDR} \ -net tap,ifname=${TAP_IF},script=no \ -usbdevice tablet \ -serial stdio \ -vga std \ " kvm_cmd=$(echo ${kvm_cmd}) # -net dump \ # -vga std \ ## This forces RAW format, but this is usually not necessary. # -drive file=${DRIVE_IMAGE},if=ide,index=0,cache=none,format=raw \ # -cdrom ${CDROM_IMAGE} \ # -daemonize \ # -serial tcp:127.0.0.1:${SERIAL_TCP_PORT},server,nowait \ print_section "KVM command-line being used:" echo " ${kvm_cmd}" ${kvm_cmd} #on_exit_cleanup_kvm } # Start here. main # # __________________ _-_ # \___=NCC-1701= __)) ____.---'---`---.____ # \_ \ \----._________.----' # \ \ / / `-_-' # __,--`-`---'-'-. # /}___ )(- # `--.____,-' # # vim:set sr et ts=4 sw=4 ft=sh:
KVM Networking
Edit the file /etc/network/interfaces:
auto eth0 iface eth0 inet static # Don't need to set eth0 since it will get bridged. auto br0 iface br0 inet dhcp bridge_ports eth0 bridge_fd 9 bridge_stp off bridge_maxage 12 bridge_hello 2
edit libvirt guest XML definition
Edit the file for your given VM. In this example the guest name is model1. In my system the full path is /etc/libvirt/qemu/model1.xml. Look for the interface section and change it to a bridge something like the following:
<interface type='bridge'> <mac address='54:52:00:1e:67:94'/> <source bridge='br0'/> </interface>
Restart the host's networking
You will notice a momentary interruption of the connection to the machine.
/etc/init.d/networking restart
Restart libvirtd
/etc/init.d/libvirt-bin restart
Start your libvirt client and test the connection
I use `virt-manager` or `virsh`.
KVM network console port
Add the following to /etc/rc.local, where 00:E0:81:2B:0C:C1 is the MAC address of the destination nc listener and 12345 is the listening port:
/sbin/modprobe netconsole netconsole=@/eth0,12345@10.0.0.1/00:E0:81:2B:0C:C1
The log client is just the NetCat command:
nc -dul 12345
It helps to disable sync logging. Edit /etc/syslog.conf:
/var/log/messages
to
-/var/log/messages
Add the following to /etc/rc.local:
echo 9 > /proc/sysrq-trigger
libvirtd
Libvirt does not do the actual virtualization itself. It is just a client-server interface to virtual machines. This allows you to manage the guests remotely through a simple GUI interface. I have not decided if this tool is worth the trouble or not. It's simple to just SSH into a server and manage it from the command-line. Libvirt is still a little rough. It's very fragile. You will find that you have to restart it often:
/etc/init.d/libvirt-bin restart
virsh
The XML config files are stored here: /etc/libvirt/qemu/*.xml.
virsh -c qemu:///system
Common commands. Note that domain is the guest name. Also note that list should really be called list_only_running; you want the --all option to make it actually list all the domains no matter what state they are in.
help list --all domstate [domain] start [domain] destroy [domain]
KVM Remote via VNC
Set VNC password via the interactive monitor
Some VNC clients require a password (OS X built-in). You can set a VNC password on the KVM server. This requires that you start KVM with both he -vnc :0" option and set the -monitor stdio option, which will start an interactive monitor to control the guest. From the monitor command-line you can set the monitor password with the change vmc password command.
# qemu-system-x86_64 -m 1024 -smp 1 -hdb /dev/sdc -vga std -k en-us -boot menu=on -vnc :0 -monitor stdio QEMU 1.4.0 monitor - type 'help' for more information (qemu) change vnc password Password: ******** (qemu)
Set VNC password via script by talking to the monitor through a FIFO named pipe
You can plug the monitor into a standard Linux FIFO pipe. This makes it easy to change the VNC password from a script. You don't have to fool around with UNIX or network sockets. Easy!
PASSWORD="password" mkfifo /tmp/guest.monitor.{in,out} qemu-system-x86_64 -m 1024 -smp 1 -hda /dev/sdb -vga std -vnc :0 -chardev pipe,id=monitor,path=/tmp/guest.monitor -monitor chardev:monitor –daemonize echo "change vnc password ${PASSWORD}" >> /tmp/guest.monitor.in rm /tmp/guest.monitor.{in,out}
localhost
Multiple VNC screen sessions can run on the same machine. By default each session starts at port 5900+session_number. So if you are running just one VNC screem it will usually be mapped to port 5900; the next one will be mapped to 5901; and the next one 5902 and so on.
ssh -N -f -L 5900:192.168.1.13:5900 username@gateway.example.com vncviewer AutoSelect=0 FullColor=1 localhost
Or like this to view VNC session 0 on the machine 192.168.1.13.
vncviewer AutoSelect=0 FullColor=1 -via username@gateway.example.com 192.168.1.13:0
session 1 on the machine 192.168.1.13:
vncviewer AutoSelect=0 FullColor=1 -via username@gateway.example.com 192.168.1.13:1
Note that when you use via the names and addresses are from the point of view of the gateway machine. So machine 192.168.1.13 on the LAN visible to the gateway machine. You could also just say localhost if you want to connect to a VNC session running on the gateway machine itself.
vncviewer AutoSelect=0 FullColor=1 -via username@gateway.example.com localhost
So why use via at all in that case? Why not just do this?
vncviewer AutoSelect=0 FullColor=1 username@gateway.example.com
Remember that we may still be going through a firewall. Firewalls rarely allow the VNC ports 5900 through. The via option says to go through SSH (port 22), which most firewalls will allow -- plus this adds SSH encryption to the session.
remote KVM server
sudo virt-install --connect qemu:///system --name ubuntu910 --force --ram 512 --file ubuntu910.qcow2 --file-size 12 --cdrom ubuntu-9.10-desktop-i386.iso --vnc --noautoconsole --accelerate --os-type linux --os-variant generic26
Error: Rect too big
This is a bug in the vncviewer. Add these options to the command-line:
AutoSelect=0 FullColor=1
vncviewer AutoSelect=0 FullColor=1 -via username@gateway.example.com localhost:0
Boot from CD-ROM image
Testing a CD-ROM image is as simple as the following command.
kvm -m 1024 -cdrom ubuntu-12.10-desktop-i386.iso
If you want to boot a CD-ROM to install onto a drive image, use the following command.
kvm -m 1024 -drive file=disk.img -cdrom ubuntu-12.10-desktop-i386.iso -boot d
It's important to use the -m 1024 because the default amount of RAM is only 128 MB, which is far too small to run a GUI installer.
Boot from USB drive
This is useful for testing Linux distros on USB flash drives.
This example assumes that your physical USB drive shows up on your host as /dev/sdc. This is a "full" example that also shows creating a virtual disk to install Linux onto, and creating a named pip fifo to set the VNC password.
qemu-img create -f qcow disk.qcow 4G mkfifo /tmp/guest.monitor.{in,out} qemu-system-x86_64 -m 1024 -smp 1 -drive file=/root/disk.qcow,index=0,media=disk -drive file=/dev/sdc,index=1,media=cdrom -vga std -vnc :0 -chardev pipe,id=monitor,path=/tmp/guest.monitor -monitor chardev:monitor -daemonize rm /tmp/guest.monitor.{in,out} xvnc4viewer localhost:0
Here are some other basic examples.
Determine where your USB drive device node file is located. In this example the device node is /dev/sdb1. I think this is wrong. It should be /dev/sdb. You might also want to enable VNC. See the example after this one for an alternative.
kvm -m 1024 -smp 1 -hdb /dev/sdb1 -boot menu=on
This is a better alternative (and at the qemu command-line run command, change vmc password):
qemu-system-x86_64 -m 1024 -smp 1 -hdb /dev/sdb -vga std -usb -usbdevice tablet -vnc :0 -monitor stdio
Or this:
qemu-system-x86_64 -m 1024 -smp 1 -hdb /dev/sdb -curses
Sometimes this does not work. The following has been suggested; however, I have not been able to get this to work yet:
kvm -m 1024 -smp 1 -usb -usbdevice host:0781:5406 -boot menu=on
Error: "Warning: KVM is not available."
The virt-manager program may give the following warning when you try to create a new virtual machine.
Warning: KVM is not available. This may mean the KVM package is not installed, or the KVM kernel modules are not loaded.
The most likely cause is that virtualization has been disabled in your BIOS. Look for problems in dmesg.
dmesg | grep kvm
You may see messages like the following:
[ 146.864343] kvm: disabled by bios [ 146.897394] init: qemu-kvm pre-start process (1656) terminated with status 1
If you see these messages then you will need to reboot and update your BIOS settings. On my Lenovo Thinkpad T430s it's under Security | Virtualization. The virtualization settings should all be enabled.
Virtualization Intel (R) Virtualization Technology [Enabled] Intel (R) VT-d Feature [Enabled]
Check to make sure that your CPU actually supports virtualization. This is rarely a problem since almost all modern CPUs support Intel vmx or AMD svm. Atom processors are the notable exception.
grep -e 'vmx' -e 'svm' /proc/cpuinfo