Live migration xen

I'll share it how to do a live migration with xen. First, you need to known that to do a live migration for a domU, the back-storage of your virtual machine has to be on a shared storage, like iSCSI , AoE, or any other things... I explain here how to set-up AoE on a Linux Debian, and if you prefer iSCSI, I explain it here. Second, to do the live migration between two host, these hosts needs to be on the same subnet, I means on the same layer 2 network, otherwise it won't work.
To let things to be as easier as possible on each host :

• host_not_virtual_1 : is the host where the vm is at the beginning, I means before the live migration
• host_not_virtual_2 : is the host where the vm is at the end, I means after the live migration
• host_storage_area : is the host running the storage area, it doesn't known anything about the xen or live migration, it only has the data.
• host_virtual : is the virtual machine that we want to migrate from one computer to another.

To summarize these things, we want to migrate host_virtual from host_not_virtual_1 to host_not_virtual_2.
I did choose to share the xen vm on a iSCSI device. The procedure to do it on a AoE device, will be mostly the same. So, in my configuration to do this live migration, there is three host, between two of them we will do the live migration, and the third host run the storage area (iSCSI storage area).
First, be sure to install properly Xen on your system, and set up some option, I explain how to do that [here].
I did suppose too that they were already a vm running on :

• host_not_virtual_1

So, here are the step that I did follow to do my live migration, I was inspired by the external links provided at the end of this web page.

Contents [show] 1 Enable xen to do a live migration 2 Create a iSCSI device to be available on the network 3 Enable iSCSI storage on the first host running the live migration 4 Put the vm data on this iSCSI storage area 5 Configure the first host 6 Configure the second host 7 Do the live migration 8 External links

Enable xen to do a live migration

With a default install of xen on a debian Linux, you won't be able to do a live migration. You have to first set some options in the main configuration file of xen :

/etc/xen/xend-config.sxp

You can find this 4 lines :

# (xend-relocation-hosts-allow )
# (xend-relocation-port 8002)
# (xend-relocation-address )
# (xend-relocation-server yes)

they are commented, with a :

#

in front of them. You have to erase the :

#

character in from of them, to uncomment the lines. This enable [xen] to do a live migration. So, the last 4 lines will become :

(xend-relocation-hosts-allow )
(xend-relocation-port 8002)
(xend-relocation-address )
(xend-relocation-server yes)

You now have to restart the xen daemon :

root@host_not_virtual_1 # xend restart

Create a iSCSI device to be available on the network

On the host :

• host_storage_area

configure an iSCSI disc to put the data of the vm. I used a debian Linux to do this. You can find [here] how to set up the server and client side. This boxe doesn't need to known anything about xen and live migration, it only store the data, that's all !

Enable iSCSI storage on the first host running the live migration

Let's use this iSCSI device on

• host_not_virtual_1.

Follow this [link] if you still don't known how to attach an iSCSI device on your host.
On the system, let's say that the iSCSI device got the device block :

/dev/sda

I first had to configure this new block device on the system, let's use :

fdisk

and create one primary partition to be used by LVM, so type number of :

8e

Initialise the partition to be used with LVM :

root@host_not_virtual_1 # pvcreate /dev/sda1

then create the volume group :

root@host_not_virtual_1 # vgcreate vg_xen /dev/sda1

create the logical volume needed for the virutal machine, 3GO will be enought for me, and don't forget to create also a swap device :

root@host_not_virtual_1 # lvcreate -L 3G -n lvm-xen vg_xen
root@host_not_virtual_1 # lvcreate -L 1G -n lvm-xen-swap vg_xen

create the file system and enable the swap device :

root@host_not_virtual_1 # mkfs.ext3 /dev/vg_xen/lvm-xen
root@host_not_virtual_1 # mkswap /dev/vg_xen/lvm-xen-swap
root@host_not_virtual_1 # swapon /dev/vg_xen/lvm-xen-swap

Put the vm data on this iSCSI storage area

Work related to block device and file system are finished at this point.
Let's now mount the partition where the data of VM host_virtual was on our local disk :

root@host_not_virtual_1 # mount /dev/disk-1/debian-base-xen /mnt/old-vm

Note that is important that the vm is not actively used. So, shutdown the VM if is currently used or do a snapshot if it is used and on a LVM. We will now mount the iSCSI volume :

root@host_not_virtual_1 # mount /dev/vg_xen/lvm-xen /mnt/new-vm

And now, copy all the the data of vm from one place to another place :

root@host_not_virtual_1 # cp -ax /mnt/old-vm/* /mnt/new-vm/

When this is done, you can launch your virtual machine through your local disk or through your iSCSI device. You only have to change the relevant line of your vm configuration file, so related to disk parameter.

Configure the first host

You have to edit a line within the xen configuration file of this vm, the line that looks like that one :

disk   =       ['phy:/dev/disk-1/debian-base-xen,sda1,w', 'phy:/dev/disk-1/debian-base-xen-swap,sda2,w']

As the data of your vm are also place now on (we did a copy at the [last step] ) :

/dev/vg_xen/lvm-xen

and the swap device on :

/dev/vg_xen/lvm-xen-swap

change the line to this one :

disk   =       ['phy:/dev/vg_xen/lvm-xen,sda1,w', 'phy:/dev/vg_xen/lvm-xen-swap,sda2,w']

Now, you can launch your vm on host_not_virtual_1, there won't be any difference, except that the vm host_virtual data are on a shared storage, but you won't notice anything.

Configure the second host

The configuration xen file for host_virtual is :

/etc/xen/debian-base-xen.cfg

Let's now put the configuration file done on __host_not_virtual_1__ on the other host that will run the live migration ( host_not_virtual_2 ) :

root@host_not_virtual_1 # scp /etc/xen/debian-base-xen.cfg root@host_not_virtual_2:/etc/xen/

Let's do some work on this other host ( host_not_virtual_2 ). We first had to add/attach the iSCSI device available for him, you can find [here] how you can do it. I suppose that the new iSCSI device on host_not_virtual_2 is accessible through the block device :

/dev/sdb

At this point host_not_virtual_2 still don't known that :

/dev/sdb

has a logical volume on it, 2 step needs to be done :

root@host_not_virtual_2 # vgscan

and

root@host_not_virtual_2 # vgchange -a y

Now, if you do on host_not_virtual_host_2 a :

root@host_not_virtual_2 # vgdisplay vg_xen

you will see the logical volume that we did configure on host_not_virtual_1.

Do the live migration

And finally, you can now enjoy doing a live migration. First, find the vm id of the vm that you want to migrate from :

host_not_virtual_1

to

host_not_virtual_2

This can be done with :

# xm list

Let's say that the vm id was :

2

I did this command to migrate the vm :

root@host_not_virtual_1 # xm migrate --live 2 host_not_virtual_2

And after a moment, you won't see anymore this vm on :

host_not_virtual_1

but actually, it will be on :

host_not_virtual_2