XEN: How to remove an old-school EXT3 Storage Repository and create a local LVM based SR with RAID 1

With newer XenServer Version (> 5.x) you cannot longer use the old EXT3 based local storage repositories to store your virtual disks. Citrix has introduced LVM based storage repositories instead.

Caution: Make a backup before executing the following steps, otherwise you’ll lose all your data on the disk!

I assume, we already have a configured RAID1 mirror with a XSLocalEXT storage repository due to a XenServer upgrade. If you want to create a new software mirror with two new installed local disks, you have to create a new md-device with mdadm (this is not part of this documentation).

First, identify the status and devices of the mirror

If the RAID is in “rebuild” status, wait until it’s in “active” status.

[root@xenserver ~]# cat /proc/mdstat
Personalities : [raid1]
md126 : active raid1 sda[1] sdb[0]
488383488 blocks super external:/md127/0 [2/2] [UU]

md127 : inactive sdb[1](S) sda[0](S)
5928 blocks super external:imsm
unused devices: <none>

Get more details about the Software Mirror

mdadm --detail /dev/md126
mdadm --examine --brief --scan  --config=partitions

Identify and remove old logical volume from RAID1

[root@xenserver ~]# lvscan
  ACTIVE            '/dev/VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e/MGT' [4.00 MiB] inherit
  ACTIVE            '/dev/VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e/VHD-9ac571bf-5809-4f1e-a767-9fcb0c1d0b88' [19.57 GiB] inherit
  ACTIVE            '/dev/VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e/VHD-a5b9ff3b-3bf2-4ea8-a53b-7fc086f4b4ab' [80.16 GiB] inherit
  ACTIVE            '/dev/VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e/VHD-186f9092-9d98-410c-8839-024172512a36' [78.31 GiB] inherit
  ACTIVE            '/dev/VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e/VHD-6fb6374a-1b83-48c6-bd26-277f19dcf4a8' [34.78 GiB] inherit
  inactive          '/dev/VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e/VHD-940e675e-aa13-4aa1-b694-41b6516b7e28' [48.93 GiB] inherit
  inactive          '/dev/VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e/VHD-60d3bd98-f81d-412b-86f6-d1d65ae6c7d2' [34.78 GiB] inherit
  ACTIVE            '/dev/VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e/VHD-ac10a162-e39e-4e61-a40c-3221413da002' [65.13 GiB] inherit
  inactive          '/dev/XSLocalEXT-6123fd9a-b126-2b74-a476-5a120175a1d9/6123fd9a-b126-2b74-a476-5a120175a1d9' [465.75 GiB] inherit

We want to remove the device /dev/XSLocalEXT …

First deactivate the LV if not already done

[root@xenserver ~]# lvchange -a n /dev/XSLocalEXT-6123fd9a-b126-2b74-a476-5a120175a1d9/6123fd9a-b126-2b74-a476-5a120175a1d9

Now you can remove the LV. You must add the metadata_read_only option, because Xenserver sets a read-only flag.

lvremove /dev/XSLocalEXT-6123fd9a-b126-2b74-a476-5a120175a1d9/6123fd9a-b126-2b74-a476-5a120175a1d9 --config global{metadata_read_only=0}

Remove also the empty volume group

[root@xenserver ~]# vgs

VG #PV #LV #SN Attr VSize VFree
VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e 1 8 0 wz--n- 890.00g 528.33g
XSLocalEXT-6123fd9a-b126-2b74-a476-5a120175a1d9 1 0 0 wz--n- 465.75g 465.75g

vgremove XSLocalEXT-6123fd9a-b126-2b74-a476-5a120175a1d9 --config global{metadata_read_only=0}

Check also, that the physical volume has no more corresponding logical volume

[root@xenserver ~]# pvscan
  PV /dev/sdd3      VG XSLocalEXT-fa330a8c-3f42-42f5-9935-3e4b40c03be3      lvm2 [457.75 GiB / 0    free]
  PV /dev/sdc3      VG VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e   lvm2 [890.00 GiB / 528.33 GiB free]
  PV /dev/md126p1                                                           lvm2 [465.76 GiB]
  Total: 3 [1.77 TiB] / in use: 2 [1.32 TiB] / in no VG: 1 [465.76 GiB]

[root@xenserver ~]
# pvs PV VG Fmt Attr PSize PFree /dev/md126p1 lvm2 --- 465.76g 465.76g /dev/sdc3 VG_XenStorage-04cd13e0-8237-7754-2d66-4a6a10c5137e lvm2 a-- 890.00g 528.33g /dev/sdd3 XSLocalEXT-fa330a8c-3f42-42f5-9935-3e4b40c03be3 lvm2 a-- 457.75g 0

Reusing the local RAID1 by creating a new local storage repository with type=lvm

First, identify the MD-Partition with it’s uuid

ll /dev/disk/by-id/

lrwxrwxrwx 1 root root 13 Dec 29 09:26 md-uuid-27052c74:af76a037:e50f003e:65d1ffb8-part1 -> ../../md126p1

Then create the new LVM based storage repository

xe sr-create type=lvm content-type=user device-config:device=/dev/disk/by-id/md-uuid-27052c74:af76a037:e50f003e:65d1ffb8-part1 name-label="Local Disk RAID1"

Check, that the SR was successfully created

[root@xenserver ~]# lsblk
NAME                                                                                                   MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                                                                                                      8:0    0 465.8G  0 disk
└─md126                                                                                                  9:126  0 465.8G  0 raid1
  └─md126p1                                                                                            259:0    0 465.8G  0 md
    └─VG_XenStorage--d5e61586--e747--dac1--f971--29281873a18c-MGT                                      253:2    0     4M  0 lvm
sdb                                                                                                      8:16   0 465.8G  0 disk
└─md126                                                                                                  9:126  0 465.8G  0 raid1
  └─md126p1                                                                                            259:0    0 465.8G  0 md
    └─VG_XenStorage--d5e61586--e747--dac1--f971--29281873a18c-MGT                                      253:2    0     4M  0 lvm

You can now use the new created SR in Xencenter to store your VMs.