Cloning the OS under encrypted LVM to a smaller disk

In workflows, cloning Linux hosts has become commonplace for me. But one day I had to clone a server with LVM and LUKS encryption to a smaller disk. And it turned out not to be so simple.

Please note, this method worked for me exclusively on iron servers. This did not work on KVM with qcow2 disks, the system after unlocking LUKS fell into initframfs with errors, which is missing /bin And /sbin. These are the consequences of cloning with errors, you may need to use other tools, for example dd.

Errors I encountered on virtual machines

Given:

  • Debian Server debian-src with encrypted LVM on disk sda 20GB, which is occupied by ~ 3GB.

  • blank disk sdb 15GB to which you want to transfer debian-src

What’s wrong with encrypted LVM?

At the very beginning, I didn’t even think that something could go wrong, so I just booted from Clonezilla, indicated the flag through the Expert mode -icds, which skips checking the size of the destination drive. This allows you to clone a large disk to a smaller disk, as long as the larger disk uses no more space than the smaller disk.

But at the end of cloning, I ran into an error of insufficient space on the destination disk:

And then it became clear: LVM does not allow you to simply clone a disk, it needs to be reduced, the presence of LUKS encryption added complexity.

How to shrink encrypted LVM?

First, boot from the Live-CD to debian-src (I had Linux Mint on hand)

  1. Deciphering LUKS /dev/sda5give it a name enc and display information on PV (physical volumes) and LV (logical volumes)

root@mint:~# cryptsetup open /dev/sda5 enc
Enter passphrase for /dev/sda5: 

root@mint:~# pvs
  PV              VG            Fmt  Attr PSize  PFree
  /dev/mapper/enc debian-src-vg lvm2 a--  19.50g 44.00m

root@mint:~# lvs
  LV     VG            Attr       LSize
  root   debian-src-vg -wi-ao---- <18.51g
  swap_1 debian-src-vg -wi-a----- 976.00m

root@mint:~# df -h /dev/mapper/debian--src--vg-root
Filesystem                        Size  Used Avail Use%
/dev/mapper/debian--src--vg-root   19G  1.3G   16G   8%

We see that the PV is completely filled, and 16GB is free in the LV of the root, it is necessary to reduce the PV.

  1. Starting to decrease the volume of LV debian--src--vg-root. We will reduce everything by 10GB.

root@mint:~# lvresize --resize -y -L -10G /dev/debian-src-vg/root
Do you want to unmount "/media/mint/39551c0b-2da8-4303-8de5-65e7d6c552e9" ? [Y|n] y
fsck from util-linux 2.34
resize2fs 1.45.5 (07-Jan-2020)
Resizing the filesystem on /dev/mapper/debian--src--vg-root to 2230272 (4k) blocks.
The filesystem on /dev/mapper/debian--src--vg-root is now 2230272 (4k) blocks long.

  Size of logical volume debian-src-vg/root changed from <18.51 GiB (4738 extents) to <8.51 GiB (2178 extents).
  Logical volume debian-src-vg/root successfully resized.

Key -y allows to unmount LV before resizing

  1. If you have a swap, then after step 2 we will have free space between LV root and swap. Let’s remove it.

root@mint:~# pvs -v --segments
  PV              VG            Fmt  Attr PSize  PFree  Start SSize LV     Start Type   PE Ranges                
  /dev/mapper/enc debian-src-vg lvm2 a--  19.50g 10.04g     0  2178 root       0 linear /dev/mapper/enc:0-2177   
  /dev/mapper/enc debian-src-vg lvm2 a--  19.50g 10.04g  2178  2560            0 free                            
  /dev/mapper/enc debian-src-vg lvm2 a--  19.50g 10.04g  4738   244 swap_1     0 linear /dev/mapper/enc:4738-4981
  /dev/mapper/enc debian-src-vg lvm2 a--  19.50g 10.04g  4982    11            0 free                            

root@mint:~# pvmove --alloc anywhere /dev/mapper/enc:4738-4981
  /dev/mapper/enc: Moved: 2.87%
  /dev/mapper/enc: Moved: 100.00%

Pay attention to PE Ranges and we see that between LV root And swap_1 there is free space, move swap /dev/mapper/enc:4738-4981 close to root.

After moving, we see that the free space has moved to the end:

root@mint:~# pvs -v --segments
  PV              VG            Fmt  Attr PSize  PFree  Start SSize LV     Start Type   PE Ranges                
  /dev/mapper/enc debian-src-vg lvm2 a--  19.50g 10.04g     0  2178 root       0 linear /dev/mapper/enc:0-2177   
  /dev/mapper/enc debian-src-vg lvm2 a--  19.50g 10.04g  2178   244 swap_1     0 linear /dev/mapper/enc:2178-2421
  /dev/mapper/enc debian-src-vg lvm2 a--  19.50g 10.04g  2422  2571            0 free

  1. We begin to reduce the LUKS partition. Let’s look at the size first:

root@mint:~# cryptsetup status enc
/dev/mapper/enc is active and is in use.
  type:    LUKS2
  cipher:  aes-xts-plain64
  keysize: 512 bits
  key location: keyring
  device:  /dev/sda5
  sector size:  512
  offset:  32768 sectors
  size:    40906752 sectors
  mode:    read/write

The value we need is size: 40906752 sectors
Let’s use the formula for subtracting the volume of a LUKS partition:
LUKS_size_sectors - значение_уменьшения_в_гб * 1024 * 1024 * 2 = значение для cryptsetup resize

I reduce LUKS by 10GB, the values ​​in my case are: 40906752-10*1024*1024*2=19935232

root@mint:~# cryptsetup resize enc -b 19935232
Enter passphrase for /dev/sda5:

We will display information about the LUKS partition, it will be useful to us later. We are interested in offset And size:

root@mint:~# cryptsetup status enc
/dev/mapper/enc is active and is in use.
  ...
  offset:  32768 sectors
  size:    19935232 sectors
  ...

  1. Deactivate VG (Volume Group) and close LUKS.

root@mint:~# vgchange -a n debian-src-vg
  WARNING: Device /dev/mapper/enc has size of 19935232 sectors which is smaller than corresponding PV size of 40906752 sectors. Was device resized?
  WARNING: One or more devices used as PVs in VG debian-src-vg have changed sizes.
  0 logical volume(s) in volume group "debian-src-vg" now active

root@mint:~# cryptsetup close enc

Note that all LVs must be unmounted otherwise you will get an error.
If you mount LUKS back after this step, you will need to repeat steps 4 and 5.

We see a warning that the LUKS partition is smaller than the PV in VG debian-src-vg. We’ll fix this a bit later.

  1. We begin to reduce the disk partition itself sda5because the root of the system is located there.

root@mint:~# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0    7:0    0  1.9G  1 loop /rofs
sda      8:0    0   20G  0 disk 
├─sda1   8:1    0  487M  0 part 
├─sda2   8:2    0    1K  0 part 
└─sda5   8:5    0 19.5G  0 part

root@mint:~# parted /dev/sda
GNU Parted 3.3
Using /dev/sda
(parted) unit s                                                           
(parted) p                                                                
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sda: 41943040s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start     End        Size       Type      File system  Flags
 1      2048s     999423s    997376s    primary   ext2         boot
 2      1001470s  41940991s  40939522s  extended
 5      1001472s  41940991s  40939520s  logical

Let’s use the formula for calculating the new partition size: NEW_PARTITION_SECTOR_END = PARTITION_SECTOR_START + (LUKS_SIZE_SECTORS + LUKS_OFFSET_SECTORS) - 1

Values ​​in my case: 1001472+(19935232+32768)-1=20969471

(parted) resizepart 5 20969471
Warning: Shrinking a partition can cause data loss, are you sure you want to continue?
Yes/No? yes                                                               
(parted) q                                                                
Information: You may need to update /etc/fstab.

  1. We mount LUKS, activate VG and see that we managed to compress the disks. But the error is that the LUKS partition is smaller than the PV in debian-src-vg left, now we will eliminate it.

root@mint:~# cryptsetup luksOpen /dev/sda5 enc
Enter passphrase for /dev/sda5: 
root@mint:~# vgchange -a y debian-src-vg
  WARNING: Device /dev/mapper/enc has size of 19935232 sectors which is smaller than corresponding PV size of 40906752 sectors. Was device resized?
  WARNING: One or more devices used as PVs in VG debian-src-vg have changed sizes.
  2 logical volume(s) in volume group "debian-src-vg" now active

root@mint:~# lsblk
NAME                         MAJ:MIN RM  SIZE RO TYPE
sda                            8:0    0   20G  0 disk  
├─sda1                         8:1    0  487M  0 part  
├─sda2                         8:2    0    1K  0 part  
└─sda5                         8:5    0  9.5G  0 part  
  └─enc                      253:0    0  9.5G  0 crypt 
    ├─debian--src--vg-root   253:1    0  8.5G  0 lvm
    └─debian--src--vg-swap_1 253:2    0  976M  0 lvm

  1. We reduce the size of the PV, and also expand the LV root as much as possible within the reduced PV.

root@mint:~# pvresize /dev/mapper/enc
  WARNING: Device /dev/mapper/enc has size of 19935232 sectors which is smaller than corresponding PV size of 40906752 sectors. Was device resized?
  WARNING: One or more devices used as PVs in VG debian-src-vg have changed sizes.
  Physical volume "/dev/mapper/enc" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized              
                                 
root@mint:~# lvresize -l +100%FREE /dev/debian-src-vg/root 
  Size of logical volume debian-src-vg/root changed from <8.51 GiB (2178 extents) to 8.55 GiB (2189 extents).
  Logical volume debian-src-vg/root successfully resized.
  
root@mint:~# e2fsck -f /dev/debian-src-vg/root
e2fsck 1.45.5 (07-Jan-2020)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/debian-src-vg/root: 37219/561936 files (1.4% non-contiguous), 408261/2230272 blocks

root@mint:~# resize2fs /dev/debian-src-vg/root
resize2fs 1.45.5 (07-Jan-2020)
Resizing the filesystem on /dev/debian-src-vg/root to 2241536 (4k) blocks.
The filesystem on /dev/debian-src-vg/root is now 2241536 (4k) blocks long.

  1. Let’s see how the size of our disks, PV and LV has changed. From 19.5G we reduced the drive to 9.5G, exactly 10 gigabytes.

root@mint:~# lsblk
NAME                         MAJ:MIN RM   SIZE RO TYPE
sda                            8:0    0    20G  0 disk  
├─sda1                         8:1    0   487M  0 part  
├─sda2                         8:2    0     1K  0 part  
└─sda5                         8:5    0   9.5G  0 part  
  └─enc                      253:0    0   9.5G  0 crypt 
    ├─debian--src--vg-root   253:1    0   8.6G  0 lvm   
    └─debian--src--vg-swap_1 253:2    0   976M  0 lvm   

root@mint:~# pvs
  PV              VG            Fmt  Attr PSize PFree
  /dev/mapper/enc debian-src-vg lvm2 a--  9.50g    0 

root@mint:~# lvs
  LV     VG            Attr       LSize   
  root   debian-src-vg -wi-a-----   8.55g
  swap_1 debian-src-vg -wi-a----- 976.00m

  1. On this we have finished all the manipulations with the dimension of the disks. The only thing left is to take the Clonezilla Live CD and, as usual, clone the disk in expert mode with the flag -icds

Cloning process

  1. Boot into Clonezilla, select clone device-deviceExpert mode, disk_to_local_disk

  2. Choose our reduced sda as source disk.

  1. We put the flag -icds

Allows you to clone a large disk to a smaller disk, as long as the larger disk uses space that does not exceed the size of the smaller disk.

Allows you to clone a large disk to a smaller disk, as long as the larger disk uses space that does not exceed the size of the smaller disk.

  1. When cloning the partition table, select the -k0 flag to clone the table from the Source disk.

  2. We confirm that we want to clone to the sdb disk of our choice and wait for the cloning to finish.

Increasing disk size

After cloning, you need to expand the second disk to which we cloned the OS (in my case, this sdb)

  1. We boot into the Live CD, unlock LUKS and check the disks. We see that the partition table and the system were copied.

root@mint:~# cryptsetup open /dev/sdb5 enc
Enter passphrase for /dev/sdb5:

root@mint:~# lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE 
sda       8:0    0    20G  0 disk  
├─sda1    8:1    0   487M  0 part  
├─sda2    8:2    0     1K  0 part  
└─sda5    8:5    0   9.5G  0 part  
sdb       8:16   0    15G  0 disk  
├─sdb1    8:17   0   487M  0 part  
├─sdb2    8:18   0     1K  0 part  
└─sdb5    8:21   0   9.5G  0 part  
  └─enc 253:0    0   9.5G  0 crypt

  1. Let’s try to expand LUKS to the maximum.

root@mint:~# cryptsetup -v resize 'enc'
Enter passphrase for /dev/sdb5: 
Key slot 0 unlocked.
Command successful.

root@mint:~# lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE
...
sdb       8:16   0    15G  0 disk  
├─sdb1    8:17   0 365.3M  0 part  
├─sdb2    8:18   0     1K  0 part  
└─sdb5    8:21   0  14.7G  0 part  
  └─enc 253:0    0  14.6G  0 crypt

Now we see how sdb5 And enc expanded as far as possible

Spoiler: you won’t be able to expand via GParted – it gives an error on the command cryptsetup -v resize

  1. Let’s look at the information about PV and notice that PV enc still remained at 9.5GiB. We need to fix this.

root@mint:~# pvdisplay -m
  --- Physical volume ---
  PV Name               /dev/mapper/enc
  VG Name               debian-src-vg
  PV Size               9.50 GiB / not usable 0
...

  1. Change the PV size.

root@mint:~# pvresize /dev/mapper/enc 
  Physical volume "/dev/mapper/enc" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized

  1. We check the size of the LV, we see that the root has not yet been expanded either. Resizing the LV.

root@mint:~# lvs
  LV     VG            Attr       LSize   
  root   debian-src-vg -wi-------   8.55g                                                    
  swap_1 debian-src-vg -wi------- 976.00m

root@mint:~# lvresize -l +100%FREE /dev/debian-src-vg/root
  Size of logical volume debian-src-vg/root changed from 8.55 GiB (2189 extents) to 13.67 GiB (3500 extents).
  Logical volume debian-src-vg/root successfully resized.

  1. Checking the file system and resizing the root

root@mint:~# e2fsck -y /dev/debian-src-vg/root
...
/dev/debian-src-vg/root: ***** FILE SYSTEM WAS MODIFIED *****
/dev/debian-src-vg/root: 24777/561936 files (1.5% non-contiguous), 293922/2241536 blocks

root@mint:~# resize2fs /dev/debian-src-vg/root
resize2fs 1.45.5 (07-Jan-2020)
Resizing the filesystem on /dev/debian-src-vg/root to 3584000 (4k) blocks.
The filesystem on /dev/debian-src-vg/root is now 3584000 (4k) blocks long.

  1. Check disk size, PV and LV

root@mint:~# lsblk
NAME                         MAJ:MIN RM   SIZE RO TYPE
...
sdb                            8:16   0    15G  0 disk  
├─sdb1                         8:17   0 365.3M  0 part  
├─sdb2                         8:18   0     1K  0 part  
└─sdb5                         8:21   0  14.7G  0 part  
  └─enc                      253:0    0  14.6G  0 crypt 
    ├─debian--src--vg-root   253:1    0  13.7G  0 lvm   
    └─debian--src--vg-swap_1 253:2    0   976M  0 lvm   

root@mint:~# pvs
  PV              VG            Fmt  Attr PSize  PFree
  /dev/mapper/enc debian-src-vg lvm2 a--  14.62g    0 

root@mint:~# lvs
  LV     VG            Attr       LSize
  root   debian-src-vg -wi-a-----  13.67g
  swap_1 debian-src-vg -wi-a----- 976.00m

All! Now we can start from the new disk. I have a couple of servers that I copied in this way and they have been working stably for more than six months.
I decided to post this article on Habr because of the large volume, I leave smaller notes in my tg channel https://t.me/sdnv_funkhole.

Links to resources that helped me with this task:

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *