Setting up local synchronous replication using DRDB9
DRBD (Distributed Replicated Block Device) is a distributed, flexible and universally replicated storage solution for Linux.
DRBD has a superficial similarity to RAID-1 in that it includes a copy of the data on two storage devices so that in the event of a failure, the data on the other can be used. However, it works very differently than RAID and even network RAID.
In this article, I will share my experience on how to create drbd replication locally, not using drbdadm, but using drbdsetup and drbdmeta.
-
drbdadm: DRBD high level administration tool. It is used over drbdsetup and drbdmeta
-
drbdsetup: A lower-level administration tool for connecting DRBD devices to their backup devices, configuring pairs of DRBD devices to reflect their backup devices, and for checking the configuration of running DRBD devices.
-
drbdmeta: metadata management tool.
Test environment
Ubuntu 18.04, kernel version 4.19 and higher
Step 1. Installing and downloading the module
Vanilla kernels contain only the eighth version of DRBD, it is advisable to rebuild the kernels without this module, and install the ninth version from the repository.
Adding repositories to your sources.list
deb http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu bionic main
deb-src http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu bionic main
Update cache and install drbd packages
apt update
apt install drbd-dkms drbd-utils
Checking the module version and loading it with parameters
modinfo drbd
modprobe drbd usermode_helper=/bin/true minor_count=36
Step 2Setting up global_common.conf
I use the following config parameters, all the rest can be set when configuring the resource
# DRBD is the result of over a decade of development by LINBIT.
# In case you need professional services for DRBD or have
# feature requests visit http://www.linbit.com
global {
usage-count no;
}
common {
disk {
c-delay-target 50;
c-fill-target 0;
}
net {
csums-alg hash-alg;
}
}
You can read more about all the options here
Step 3Configure the DRBD9 Resource
What will be required:
-
Two block devices (source and destination)
-
Two logical volumes for metadata (I used 2 GB each)
Process of creation:
-
Create two resources
drbdsetup new-resource drbdres0 0
drbdsetup new-resource drbdres1 1
where:
drbdres0 and drbdres1 – resource name
0 and 1 – peer node id
-
Create drbd devices
drbdsetup new-minor drbdres0 0 0
drbdsetup new-minor drbdres1 1 0
where:
drbdres0 and drbdres1 – resources created earlier
the second argument (minor) is the number of the created device, i.e. drbd0, drbd1
the third argument is the volume number
-
Creating metadata on volumes for metadata
drbdmeta --force 0 v09 '/dev/zd0' flex-external create-md 1
drbdmeta --force 1 v09 '/dev/zd16' flex-external create-md 1
-
Add disks to previously created drbd devices
drbdsetup attach 0 '/dev/zd32' '/dev/zd0' flexible --disk-flushes=no --disk-barrier=no --al-extents=3389
drbdsetup attach 1 '/dev/zd64' '/dev/zd16' flexible --disk-flushes=no --disk-barrier=no --al-extents=3389
where:
the first argument (0 and 1) is the number of the created device (minor)
‘/dev/zd32’ and ‘/dev/zd64’ – source and destination
‘/dev/zd0’ and ‘/dev/zd16’ are previously created volumes for metadata and then parameters follow
-
We create new peers
drbdsetup new-peer drbdres0 1 --protocol=C --max-buffers=36K --max-epoch-size=36K --sndbuf-size=1024K --rcvbuf-size=2048K
drbdsetup new-peer drbdres1 0 --protocol=C --max-buffers=36K --max-epoch-size=36K --sndbuf-size=1024K --rcvbuf-size=2048K
where:
drbdres0 and drbdres1 – previously created resources
the second parameter (1 and 0) is the peer node id followed by the connection parameters
-
Create connection endpoints
drbdsetup new-path drbdres0 1 127.0.0.1:7788 127.0.0.1:7789
drbdsetup new-path drbdres1 0 127.0.0.1:7789 127.0.0.1:7788
Note: If you want to create even more resources locally, you will need to use other ports. for example: 127.0.0.1:7790 127.0.0.1:7791 and so on
-
Adding Options
drbdsetup peer-device-options drbdres1 0 0 --c-plan-ahead=0 --resync-rate=80M
drbdsetup peer-device-options drbdres0 1 0 --c-plan-ahead=0 --resync-rate=80M
resync-rate is the maximum sync rate. Keep in mind that if you set the speed to high, then drbd will use the entire resource of the disk or logical volume, and if some application is running on it, then it will not get access to the disk until the synchronization is completed.
-
Making the role primary for the first resource
drbdsetup primary drbdres0 --force
-
Making a connection
drbdsetup connect drbdres0 1
drbdsetup connect drbdres1 0
After the performed actions when displaying the status
drbdsetup status -v
synchronization should start, after the end of synchronization, both resources should have UpToDate. You can read more about statuses here in the “Theory” section. Official site with documentation. Further, it is necessary to use newly created devices drbd0 / drbd1 for work, depending on which primary.