Skip to content
Fragmented Development

Creating an encrypted RAID-5 array with MDADM, LUKS and LVM

I run a 12TB RAID 5 array comprised of three 6TB hard drives. My RAID setup is defined in software, not hardware - which has served me well, as it has been migrated between three installs and two separate sets of server hardware. The raw drives have been added to a MDADM software RAID array, which is encrypted with LUKS, and partitions are managed by LVM.

Convoluted? Yes. Performant? No. A first draft that taught me a large amount about software RAID on Linux, and has served me well? Mostly.

The main issue is, I created this array several years ago, and don't remember a lot about how I set it up. Luckily, I wrote all of the relevant commands down in a shell script, which I will post here for posterity and preservation.

This is not how I would do this again. It is, however, an interesting historical document that I will build on the next time I have the opportunity to build a RAID array. Judging by SSD prices, I am not holding my breath.


#!/usr/bin/env bash

# Create partition information. fdisk commands were:
# g - convert disk to GPT
# n - new partition
# Return - 1st partition
# Return - Start at the beginning of the disk
# -100M - leave 100M wiggle room at the end of the disk
# t - Change partition type
# 29 - Partition type should be "Linux RAID"
# w - write changes and exit
sudo fdisk /dev/sdb; sudo fdisk /dev/sdc; sudo fdisk /dev/sdd

# Create the actual array. Will start in a degraded state, and rebuild. SLOWLY.
sudo mdadm --create /dev/md0 --level=5 -n 3 /dev/sdb1 /dev/sdc1 /dev/sdd1

# Copy details from this command into /dev/mdadm/mdadm.conf
sudo mdadm --detail --scan

# View status of array, and rebuild percentage
sudo mdadm --detail /dev/md0

# Write encrypted data to that bad boy
pv /dev/urandom | sudo dd of=/dev/md0 bs=512K

# Encrypt the new logical volume, and mount it
sudo cryptsetup --verbose -c twofish-xts-plain64 -h whirlpool --key-size 512 \
	luksFormat /dev/md0
sudo cryptsetup open /dev/md0 raid_crypt

# Initialize the raid array for LVM use
sudo pvcreate /dev/mapper/raid_crypt
sudo vgcreate raid-vg /dev/mapper/raid_crypt

# Create the logical volumes
sudo lvcreate -l 10%VG -n home raid-vg /dev/mapper/raid_crypt
sudo lvcreate -l 10%VG -n var raid-vg /dev/mapper/raid_crypt
sudo lvcreate -l 80%VG -n srv raid-vg /dev/mapper/raid_crypt

# Create the ext4 filesystems
sudo mkfs.ext4 /dev/raid-vg/home
sudo mkfs.ext4 /dev/raid-vg/srv
sudo mkfs.ext4 /dev/raid-vg/var

# Voilla! Now you can move everything to those volumes from a live CD, and
# move the old directory to a new location... just in case!
cp -apx /[old-dir] /[new-device]
mv /[old-dir] /[old-dir].old

# Generate a random keyfile for unlocking drives
sudo dd bs=4M count=1 if=/dev/random \
	of=/etc/cryptsetup-initramfs/raid-encryption.key
sudo chmod 600 /etc/cryptsetup-initramfs/raid-encryption.key

# Add key file to keyslot on RAID volume
sudo cryptsetup luksAddKey /dev/md0 \
	/etc/cryptsetup-initramfs/raid-encryption.key

# Get the RAID uuid
sudo blkid | grep 'md0'

# Automatically unlock + mount RAID on boot
echo 'raid_crypt UUID=[ UUID of the luks2 partition ] /etc/cryptsetup-initramfs/raid-encryption.key luks,discard' | \
	sudo tee -a /etc/crypttab

# Add lines for each partition
echo '/dev/mapper/raid_crypt--vg-home	/home	ext4	errors=remount-ro	0	2' | sudo tee -a /etc/fstab
echo '/dev/mapper/raid_crypt--vg-var	/var	ext4	errors=remount-ro	0	2' | sudo tee -a /etc/fstab
echo '/dev/mapper/raid_crypt--vg-srv	/srv	ext4	errors=remount-ro	0	2' | sudo tee -a /etc/fstab

Tags: linux


Add Your Comment