DB-유

Adding and Mounting a New Disk on Linux

· linux, storage, filesystem

I add disks often enough, a second virtual disk on a VMware guest, a cloud volume, a drive in a rack server, that I stopped trusting my memory and follow the same sequence every time: find the device, partition, format, mount, make it persistent. The last step is where it hurts. One typo in /etc/fstab and the box sits in emergency mode at the next reboot, usually when nobody's near the console.

This is the sequence on CentOS 7 and Ubuntu 16.04, using a new disk for a MySQL data directory as the example. I use XFS on CentOS 7 (its default) and ext4 on Ubuntu unless there's a reason not to.

1. Identify the new device

lsblk
fdisk -l
dmesg | grep -i 'sd[b-z]'

lsblk shows the tree of block devices. The new disk is the one with no partitions and no mount point, typically /dev/sdb on the first added disk, or /dev/vdb on KVM guests with virtio. If the disk was hot-added and does not appear, rescan the SCSI bus:

echo "- - -" > /sys/class/scsi_host/host0/scan

Repeat for host1, host2 if there are several. Double check the size matches what you expect before going any further; formatting the wrong disk is not something you can undo.

2. Partition

For disks under 2 TB, fdisk with an MBR table is fine. For anything larger, or if you simply prefer GPT, use parted. I default to parted and GPT now:

parted /dev/sdb mklabel gpt
parted -a optimal /dev/sdb mkpart primary 0% 100%
lsblk /dev/sdb

You should now see /dev/sdb1. The -a optimal flag aligns the partition to the disk's reported optimal I/O size, which matters on SSDs and on storage arrays with 4K sectors.

If you would rather skip partitioning and put the filesystem directly on /dev/sdb, that works too, but some tools (and some people) get confused by an unpartitioned disk, so I always create the one partition.

3. Filesystem

On CentOS 7 with XFS:

mkfs.xfs -L mysqldata /dev/sdb1

On Ubuntu with ext4:

mkfs.ext4 -L mysqldata /dev/sdb1

The -L label is optional but useful: you can mount by label later, and lsblk -f shows it. For ext4 on a disk that will hold a small number of very large files (database files, VM images), consider -T largefile4 to reduce the inode count and speed up mkfs.

4. Mount it and copy the data

mkdir -p /var/lib/mysql-new
mount /dev/sdb1 /var/lib/mysql-new
df -h /var/lib/mysql-new

Moving an existing data directory onto the new disk means stopping the service, copying while preserving ownership and permissions, then swapping the mount:

systemctl stop mariadb
rsync -avP /var/lib/mysql/ /var/lib/mysql-new/
umount /var/lib/mysql-new
mv /var/lib/mysql /var/lib/mysql.old
mkdir /var/lib/mysql
mount /dev/sdb1 /var/lib/mysql

On CentOS 7 with SELinux enforcing, the copied files carry the right context because rsync preserved it, but the new mount point directory itself needs it too:

restorecon -Rv /var/lib/mysql

Then systemctl start mariadb and check the log. Once you are satisfied, remove /var/lib/mysql.old.

5. fstab

I always use the UUID, never /dev/sdb1. Device names can change when disks are added or removed, or when the hypervisor enumerates them in a different order.

blkid /dev/sdb1

Copy the UUID="..." value into /etc/fstab:

UUID=3f2b9c1e-8a4d-4b7e-9f0a-1c2d3e4f5a6b  /var/lib/mysql  xfs  defaults,noatime  0 0

For ext4 the fifth and sixth fields would normally be 0 2 so fsck checks it at boot. noatime stops the kernel updating the access timestamp on every read, which is pointless overhead for a database directory. If losing the disk should not block boot (for example a backup volume), add nofail to the options.

Test fstab before you reboot

The critical test: unmount and remount using only fstab, which is exactly what will happen at boot.

umount /var/lib/mysql
mount -a
df -h /var/lib/mysql
findmnt /var/lib/mysql

If mount -a prints nothing and df shows the new disk, fstab is correct. Only then reboot, if you want to be thorough. findmnt --verify (util-linux 2.30 and later, so available on newer distributions but not on CentOS 7) checks the whole fstab for problems without mounting anything.

Problems

  • mount: wrong fs type, bad option, bad superblock on /dev/sdb1: usually a typo in the filesystem type in fstab (ext4 written as ext3, or xfs on a disk you formatted ext4). blkid shows the actual TYPE.
  • Server drops to emergency mode at boot with Dependency failed for /var/lib/mysql: fstab references a UUID or device that does not exist. Log in on the console, mount -o remount,rw /, fix or comment out the line, and reboot. This is why mount -a before rebooting matters.
  • mysqld fails after the move with Can't create/write to file or Permission denied: ownership was lost (you used cp without -a) or SELinux context is wrong. ls -laZ /var/lib/mysql shows both.
  • XFS cannot be shrunk. If you may need to shrink a volume later, use ext4 or put LVM underneath it and leave free space in the volume group.

If I were starting fresh, I'd put LVM under it. pvcreate, vgcreate and lvcreate take a few minutes more, and lvextend -r beats repeating the copy dance when the next disk arrives.