DB-유

NFS Server and Client Setup on Linux

· linux, nfs, storage

Every time two web nodes need the same uploads directory, I end up back at NFS. Maybe there are fancier answers, but it's in the kernel, needs almost nothing on the client, and NFSv4 got rid of most of the portmapper firewall headaches of the old days.

I'll set up an NFSv4 server on CentOS 7 and mount it from an Ubuntu 18.04 client, with 10.0.0.10 as the server and 10.0.0.20 as the client. One warning up front: NFS without Kerberos has no real authentication, it trusts client IPs and UIDs. Keep it on a private subnet or behind a VPN.

Server install

On CentOS 7:

sudo yum install -y nfs-utils
sudo systemctl enable --now nfs-server

On an Ubuntu server the package is nfs-kernel-server and the unit is the same name.

The export directory

Decide who owns it before anything else. For a shared upload directory I create a group with a fixed GID on both sides so ownership maps cleanly (mismatched UIDs have cost me an afternoon before):

sudo groupadd -g 5000 webdata
sudo mkdir -p /srv/nfs/uploads
sudo chown root:webdata /srv/nfs/uploads
sudo chmod 2775 /srv/nfs/uploads

The setgid bit (the leading 2) makes files created in the directory belong to webdata regardless of the creating user's primary group.

/etc/exports

sudo nano /etc/exports

/srv/nfs/uploads  10.0.0.0/24(rw,sync,no_subtree_check,root_squash)
/srv/nfs/backups  10.0.0.20(ro,sync,no_subtree_check,all_squash,anonuid=65534,anongid=65534)

The options that matter:

  • rw / ro: read-write or read-only for that client spec.
  • sync: the server acknowledges writes only after committing them. async is faster and unsafe on a crash.
  • no_subtree_check: disables a check that causes problems when files are renamed; the exportfs man page recommends it.
  • root_squash: root on the client becomes nfsnobody on the server. This is the default, and I leave it alone unless I have a specific reason.
  • all_squash: every client user is mapped to anonuid/anongid, useful for read-only shares.

Apply without restarting:

sudo exportfs -rav
sudo exportfs -s

Firewall

NFSv4 needs only TCP 2049. If you must support NFSv3 clients, mountd and rpcbind are needed too. On CentOS 7 with firewalld:

sudo firewall-cmd --permanent --add-service=nfs
sudo firewall-cmd --permanent --add-service=rpc-bind
sudo firewall-cmd --permanent --add-service=mountd
sudo firewall-cmd --reload

For a v4-only setup you can limit that to --add-service=nfs. To disable v3 entirely on the server, set RPCNFSDARGS="-N 3" in /etc/sysconfig/nfs and restart nfs-server.

Client mount

On Ubuntu 18.04:

sudo apt install -y nfs-common
sudo mkdir -p /var/www/uploads
sudo mount -t nfs4 10.0.0.10:/srv/nfs/uploads /var/www/uploads

Note that with NFSv4 the path is still the full server path in this configuration, because CentOS 7 exports a pseudo-root at / by default. To make it permanent, add to /etc/fstab:

10.0.0.10:/srv/nfs/uploads  /var/www/uploads  nfs4  defaults,_netdev,noatime,nofail  0  0

_netdev tells systemd to wait for the network, and nofail prevents a missing NFS server from blocking boot. Test the fstab entry with sudo umount /var/www/uploads && sudo mount -a.

Create the same group on the client and add the web server user to it:

sudo groupadd -g 5000 webdata
sudo usermod -aG webdata www-data

Did it work?

On the client:

mount | grep nfs4
df -h /var/www/uploads
sudo -u www-data touch /var/www/uploads/test.txt
ls -ln /var/www/uploads/test.txt

The file should be created and ls -ln should show GID 5000. On the server, ls -l /srv/nfs/uploads shows the same file with group webdata. nfsstat -c on the client and nfsstat -s on the server show operation counters increasing.

Gotchas

  • mount.nfs4: access denied by server while mounting 10.0.0.10:/srv/nfs/uploads. The client address does not match any entry in /etc/exports, or you forgot exportfs -ra. exportfs -v on the server shows what is actually exported and to whom.
  • mount.nfs4: Connection timed out. Port 2049 is blocked. nc -zv 10.0.0.10 2049 from the client; if that fails, it is the firewall.
  • Files show up owned by nobody:nogroup on the client. UIDs and GIDs differ between the hosts, or all_squash is in effect. Either align the numeric IDs as done with the webdata group above, or accept the squashing for read-only shares.
  • Client hangs on ls when the server is down. That is NFS's default hard mount behaviour, which protects against data loss. Add soft,timeo=50,retrans=3 to the fstab options for non-critical shares where an I/O error is preferable to a hang.

Always set nofail in fstab. A storage hiccup should never turn into an unbootable web server.