Sharing files between servers
Two machines that need the same files is a common problem with several answers, and the obvious one has sharp edges.
Consider object storage first
If the files are uploads, media, backups or anything addressed by name rather than by path, an S3-compatible store is the better fit: no locking, no hanging mounts, and it scales past one machine. See S3-compatible object storage on your own server.
Use a shared filesystem when software genuinely requires a POSIX path — many legacy applications and most control panels do.
NFS, done carefully
On the server, in /etc/exports:
/srv/share 10.0.0.0/24(rw,sync,no_subtree_check)
Export only to your private network, never to a public address. NFSv3 has effectively no authentication: whoever can reach the port is trusted. See connecting your own servers.
On the client:
10.0.0.5:/srv/share /mnt/share nfs4
rw,hard,intr,timeo=600,retrans=2,_netdev 0 0
The mount options that matter
- hard vs soft
hardretries forever if the server disappears — processes block but data is never silently lost.softreturns an error, which corrupts applications that do not check. Usehard, and accept that an NFS server going away hangs its clients.- _netdev
- Tells systemd this mount needs the network. Without it, boot can stall waiting for a filesystem that cannot exist yet.
- sync vs async
- On the server,
asyncis much faster and can lose recent writes on a crash. Decide deliberately.
The failure to plan for
When the NFS server stops, every client with a hard mount blocks, and
processes pile up in uninterruptible sleep where even kill -9 does
nothing. Your load average climbs into the hundreds while the CPUs sit idle.
That is NFS behaving correctly. The lesson is that the NFS server becomes a single point for everything mounting it — plan for that, or do not centralise something critical on it. See high availability, honestly.
User ids must match
NFS maps by numeric id, not by name. If www-data is 33 on one machine
and 1001 on another, permissions will be wrong in ways that look like madness. Keep
ids consistent, which is an argument for
configuration management.
Alternatives worth knowing
- SSHFS — trivial to set up, slow. Fine for occasional access.
- GlusterFS or Ceph — replicated and distributed. Powerful, and real operational work.
- rsync on a timer — not shared at all, but for read-mostly data that tolerates a delay it removes the whole class of problem.
Use jumbo frames on the storage network if both ends agree — see jumbo frames.
Still stuck? Mail support@novogara.com — an engineer answers, at any hour. Back to the knowledge base