tooolbox/blog/Diagnosing NFS Mount Failures on Linux
troubleshooting2025-05-25Β·9Β·52/348

Diagnosing NFS Mount Failures on Linux

Comprehensive guide to diagnosing and resolving NFS mount failures on Linux, including network issues, permission errors, and configuration problems.

Diagnosing NFS Mount Failures on Linux

NFS (Network File System) is widely used for shared storage, but mount failures can be complex to diagnose. When my data processing cluster couldn't mount the shared NFS volume, I had to systematically check network, server, and client configurations.

Environment

  • Client: Ubuntu 22.04 LTS
  • Server: Ubuntu 22.04 LTS
  • NFS version: NFSv4
  • Network: Same subnet (192.168.1.0/24)
# Client
$ mount.nfs --version
mount.nfs: (util-linux 2.37.2)

# Server
$ sudo dpkg -l | grep nfs-kernel-server
ii  nfs-kernel-server    1.3.3-2build2    amd64    NFS server kernel module and tools

The Problem

$ sudo mount -t nfs4 192.168.1.100:/exports/data /mnt/nfs
mount.nfs: Connection timed out

# Alternative with v3
$ sudo mount -t nfs 192.168.1.100:/exports/data /mnt/nfs
mount.nfs: access denied by server while mounting 192.168.1.100:/exports/data

Analysis

Step 1: Check NFS Server Status

# On the NFS server
$ sudo systemctl status nfs-kernel-server
● nfs-kernel-server.service - NFS server and services
     Loaded: loaded (/lib/systemd/system/nfs-kernel-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2025-05-25 08:00:00 WET; 2h ago

# Check running NFS services
$ sudo rpcinfo -p localhost
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100005    3   tcp  20048  mountd
    100003    4   tcp   2049  nfs
    100003    4   udp   2049  nfs

Step 2: Check Exports Configuration

# Check exported directories
$ sudo cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
#       to NFS clients.  See exports(5).
/exports/data 192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)

# Verify exports are loaded
$ sudo exportfs -v
/exports/data
        192.168.1.0/24(rw,sync,wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check)

Step 3: Test Network Connectivity

# From client
$ ping 192.168.1.100
PING 192.168.1.100 (192.168.1.100) 56(84) bytes of data.
64 bytes from 192.168.1.100: icmp_seq=1 ttl=64 time=0.5 ms

# Check if NFS port is open
$ nmap -p 2049 192.168.1.100
PORT     STATE SERVICE
2049/tcp open  nfs

# Check mountd port
$ nmap -p 20048 192.168.1.100
PORT     STATE SERVICE
20048/tcp open  mountd

Step 4: Check Firewall

# On server
$ sudo iptables -L -n | grep 2049
# Should show ACCEPT rules for port 2049

# Check ufw
$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
2049/tcp                   ALLOW IN    192.168.1.0/24
20048/tcp                  ALLOW IN    192.168.1.0/24

Step 5: Check Client Configuration

# Check if client has NFS utilities
$ dpkg -l | grep nfs-common
ii  nfs-common    1.3.3-2build2    amd64    NFS support files common to client and server

# Check NFS client services
$ sudo systemctl status nfs-common
● nfs-common.service - NFS support files common to client and server
     Active: active (running) since Sat 2025-05-25 08:00:00 WET; 2h ago

Solution

Fix 1: Re-export NFS Shares

# On server - re-export after changing /etc/exports
$ sudo exportfs -ra
$ sudo exportfs -v

# Verify
$ sudo exportfs -v
/exports/data
        192.168.1.0/24(rw,sync,wdelay,hide,nocrossmnt,secure,no_root_squash,no_all_squash,no_subtree_check)

Fix 2: Fix NFS Server Firewall

# Allow NFS ports
$ sudo ufw allow 2049/tcp
$ sudo ufw allow 20048/tcp
$ sudo ufw allow 111/tcp
$ sudo ufw allow 111/udp

# Or using iptables
$ sudo iptables -A INPUT -p tcp -m multiport --dports 111,2049,20048 -j ACCEPT
$ sudo iptables -A INPUT -p udp -m multiport --dports 111,2049,20048 -j ACCEPT

Fix 3: Use NFSv4 with Fixed Port

# On server - configure NFS to use fixed ports
$ sudo nano /etc/nfs.conf
[nfsd]
port=2049

[mountd]
port=20048

# Restart NFS server
$ sudo systemctl restart nfs-kernel-server

Fix 4: Mount with Specific Options

# Mount with NFSv4
$ sudo mount -t nfs4 -o vers=4,hard,timeo=600,retrans=2 \
    192.168.1.100:/exports/data /mnt/nfs

# Mount with NFSv3
$ sudo mount -t nfs -o vers=3,tcp,nolock \
    192.168.1.100:/exports/data /mnt/nfs

# Mount with specific ports
$ sudo mount -t nfs -o port=2049,mountport=20048 \
    192.168.1.100:/exports/data /mnt/nfs

Fix 5: Fix Permissions on Server

# On server - check directory permissions
$ ls -la /exports/
drwxr-xr-x  3 root root 4096 Jun 25 10:00 data

# Ensure NFS user can access
$ sudo chmod 755 /exports/data
$ sudo chown nobody:nogroup /exports/data

Make NFS Mount Persistent

# Add to /etc/fstab
$ echo "192.168.1.100:/exports/data /mnt/nfs nfs4 defaults,_netdev,nofail 0 0" | sudo tee -a /etc/fstab

# Test fstab entry
$ sudo mount -a

# Verify
$ df -h /mnt/nfs
Filesystem                Size  Used Avail Use% Mounted on
192.168.1.100:/exports/data 100G   45G   55G  45% /mnt/nfs

Troubleshooting Commands

# Show NFS server exports
$ showmount -e 192.168.1.100

# Show NFS statistics
$ nfsstat

# Check NFS client connections
$ sudo nfsstat -c

# Monitor NFS traffic
$ sudo tcpdump -i eth0 port 2049

Lessons Learned

  1. Always check firewall rules - NFS requires multiple ports (111, 2049, mountd).
  2. Use exportfs -ra after modifying /etc/exports on the server.
  3. NFSv4 is simpler than NFSv3 - uses only port 2049.
  4. _netdev in fstab ensures mount waits for network to be ready.
  5. Monitor NFS performance with nfsstat and adjust mount options accordingly.

This blog does not accept any external sponsorships, affiliate marketing, or ad revenue.