sysctl Parameter Tuning Guide: Optimizing Linux Kernel Performance

Introduction

The Linux kernel exposes hundreds of tunable parameters through the /proc/sys/ filesystem and the sysctl command. These parameters control everything from TCP buffer sizes to virtual memory behavior to filesystem limits. Default values are conservative and optimized for general-purpose use. For specific workloads like high-traffic web servers, databases, or trading systems, tuning these parameters can yield significant performance improvements.

Environment

The tuning parameters discussed here were tested on a Ubuntu 22.04 LTS server with 32 GB RAM, 10 Gbps network interface, and NVMe storage. The workload is a high-traffic web application serving 10,000+ requests per second.

cat /proc/version
# Linux version 5.15.0-91-generic (buildd@lcy02-amd64-045)

free -h
#               total        used        free      shared  buff/cache   available
# Mem:           31Gi       12Gi        8Gi       1.2Gi       11Gi       17Gi

Problem

Under peak load, the server shows:

High TCP retransmission rates
Connection timeouts under load
Slow disk I/O response times
Out-of-memory conditions despite free memory in buffers

The default kernel parameters are not tuned for this workload profile.

Analysis

sysctl parameters are stored in /proc/sys/ and can be read or written at runtime:

# Read a parameter
sysctl net.core.somaxconn
# net.core.somaxconn = 128

# Write a parameter (temporary, lost on reboot)
sysctl -w net.core.somaxconn=4096

Permanent changes go in /etc/sysctl.conf or files under /etc/sysctl.d/.

Solution

1. Network connection handling

For high-traffic servers, increase the connection backlog:

# /etc/sysctl.d/99-network-tuning.conf
# Increase the maximum number of connections waiting to be accepted
net.core.somaxconn = 65535

# Increase the maximum number of connections tracked
net.netfilter.nf_conntrack_max = 1048576

# Increase the local port range
net.ipv4.ip_local_port_range = 1024 65535

# Allow reuse of TIME_WAIT sockets
net.ipv4.tcp_tw_reuse = 1

# Increase the maximum number of queued connections
net.core.netdev_max_backlog = 65536

2. TCP buffer sizes

# /etc/sysctl.d/99-network-tuning.conf
# Maximum receive/send buffer sizes
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

# TCP receive/send buffer defaults and maximums (min default max)
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

# Enable TCP window scaling
net.ipv4.tcp_window_scaling = 1

# Enable TCP timestamps
net.ipv4.tcp_timestamps = 1

# Enable selective acknowledgments
net.ipv4.tcp_sack = 1

3. Virtual memory tuning

# /etc/sysctl.d/99-memory-tuning.conf
# Swap usage (0 = avoid swapping when possible)
vm.swappiness = 10

# Dirty page ratios (% of total memory)
vm.dirty_ratio = 15
vm.dirty_background_ratio = 5

# Maximum memory map areas per process
vm.max_map_count = 262144

# How often to flush dirty pages (centiseconds)
vm.dirty_expire_centisecs = 3000
vm.dirty_writeback_centisecs = 500

4. File system limits

# /etc/sysctl.d/99-fs-tuning.conf
# Maximum number of file handles
fs.file-max = 2097152

# Inotify limits
fs.inotify.max_user_watches = 524288
fs.inotify.max_user_instances = 8192

# AIO limits
fs.aio-max-nr = 1048576

5. Apply all changes

# Apply immediately
sudo sysctl -p /etc/sysctl.d/99-network-tuning.conf
sudo sysctl -p /etc/sysctl.d/99-memory-tuning.conf
sudo sysctl -p /etc/sysctl.d/99-fs-tuning.conf

# Or apply all at once
sudo sysctl --system

6. Verify changes

sysctl net.core.somaxconn vm.swappiness fs.file-max
# net.core.somaxconn = 65535
# vm.swappiness = 10
# fs.file-max = 2097152

7. Monitor the impact

# Check TCP retransmissions
ss -ti | grep -E "retrans|rtt"
# cubic wscale:7,7 rto:204 rtt:0.5/0.25 ato:40 mss:1448 pmtu:1500
# rcvmss:1448 advmss:1448 retrans:0/0 bytes_acked:123456

# Check connection states
ss -s
# Total: 12345
# TCP:   12000 (estab 8000, closed 500, orphaned 200, timewait 300)

# Monitor memory
vmstat 1 5
# procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
#  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
#  5  0  10240 8192000 262144 8388608    0    0     0     0    0    0 45  5 50  0

8. Create workload-specific profiles

# For a database server
cat > /etc/sysctl.d/99-database.conf << 'EOF'
vm.swappiness = 1
vm.dirty_ratio = 40
vm.dirty_background_ratio = 10
vm.overcommit_memory = 0
vm.overcommit_ratio = 80
net.core.somaxconn = 4096
net.ipv4.tcp_max_syn_backlog = 4096
EOF

# For a web server
cat > /etc/sysctl.d/99-webserver.conf << 'EOF'
net.core.somaxconn = 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
net.core.netdev_max_backlog = 65536
fs.file-max = 2097152
EOF

9. Use sysctl.conf for legacy compatibility

# /etc/sysctl.conf
# Add custom parameters at the end
net.core.somaxconn = 65535
vm.swappiness = 10

Lessons Learned

sysctl tuning is not a one-size-fits-all exercise. Always benchmark before and after changes. Start with network and memory parameters, as these typically yield the most visible improvements. Use separate files in /etc/sysctl.d/ for different workload profiles. And always test changes on a staging system first. A wrong parameter value can degrade performance rather than improve it.

This blog does not accept any external sponsorships, affiliate marketing, or ad revenue.