troubleshooting2024-03-15·10 min·318/348

Linux Process Management: kill, nice, and renice

Managing Linux processes with kill, nice, and renice, including signal types, priority adjustment, and troubleshooting unresponsive processes.

Introduction

Managing processes is a fundamental Linux skill. When a process becomes unresponsive, consumes too much CPU, or needs to be terminated, knowing the right signals and priority adjustments is essential. This post covers the kill command, process signals, and priority management with nice and renice.

Environment

The examples use Ubuntu 22.04 LTS, managing processes in a multi-user development environment.

ps --version
# procps-ng 3.3.17

Problem

A process is consuming 100% CPU and making the system unresponsive:

top
# top - 14:30:00 up 10 days,  2:30,  1 user,  load average: 4.50, 4.20, 3.80
# Tasks: 250 total,   3 running, 247 sleeping,   0 stopped,   0 zombie
# %Cpu(s): 25.0 us,  5.0 sy,  0.0 ni, 70.0 id,  0.0 wa,  0.0 hi,  0.0 si
# MiB Mem:  32813.5 total,   2345.7 free,  28456.8 used,   2011.0 buff/cache
#
#   PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
# 12345 user      20   0  1234567 890123  12345 R 100.0   2.6   45:23.45 myapp

The myapp process is consuming 100% CPU and needs to be either killed or have its priority lowered.

Analysis

Linux signals are the primary mechanism for process control:

SignalNumberDescription
SIGHUP1Reload configuration
SIGINT2Interrupt (Ctrl+C)
SIGQUIT3Quit with core dump
SIGKILL9Force kill (cannot be caught)
SIGTERM15Graceful termination (default)
SIGUSR110User-defined signal 1
SIGUSR212User-defined signal 2
SIGSTOP19Stop process
SIGCONT18Continue stopped process

Process priority uses nice values from -20 (highest priority) to 19 (lowest priority).

Solution

1. Send SIGTERM to a process

# Find the PID
pidof myapp
# 12345

# Send SIGTERM (graceful shutdown)
kill 12345

# Or use the process name
killall myapp

2. Force kill with SIGKILL

# If SIGTERM does not work
kill -9 12345

# Or
kill -SIGKILL 12345

# Find and kill by name
pkill -9 myapp

3. Check if a process is still running

# Check by PID
ps -p 12345
#   PID TTY          TIME CMD
# 12345 ?        00:45:23 myapp

# Check by name
pgrep myapp
# 12345

4. Lower process priority with nice

# Start a process with lower priority
nice -n 10 myapp

# Start with highest priority (requires root)
sudo nice -n -20 myapp

5. Change priority of a running process with renice

# Lower priority of running process
renice 10 -p 12345
# 12345 (process ID) old priority 0, new priority 10

# Set to lowest priority
sudo renice 19 -p 12345

# Check current nice value
ps -o pid,ni,comm -p 12345
#   PID  NI COMMAND
# 12345  10 myapp

6. Send SIGHUP to reload configuration

# Reload Nginx configuration
kill -HUP $(pidof nginx)

# Reload systemd service
sudo systemctl reload myapp

7. Stop and resume processes

# Stop a process
kill -STOP 12345

# Resume a stopped process
kill -CONT 12345

# Or use bg/fg in shell
bg %1  # Resume job in background
fg %1  # Bring job to foreground

8. Kill all processes of a user

# Kill all processes for a specific user
sudo killall -u username

# Or
sudo pkill -u username

9. Find and kill zombie processes

# Find zombie processes
ps aux | grep -w Z
# user  12345  0.0  0.0      0     0 ?        Z    10:00   0:00 [myapp] 

# Kill the parent process
kill -9 $(ps -o ppid= -p 12345)

10. Use timeout for processes that may hang

# Kill after 60 seconds
timeout 60 myapp

# Send specific signal after timeout
timeout -s SIGTERM 120 myapp

11. Monitor process resource usage

# Real-time process monitoring
top -u username

# Detailed process info
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -20

# Check open files
lsof -p 12345

12. Batch kill processes by pattern

# Kill all Python processes
pkill python

# Kill processes matching a pattern
pgrep -f "myapp.*debug" | xargs kill -9

# Safe version with confirmation
pgrep -f "myapp" | while read pid; do
  kill -0 $pid 2>/dev/null && echo "Kill $pid? (y/n)" && read -n 1 -r
  [[ $REPLY =~ ^[Yy]$ ]] && kill -15 $pid
done

Lessons Learned

Always try SIGTERM before SIGKILL. SIGTERM allows the process to clean up, while SIGKILL terminates immediately. Use nice and renice to manage CPU-intensive processes without killing them. And for automated process management, consider using systemd service files with Restart= and resource limits.


This blog does not accept any external sponsorships, affiliate marketing, or ad revenue.