tooolbox/blog/Redis Memory Usage Monitoring: Eviction Policies and Memory Analysis
performance2024-08-01Β·8Β·284/348

Redis Memory Usage Monitoring: Eviction Policies and Memory Analysis

Monitoring and managing Redis memory usage through eviction policies, memory analysis tools, and optimization techniques for large datasets.

Introduction

Redis is fast because it keeps everything in memory, but memory is a finite resource. I discovered this when my Redis instance, which was supposed to hold about 2GB of data, started consuming 6GB and causing the OOM killer to terminate the process. The cause was a combination of storing large values (serialized JSON objects up to 500KB each) and an eviction policy that was not configured for my workload.

Redis memory management requires active monitoring and tuning. This post covers how I set up memory monitoring, identified the causes of excessive memory usage, and configured the system to stay within bounds.

Environment

  • Redis version: 7.2.3
  • Deployment: Single instance on Ubuntu 22.04
  • Available RAM: 8GB (shared with other services)
  • Redis maxmemory: 4GB
  • Data type: Mix of hashes, sorted sets, and strings
  • Use case: Trading signal cache and session store

Problem

The Redis instance was growing unbounded despite having a maxmemory limit configured:

redis-cli info memory
# used_memory:6442450944
# used_memory_human:6.00G
# maxmemory:4294967296
# maxmemory_human:4.00G
# mem_fragmentation_ratio:2.34

The used_memory was 6GB despite maxmemory being set to 4GB. The fragmentation ratio of 2.34 indicated severe memory fragmentation β€” Redis was using 2.34x more memory than the actual data required.

Additionally, the eviction policy was set to noeviction, which means Redis refuses new writes when maxmemory is reached instead of evicting old keys:

redis-cli config get maxmemory-policy
# maxmemory-policy
# noeviction

This caused write errors in the application:

OOM command not allowed when used memory > 'maxmemory'.

Analysis

I used Redis's memory analysis tool to identify the largest keys:

redis-cli --bigkeys
# ...Biggest string found so far: 'cache:trade:12345' with 512000 bytes
# ...Biggest hash found so far: 'session:user:67890' with 245760 bytes
# ...Biggest zset found so far: 'signals:momentum' with 131072 members

The cache:trade:* keys were the biggest offenders β€” each one was a serialized JSON object averaging 500KB. There were about 5,000 of them, consuming 2.5GB of memory.

I also checked the memory fragmentation:

redis-cli memory doctor
# Hi, my name is Redis, and I'm a very nervous robot.
# ...
# Currently I have 6.00G of memory used out of 8.00G available.
# ...
# The peak memory usage was 6.23G, which is 77.88% of the available memory.
# ...
# Memory fragmentation is 2.34, which is above the recommended threshold of 1.5.

The fragmentation was caused by Redis allocating memory in chunks that were larger than needed. When keys of varying sizes are created and deleted, the memory allocator leaves gaps that cannot be reused efficiently.

I also checked the key expiration statistics:

redis-cli info keyspace
# db0:keys=45000,expires=32000,avg_ttl=3600000

Only 32,000 out of 45,000 keys had expiration times. The remaining 13,000 keys (mostly cache:trade:* keys) had no TTL and would never expire.

Solution

I implemented a comprehensive memory management strategy:

1. Configure an appropriate eviction policy:

redis-cli config set maxmemory 4gb
redis-cli config set maxmemory-policy allkeys-lru
redis-cli config set maxmemory-samples 10

allkeys-lru evicts the least recently used keys across all keys when maxmemory is reached. This is appropriate for a cache workload where older data is less valuable.

For persistent keys that should not be evicted, I used a separate Redis database:

# DB 0: Cache with LRU eviction
# DB 1: Persistent data with noeviction

2. Set TTLs on all cache keys:

import redis

r = redis.Redis()

def cache_trade(trade_id, trade_data, ttl=3600):
    """Cache trade data with a 1-hour TTL."""
    key = f"cache:trade:{trade_id}"
    serialized = json.dumps(trade_data)
    r.setex(key, ttl, serialized)

def get_cached_trade(trade_id):
    """Retrieve cached trade data."""
    key = f"cache:trade:{trade_id}"
    data = r.get(key)
    if data:
        return json.loads(data)
    return None

3. Reduce value sizes:

I restructured the trade data to store only essential fields in Redis:

# Before: Full trade object (~500KB)
trade_full = {
    "id": 12345,
    "symbol": "AAPL",
    "strategy": "momentum",
    "entry_price": 150.25,
    "exit_price": 155.80,
    "quantity": 100,
    "pnl": 555.00,
    "timestamp": "2024-08-01T10:30:00Z",
    "indicators": { ... },  # Large technical indicators
    "market_data": { ... },  # Full market snapshot
    "metadata": { ... }  # Additional metadata
}

# After: Compact trade object (~2KB)
trade_compact = {
    "id": 12345,
    "s": "AAPL",
    "st": "mom",
    "ep": 150.25,
    "xp": 155.80,
    "q": 100,
    "pnl": 555.00,
    "t": "2024-08-01T10:30:00Z"
}

4. Configure memory optimization settings:

# Use jemalloc allocator (better fragmentation handling)
redis-cli config set activedefrag yes

# Tune jemalloc background defragmentation
redis-cli config set active-defrag-enabled yes
redis-cli config set active-defrag-threshold-lower 10
redis-cli config set active-defrag-threshold-upper 100
redis-cli config set active-defrag-cycle-min 1
redis-cli config set active-defrag-cycle-max 25

5. Monitor memory usage with a script:

#!/bin/bash
# redis-memory-monitor.sh

THRESHOLD_GB=3.5
REDIS_HOST="localhost"

while true; do
    USED_BYTES=$(redis-cli -h $REDIS_HOST info memory | grep "used_memory:" | cut -d: -f2 | tr -d '\r')
    USED_GB=$(echo "scale=2; $USED_BYTES / 1073741824" | bc)

    FRAG_RATIO=$(redis-cli -h $REDIS_HOST info memory | grep "mem_fragmentation_ratio:" | cut -d: -f2 | tr -d '\r')

    KEYS=$(redis-cli -h $REDIS_HOST dbsize)

    echo "$(date): Memory: ${USED_GB}GB | Frag: ${FRAG_RATIO} | ${KEYS}"

    if (( $(echo "$USED_GB > $THRESHOLD_GB" | bc -l) )); then
        echo "$(date): WARNING: Memory usage ${USED_GB}GB exceeds threshold ${THRESHOLD_GB}GB"
    fi

    sleep 60
done

After implementing all changes:

redis-cli info memory
# used_memory:1572864000
# used_memory_human:1.46G
# maxmemory:4294967296
# maxmemory_human:4.00G
# mem_fragmentation_ratio:1.12

Memory usage dropped from 6GB to 1.46GB, and the fragmentation ratio improved from 2.34 to 1.12.

Lessons Learned

  1. Always set maxmemory β€” Without a maxmemory limit, Redis can consume all available memory and cause the OOM killer to terminate the process.
  2. Choose the right eviction policy β€” allkeys-lru for caches, volatile-lru when only some keys should be evicted, noeviction for persistent data.
  3. Set TTLs on cache keys β€” Keys without expiration accumulate forever. Set appropriate TTLs on all cache data.
  4. Monitor fragmentation ratio β€” A ratio above 1.5 indicates significant fragmentation. Enable active defragmentation to mitigate this.
  5. Compress values β€” Large values consume disproportionate memory. Use compact serialization formats and store only essential data in Redis.

This blog does not accept any external sponsorships, affiliate marketing, or ad revenue.