tooolbox/blog/Resolving Socket.io Connection Issues in Production
troubleshooting2025-01-10Β·13Β·234/348

Resolving Socket.io Connection Issues in Production

Troubleshooting Socket.io connection failures, reconnection logic, rooms and namespaces configuration, and scaling WebSocket connections across multiple servers.

Introduction

Socket.io simplifies WebSocket communication with automatic fallbacks and reconnection logic, but production deployments reveal connection issues that do not appear in development. After deploying Socket.io across multiple server instances behind a load balancer, I encountered connection drops, message ordering issues, and scaling bottlenecks that required fundamental architecture changes.

This post covers the most common Socket.io production issues and the solutions that maintain reliable real-time communication at scale.

Environment

node --version
# v20.11.0

npm list socket.io socket.io-client @socket.io/redis-adapter
# socket.io@4.7.4
# socket.io-client@4.7.4
# @socket.io/redis-adapter@8.2.1

# Redis for multi-server pub/sub
redis-cli --version
# redis-cli 7.2.3

The deployment architecture:

Load Balancer (nginx)
    β”œβ”€β”€ Server 1 (Node.js + Socket.io)
    β”œβ”€β”€ Server 2 (Node.js + Socket.io)
    └── Server 3 (Node.js + Socket.io)
         ↕ Redis (pub/sub adapter)

Problem

The first issue was that connections dropped every 30 seconds behind the nginx load balancer. The WebSocket connection would establish successfully, transmit data for a few seconds, then disconnect:

// Client-side error
Error: websocket connection closed
    at WebSocket onClose (socket.io-client.js:1234)
    // Code: 1006 (Abnormal Closure)

The second issue was that messages sent from clients connected to different server instances were not received by other clients. Server 1 could communicate with clients on Server 1, but messages were not forwarded to clients on Server 2:

// Client A connects to Server 1
// Client B connects to Server 2
// Client A emits "message" event
// Client B never receives it

The third issue was that the reconnection logic caused duplicate messages. When a client reconnected after a network interruption, the server replayed all missed events, resulting in the client processing the same events twice:

// Client receives duplicate events after reconnection
socket.on("chat-message", (message) => {
  // This handler is called multiple times for the same message
  // because the server stored events in memory
  addMessageToChat(message);
});

Analysis

The nginx WebSocket timeout issue is caused by nginx's default proxy timeout of 60 seconds. WebSocket connections are long-lived, and nginx closes connections that have not received data within the timeout period. The Socket.io polling fallback complicates this because nginx may not properly upgrade the connection to WebSocket.

The multi-server messaging issue occurs because Socket.io events are stored in memory by default. When a client connects to Server 1 and another to Server 2, events emitted by one server are not automatically forwarded to the other. The Redis adapter solves this by using Redis pub/sub to broadcast events across all server instances.

The duplicate message issue is caused by Socket.io's default behavior of storing undelivered events in memory. When a client disconnects and reconnects, the server sends all events that occurred during the disconnection. If the client already received some of these events before the disconnection, duplicates result.

Solution

Configure nginx for WebSocket support:

# /etc/nginx/conf.d/socketio.conf
upstream socket_nodes {
    ip_hash; # Sticky sessions based on IP
    server server1:3000;
    server server2:3000;
    server server3:3000;
}

server {
    listen 80;
    server_name example.com;

    location /socket.io/ {
        proxy_pass http://socket_nodes;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Timeout settings for WebSocket
        proxy_connect_timeout 7d;
        proxy_send_timeout 7d;
        proxy_read_timeout 7d;

        # Disable buffering
        proxy_buffering off;
        proxy_cache off;
    }
}

Configure Redis adapter for multi-server communication:

// server.ts
import { Server } from "socket.io";
import { createAdapter } from "@socket.io/redis-adapter";
import { createClient } from "redis";

const io = new Server(3000, {
  cors: {
    origin: ["http://localhost:3000", "https://example.com"],
    methods: ["GET", "POST"],
  },
  pingTimeout: 60000,
  pingInterval: 25000,
  transports: ["websocket", "polling"],
});

// Redis clients for pub/sub
const pubClient = createClient({ url: process.env.REDIS_URL });
const subClient = pubClient.duplicate();

await Promise.all([pubClient.connect(), subClient.connect()]);

io.adapter(createAdapter(pubClient, subClient));

console.log("Socket.io server with Redis adapter ready");

Implement proper reconnection handling:

// Client-side connection with reconnection
import { io, Socket } from "socket.io-client";

const socket = io("https://example.com", {
  transports: ["websocket"],
  reconnection: true,
  reconnectionAttempts: 10,
  reconnectionDelay: 1000,
  reconnectionDelayMax: 5000,
  timeout: 20000,
});

socket.on("connect", () => {
  console.log("Connected:", socket.id);

  // Request missed events from server
  socket.emit("sync-events", {
    lastEventId: localStorage.getItem("lastEventId"),
  });
});

socket.on("disconnect", (reason) => {
  console.log("Disconnected:", reason);

  if (reason === "io server disconnect") {
    // Server forcibly disconnected - reconnect manually
    socket.connect();
  }
  // Other reasons (transport close, ping timeout) auto-reconnect
});

socket.on("connect_error", (error) => {
  console.error("Connection error:", error.message);

  // Fallback to polling if WebSocket fails
  if (socket.active) {
    setTimeout(() => {
      socket.connect();
    }, 2000);
  }
});

Implement event deduplication on the server:

// server/middleware/deduplication.ts
import { Socket } from "socket.io";

// In-memory deduplication with TTL
const processedEvents = new Map();
const EVENT_TTL = 30000; // 30 seconds

setInterval(() => {
  const now = Date.now();
  for (const [key, timestamp] of processedEvents.entries()) {
    if (now - timestamp > EVENT_TTL) {
      processedEvents.delete(key);
    }
  }
}, 5000);

export function deduplicationMiddleware(socket: Socket, next: (err?: Error) => void) {
  const originalEmit = socket.emit;

  socket.emit = function (event: string, ...args: any[]) {
    const eventId = args[0]?.eventId;

    if (eventId) {
      if (processedEvents.has(eventId)) {
        // Skip duplicate event
        return;
      }
      processedEvents.set(eventId, Date.now());
    }

    return originalEmit.apply(this, [event, ...args]);
  };

  next();
}

io.use(deduplicationMiddleware);

Use rooms for targeted message delivery:

// Server-side room management
io.on("connection", (socket) => {
  console.log("Client connected:", socket.id);

  // Join room based on user ID
  socket.on("join-room", (roomId: string) => {
    socket.join(roomId);
    console.log(`Socket ${socket.id} joined room ${roomId}`);

    // Notify room members
    io.to(roomId).emit("user-joined", {
      userId: socket.data.userId,
      socketId: socket.id,
    });
  });

  // Leave room
  socket.on("leave-room", (roomId: string) => {
    socket.leave(roomId);
    io.to(roomId).emit("user-left", {
      userId: socket.data.userId,
    });
  });

  // Send message to specific room
  socket.on("room-message", (roomId: string, message: string) => {
    // Verify user is in the room
    if (socket.rooms.has(roomId)) {
      io.to(roomId).emit("new-message", {
        userId: socket.data.userId,
        message,
        timestamp: Date.now(),
      });
    }
  });

  socket.on("disconnect", () => {
    // Notify all rooms the user was in
    for (const room of socket.rooms) {
      if (room !== socket.id) {
        io.to(room).emit("user-left", {
          userId: socket.data.userId,
        });
      }
    }
  });
});

Handle scaling with Redis sticky sessions:

// server/load-balanced.ts
import { Server } from "socket.io";
import { createAdapter } from "@socket.io/redis-adapter";
import { createClient } from "redis";
import express from "express";

const app = express();
const httpServer = createServer(app);

const io = new Server(httpServer, {
  adapter: createAdapter(pubClient, subClient),
  cookie: {
    name: "io",
    httpOnly: true,
    maxAge: 86400000, // 24 hours
  },
  // Enable sticky sessions in load balancer
  // to ensure consistent connection to same server
  transports: ["websocket"],
});

// Health check endpoint
app.get("/health", (req, res) => {
  res.json({
    status: "ok",
    connections: io.engine.clientsCount,
    rooms: Array.from(io.sockets.adapter.rooms.keys()),
  });
});

httpServer.listen(3000);

Lessons Learned

  1. Configure nginx with proper WebSocket timeouts. The default 60-second timeout is too short for long-lived connections. Set proxy_read_timeout to at least 7 days.

  2. Use the Redis adapter for any multi-server deployment. Without it, events are isolated to the server instance that received them.

  3. Implement event deduplication to handle reconnection scenarios where clients may receive duplicate events.

  4. Use rooms for targeted messaging instead of broadcasting to all connected clients. This reduces unnecessary network traffic and client-side processing.

  5. Monitor connection counts per server to detect load imbalance. The health check endpoint provides this information for load balancer configuration.

This blog does not accept any external sponsorships, affiliate marketing, or ad revenue.