Scaling WebSocket Servers

Scaling WebSocket Infrastructure with Socket.IO, Redis, and Traefik

Challenges in Scaling WebSocket

Scaling HTTP

Scaling WebSocket

Theoretical Analysis

Objective

Problems

• Cross-server communication: Multiple WebSocket servers must coordinate with one another. When a server receives a message from a client, it must ensure that all relevant clients connected to any server in the cluster can receive that message.

• Connection affinity: Once a client establishes a connection with a particular server, its subsequent communication must continue through that connection. Otherwise, another server may not recognize the client's existing session or connection state.

Solutions

const redisAdapter = require("socket.io-redis");

const redisHost = process.env.REDIS_HOST || "localhost";

io.adapter(
  redisAdapter({
    host: redisHost,
    port: 6379,
  })
);

Summary

• Each client establishes a connection to a specific WebSocket server instance.

• Messages sent over an established WebSocket connection continue through that connection to the same server instance.

• When a server receives an event that needs to be broadcast, the adapter distributes it to other server instances, which forward it to their connected clients.

Tutorials

Step 1: Set Up a WebSocket Server with a Redis Adapter

const os = require("os");
const ifaces = os.networkInterfaces();

const privateIp = (() => {
  return Object.values(ifaces)
    .flat()
    .find((val) => val.family === "IPv4" && !val.internal)
    .address;
})();

const randomOffset = Math.floor(Math.random() * 10);
const intervalOffset = (30 + randomOffset) * 1000;

// WebSocket server
const socketPort = 5000;
const socketServer = require("http").createServer();

const io = require("socket.io")(socketServer, {
  path: "/",
});

// Redis adapter
const redisAdapter = require("socket.io-redis");
const redisHost = process.env.REDIS_HOST || "localhost";

io.adapter(
  redisAdapter({
    host: redisHost,
    port: 6379,
  })
);

// Event handlers
io.on("connection", (client) => {
  console.log("New incoming connection from", client.id);

  client.on("test000", (message) => {
    console.log("Message from client:", client.id, "->", message);
  });
});

setInterval(() => {
  const log0 = `I am the host: ${privateIp}. I am healthy.`;
  console.log(log0);
  io.emit("okok", log0);
}, intervalOffset);

// Start listening
socketServer.listen(socketPort);

Step 2: Build a Docker Image for the WebSocket Server

# Stage 1: Install dependencies
FROM node:13-alpine AS node-compiled

ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}

RUN apk --no-cache add python make g++

COPY package*.json ./
RUN npm install

# Stage 2: Build the application image
FROM node:13-alpine

WORKDIR /usr/src/app

COPY . .
COPY --from=node-compiled node_modules node_modules

# EXPOSE 5000

CMD ["npm", "run", "start"]

docker build . -t ws_local

Step 3: Create the docker-compose.yaml File

version: "3.2"

services:
  socket-server:
    # build: .
    # image: sw360cab/wsk-base:0.1.1
    image: websocket_local
    # restart: always
    deploy:
      mode: replicated
      replicas: 2
    environment:
      - "REDIS_HOST=redis"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.socket-router.rule=PathPrefix(`/wsk`)"
      - "traefik.http.services.service01.loadbalancer.server.port=5000"
      - "traefik.http.services.service01.loadbalancer.sticky.cookie=true"
      - "traefik.http.services.service01.loadbalancer.sticky.cookie.name=io"
      - "traefik.http.services.service01.loadbalancer.sticky.cookie.httponly=true"
      - "traefik.http.services.service01.loadbalancer.sticky.cookie.secure=true"
      - "traefik.http.services.service01.loadbalancer.sticky.cookie.samesite=lax"
      # Replace the /wsk prefix
      - "traefik.http.middlewares.socket-replaceprefix.replacepath.path=/"
      # Attach the middleware to the router
      - "traefik.http.routers.socket-router.middlewares=socket-replaceprefix@docker"

  traefik-reverse-proxy:
    image: traefik:v2.2
    command:
      - "--api.insecure=true"
      - "--providers.docker.exposedByDefault=false"
      - "--accesslog"
      # - "--entryPoints.web.address=:80"
      # - "--entryPoints.web.forwardedHeaders.insecure=true"
    ports:
      - "80:80"
      # Web UI (enabled by --api.insecure=true)
      - "8080:8080"
    volumes:
      # Allow Traefik to monitor Docker events
      - /var/run/docker.sock:/var/run/docker.sock

  redis:
    image: redis:5.0

Understanding the Configuration

traefik-reverse-proxy:
  image: traefik:v2.2
  command:
    - "--api.insecure=true"
    - "--accesslog"
  ports:
    - "80:80"
    # Web UI (enabled by --api.insecure=true)
    - "8080:8080"
  volumes:
    # Allow Traefik to monitor Docker events
    - /var/run/docker.sock:/var/run/docker.sock

socket-server:
  image: sw360cab/wsk-base:0.1.1
  restart: always
  deploy:
    mode: replicated
    replicas: 2
  environment:
    - "REDIS_HOST=redis"
  labels:
    - "traefik.http.routers.socket-router.rule=PathPrefix(`/wsk`)"
    - "traefik.http.services.service01.loadbalancer.server.port=5000"
    - "traefik.http.services.service01.loadbalancer.sticky.cookie=true"
    - "traefik.http.services.service01.loadbalancer.sticky.cookie.name=io"
    - "traefik.http.services.service01.loadbalancer.sticky.cookie.httponly=true"
    - "traefik.http.services.service01.loadbalancer.sticky.cookie.secure=true"
    - "traefik.http.services.service01.loadbalancer.sticky.cookie.samesite=lax"
    - "traefik.http.middlewares.socket-replaceprefix.replacepath.path=/"
    - "traefik.http.routers.socket-router.middlewares=socket-replaceprefix@docker"

• Router: The router matches incoming requests whose paths begin with /wsk.

• Service: The service forwards requests to port 5000, where the WebSocket application listens. It also configures sticky sessions through cookies.

• Middleware: The middleware replaces the incoming path with /, matching the path expected by the WebSocket application.

- "traefik.http.routers.socket-router.middlewares=socket-replaceprefix@docker"

Step 4: Deploy the Docker Swarm Stack

docker stack deploy --compose-file stack/docker-compose.yml wsk

Step 5: Launch the WebSocket Client

• 5000 — Direct connection to the WebSocket server.

• 80 — Connection through an HTTP reverse proxy such as Traefik or HAProxy.

• 30000 — Example NodePort when using Kubernetes.

• 30080 — Example port when using Traefik Ingress in Kubernetes.

const uuid = require("uuid");
const io = require("socket.io-client");

const client = io("http://localhost:80", {
  // port: 5000 - direct connection
  // port: 80 - through an HTTP reverse proxy
  // port: 30000 - through a Kubernetes NodePort
  // port: 30080 - example Traefik Ingress port
  path: "/wsk", // when using the Traefik configuration above
  // path: "/", // for other configurations
  reconnection: true,
  reconnectionDelay: 500,
  transports: ["websocket"],
});

const clientId = uuid.v4();
let disconnectTimer;

client.on("connect", () => {
  console.log("Connected!", clientId);

  setTimeout(() => {
    console.log("Sending first message");
    client.emit("test000", clientId);
  }, 500);

  // Clear the disconnection timeout
  clearTimeout(disconnectTimer);
});

client.on("okok", (message) => {
  console.log("The server has a message for you:", message);
});

client.on("disconnect", () => {
  console.log("Disconnected!");

  disconnectTimer = setTimeout(() => {
    console.log("Not reconnecting in 10 seconds. Exiting...");
    process.exit(0);
  }, 10000);
});

client.on("connect_error", (err) => {
  console.error(err);
});

setInterval(() => {
  console.log("Sending repeated message");
  client.emit("test000", clientId);
}, 5000);

node client_socket.js

Conclusion

References

• sw360cab/websockets-scaling: A tutorial to scale WebSockets with Docker Swarm and Kubernetes

• Concepts – Traefik

• Docker Provider – Traefik

• Docker Configuration Reference – Traefik

• Kubernetes IngressRoute – Traefik

• Middlewares – Traefik

• Sticky Sessions – Traefik