When a Node.js API becomes slow, adding replicas feels like the fastest response. But if the event loop is blocked by synchronous work, each replica can reproduce the same stall. Measure the queue before scaling it.

What to measure

Event-loop delay measures how late callbacks run relative to their expected schedule. Event-loop utilization measures how busy the loop has been. Neither is a direct CPU percentage, and a high value alone does not prove the root cause.

const { monitorEventLoopDelay, performance } = require('node:perf_hooks');
const histogram = monitorEventLoopDelay({ resolution: 20 });
histogram.enable();
let previous = performance.eventLoopUtilization();
setInterval(() => {
  const delta = performance.eventLoopUtilization(previous);
  previous = performance.eventLoopUtilization();
  console.log({
    p99DelayMs: Number((histogram.percentile(99) / 1e6).toFixed(2)),
    utilization: Number(delta.utilization.toFixed(3))
  });
  histogram.reset();
}, 10000).unref();

Export these values to metrics instead of logging them in production. Record p50 and p99 request latency, throughput, CPU, memory, garbage collection, and downstream dependency latency alongside them. A ten-second aggregation is a starting point, not a universal alert threshold.

Find the expensive callback

Start with a CPU profile under representative load. Look for synchronous JSON parsing or serialization of huge payloads, regular-expression backtracking, synchronous crypto, filesystem calls, compression, and expensive application loops. Node's built-in CPU profiler can help identify hot stacks, but profiling itself has overhead. Test with representative traffic and compare before and after.

Also inspect garbage collection pauses and allocation churn. An overloaded process may alternate between CPU-heavy application work and garbage collection. Scaling replicas can lower load per process, but it will not fix a pathological operation triggered by every request.

Decide between optimization, offloading, and scaling

Worker threads have startup, serialization, memory, and coordination costs. For occasional tiny jobs, the overhead can exceed the work. For long-running CPU-heavy jobs, a managed pool and backpressure are often more predictable than spawning a thread for every request.

Use an experiment, not a guess

  1. Capture request p95/p99, event-loop delay, utilization, CPU, GC, and dependency timings during the same time window.
  2. Reproduce the issue with representative payloads and concurrency.
  3. Take a CPU profile, identify the hot path, and verify whether the stall correlates with traffic or particular endpoints.
  4. Change one factor: optimize the synchronous path, bound work, or offload it.
  5. Load-test again, including failure modes and queue saturation.
  6. Only then tune Kubernetes CPU limits, autoscaling signals, and replica counts.

Be especially cautious when using CPU-based HPA alone. An event-loop stall can be latency-critical even before average CPU utilization looks dramatic. Conversely, high utilization may be acceptable for batch-oriented workers with healthy queues and latency targets.

Production checklist

Track event-loop delay percentiles per process, not just fleet averages. Alert on sustained symptoms that correlate with user-facing latency. Preserve a profiling workflow for incidents. Bound worker queues, apply timeouts, and keep a rollback path for performance changes. If a single request can monopolize the loop, isolate or reject that work before buying more replicas.

Further reading

Related articles