Testing circuit breakers — slowdep guides

Background

What a circuit breaker is

A circuit breaker is a resiliency pattern that protects your application from cascading failures. It sits in front of a dependency and monitors calls. When failures exceed a threshold, it "opens" the circuit — subsequent calls fail immediately without hitting the downstream service. After a cooldown period, it enters "half-open" state, allowing a probe request through. If the probe succeeds, the circuit closes and normal operation resumes.

CLOSED ? N failures ? OPEN ? cooldown ? HALF-OPEN ? success ? CLOSED

The key insight

Why circuit breakers need slow failures

A circuit breaker doesn't trip on fast errors alone. It trips when the system is under stress — meaning errors are frequent and responses are slow. A service that fails instantly is actually less dangerous than one that fails after 3 seconds, because fast failures don't exhaust your connection pool or block your threads.

This is why mocks that throw immediately can't test circuit breaker behavior accurately. A mock that throws in 0ms won't exhaust your connection pool. A mock that throws after 3 seconds will. slowdep gives you the latency + error combination that actually triggers circuit breaker thresholds.

The dangerous combination

High error rate + high latency = circuit breaker trip condition. slowdep lets you control both. errorRate: 0.3 with p99: 3000 simulates a degraded service that's slow to fail.

Simulating the trip

Simulating the trip condition

Configure slowdep with enough error rate and latency to trip your circuit breaker's thresholds. A 30% error rate with p99 of 3s is typical of a degraded database or downstream service.

import { withLatency } from 'slowdep';

// simulates a degraded service: slow AND error-prone
const degradedService = withLatency(realService.call, {
  p50: 500,      // 500ms median — already slow
  p99: 3000,     // 3s p99 — tail exhausts connection pools
  errorRate: 0.3, // 30% fail — above typical circuit breaker threshold
});

// fire requests until the circuit trips
async function tripCircuit(cb: CircuitBreaker, service: Function) {
  const calls = Array.from({ length: 20 }, () =>
    cb.execute(() => service()).catch(() => {}) // suppress individual errors
  );
  await Promise.all(calls);
}

Open state

Testing the open state

After the circuit opens, subsequent calls must fail immediately — not after a timeout. This is the core value of the pattern: once the circuit is open, no time is wasted waiting for a downstream that's known to be unhealthy.

it('fails fast when circuit is open', async () => {
  const cb = new SimpleCircuitBreaker({ threshold: 5, timeout: 10000 });
  const degraded = withLatency(
    async () => { throw new Error('downstream error'); },
    { p50: 500, p99: 3000, errorRate: 1.0 }
  );

  // trip the circuit with 5 failures
  for (let i = 0; i < 5; i++) {
    await cb.execute(() => degraded()).catch(() => {});
  }

  expect(cb.state).toBe('open');

  // open circuit must fail immediately
  const start = Date.now();
  await expect(cb.execute(() => degraded()))
    .rejects.toThrow('circuit open');
  const elapsed = Date.now() - start;

  expect(elapsed).toBeLessThan(10); // must be near-instant, not 500ms+
});

Recovery

Testing the half-open recovery

After the cooldown period, the circuit enters half-open state. Set the service back to healthy (errorRate: 0) and verify the circuit closes after a successful probe.

it('closes circuit after successful probe', async () => {
  jest.useFakeTimers();
  const cb = new SimpleCircuitBreaker({ threshold: 3, cooldown: 5000 });

  // wrap a function we can control
  let shouldFail = true;
  const service = withLatency(
    async () => {
      if (shouldFail) throw new Error('down');
      return 'ok';
    },
    { p50: 200, p99: 2000, errorRate: 0 } // errorRate=0; failure controlled above
  );

  // trip the circuit
  for (let i = 0; i < 3; i++) {
    await cb.execute(() => service()).catch(() => {});
  }
  expect(cb.state).toBe('open');

  // advance past cooldown ? half-open
  jest.advanceTimersByTime(5001);
  expect(cb.state).toBe('half-open');

  // "service recovers"
  shouldFail = false;

  // probe succeeds ? circuit closes
  const result = await cb.execute(() => service());
  expect(result).toBe('ok');
  expect(cb.state).toBe('closed');

  jest.useRealTimers();
});

Full example

A minimal circuit breaker with full tests

Here's a minimal circuit breaker implementation and tests for all three states — closed, open, and half-open — using slowdep to control failure conditions.

// SimpleCircuitBreaker.ts
type State = 'closed' | 'open' | 'half-open';

export class SimpleCircuitBreaker {
  state: State = 'closed';
  private failures = 0;
  private openedAt = 0;
  constructor(private opts: { threshold: number; cooldown: number }) {}

  async execute<T>(fn: () => Promise<T>): Promise<T> {
    if (this.state === 'open') {
      const elapsed = Date.now() - this.openedAt;
      if (elapsed < this.opts.cooldown) throw new Error('circuit open');
      this.state = 'half-open';
    }
    try {
      const result = await fn();
      this.failures = 0;
      this.state = 'closed';
      return result;
    } catch (err) {
      this.failures++;
      if (this.failures >= this.opts.threshold) {
        this.state = 'open';
        this.openedAt = Date.now();
      }
      throw err;
    }
  }
}

import { withLatency } from 'slowdep';
import { SimpleCircuitBreaker } from './SimpleCircuitBreaker';

describe('SimpleCircuitBreaker', () => {
  it('stays closed on success', async () => {
    const cb = new SimpleCircuitBreaker({ threshold: 3, cooldown: 1000 });
    const healthy = withLatency(async () => 'ok', { p50: 5, p99: 50 });
    await cb.execute(() => healthy());
    expect(cb.state).toBe('closed');
  });

  it('opens after threshold failures with realistic latency', async () => {
    const cb = new SimpleCircuitBreaker({ threshold: 3, cooldown: 1000 });
    const degraded = withLatency(
      async () => { throw new Error('down'); },
      { p50: 500, p99: 3000, errorRate: 1.0 }
    );
    for (let i = 0; i < 3; i++) {
      await cb.execute(() => degraded()).catch(() => {});
    }
    expect(cb.state).toBe('open');
  });

  it('transitions to half-open after cooldown', async () => {
    jest.useFakeTimers();
    const cb = new SimpleCircuitBreaker({ threshold: 1, cooldown: 5000 });
    const failing = withLatency(
      async () => { throw new Error(); },
      { p50: 100, p99: 500, errorRate: 1.0 }
    );
    await cb.execute(() => failing()).catch(() => {});
    expect(cb.state).toBe('open');
    jest.advanceTimersByTime(5001);
    const healthy = withLatency(async () => 'ok', 'redis');
    await cb.execute(() => healthy());
    expect(cb.state).toBe('closed');
    jest.useRealTimers();
  });
});