slowdep + OpenAI — integration

Preset

openai latency profile

LLM APIs are the slowest dependency most applications deal with. The 'openai' preset reflects real-world gpt-4o latency — 800ms for short responses, up to 8s for longer outputs. The 0.5% error rate captures rate limits and transient 500s.

openai 800ms 4s 8s 0.5%

Why this matters more for AI features

Most services have a 30s request timeout. OpenAI p99 can reach 8s for long completions — that's 27% of your budget on one dependency. Without testing against realistic latency, your timeout thresholds, loading UX, and retry budgets are guesses.

Quick start

Wrap chat completions

Wrap client.chat.completions.create directly. The OpenAI SDK's nested structure means withLatencyAll on the top-level client won't reach deeply nested methods — wrapping the specific method you use is the right approach.

import OpenAI from 'openai';
import { withLatency } from 'slowdep';

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Wrap the specific method — bind to preserve the internal `this` context
const slowCreate = withLatency(
  client.chat.completions.create.bind(client.chat.completions),
  'openai'
);

// Use slowCreate exactly like client.chat.completions.create
const completion = await slowCreate({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Summarize the key points of async programming.' }
  ],
  max_tokens: 500
});

console.log(completion.choices[0].message.content);

Wrapping a client

The fake response pattern

Combine slowdep with a Jest mock to return controlled fake completions while applying realistic latency. This is the most useful pattern for unit tests — you get deterministic output AND realistic timing, without spending tokens.

import { withLatency } from 'slowdep';
import type { ChatCompletion } from 'openai/resources';

// A factory that returns a fake ChatCompletion with the given content
function fakeCompletion(content: string): ChatCompletion {
  return {
    id: 'chatcmpl-test',
    object: 'chat.completion',
    created: Math.floor(Date.now() / 1000),
    model: 'gpt-4o',
    choices: [{
      index: 0,
      message: { role: 'assistant', content, refusal: null },
      logprobs: null,
      finish_reason: 'stop'
    }],
    usage: { prompt_tokens: 20, completion_tokens: 40, total_tokens: 60 }
  };
}

// Fake create: returns a deterministic response with realistic latency
export const mockCreate = withLatency(
  jest.fn().mockResolvedValue(fakeCompletion('Async programming separates task initiation from completion.')),
  'openai'
);

Testing

Test timeout and retry logic

The two most important AI-feature tests: does your timeout fire correctly, and does your retry handler deal with the 0.5% error rate gracefully?

import { withLatency } from 'slowdep';
import { summarizeText } from '../src/ai/summarize';

test('summarizeText throws TimeoutError when OpenAI takes > 10s', async () => {
  // Simulate a very slow completion — e.g. long output, high load
  const slowCreate = withLatency(
    jest.fn().mockResolvedValue(fakeCompletion('summary')),
    { p50: 12000, p99: 20000 }
  );

  await expect(
    summarizeText('long article...', { create: slowCreate, timeoutMs: 10000 })
  ).rejects.toThrow('TimeoutError');
}, 15000);

import { withLatency } from 'slowdep';

test('retries once on OpenAI 429 and succeeds', async () => {
  const mockCreate = jest.fn()
    .mockRejectedValueOnce(Object.assign(new Error('Rate limit exceeded'), { status: 429 }))
    .mockResolvedValueOnce(fakeCompletion('success on retry'));

  // Apply realistic latency to both the failure and the success
  const slowCreate = withLatency(mockCreate, 'openai');

  const result = await callWithRetry(slowCreate, {
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'hello' }]
  });

  expect(result.choices[0].message.content).toBe('success on retry');
  expect(mockCreate).toHaveBeenCalledTimes(2);
}, 20000);

import { render, screen, waitFor } from '@testing-library/react';
import { withLatency } from 'slowdep';
import { SummaryWidget } from '../src/components/SummaryWidget';

test('shows loading spinner while waiting for completion', async () => {
  const slowCreate = withLatency(
    jest.fn().mockResolvedValue(fakeCompletion('The summary is...')),
    { p50: 1500, p99: 3000 }
  );

  render(<SummaryWidget text="article..." createFn={slowCreate} />);

  // Loading state is visible immediately
  expect(screen.getByRole('progressbar')).toBeInTheDocument();

  // After the (simulated) slow response, summary appears
  await waitFor(() => {
    expect(screen.getByText(/The summary is/)).toBeInTheDocument();
  }, { timeout: 5000 });
});

Individual wrapping

Wrap embeddings and other endpoints

The same pattern works for embeddings, moderation, and fine-tuning — wrap the specific method you use. Embeddings are notably faster than chat completions; use a custom profile.

import OpenAI from 'openai';
import { withLatency } from 'slowdep';

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Chat completions — use the openai preset
const slowChat = withLatency(
  client.chat.completions.create.bind(client.chat.completions),
  'openai'
);

// Embeddings are ~10x faster than chat completions
const slowEmbed = withLatency(
  client.embeddings.create.bind(client.embeddings),
  { p50: 80, p99: 400 }
);

// Moderation is fast, similar to embeddings
const slowModerate = withLatency(
  client.moderations.create.bind(client.moderations),
  { p50: 100, p99: 500 }
);

const embedding = await slowEmbed({
  model: 'text-embedding-3-small',
  input: 'What is the meaning of async programming?'
});

Production safety

Never slow real API calls

import OpenAI from 'openai';
import { withLatency } from 'slowdep';

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const rawCreate = client.chat.completions.create.bind(client.chat.completions);

// In tests, always use a mock + slowdep — never hit the real API
// In development, you can optionally add latency to a real client for manual UX testing
export const chatCreate = process.env.NODE_ENV === 'test'
  ? withLatency(jest.fn().mockResolvedValue(fakeCompletion('...')), 'openai')
  : rawCreate;  // always use real client in production and development

Never add latency to real API calls

OpenAI calls already have real latency. Only use slowdep when wrapping a mock function (e.g. jest.fn()) that returns a fake response. If you accidentally wrap a real API call, you'll be adding fake latency on top of real latency.

What to test

Scenarios this integration helps you verify

Request timeout calibration — with p99 of 8s, is your 10s server timeout actually enough margin? What about chained LLM calls?
Loading state UX — does your UI show a spinner within 200ms? Does it show a "still working..." message after 3s?
Retry budget — with 0.5% error rate and 800ms p50, three retries with backoff can take 5+ seconds — is that within your total budget?
Parallel vs sequential calls — two parallel completions at p50 = 800ms is very different from two sequential calls at p50 = 1.6s
Circuit breaker behavior — after several slow responses, does your circuit breaker open and return a cached/fallback response?
Cost guard — for features with usage limits, realistic latency helps you simulate what happens when the token budget is exhausted mid-request

TypeScript

Typed completion wrapper

import OpenAI from 'openai';
import type { ChatCompletion, ChatCompletionCreateParamsNonStreaming } from 'openai/resources';
import { withLatency } from 'slowdep';

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Return type is inferred as Promise<ChatCompletion> — no casting needed
const slowCreate = withLatency(
  (params: ChatCompletionCreateParamsNonStreaming): Promise<ChatCompletion> =>
    client.chat.completions.create(params) as Promise<ChatCompletion>,
  'openai'
);

// Full type inference on the result
const result: ChatCompletion = await slowCreate({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }]
});

const text: string = result.choices[0].message.content ?? '';