AI API Error Handling Guide: Retry Logic, Timeouts, and Fallbacks

AI API error handling is the practice of gracefully managing the six failure modes of LLM APIs — rate limits, timeouts, provider outages, malformed responses, content filter rejections, and invalid key errors. Production AI applications need retry logic with exponential backoff, circuit breakers, and fallback models to maintain reliability above 99.9%. This guide covers the complete error handling stack with production-ready code.

The 6 AI API Error Types You Must Handle

Every LLM provider returns variations of the same six error categories. Understanding them is the foundation of resilient AI applications:

Production Retry Logic with Exponential Backoff

The single most impactful pattern: retry transient failures with exponentially increasing delays plus random jitter to prevent thundering herds:

import openai, time, random
from openai import OpenAI

client = OpenAI(
    api_key="your-drai-key",
    base_url="https://api.dr-ai.top/v1"
)

def chat_with_retry(messages, max_retries=5, model="gpt-5"):
    """Chat completion with exponential backoff + jitter."""
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model=model, messages=messages, timeout=30
            )
        except openai.RateLimitError:
            # 429: wait and retry — respecting Retry-After if present
            delay = min(2 ** attempt + random.uniform(0, 1), 60)
            time.sleep(delay)
        except openai.APITimeoutError:
            # Timeout: retry with shorter output budget
            if attempt >= 2:
                return fallback_model(messages)
            time.sleep(1)
        except openai.APIStatusError as e:
            if e.status_code >= 500 and attempt < max_retries - 1:
                time.sleep(2 ** attempt)
                continue
            raise  # 4xx errors: don't retry
    raise RuntimeError(f"Failed after {max_retries} attempts")

The jitter (random.uniform(0, 1)) matters at scale: when 1,000 concurrent requests hit a 429 simultaneously, synchronized retries create a second outage. Jitter spreads them out.

Circuit Breakers: Stop Hammering a Dead Provider

Retrying a provider that's down wastes latency and money. A circuit breaker tracks failures and skips the provider entirely when it trips:

class CircuitBreaker:
    def __init__(self, failure_threshold=5, reset_timeout=60):
        self.failures = 0
        self.threshold = failure_threshold
        self.reset_timeout = reset_timeout
        self.last_failure = 0
        self.state = "closed"  # closed | open | half-open

    def can_request(self):
        if self.state == "open":
            if time.time() - self.last_failure > self.reset_timeout:
                self.state = "half-open"  # try one request
                return True
            return False
        return True

    def record_success(self):
        self.failures = 0
        self.state = "closed"

    def record_failure(self):
        self.failures += 1
        self.last_failure = time.time()
        if self.failures >= self.threshold:
            self.state = "open"

# Usage: GPT-5 → Claude Sonnet 4 → DeepSeek Chat chain
breakers = {"gpt-5": CircuitBreaker(), "claude-sonnet-4": CircuitBreaker()}
fallbacks = {"gpt-5": "claude-sonnet-4", "claude-sonnet-4": "deepseek-chat"}

DrAI implements exactly this pattern server-side: breakers trip after 5 consecutive failures and half-open after 60 seconds. During two OpenAI outages in 2026, requests automatically routed to Claude and DeepSeek with zero user-visible errors.

Automatic Model Fallback Chains

Combine retries with fallbacks for maximum resilience. The pattern: retry the same model for transient errors, switch models for provider outages:

FALLBACK_CHAIN = ["gpt-5", "claude-sonnet-4", "deepseek-chat"]

def resilient_chat(messages):
    last_error = None
    for model in FALLBACK_CHAIN:
        try:
            return chat_with_retry(messages, model=model)
        except Exception as e:
            last_error = e
            log.warning(f"Model {model} failed: {e}, trying next")
    raise last_error

With DrAI this works out of the box — configure fallbacks per API key in the dashboard and the gateway handles the chain server-side, saving you the client-side complexity and the extra latency of a failed first attempt.

Handling Streaming Errors Mid-Stream

Streaming responses can fail after partial delivery — the worst case. Your code must handle a stream that dies at token 500 of 1,000:

def stream_with_recovery(messages):
    collected = []
    try:
        stream = client.chat.completions.create(
            model="gpt-5", messages=messages, stream=True
        )
        for chunk in stream:
            delta = chunk.choices[0].delta.content
            if delta:
                collected.append(delta)
                yield delta
    except Exception as e:
        # Option 1: regenerate with collected context as prefix
        # Option 2: yield an error marker and let UI handle it
        yield f"\n[stream interrupted: regenerating...]"

# Key point: always persist collected tokens so a retry
# can continue rather than restart from scratch

Error Monitoring: What to Measure

Instrument these four metrics per model and per provider:

• Error rate by type — 429s vs 5xx vs timeouts tell different stories. Rising 429s mean you need higher limits; rising 5xx means provider trouble.
• p95 retry latency — the cost of your retry policy. If p95 exceeds 10s, your timeouts are too generous.
• Fallback trigger rate — how often the chain activates. Above 2% weekly indicates a provider reliability problem.
• Circuit breaker state changes — each trip is an incident worth reviewing.

Timeout Budgets: Set Them Explicitly

Default SDK timeouts (10 minutes in some clients) are production suicide. Set explicit budgets per use case:

The Complete Pattern, Summarized

1. Classify errors: retryable (429, 5xx, timeout) vs not (400, 401)
2. Exponential backoff with jitter for retryable errors, max 5 attempts
3. Circuit breaker per provider: trip at 5 failures, reset after 60s
4. Fallback chain across providers for outage resilience
5. Explicit timeout budgets per use case
6. Monitor error rate, retry latency, and fallback frequency

DrAI handles steps 2-4 server-side — your client code only needs sensible timeouts and clean error surfaces. Read our rate limiting best practices and the LLM security guide for adjacent hardening, or check DrAI pricing to try the gateway free.