Streaming AI Responses: Server-Sent Events vs WebSocket Implementation

Published 2026-07-26 · 14 min read

Nobody likes staring at a loading spinner for 15 seconds while an AI generates a long response. Streaming changes the experience entirely — tokens appear as they're generated, making even complex queries feel fast and interactive. But implementing streaming correctly involves choosing the right protocol (SSE vs WebSocket), handling partial responses, and managing client-side state. This guide covers everything you need to build production-grade streaming AI interfaces.

Why Streaming Matters for AI Applications

Without streaming, the user experience is: type message → wait 5-20 seconds → entire response appears at once. With streaming: type message → first word appears in 200ms → text flows in real-time. This isn't just a cosmetic improvement — it fundamentally changes how users perceive your application's speed.

Studies show that perceived latency (time to first token) matters more than total latency. A response that starts streaming in 200ms and takes 10 seconds to complete feels faster than a response that takes 5 seconds to return all at once. Streaming gives you that 200ms first-token advantage.

Protocol Choice: SSE vs WebSocket

The two main protocols for streaming AI responses:

Recommendation: Use SSE for AI chat applications. The OpenAI API (and DrAI) uses SSE for streaming, and it's the industry standard. WebSocket is overkill unless you need bidirectional real-time communication (like collaborative editing).

How SSE Streaming Works in the OpenAI/DrAI API

When you set "stream": true in your API request, the response switches from a single JSON object to a stream of SSE events:

# Request with streaming enabled
curl https://ai.dr-ai.top/v1/chat/completions \
  -H "Authorization: Bearer sk-your-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5-mini",
    "messages": [{"role":"user","content":"Write a haiku"}],
    "stream": true
  }'

# Response is a stream of SSE events:
data: {"id":"1","choices":[{"delta":{"role":"assistant"}}]}
data: {"id":"1","choices":[{"delta":{"content":"Silent"}}]}
data: {"id":"1","choices":[{"delta":{"content":" morning"}}]}
data: {"id":"1","choices":[{"delta":{"content":" light,"}}]}
data: {"id":"1","choices":[{"delta":{"content":"
Dew drops"}}]}
data: {"id":"1","choices":[{"delta":{},"finish_reason":"stop"}]}
data: [DONE]

Each data: line is a JSON object containing a delta — the incremental text generated since the last event. Concatenate all deltas to build the complete response. The stream ends with data: [DONE].

Implementing SSE Streaming in the Browser

For browser-based applications, the Fetch API with ReadableStream is the most flexible approach:

async function streamChat(messages, onToken, onComplete, onError) {
    const response = await fetch('https://ai.dr-ai.top/v1/chat/completions', {
        method: 'POST',
        headers: {
            'Authorization': 'Bearer sk-your-key',
            'Content-Type': 'application/json'
        },
        body: JSON.stringify({
            model: 'gpt-5-mini',
            messages: messages,
            stream: true
        })
    });
    
    if (!response.ok) {
        throw new Error(`HTTP ${response.status}`);
    }
    
    const reader = response.body.getReader();
    const decoder = new TextDecoder();
    let buffer = '';
    let fullResponse = '';
    
    try {
        while (true) {
            const { done, value } = await reader.read();
            if (done) break;
            
            buffer += decoder.decode(value, { stream: true });
            const lines = buffer.split('\n');
            buffer = lines.pop(); // Keep incomplete line in buffer
            
            for (const line of lines) {
                const trimmed = line.trim();
                if (!trimmed || !trimmed.startsWith('data: ')) continue;
                
                const data = trimmed.slice(6);
                if (data === '[DONE]') {
                    onComplete(fullResponse);
                    return;
                }
                
                try {
                    const parsed = JSON.parse(data);
                    const delta = parsed.choices[0]?.delta?.content;
                    if (delta) {
                        fullResponse += delta;
                        onToken(delta, fullResponse);  // Update UI
                    }
                } catch (e) {
                    console.warn('Parse error:', e);
                }
            }
        }
    } catch (error) {
        onError(error);
    }
}

// Usage: stream tokens directly to the DOM
const output = document.getElementById('chat-output');
await streamChat(
    [{role: 'user', content: 'Explain quantum computing'}],
    (token, full) => { output.textContent = full; },           // onToken
    (full) => { console.log('Complete:', full); },              // onComplete
    (err) => { console.error('Error:', err); }                   // onError
);

Implementing SSE Streaming in Python

import httpx
import asyncio
import json

async def stream_chat(api_key, messages, model="gpt-5-mini"):
    async with httpx.AsyncClient() as client:
        async with client.stream(
            "POST",
            "https://ai.dr-ai.top/v1/chat/completions",
            headers={
                "Authorization": f"Bearer {api_key}",
                "Content-Type": "application/json"
            },
            json={
                "model": model,
                "messages": messages,
                "stream": True
            },
            timeout=60.0
        ) as response:
            async for line in response.aiter_lines():
                if line.startswith("data: "):
                    data = line[6:]
                    if data == "[DONE]":
                        break
                    chunk = json.loads(data)
                    delta = chunk["choices"][0]["delta"].get("content", "")
                    if delta:
                        print(delta, end="", flush=True)
            print()  # Final newline

# Run it
asyncio.run(stream_chat("sk-your-key", [
    {"role": "user", "content": "Write a Python web scraper tutorial"}
]))

Proxying SSE Through Your Backend

Never expose your API key in frontend code. Proxy the stream through your backend, which adds the Authorization header and forwards the SSE stream to the client. Here's a Node.js/Express implementation:

const express = require('express');
const app = express();

app.post('/api/chat/stream', async (req, res) => {
    // Set SSE headers
    res.setHeader('Content-Type', 'text/event-stream');
    res.setHeader('Cache-Control', 'no-cache');
    res.setHeader('Connection', 'keep-alive');
    res.setHeader('X-Accel-Buffering', 'no'); // Disable Nginx buffering
    
    try {
        const response = await fetch('https://ai.dr-ai.top/v1/chat/completions', {
            method: 'POST',
            headers: {
                'Authorization': `Bearer ${process.env.DRAI_API_KEY}`,
                'Content-Type': 'application/json'
            },
            body: JSON.stringify({
                ...req.body,
                stream: true
            })
        });
        
        // Pipe the upstream stream directly to the client
        const reader = response.body.getReader();
        const decoder = new TextDecoder();
        
        while (true) {
            const { done, value } = await reader.read();
            if (done) break;
            res.write(decoder.decode(value));
        }
        
        res.end();
    } catch (error) {
        console.error('Stream error:', error);
        res.write(`data: ${JSON.stringify({error: error.message})}\n\n`);
        res.end();
    }
});

Critical Nginx config: If you're behind Nginx, you MUST disable proxy buffering for SSE endpoints, or Nginx will accumulate the entire response before sending anything to the client:

# Nginx config for SSE proxying
location /api/chat/stream {
    proxy_pass http://127.0.0.1:3000;
    proxy_http_version 1.1;
    proxy_set_header Connection '';
    proxy_buffering off;        # CRITICAL - without this, no streaming!
    proxy_cache off;
    proxy_read_timeout 300s;   # Allow long-running streams
    chunked_transfer_encoding on;
}

Handling Edge Cases

Partial Chunks and Buffer Management

SSE data can arrive split across TCP packets. Always buffer incomplete lines and process them only when you see a complete data: line ending with a newline. The buffer pattern in the examples above handles this correctly.

Connection Drops Mid-Stream

If the connection drops during streaming, you have two options: show the partial response with an error indicator, or attempt to reconnect and resume. For most applications, showing the partial response with a "connection lost" message is simpler and sufficient. The EventSource API (native SSE in browsers) auto-reconnects, but the OpenAI API doesn't support stream resumption.

Timeout Handling

Long responses can take 30+ seconds to fully stream. Set generous timeouts on both client and server. For DrAI, the default stream timeout is 300 seconds — sufficient for even the longest responses.

Concurrent Streams

If multiple users are streaming simultaneously, ensure your server can handle concurrent SSE connections. Each SSE connection holds an HTTP connection open — traditional servers (PHP-FPM, default Express) handle limited concurrent connections. Use async frameworks (FastAPI, async Express) or a dedicated streaming server.

Advanced: Function Calling with Streaming

When the model calls functions (tools) during a streamed response, the function arguments arrive incrementally. You need to buffer them until complete, then parse the JSON:

// Handle streamed function calls
let functionArgs = '';
for (const chunk of stream) {
    const delta = chunk.choices[0].delta;
    
    if (delta.tool_calls) {
        for (const tc of delta.tool_calls) {
            if (tc.function.arguments) {
                functionArgs += tc.function.arguments;
                // Don't parse yet - arguments are incomplete
            }
        }
    } else if (delta.content) {
        // Regular text content - display immediately
        displayText(delta.content);
    }
}
// After stream ends, parse the complete function arguments
const args = JSON.parse(functionArgs);
executeFunction(args);

Performance Optimization

Token-by-token vs chunk-by-chunk: Some UIs update the DOM for every single token, which can cause performance issues with fast models. Consider batching DOM updates using requestAnimationFrame to update every 16ms instead of every token.

Compression: SSE streams can be gzip-compressed by the server, reducing bandwidth by 60-80% for text-heavy responses. However, compression adds latency — enable it for long responses, disable for short ones.

Conclusion

Streaming is no longer optional for AI applications — it's the baseline user expectation. Server-Sent Events (SSE) is the right protocol for 95% of use cases, and the OpenAI-compatible API (used by DrAI) supports it natively. Implement it with proper buffering, backend proxying, and Nginx configuration, and your application will feel instant even when generating complex responses.

For a complete chatbot implementation with streaming, see our chatbot integration guide. For handling the infrastructure side (rate limits, failover), check our rate limiting guide. Ready to build? Get started with DrAI.