OpenAI Streaming in PHP: Surviving the Half-Written Response

Streaming from the OpenAI API feels like a solved problem until it isn't. The docs are clean, the happy path works first try, and then you deploy to production and start getting support tickets about responses that just... stop. Mid-word. Mid-list. Sometimes mid-JSON if you're doing structured output. The timeout problem is real, it's silent, and the default PHP setup almost guarantees you'll hit it.

What Streaming Actually Buys You

The non-streaming OpenAI call is simple: send a request, wait up to 30-60 seconds, get back a big JSON blob. That works fine for short completions. It falls apart the moment a user asks for anything that takes more than a few seconds to generate — a long document, a detailed analysis, a multi-step plan. You're either staring at a spinner, or your web server has already given up and returned a 504.

Streaming flips the model. The API sends you server-sent events (SSE) — little data: chunks as tokens are generated — and you forward them to the browser in real time. The user sees words appearing as they're written. Perceived latency drops to near zero. It's genuinely better UX for anything beyond a one-liner.

The problem is that PHP and most web servers were not designed with long-running streaming connections in mind. And OpenAI's API can take 30, 60, even 90+ seconds to finish a long completion. Those two facts collide in ugly ways.

The Timeout Stack You're Fighting

There isn't one timeout. There are four, and they're independent:

PHP max_execution_time — default 30 seconds. Kills the process hard.
default_socket_timeout — controls how long stream wrappers wait. Often 60 seconds.
Nginx/Apache upstream timeout — fastcgi_read_timeout in Nginx defaults to 60 seconds.
Your HTTP client's own timeout — Guzzle, Symfony HttpClient, cURL — whatever you're using to call OpenAI.

You can get bit by any one of these independently. I had a client's healthcare portal — a clinical notes assistant — where the PHP process was fine, but Nginx was silently closing the upstream connection at 60 seconds. The response to the browser just stopped. No error. No exception in the logs. The frontend JavaScript's EventSource closed cleanly because the server closed the connection cleanly. From the user's perspective the AI just got tired and quit.

A Working Streaming Endpoint in Laravel

Here's the actual pattern I use. This is a Laravel route, streaming an OpenAI chat completion directly to the browser as SSE.

<?php

use Illuminate\Http\Request;
use Illuminate\Support\Facades\Route;
use OpenAI\Laravel\Facades\OpenAI;

Route::post('/api/stream-completion', function (Request $request) {
    $request->validate([
        'messages' => 'required|array',
        'messages.*.role' => 'required|in:user,assistant,system',
        'messages.*.content' => 'required|string|max:32000',
    ]);

    // Extend PHP execution time for this request only.
    // 0 = no limit. Use a sane ceiling in production.
    set_time_limit(120);

    return response()->stream(function () use ($request) {
        $stream = OpenAI::chat()->createStreamed([
            'model' => 'gpt-4o',
            'messages' => $request->input('messages'),
            'max_tokens' => 2048,
        ]);

        $buffer = '';
        $finishReason = null;

        try {
            foreach ($stream as $response) {
                $delta = $response->choices[0]->delta->content ?? '';
                $finishReason = $response->choices[0]->finishReason;

                if ($delta !== '') {
                    $buffer .= $delta;
                    // SSE format: "data: ...\n\n"
                    echo 'data: ' . json_encode(['token' => $delta]) . "\n\n";
                    ob_flush();
                    flush();
                }
            }

            // Signal clean completion to the client.
            echo 'data: ' . json_encode([
                'done' => true,
                'finish_reason' => $finishReason,
            ]) . "\n\n";
            ob_flush();
            flush();

        } catch (\Exception $e) {
            // Tell the client something went wrong — don't just close silently.
            echo 'data: ' . json_encode([
                'error' => true,
                'message' => 'Stream interrupted. Please try again.',
            ]) . "\n\n";
            ob_flush();
            flush();

            \Log::error('OpenAI stream error', [
                'message' => $e->getMessage(),
                'buffer_length' => strlen($buffer),
            ]);
        }
    }, 200, [
        'Content-Type' => 'text/event-stream',
        'Cache-Control' => 'no-cache',
        'X-Accel-Buffering' => 'no', // Critical for Nginx.
    ]);
});

That X-Accel-Buffering: no header is not optional. Nginx buffers proxy responses by default. Without it, your tokens accumulate in Nginx's buffer and get flushed in unpredictable batches — or not until the connection closes. I have spent an embarrassing amount of time staring at a frozen UI before I remembered this header.

The Gotchas That Will Bite You

ob_flush() before flush(). PHP has its own output buffer on top of the system buffer. You need both. Miss ob_flush() and your tokens sit in PHP's buffer. Miss flush() and they sit in the SAPI layer. Some setups also need ob_implicit_flush(true) at the top of the script, though in my experience the explicit pair handles it.

The half-written response has no exception. This is the insidious one. If Nginx cuts the upstream connection mid-stream, the OpenAI PHP client doesn't always throw. The foreach just ends early. $finishReason is null. $buffer has partial content. From PHP's perspective, everything was fine. You need to check $finishReason at the end and treat null as a failure state.

set_time_limit() resets the clock, not the limit. Call it at the start of the stream callback, not before. And set it to something real — I use 120 seconds as a ceiling. 0 (unlimited) is tempting but it means a runaway request can hold a PHP-FPM worker forever.

Nginx's fastcgi_read_timeout. You need to set this in your server block:

location ~ \.php$ {
    fastcgi_read_timeout 120s;
    # rest of your fastcgi config
}

Make sure it matches or exceeds your set_time_limit() value. I've been burned by mismatches — PHP is happy to keep running but Nginx already hung up.

The OpenAI client's own cURL timeout. The openai-php/client library uses Guzzle under the hood. If you're configuring it manually, check that timeout and read_timeout aren't set too conservatively. For streaming, I set read_timeout to 0 (wait forever for the next chunk) and rely on the overall max_execution_time as the backstop.

Recovery on the Frontend

Even with all of the above locked down, networks are unreliable. My frontend always tracks whether it received a clean done: true event. If the EventSource closes without it, I show a "Response may be incomplete" warning and offer a retry button. Simple, but clients notice and appreciate it.

const source = new EventSource('/api/stream-completion', { /* ... */ });
let receivedDone = false;
let fullText = '';

source.onmessage = (event) => {
    const data = JSON.parse(event.data);
    if (data.done) {
        receivedDone = true;
        source.close();
        return;
    }
    if (data.error) {
        showRetryUI('Stream error. Try again.');
        source.close();
        return;
    }
    fullText += data.token;
    renderText(fullText);
};

source.onerror = () => {
    source.close();
    if (!receivedDone) {
        showRetryUI('Response may be incomplete.');
    }
};

Note that EventSource doesn't support POST natively, which is annoying for sending message history. I work around this by writing the conversation to a short-lived server-side session key first, then firing the GET-based SSE request with that key. Alternatively, use fetch() with a ReadableStream — more setup, but it handles POST cleanly and gives you more control.

When I'd Reach for This (and When I Wouldn't)

I use streaming for anything where the completion might take more than three or four seconds and a human is waiting on it. Clinical note drafting, long-form content generation, detailed code explanations. The UX improvement is real and clients notice immediately.

I don't stream for background jobs. If I'm generating a report that goes into a database or a PDF that gets emailed, there's no human watching a cursor blink. In that case I queue a job, call the non-streaming endpoint with a generous timeout, and move on. Simpler, easier to retry, easier to log.

I also don't stream when I need structured JSON output. Partial JSON is worse than no JSON — it's harder to detect failure and you can't parse it incrementally without a streaming JSON parser, which is more complexity than the UX benefit is usually worth. I call the non-streaming endpoint, validate the full response, and cache aggressively.

The Bottom Line

Streaming OpenAI responses in PHP is not hard, but there are four independent timeout layers that can each silently kill your connection, and the failure mode is a clean-looking truncation rather than an error you can catch. Get your headers right, extend the right timeouts at the right layer, and always signal completion explicitly so the client knows whether to trust what it received. The half-written response problem is entirely solvable — you just have to know it's coming.