nginx Keepalive Defaults Are Quietly Wasting Your Connections

I spent an afternoon last year chasing phantom 502s on a biotech client's internal API server. Load wasn't high. The upstream Laravel app was healthy. But every few minutes, a handful of requests from their Python data pipeline were dying. The culprit was keepalive_requests, and its default value of 1000 — a number that sounds big until you have a client making 50 requests per second.

What keepalive Actually Buys You

HTTP keepalive (or persistent connections) lets a client reuse a single TCP connection for multiple requests instead of paying the three-way handshake tax on every one. For a web browser loading a page, the savings are modest. For an API client in a tight loop — a data pipeline, a webhook processor, a queue worker — the savings are real. You're eliminating syscalls, kernel state, and TLS negotiation overhead on every request.

nginx has two settings that control how long and how hard it lets clients use a persistent connection:

• keepalive_timeout — how long nginx holds an idle connection open waiting for the next request
• keepalive_requests — the maximum number of requests nginx will serve over a single connection before closing it

The defaults ship as:

keepalive_timeout 75s;
keepalive_requests 1000;

keepalive_requests is the one that bites API shops. 1000 sounds like a lot. But a Python worker making 50 requests per second will exhaust that in 20 seconds. nginx closes the connection, the client gets a reset or a 502 depending on timing, and you spend an afternoon reading access logs wondering what went wrong.

The Actual Problem in Practice

Here's what happens at the socket level. nginx serves request number 1000 on a connection and then sends a Connection: close header on the response. The client, if it's well-behaved (like Python's requests with a Session, or any HTTP/1.1 client that reads headers), should notice this and open a new connection cleanly. But "should" does a lot of work in that sentence.

I've seen gRPC-transcoding proxies, internal Python scripts written by scientists who aren't web developers, and legacy PHP CLI jobs that don't handle the Connection: close case gracefully. They try to reuse the closed socket, get a RST from the kernel, and either retry (if you're lucky) or surface a 502 or a cryptic connection error (if you're not).

The other scenario: clients that do handle reconnection correctly still pay the TCP + TLS overhead every 1000 requests. If your API clients are over TLS — and they should be, even on an internal network — that's a full TLS handshake every 20 seconds at 50 req/s. Not catastrophic, but measurable.

How I Tune This

For a server that's primarily serving long-lived API clients (as opposed to a public-facing site with browser traffic), I change both values:

http {
    # How long to hold an idle connection open.
    # 75s is fine for browsers. For API clients that batch work,
    # bumping this gives them time between bursts without reconnecting.
    keepalive_timeout 300s;

    # Allow far more requests per connection.
    # 1000 is the default and will silently close connections
    # under sustained API load. Set this high or to 0 (unlimited).
    keepalive_requests 10000;

    # ... rest of your http block
}

If you have a specific server block that's purely an API proxy — say, fronting a Laravel app that only internal services talk to — you can scope it there instead of in the http block:

server {
    listen 443 ssl;
    server_name api-internal.example.com;

    keepalive_timeout 300s;
    keepalive_requests 10000;

    location / {
        proxy_pass http://upstream_app;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

Note proxy_http_version 1.1 and proxy_set_header Connection "" — those are required if you also want keepalive on the connection between nginx and your upstream app server. Without them, nginx defaults to HTTP/1.0 for proxied requests, and keepalive to the upstream doesn't work at all. That's a separate tuning topic, but it's worth doing at the same time.

The Gotchas That Got Me

The 1000-request cliff is silent. nginx doesn't log anything when it closes a connection at the keepalive_requests limit. The Connection: close header is the only signal, and most log setups don't capture response headers. I found it by running tcpdump on the loopback interface and watching for FIN packets after exactly 1000 requests. Once you know to look for it, it's obvious. Before that, it's a ghost.

keepalive_timeout has two values. The directive accepts an optional second parameter:

keepalive_timeout 300s 120s;

The first value is how long nginx keeps the connection open internally. The second value is sent to the client in a Keep-Alive: timeout=120 header as a hint. This matters if your client respects that header, because you can tell it to close the connection a little before nginx does — avoiding a race condition where the client tries to reuse a connection nginx just decided to close. I set the client-facing hint slightly lower than the server-side timeout for this reason.

High keepalive_timeout on public-facing servers can bite you. If you're tuning a server that serves browser traffic or arbitrary third-party clients, holding connections open for 300 seconds means a lot of file descriptors parked doing nothing. Each open connection costs kernel state. Check your worker_connections and worker_rlimit_nofile settings if you push this up significantly:

worker_processes auto;
worker_rlimit_nofile 65535;

events {
    worker_connections 16384;
}

For an internal API server with a bounded number of clients, this isn't a concern — you're not going to have thousands of concurrent idle connections. For a public server, be more conservative, or scope the higher timeouts only to specific server blocks.

Upstream keepalive is a different directive entirely. keepalive_requests and keepalive_timeout in the http or server block control client-to-nginx connections. The connections from nginx to your upstream (PHP-FPM, a Node process, another service) are controlled by the keepalive directive in your upstream block:

upstream upstream_app {
    server 127.0.0.1:9000;
    keepalive 64;  # number of idle keepalive connections to cache
}

I've seen people tune the client-facing settings and wonder why their PHP-FPM server is still getting hammered with connection overhead. It's because they never set up upstream keepalive. Both ends need attention.

When I'd Reach for This Tuning

I do this for any nginx instance that's primarily serving machine-to-machine traffic: internal APIs, webhook receivers, data pipeline endpoints, background job servers. Basically anywhere the clients are code, not browsers.

Specifically:

• Queue workers hitting an internal REST API
• Python or Go data pipelines making batched calls
• Laravel Horizon jobs that call external services through an nginx proxy
• Any service-to-service traffic inside a VPC

I leave the defaults closer to stock for public-facing marketing sites or e-commerce storefronts where the client population is unpredictable and I care more about connection slot availability than minimizing per-request overhead.

I also reach for this when I see any of these symptoms: intermittent 502s under moderate load, connection reset errors in API client logs, or TLS handshake spikes in your APM that don't correspond to traffic spikes. Any of those can be the 1000-request cliff.

How to Verify It's Working

After changing the config and reloading nginx (nginx -s reload), I verify with a quick test. On the client machine, or from a bastion, open a persistent HTTP connection and count requests. With curl:

# Fire 1100 requests and watch for Connection: close in the headers
for i in $(seq 1 1100); do
  curl -s -o /dev/null -w "%{http_code} conn:%{num_connects}\n" \
    --keepalive-time 300 \
    https://api-internal.example.com/health
done | grep "conn:1"

With the default keepalive_requests 1000, you'd see a conn:1 (new connection opened) at request 1001. With keepalive_requests 10000, you should see only the very first request opening a connection. If you're seeing unexpected reconnects, something upstream is still closing the connection early — check your PHP-FPM or app server config too.

The Short Version

nginx's default of 1000 requests per keepalive connection was designed for browser traffic patterns, not machine-to-machine API workloads. If you have long-lived API clients, bump keepalive_requests to at least 10000 (or 0 for unlimited), extend keepalive_timeout to match your client's usage pattern, and while you're in there, set up upstream keepalive too. The default that ships in every nginx install is quietly making your API clients reconnect far more than they need to — and on TLS endpoints, that overhead adds up fast.