log in
consulting hosting industries the daily tools about contact
← All posts
· 6 min read · Docker healthchecks

Docker Healthchecks That Actually Mean Something

Your container says healthy. Your app is throwing 500s. Here's why that happens and how to write healthchecks that actually gate traffic.

Your container is healthy. Your app is throwing 500s on every request. Your load balancer is happily routing traffic into the void. Welcome to the false confidence of a default Docker healthcheck.

I've debugged this exact situation more than once. Most recently on a Laravel app running behind Traefik for a healthcare client — the container was "healthy" for six minutes while the database connection pool was exhausted and every user hit a blank white page. Nobody got alerted because the orchestrator saw green. The healthcheck was lying.

What a Docker Healthcheck Actually Does

A Docker HEALTHCHECK instruction tells the daemon to periodically run a command inside the container. If the command exits 0, the container is healthy. Non-zero means unhealthy. That's the entire mechanism. Docker doesn't care what the command does — it just cares about the exit code.

The failure mode is obvious once you see it: people write healthchecks that test whether the process is running, not whether the application is functional. Those are very different things.

A PHP-FPM process can be alive and accepting connections while:

  • The database is unreachable
  • Redis has dropped and your session layer is broken
  • A queue worker died and jobs are silently piling up
  • Your app's .env got nuked on a bad deploy and config is half-loaded
  • An upstream API your app depends on is down and every request is hanging

The process is fine. The app is on fire. The healthcheck returns 0.

The Lazy Default (and Why Everyone Uses It)

The most common healthcheck I see in the wild:

HEALTHCHECK CMD curl -f http://localhost/ || exit 1

This checks that something responded on port 80. If nginx is up and serves your 500 error page, this passes. If your app throws an exception on the index route but still returns a 200 (yes, this happens — I've seen Laravel catch exceptions in a top-level handler and return 200 OK with an error body), this passes.

It's not useless. It's better than nothing. But it's nowhere near good enough if you're using health status to gate traffic in a production cluster.

Write a Real Health Endpoint

The fix is straightforward: write an actual health endpoint in your application that checks the things your application actually needs. Then point your healthcheck at that.

Here's what I ship in Laravel projects:

// routes/web.php or routes/api.php
Route::get('/healthz', function () {
    $checks = [];
    $status = 200;

    // Database
    try {
        DB::connection()->getPdo();
        $checks['db'] = 'ok';
    } catch (\Exception $e) {
        $checks['db'] = 'fail';
        $status = 503;
    }

    // Redis / Cache
    try {
        Cache::store('redis')->put('healthz', 1, 10);
        $checks['cache'] = 'ok';
    } catch (\Exception $e) {
        $checks['cache'] = 'fail';
        $status = 503;
    }

    // Storage writable
    try {
        $path = storage_path('app/.healthz');
        file_put_contents($path, time());
        unlink($path);
        $checks['storage'] = 'ok';
    } catch (\Exception $e) {
        $checks['storage'] = 'fail';
        $status = 503;
    }

    return response()->json($checks, $status);
})->middleware('throttle:60,1');

Then in the Dockerfile:

HEALTHCHECK --interval=15s --timeout=5s --start-period=30s --retries=3 \
  CMD curl -sf -o /dev/null -w "%{http_code}" http://localhost/healthz | grep -q 200 || exit 1

That -sf on curl suppresses output and fails silently on HTTP errors. The -w "%{http_code}" writes just the status code to stdout, and grep -q 200 means only an actual 200 passes. A 503 from your health endpoint fails the check. A hung connection that times out fails the check. A 500 from an unhandled exception fails the check.

Note --start-period=30s. That's critical. Without it, Docker starts checking immediately and marks your container unhealthy before PHP-FPM or your queue workers have finished initializing. I've had deploys fail in staging because the healthcheck fired before Octane finished booting. Give it breathing room.

The Gotchas That Will Bite You

1. Don't check things you can't fix.

If an upstream third-party API is down — say, an insurance eligibility verification service — and you include that in your healthcheck, your container goes unhealthy through no fault of your own. The orchestrator kills and restarts it. The new container also fails the healthcheck. You just took your whole app offline because a vendor had a bad morning. Check your own infrastructure dependencies only. Log upstream failures differently.

2. Timeouts compound.

If your healthcheck --timeout is 5s and your DB check can hang for 30s waiting for a TCP timeout, the healthcheck itself will time out and get killed — but the DB::connect() call might hold a connection slot or leave a zombie. Keep your health endpoint fast. Set explicit short timeouts on your dependency checks:

// Force a short socket timeout on the DB check
try {
    DB::connection()->statement('SELECT 1');
    $checks['db'] = 'ok';
} catch (\Exception $e) {
    $checks['db'] = 'fail';
    $status = 503;
}

For the DB specifically, I'll sometimes set PDO::ATTR_TIMEOUT on the connection or use a raw socket connect with stream_socket_client() and a 2s timeout instead of going through Eloquent at all.

3. The health endpoint itself can break things.

A healthcheck fires every 15 seconds per container. If you run 10 containers, that's 40 DB queries per minute just from health probes — more if you're also running Kubernetes liveness and readiness probes separately. On a tight connection pool, that matters. Throttle the route (I do in the example above). Consider using a read replica or a dedicated low-privilege health user that can only run SELECT 1.

4. Swarm and Kubernetes handle unhealthy differently.

In Docker Swarm, an unhealthy container gets restarted. In Kubernetes, it depends on whether you're using a liveness probe (restart the pod) or a readiness probe (pull it from the load balancer but don't restart). Those are different tools for different failure modes. I use liveness for "is the process dead" and readiness for "is the app ready to serve traffic" — and they can point at different endpoints or have different thresholds. Don't conflate them.

5. --retries is not the same as a grace period.

If you set --retries=3 and --interval=15s, a container needs to fail three consecutive checks before Docker marks it unhealthy. That's 45 seconds of routing traffic to a broken container before anything reacts. For most apps that's fine. For a payment processing endpoint, it's not. Tune these numbers to your actual tolerance, not the defaults.

When This Really Matters

If you're running a single container on a single VPS and you SSH in when things break, your healthcheck being sloppy is an inconvenience. Write a better one anyway, but the blast radius is small.

Where this becomes genuinely critical:

  • Rolling deploys. If your orchestrator uses health status to decide when to cut traffic to the new container, a lying healthcheck means you cut over before the app is ready. I watched a Laravel app with a multi-minute migration fail silently in production because the healthcheck passed before php artisan migrate finished running in the entrypoint script.
  • Auto-scaling. New instances that aren't actually ready get traffic. Users get errors. The autoscaler thinks it's a load problem and adds more broken instances.
  • Multi-region / active-active setups. A broken container in one region that reports healthy skews traffic toward it instead of away. The opposite of what you want.

For the biotech client I mentioned at the top — once I put a real healthcheck in place that tested the DB connection, the next time the connection pool exhausted (it happened again three weeks later, different root cause), Traefik saw the 503s, pulled the container from rotation within one check interval, and the on-call engineer got a clean alert instead of a wall of user complaints.

When I'd Skip the Fancy Health Endpoint

Simple static sites or apps with no runtime dependencies — just nginx serving compiled assets. In that case, curl -f http://localhost/ is fine. If nginx is up, you're healthy.

Also: don't over-engineer this into a full observability platform. I've seen health endpoints that run 15 checks, do DNS lookups, ping CDN edges, and take 4 seconds to respond. That's a monitoring tool, not a healthcheck. Keep it fast, keep it focused on the dependencies that will actually take your app down.

The Takeaway

A healthcheck that only proves your process is running is a false signal dressed up as reliability. Write one that actually exercises your critical path, return a real HTTP status code from your app, and tune the timing parameters for your specific deploy workflow. It takes an hour to do right and it will save you at least one 2am incident a year. That's a good trade.

docker devops containers healthchecks php laravel infrastructure

Need help shipping something like this? Get in touch.