Laravel Horizon: Size Workers by Memory, Not Queue Depth

Most teams ship Horizon with the default config, bump numProcesses when queues back up, and consider the job done. That worked fine until I had a queue worker OOM-kill itself mid-job and corrupt a batch of records for a healthcare client. That's when I started actually reading what memory in the processes supervisor config does — and realized I'd been thinking about worker sizing wrong for years.

The Problem Horizon Actually Solves Here

Horizon's process manager operates in two modes: simple and auto. Most people use auto because it scales worker count based on queue throughput. What it does not automatically account for is how much RAM each individual job needs to do its work.

A numProcesses of 5 means Horizon can spin up to 5 workers. Fine. But if each of those workers is processing a job that loads a 50MB CSV, parses it, and runs transformations — you've now got 250MB of working memory potentially live at once, on top of PHP's own baseline, on top of whatever else is running on the box. Multiply that by a few supervisor groups and you'll hit the wall.

The memory key in your supervisor configuration is the per-process memory limit that tells Horizon when to retire a worker. The maxProcesses key caps how many it'll spawn. These two numbers together determine your actual memory envelope. Almost every team I've seen sets memory to 128 and never revisits it.

What pmset Actually Controls

Horizon uses a concept it calls ProcessPool management, and the memory config maps to the --memory option passed to the underlying queue:work command. When a worker exceeds that limit after finishing a job, it exits gracefully and Horizon restarts it. That's the retirement mechanism.

The balance strategy with auto or simple affects how many processes Horizon targets. The memory limit affects when each process is recycled. These are separate axes and you need to tune both.

Here's a typical supervisor block that most teams ship:

'supervisor-1' => [
    'connection' => 'redis',
    'queue' => ['default'],
    'balance' => 'auto',
    'minProcesses' => 1,
    'maxProcesses' => 10,
    'memory' => 128,
    'tries' => 3,
    'timeout' => 60,
],

Fine for lightweight jobs. Completely wrong for anything that touches large payloads.

Sizing by Job Memory Footprint

The right approach is to measure your actual jobs first, then set limits that reflect reality.

I use a simple job middleware to capture peak memory:

<?php

namespace App\Jobs\Middleware;

use Illuminate\Support\Facades\Log;

class MeasureMemory
{
    public function handle(mixed $job, callable $next): void
    {
        $before = memory_get_usage(true);

        $next($job);

        $after = memory_get_peak_usage(true);
        $delta = ($after - $before) / 1024 / 1024;

        Log::channel('horizon_sizing')->info('job_memory', [
            'job' => get_class($job),
            'peak_mb' => round($delta, 2),
        ]);
    }
}

Drop that on your job classes during a profiling run in staging:

public function middleware(): array
{
    return [new MeasureMemory()];
}

Run representative load and collect the logs. You want the 95th percentile, not the average — the outliers are what kill you.

Once you have real numbers, structure your Horizon config by job class profile rather than by queue name alone:

'supervisors' => [

    // Lightweight jobs: notifications, webhooks, small DB writes
    'lightweight' => [
        'connection' => 'redis',
        'queue' => ['notifications', 'webhooks', 'default'],
        'balance' => 'auto',
        'minProcesses' => 2,
        'maxProcesses' => 20,
        'memory' => 96,
        'tries' => 3,
        'timeout' => 30,
    ],

    // Heavy jobs: report generation, CSV imports, PDF rendering
    'heavy' => [
        'connection' => 'redis',
        'queue' => ['reports', 'imports'],
        'balance' => 'simple',
        'numProcesses' => 3,
        'memory' => 512,
        'tries' => 2,
        'timeout' => 300,
    ],

    // Very heavy: LIMS data sync, large image processing
    'bulk' => [
        'connection' => 'redis',
        'queue' => ['bulk-sync'],
        'balance' => 'simple',
        'numProcesses' => 1,
        'memory' => 1024,
        'tries' => 1,
        'timeout' => 900,
    ],

],

Notice I'm using simple balance for the heavy groups and capping numProcesses deliberately. auto balance with a high maxProcesses on a memory-intensive queue is how you accidentally spawn 8 workers that each need 400MB and watch your server swap itself to death during a traffic spike.

The Gotchas That Bit Me

The memory limit is checked after job completion, not during. If a single job blows past your memory limit mid-execution, the worker doesn't get killed by Horizon — it gets killed by the OS or PHP's own memory_limit in php.ini. The Horizon memory config is a retirement trigger, not a hard cap. You need both: a Horizon memory value that causes graceful recycling, and a php.ini memory_limit that acts as the hard ceiling.

For the heavy group above, I'd set memory_limit = 768M in PHP and memory => 512 in Horizon. The Horizon limit triggers normal retirement after a job finishes cleanly. PHP's limit is the emergency brake.

maxProcesses is per supervisor, not global. If you have three supervisor groups each with maxProcesses => 10, you can have 30 workers alive simultaneously. Do the math against your available RAM before you deploy.

The balance timing matters. With auto, Horizon rebalances every second by default. If a heavy job takes 4 minutes, Horizon may try to spin up more processes during that window because the queue depth looks wrong mid-execution. For long-running jobs, simple with a fixed numProcesses is more predictable.

Restart behavior after memory retirement. When a worker exits because it hit the memory limit, Horizon replaces it immediately. That's usually fine, but if your job itself has a memory leak and every run pushes the worker over the limit, you'll burn through Redis connections and spawn cycles in a tight loop. I've seen this happen with jobs that were holding open large Eloquent collections without chunking. The symptom is a Horizon worker that shows a very high restart count in the dashboard.

The timeout and memory interact. A job that hangs near the memory limit won't get retired until it finishes (or times out). If timeout is set too high, a leaky job sits there consuming RAM for the full duration. Keep timeouts tight and proportional to what the job actually needs.

When I'd Reach for This

If your queue jobs are all thin — sending emails, updating a few rows, dispatching notifications — the default config is probably fine. You won't hit these problems.

The moment you're processing anything with meaningful payload size — document parsing, image or PDF generation, data sync from an external API that returns large collections, anything involving LIMS or EHR data exports — you need to think about this. I've integrated a handful of LIMS systems for biotech clients and the response payloads can be brutal. One system returned 40,000-row result sets in a single call. Dumping that into a queue job without tuning the memory envelope meant I was playing Russian roulette with worker stability.

Real estate MLS feed processing is another one. Full feed syncs can hit tens of thousands of listings with nested data. I've had to drop those onto a dedicated supervisor group with a 768MB Horizon memory limit, PHP capped at 1GB, one process, and a 15-minute timeout. Sounds extreme, but it runs reliably and doesn't starve other queues.

I wouldn't over-segment supervisor groups for the sake of it. Two or three tiers — light, heavy, and maybe bulk — covers most real applications. More than that and you're maintaining config complexity that rarely pays off.

Closing

Queue configuration is infrastructure sizing, and you can't size infrastructure you haven't measured. Run the memory middleware in staging, look at real numbers, and set your Horizon config to reflect what your jobs actually do — not what you hope they do. The default 128MB memory limit was picked to be safe for simple jobs, not for yours.