Shipping Laravel Logs to Self-Hosted Loki on the Same Box

Loki is not Elasticsearch. That's the first thing worth understanding, because if you approach it expecting Elasticsearch's full-text indexing model, you'll either fight it or give up. Once I stopped fighting it, I got a log aggregation stack running on the same $40 VPS as a Laravel app — no Grafana Cloud account, no separate observability box, no per-seat pricing call with a sales rep.

This post is about that stack: what it is, how to wire it to Laravel's logging system, and where it'll bite you if you're not paying attention.

What Problem Loki Actually Solves

I run managed hosting for a handful of clients — e-commerce, a couple healthcare practices, a biotech doing sample tracking. Each one generates logs. Laravel's default behavior is to write those logs to storage/logs/laravel.log, which works fine until you need to answer "what was happening on the app server at 2:14 AM when the order processor started failing?" At that point you're SSH'd in, running grep through a rotated log file, and cursing.

The standard answer is "ship your logs to a central service." Datadog, Papertrail, Loggly, Grafana Cloud — they all work. They also all have a bill attached, and for smaller clients I'm not going to eat or pass on another $40-$100/month just to search logs.

Loki's pitch: it stores logs cheaply because it doesn't index the content of log lines. It only indexes the labels you attach (app name, environment, log level, etc.). You query with LogQL, which lets you filter by label first, then grep the matching chunks. Storage is cheap object storage or local disk. CPU overhead is low. It fits on the same box.

That trade-off is real — you can't do arbitrary full-text search across all fields the way you can in Elasticsearch — but for Laravel logs, where I mostly want "show me all ERROR lines from the last 6 hours" or "find every log entry that mentions order ID 8841", it's plenty.

The Stack

Three pieces, all running as Docker containers alongside the app:

• Loki — receives and stores logs
• Promtail — tails the Laravel log file and ships lines to Loki
• Grafana — UI for querying (optional if you want to live in curl, but I don't)

I keep this in a docker-compose.logging.yml so it doesn't clutter the app's main compose file.

# docker-compose.logging.yml
version: "3.8"

services:
  loki:
    image: grafana/loki:2.9.4
    container_name: loki
    ports:
      - "127.0.0.1:3100:3100"
    volumes:
      - ./loki-config.yml:/etc/loki/local-config.yaml
      - loki_data:/loki
    command: -config.file=/etc/loki/local-config.yaml
    restart: unless-stopped

  promtail:
    image: grafana/promtail:2.9.4
    container_name: promtail
    volumes:
      - ./promtail-config.yml:/etc/promtail/config.yml
      - /var/www/html/storage/logs:/var/log/laravel:ro
    command: -config.file=/etc/promtail/config.yml
    restart: unless-stopped
    depends_on:
      - loki

  grafana:
    image: grafana/grafana:10.3.1
    container_name: grafana
    ports:
      - "127.0.0.1:3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=changeme
      - GF_AUTH_ANONYMOUS_ENABLED=false
    volumes:
      - grafana_data:/var/lib/grafana
    restart: unless-stopped
    depends_on:
      - loki

volumes:
  loki_data:
  grafana_data:

Note that both Loki and Grafana are bound to 127.0.0.1. I proxy Grafana through Nginx with HTTP auth if I need external access. Loki never needs to be externally reachable.

Loki Config

# loki-config.yml
auth_enabled: false

server:
  http_listen_port: 3100

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    instance_addr: 127.0.0.1
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2024-01-01
      store: tsdb
      object_store: filesystem
      schema: v12
      index:
        prefix: index_
        period: 24h

limits_config:
  retention_period: 720h   # 30 days

compactor:
  working_directory: /loki/compactor
  retention_enabled: true

The retention_period is the one knob that matters most on a single-box setup. 720 hours is 30 days. Tune it down if disk is tight.

Promtail Config

# promtail-config.yml
server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: laravel
    static_configs:
      - targets:
          - localhost
        labels:
          app: myapp
          env: production
          __path__: /var/log/laravel/*.log
    pipeline_stages:
      - multiline:
          firstline: '^\[\d{4}-\d{2}-\d{2}'
          max_wait_time: 3s
      - regex:
          expression: '^\[(?P<timestamp>[^\]]+)\] (?P<channel>\w+)\.(?P<level>\w+): (?P<message>.+)$'
      - labels:
          level:
          channel:

The multiline stage is not optional. Laravel stack traces span dozens of lines. Without it, Promtail ships each line as a separate log entry and your stack traces become confetti.

The regex stage pulls level and channel out of the standard Laravel log format ([2024-05-01 02:14:33] production.ERROR: ...) and promotes them to labels. This is what makes LogQL queries like {app="myapp", level="ERROR"} work.

Wiring Laravel

Laravel's logging config lives in config/logging.php. I use a stack channel that writes to the daily file driver, which is all Promtail needs.

// config/logging.php (relevant excerpt)
'channels' => [
    'stack' => [
        'driver' => 'stack',
        'channels' => ['daily'],
        'ignore_exceptions' => false,
    ],

    'daily' => [
        'driver' => 'daily',
        'path' => storage_path('logs/laravel.log'),
        'level' => env('LOG_LEVEL', 'debug'),
        'days' => 7,  // local rotation; Loki keeps the real history
    ],
],

Set LOG_CHANNEL=stack in .env and you're done on the Laravel side. No custom handler, no HTTP calls from the app, no synchronous network I/O in the request path. Promtail tails the file asynchronously. The app doesn't know Loki exists.

That's by design. I've seen setups where the app ships logs directly to a log aggregator via HTTP. If that aggregator hiccups, you've introduced a failure mode into every request. File-based tailing keeps the coupling loose.

Querying in LogQL

Once data is flowing, Grafana's Explore tab with the Loki datasource is where I spend time. A few queries I actually use:

# All errors in the last hour
{app="myapp", level="ERROR"} | line_format "{{.message}}"

# Find a specific order ID across all levels
{app="myapp"} |= "order_8841"

# Count errors by the 5-minute bucket (for a rate panel)
sum(count_over_time({app="myapp", level="ERROR"}[5m]))

# Stack traces — just look for "Exception" in the line
{app="myapp", level="ERROR"} |= "Exception"

LogQL's |= operator is a case-sensitive substring match. |~ takes a regex. Both work on the raw log line after label filtering narrows the chunk set.

The Gotchas That Bit Me

Log rotation. Laravel's daily driver rotates logs at midnight by renaming laravel.log to laravel-2024-05-01.log and creating a new laravel.log. Promtail handles this fine if your __path__ glob covers *.log. I initially had it pointing at only laravel.log by name. It stopped shipping logs every night at midnight and I didn't notice for three days.

Out-of-order entries. Loki 2.x rejects log entries with timestamps older than the max_chunk_age (default 2 hours) or that arrive out of order within a stream. If you replay or copy an old log file into the watched directory, Promtail will try to ship entries with old timestamps and Loki will drop them with a 400. You'll see errors in Promtail's own logs. Not catastrophic, but confusing the first time.

Label cardinality. This is the thing Loki docs warn about and developers ignore until it hurts. Do not make user_id or request_id a label. Labels are indexed. High-cardinality labels explode your index. Put variable data in the log line itself and use |= or |~ to search it. I made this mistake early on by parsing the log line and promoting request_id to a label. Loki's memory usage climbed steadily until I figured out what was happening.

Disk usage on busy apps. Loki's filesystem storage on a single box isn't free. I have one client's app that logs aggressively (a biotech's sample ingestion pipeline — a lot of INFO-level audit logging). Thirty days of retention at that volume was hitting 4 GB. Either tune the retention down, raise the disk, or tell the app to be quieter at INFO in production. I did all three.

Promtail permissions. The Promtail container needs read access to the Laravel log directory. If you're running PHP-FPM as www-data and the log files are 640, you'll need to either add the Promtail container's user to the right group or just chmod 644 on the log directory. I handle it with a group ACL on the host. It's annoying setup but a one-time thing.

When I'd Reach for This

This setup earns its place on single-server deployments where I own the box and the client budget doesn't support a separate observability service. It's genuinely good for:

• VPS-hosted Laravel apps with one or two servers
• Clients who want 30-day log history without a SaaS bill
• Situations where logs contain PHI or other sensitive data and I want them staying on-premise

That last point comes up in healthcare work. Shipping logs to a third-party SaaS requires a BAA and a conversation about what's in those logs. Keeping everything on the client's own server sidesteps that.

I'd skip this and pay for Grafana Cloud or Datadog if:

• The app runs across multiple servers and I need aggregation across all of them (you can do it with this stack, but it gets more complex fast)
• The team needs alerting with on-call routing — Loki + Grafana can do alerting, but it's not as mature as Datadog's incident management
• I'm already paying for a platform that includes log aggregation (some hosting platforms bundle it)

Closing

This isn't the sexiest ops setup, but it works, it's free, and I've had it running reliably on several client boxes for over a year. The multiline Promtail config and the label cardinality rule are the two things you have to get right — get those wrong and you'll either have useless log data or a Loki instance that quietly eats memory. Get them right and you've got a real log aggregation setup for the price of some disk space.