Logging Sidecar

Mimir Metrics includes an integrated Vector sidecar to collect, normalize, and forward logs from the host system.

How it works

When enabled, mimir-metrics: 1. Downloads a compatible Vector binary (if not found locally). 2. Generates a secure vector.yaml configuration in memory. 3. Starts Vector as a child process. 4. Manages the process lifecycle (stops Vector when the agent stops).

Supported Sources

1. Journald (Linux)

Captures system logs from systemd-journald.

  • Flag: -enable-logging journald
  • Config: enable_logging: "journald"

2. Docker Logs (Linux/Windows)

Captures standard output/error from all running containers.

  • Flag: -enable-logging docker
  • Config: enable_logging: "docker"
  • Socket: Defaults to /var/run/docker.sock (Linux) or named pipe (Windows). Override with -docker-socket.

3. Windows Event Logs (Windows)

Captures "System", "Application", and "Security" event logs.

  • Flag: -enable-logging windows_eventlog (or journald maps to this on Windows)
  • Config: enable_logging: "windows_eventlog"

4. Files

Tails specific log files on disk.

  • Flag: -logging-files "/var/log/nginx/*.log,/opt/app/log.txt"
  • Config: 
    logging_files:
  - "/var/log/*.log"

Ingestion Sink

Logs are normalized and sent to an HTTP endpoint.

  • URL: -logging-sink-url http://host/api/ingest
  • Auth: -logging-api-key "Bearer-Token"

Data Format

Logs are normalized to a strict JSON schema:

{
  "ts": "2023-10-27T10:00:00Z",
  "message": "Something happened",
  "workload": "docker",
  "level": "info",
  "meta": {
    "host": "server-1",
    "container": "my-app",
    "image": "nginx:latest"
  }
}

Safety Features

Disk Buffering

The sidecar uses Disk Buffering to prevent memory exhaustion if the ingest server is down. - Limit: 256 MB on disk. - Behavior: Drops oldest logs when full (drop_newest). - RAM Usage: Minimal (streaming only).

Configuration Preview

To see exactly what Vector configuration is generated:

./mimir-metrics -enable-logging journald -print-vector-config