Linux audit (auditd) source and LogStream internal logs

Audit source (source_type=audit), off until enabled in Settings > Sources:
- follows /var/log/audit/audit.log (path configurable, AUDIT_LOG), with
  rotation (end of audit.log.1 read first) and an offset checkpoint in
  /data/audit-state.json; last AUDIT_BACKFILL (1h) read on first enable
- groups the records of an event by msg=audit(time:serial), closed on EOE,
  after 2 s without a record, or once the log has moved 2 s past it
- fields audit_type, audit_serial, auid, user (enriched name, host
  /etc/passwd or uid), exe, res, key, audit_raw; app = audit_type, host =
  node= / AUDIT_HOST / /host/etc/hostname; summary "who: what [key] →
  result"; warning when res=failed / success=no, info otherwise
- hexadecimal values (proctitle, cmd, exec arguments…) decoded

Internal logs (source_type=logstream, host=logstream, app=component):
- logEvent() for VictoriaLogs unreachable / back, lost batches, spooled
  batches refused, syslog listener errors, Docker API and container
  streams lost, failing sources, start and stop
- on by default from warning; switch and minimum level in Settings >
  Sources (/data/selflog.json)
- bounded queue, never waits for the Store, same event at most once per
  30 s with a count of the repeats: a failing Store cannot loop

UI: violet (audit) and turquoise (LogStream) left edge and tint, AUDIT /
LS badge before the app, raw audit record under the summary (folded in
compact density until the row is opened), Audit and LogStream in the
source filter, FR/EN translations. docker-compose: optional
/var/log/audit, /etc/passwd and /etc/hostname mounts. README (EN/FR):
host prerequisites (log_group, ACL for UID 10001).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
cedricandClaude Opus 5.5 committed 2026-10-04 15:01:58 +02:00
1 parent f3373e1f66
commit 076c01aac4
20 files changed
+1948 -36

No files matched your search

+64
View File
@@ -220,6 +220,65 @@ on in **Settings > Sources > Kernel messages (dmesg)** (saved in `/data/dmesg.js
`DMESG_BACKFILL` ago. Messages overwritten in the kernel buffer before being read are counted
in Settings.
## Linux audit (auditd)
Logstream can follow the log of the Linux audit daemon (`/var/log/audit/audit.log`): logins,
`sudo`, watched files, syscalls matched by your `auditctl` rules… The source is **off by
default**: turn it on in **Settings > Sources > Linux audit (auditd)**, where the file path can
also be changed (saved in `/data/audit.json`; `AUDIT_LOG` sets the default path).
- **Mount**: uncomment `/var/log/audit:/var/log/audit:ro` in `docker-compose.yml`, then
`docker compose up -d`. (`/var/log` is already mounted under `/host` for the host system
logs, so `/host/var/log/audit/audit.log` also works as the path, without a new volume.)
- **Permissions** (host side): auditd writes its log as `root`, mode `0600`, in a `0700`
directory, and Logstream runs as UID `10001`. The reliable way is the group the container
already gets for the host logs (`HOST_LOGS_GID`, `adm` by default):
```bash
sudo sed -i 's/^log_group.*/log_group = adm/' /etc/audit/auditd.conf
sudo chgrp adm /var/log/audit && sudo chmod 750 /var/log/audit
sudo service auditd restart # systemctl refuses to restart auditd on some distributions
```
auditd then creates every rotated file with mode `0640` and group `adm`. An ACL also works
(`sudo setfacl -m u:10001:rx /var/log/audit && sudo setfacl -m u:10001:r /var/log/audit/audit.log`)
but must be applied again to each new file after a rotation. Settings > Sources says when the
file is missing or access is denied; it is tried again every second.
- **Events**: the records of one event (`SYSCALL`, `EXECVE`, `CWD`, `PATH`, `PROCTITLE`…) share
the same `msg=audit(time:serial)` and become **one** log, once no record came for 2 seconds.
The message is a summary — who, what, result — such as
`cedric (as root): exec apt install nginx [exec] → success` or
`bob: PAM:authentication via /usr/sbin/sshd from 203.0.113.9 on ssh → failed`; the raw
records are kept in `audit_raw` and shown under the summary (folded in compact density until
the line is opened). Hexadecimal values (`proctitle`, `cmd`, arguments with spaces) are
decoded.
- **Fields**: `audit_type` (type of the first record, also used as **app**), `audit_serial`,
`auid` (login user ID), `user` (its name, or the uid when it cannot be resolved), `exe`, `res`
(`success` / `failed`, from `res=` or `success=`) and `key` (rule key). Severity is `warning`
when the result is a failure, `info` otherwise; facility `authpriv`; source `audit`. In the
list, audit lines have a violet left edge, a light tint and an **AUDIT** badge before the app;
the **Source** filter shows them alone.
- **Names**: `user` comes from the enriched format (`log_format = ENRICHED`, the default of
recent auditd versions), else from the host's `/etc/passwd` when it is mounted as
`/host/etc/passwd`. **host** is the `node=` of the records (`name_format = hostname` in
`auditd.conf`), else `AUDIT_HOST`, else `/host/etc/hostname`, else `localhost`.
- When the source is turned on, the last hour is read first (`AUDIT_BACKFILL`, `0` = only new
events). The inode and an offset before which every event was sent are saved in
`/data/audit-state.json`: after a restart, reading resumes there; after a rotation by auditd,
the end of `audit.log.1` is read before the new file.
## LogStream internal logs
Logstream records its own events as logs, with **host** `logstream`, **app** the component
(`store`, `syslog`, `docker`, `host`, `dmesg`, `audit`, `main`) and the `logstream` source:
VictoriaLogs unreachable and back, lost batches, batches refused from the disk buffer, syslog
listener errors, Docker API or container streams lost, sources that cannot read, start and
stop. They are on by default from level **warning**; **Settings > Sources > LogStream internal
logs** turns them off or changes the minimum level (saved in `/data/selflog.json`). The same
event (component and message) is recorded at most once every 30 seconds, with the number of
repeats left out; events go through a bounded queue and never wait for the storage, so an
unreachable VictoriaLogs cannot make them pile up or loop. Everything still goes to
`docker compose logs logstream` as before. In the list, these lines have a turquoise left edge
and an **LS** badge before the app; the **Source** filter shows them alone.
## CSV export
The **Export** button (next to the log count) downloads every stored log matching the
@@ -398,6 +457,9 @@ are only known by your router or a local DNS (Pi-hole, AdGuard, Unbound…), set
| `HOST_LOGS_ROOT` | `/host` | where the host directories are mounted |
| `DMESG_BACKFILL` | `1h` | history read when the kernel messages source is turned on |
| `DMESG_DEVICE` | `/dev/kmsg` | kernel log device read by the kernel messages source |
| `AUDIT_LOG` | `/var/log/audit/audit.log` | default path of the Linux audit log (can be changed in Settings > Sources) |
| `AUDIT_BACKFILL` | `1h` | history read when the Linux audit source is turned on |
| `AUDIT_HOST` | empty | host name of the audit events when the records carry no `node=` |
| `TZ` | `Europe/Paris` | time zone for RFC 3164 timestamps (which carry none) |
| `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | ingestion tuning |
| `SPOOL_MAX_MB` | `1024` | disk buffer size for batches VictoriaLogs could not take (`0` = no buffer: retried for 15 s, then dropped) |
@@ -457,6 +519,8 @@ To update one of them:
| `syslogserver.go` | syslog listeners opened and closed from Settings > Sources |
| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower |
| `dmesg.go` | kernel messages: `/dev/kmsg` reader, record parsing, position by boot ID and sequence number |
| `audit.go` | Linux audit: `audit.log` follower (rotation, saved offset), record parsing, grouping by event, summary |
| `selflog.go` | LogStream internal logs: `logEvent`, minimum level, throttling, bounded queue |
| `tags.go` | color tag storage |
| `presets.go`, `presets.json` | color tag presets (`/api/presets`), built-in list |
| `api.go` | `/api/*` HTTP routes |