Before: LogStream could only read syslog, Docker, host system logs, dmesg and auditd; the log files of Traefik, nginx or an application stayed out of reach.
After: Settings > Log sources > Log files lists files to follow, each with a name (= app, the file name by default), the path of one file (no wildcards for now), a format (Automatic by default), an optional host and a switch. As soon as a path is typed, the file is checked and the row shows whether it was found, the detected format and its last line. Lines show in the list with a green edge and a FILE badge, and the Source filter has a Files entry.
Formats: Automatic (detected from the last 50 lines: Traefik access JSON, JSON lines, CLF or logfmt when 80 % of them follow it, else plain text), or forced: plain text (level and leading timestamp detected, indented lines joined to the previous entry), JSON lines, logfmt, access log CLF (nginx, Apache, Traefik), Traefik access log JSON, regular expression with named groups (time, level, msg, host, app, others as fields). Access logs: 5xx = err, 4xx = warning.
How: filesource.go follows each file like the audit source (inode + offset in /data/files-state.json, rotation via .1, copytruncate, FILES_BACKFILL of dated lines when turned on); fileformats.go holds the parsers and the detection. GET/PUT /api/files for the settings, POST /api/files/detect for the check of a path (POST so read-only accounts cannot read files through it). docker-compose, .env.example and both READMEs document the mount and formats.
Checked: go vet, go build, go test -race (tests for every format, detection, rotation, restart, truncation, backfill, forced format); UI checked in a static preview with stubbed API (rows, path check, auto name, FR/EN).
<!-- ccr-projects-attribution: {"github_login":"vblogio"} -->
_Requested by **Cédric**_
Before: LogStream could only read syslog, Docker, host system logs, dmesg and auditd; the log files of Traefik, nginx or an application stayed out of reach.
After: Settings > Log sources > **Log files** lists files to follow, each with a name (= app, the file name by default), the path of one file (no wildcards for now), a format (**Automatic** by default), an optional host and a switch. As soon as a path is typed, the file is checked and the row shows whether it was found, the detected format and its last line. Lines show in the list with a green edge and a **FILE** badge, and the Source filter has a **Files** entry.
Formats: Automatic (detected from the last 50 lines: Traefik access JSON, JSON lines, CLF or logfmt when 80 % of them follow it, else plain text), or forced: plain text (level and leading timestamp detected, indented lines joined to the previous entry), JSON lines, logfmt, access log CLF (nginx, Apache, Traefik), Traefik access log JSON, regular expression with named groups (`time`, `level`, `msg`, `host`, `app`, others as fields). Access logs: 5xx = err, 4xx = warning.
How: `filesource.go` follows each file like the audit source (inode + offset in `/data/files-state.json`, rotation via `.1`, copytruncate, `FILES_BACKFILL` of dated lines when turned on); `fileformats.go` holds the parsers and the detection. `GET/PUT /api/files` for the settings, `POST /api/files/detect` for the check of a path (POST so read-only accounts cannot read files through it). docker-compose, `.env.example` and both READMEs document the mount and formats.
Checked: `go vet`, `go build`, `go test -race` (tests for every format, detection, rotation, restart, truncation, backfill, forced format); UI checked in a static preview with stubbed API (rows, path check, auto name, FR/EN).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Files source (source_type=file), declared in Settings > Log sources > Log
files (/data/files.json, 50 at most): name (= app), path as mounted in the
container (file or glob, rotated copies .1/.gz left out, 200 files per
glob), format, optional host, switch.
- formats: plain text (level detected, leading timestamp read and removed),
JSON lines (msg/time/level keys, numeric pino levels, other keys as
fields, nested ones dotted), logfmt, access log CLF (nginx, Apache,
Traefik CLF with router/backend/duration), Traefik access log in JSON
(summary "GET host/path 200 · 12.5 ms · router · client"), regular
expression with named groups time/level/msg/host/app/status; 5xx = err,
4xx = warning; fields named like LogStream's own renamed field_<name>
- indented lines (text) and non-matching lines (regex) added to the
previous entry, which waits 2 s for them
- inode + offset per file in /data/files-state.json, rotation (end of
.1 read first), copytruncate, file renamed within the glob; last
FILES_BACKFILL (1h) of dated lines read when turned on, new files read
from their start
- status per file (files followed, entries read, not found / permission)
UI: rows with switch, name, path, format, host (regex field when needed),
status line; green edge, tint and FILE badge in the list, "Files" in the
source filter, HTTP fields first in the details, FR/EN. docker-compose,
.env.example and README (EN/FR): mounts, formats, FILES_BACKFILL.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- a path is one file (wildcards refused for now); the name defaults to
the file name
- format "auto" (the default): detected from the last 50 lines of the
file, the structured format followed by at least 80 % of them (Traefik
access JSON, JSON lines, CLF, logfmt), else plain text; done again when
the format changes, shown in the status ("Format detected: …"); any
other format forces it
- POST /api/files/detect {path}: found / permission / empty, detected
format and last line, shown under the row as soon as the path is typed
(POST so that read-only accounts cannot read files through it)
- README (EN/FR) updated
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Requested by Cédric
Before: LogStream could only read syslog, Docker, host system logs, dmesg and auditd; the log files of Traefik, nginx or an application stayed out of reach.
After: Settings > Log sources > Log files lists files to follow, each with a name (= app, the file name by default), the path of one file (no wildcards for now), a format (Automatic by default), an optional host and a switch. As soon as a path is typed, the file is checked and the row shows whether it was found, the detected format and its last line. Lines show in the list with a green edge and a FILE badge, and the Source filter has a Files entry.
Formats: Automatic (detected from the last 50 lines: Traefik access JSON, JSON lines, CLF or logfmt when 80 % of them follow it, else plain text), or forced: plain text (level and leading timestamp detected, indented lines joined to the previous entry), JSON lines, logfmt, access log CLF (nginx, Apache, Traefik), Traefik access log JSON, regular expression with named groups (
time,level,msg,host,app, others as fields). Access logs: 5xx = err, 4xx = warning.How:
filesource.gofollows each file like the audit source (inode + offset in/data/files-state.json, rotation via.1, copytruncate,FILES_BACKFILLof dated lines when turned on);fileformats.goholds the parsers and the detection.GET/PUT /api/filesfor the settings,POST /api/files/detectfor the check of a path (POST so read-only accounts cannot read files through it). docker-compose,.env.exampleand both READMEs document the mount and formats.Checked:
go vet,go build,go test -race(tests for every format, detection, rotation, restart, truncation, backfill, forced format); UI checked in a static preview with stubbed API (rows, path check, auto name, FR/EN).🤖 Generated with Claude Code
- a path is one file (wildcards refused for now); the name defaults to the file name - format "auto" (the default): detected from the last 50 lines of the file, the structured format followed by at least 80 % of them (Traefik access JSON, JSON lines, CLF, logfmt), else plain text; done again when the format changes, shown in the status ("Format detected: …"); any other format forces it - POST /api/files/detect {path}: found / permission / empty, detected format and last line, shown under the row as soon as the path is typed (POST so that read-only accounts cannot read files through it) - README (EN/FR) updated Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>Pull request closed