A file turned on was read from its start to find its last hour: with a
Traefik access log that holds months of lines (never rotated by
Traefik), LogStream spent minutes on old lines that were all left out,
and new lines waited behind them. Entries sent during the first poll
were also left out of the count shown in Settings.
- start of the FILES_BACKFILL history found by bisection on the dates
of the lines; files without dates: only new lines
- up to 16 reads of 4 MB per file and per poll
- status "behind" (bytes left to read), shown as "Reading the history:
N MB left"
- the status of a file exists before its first entries are counted
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- a path is one file (wildcards refused for now); the name defaults to
the file name
- format "auto" (the default): detected from the last 50 lines of the
file, the structured format followed by at least 80 % of them (Traefik
access JSON, JSON lines, CLF, logfmt), else plain text; done again when
the format changes, shown in the status ("Format detected: …"); any
other format forces it
- POST /api/files/detect {path}: found / permission / empty, detected
format and last line, shown under the row as soon as the path is typed
(POST so that read-only accounts cannot read files through it)
- README (EN/FR) updated
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Files source (source_type=file), declared in Settings > Log sources > Log
files (/data/files.json, 50 at most): name (= app), path as mounted in the
container (file or glob, rotated copies .1/.gz left out, 200 files per
glob), format, optional host, switch.
- formats: plain text (level detected, leading timestamp read and removed),
JSON lines (msg/time/level keys, numeric pino levels, other keys as
fields, nested ones dotted), logfmt, access log CLF (nginx, Apache,
Traefik CLF with router/backend/duration), Traefik access log in JSON
(summary "GET host/path 200 · 12.5 ms · router · client"), regular
expression with named groups time/level/msg/host/app/status; 5xx = err,
4xx = warning; fields named like LogStream's own renamed field_<name>
- indented lines (text) and non-matching lines (regex) added to the
previous entry, which waits 2 s for them
- inode + offset per file in /data/files-state.json, rotation (end of
.1 read first), copytruncate, file renamed within the glob; last
FILES_BACKFILL (1h) of dated lines read when turned on, new files read
from their start
- status per file (files followed, entries read, not found / permission)
UI: rows with switch, name, path, format, host (regex field when needed),
status line; green edge, tint and FILE badge in the list, "Files" in the
source filter, HTTP fields first in the details, FR/EN. docker-compose,
.env.example and README (EN/FR): mounts, formats, FILES_BACKFILL.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
cedricandClaude Opus 5.5