Stream and Time range display modes with start/end fields

A segmented control at the start of the filter bar switches between
Stream (sliding duration, live view) and Time range (start and end
dates in the chosen time zone, previous/next and zoom-out buttons).
Timeline clicks and drags switch to Time range mode; the mode and
range are kept across reloads.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.env.example

@@ -4,18 +4,52 @@ HTTP_PORT=8080
 TZ=Europe/Paris
 # How long logs are kept (e.g. 7d, 30d, 12w, 1y)
 RETENTION=30d
-# Web UI authentication (empty = disabled)
+# Web UI authentication: local (login page with the account below, or none) or oidc (OpenID Connect provider)
+AUTH_MODE=local
+# local mode: user and password (empty = no authentication)
 AUTH_USER=
 AUTH_PASS=
+# local mode: optional read-only account (can search and export, cannot change tags, sources or purge)
+AUTH_VIEWER_USER=
+AUTH_VIEWER_PASS=
+# local mode: PNG logo shown on the login page, path inside the container (empty = no logo).
+# Mount the file in docker-compose.yml, e.g. ./logo.png:/config/logo.png:ro, then LOGIN_LOGO=/config/logo.png
+LOGIN_LOGO=
+
+# Ready-made color tags offered in Settings > Filters (see docs/presets.md).
+# Empty: /data/presets.json if present, else the built-in list. To use your own file,
+# mount it in docker-compose.yml, e.g. ./presets.json:/config/presets.json:ro
+PRESETS_FILE=
+# Session lifetime, both modes (e.g. 8h, 24h)
+SESSION_TTL=12h
+# oidc mode: issuer URL exactly as the provider announces it
+# (Keycloak: https://sso.example.org/realms/<realm>, Authentik: https://auth.example.org/application/o/<slug>/)
+OIDC_ISSUER=
+OIDC_CLIENT_ID=
+OIDC_CLIENT_SECRET=
+# Callback URL of logstream, to register in the provider (path free, /auth/callback recommended)
+OIDC_REDIRECT_URL=https://logs.example.org/auth/callback
+# Requested scopes (openid is always added)
+OIDC_SCOPES=openid profile email
+# oidc mode: only members of this group are admins, the others are read-only (empty = everyone is admin).
+# The groups come from the ID token claim OIDC_GROUPS_CLAIM (default groups)
+OIDC_ADMIN_GROUP=
+OIDC_GROUPS_CLAIM=groups
 # Reverse DNS: show host names instead of IP addresses (on/off)
 RDNS=on
 # DNS server used for reverse lookups (e.g. your router: 192.168.1.1). Empty = system resolver
 DNS_SERVER=
 # Allow "Delete all logs" in Settings (true/false)
-ALLOW_PURGE=true
+ALLOW_PURGE=false
 # Maximum number of rows in a CSV export
 EXPORT_MAX=100000
 # Collect the logs of the Docker containers of this machine (on/off)
 DOCKER_LOGS=on
 # History read from a container seen for the first time (e.g. 30m, 1h, 24h; 0 = only new lines)
 DOCKER_BACKFILL=1h
+# Host system logs (enable them in Settings > Sources)
+# Group allowed to read the host logs: 4 = adm on Debian/Ubuntu; or the systemd-journal group
+# (getent group systemd-journal | cut -d: -f3)
+HOST_LOGS_GID=4
+# History read when the source is turned on (e.g. 30m, 1h, 24h; 0 = only new entries)
+HOST_LOGS_BACKFILL=1h

Dockerfile

@@ -4,7 +4,7 @@
 FROM golang:1.27.1-alpine3.24 AS build
 WORKDIR /src
 COPY go.mod ./
-COPY *.go ./
+COPY *.go presets.json ./
 COPY web ./web
 RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/logstream .
 

README.fr.md

@@ -0,0 +1,468 @@
+[English](README.md) | Français
+
+# Logstream
+
+Un collecteur syslog simple : il reçoit les logs en UDP/TCP sur le port 514, les stocke dans
+[VictoriaLogs](https://docs.victoriametrics.com/victorialogs/) et sert une interface web
+soignée (thème clair/sombre, recherche instantanée, direct, tags de couleur, interface en
+anglais et en français).
+
+```
+devices ──514 udp/tcp──▶ logstream (Go) ──HTTP batches──▶ VictoriaLogs
+                              ▲    └── SSE (live) ──▶ browser
+                              └──── API / UI ◀─────────┘
+```
+
+## Architecture
+
+![Schéma logique de Logstream](docs/architecture.png)
+
+- **Ingestion** : les messages syslog (UDP/TCP) et les logs des conteneurs Docker passent tous
+  par `sink()` (résolution DNS inverse des hôtes donnés par leur IP, sans bloquer la
+  réception), puis par la file du `Store`, qui les envoie par lots à VictoriaLogs. Quand
+  VictoriaLogs est injoignable, les lots sont gardés sur disque (`/data/spool`, jusqu'à
+  `SPOOL_MAX_MB`) et renvoyés, les plus anciens d'abord, dès son retour.
+- **Direct** : `sink()` publie aussi chaque message dans le `Hub`, qui le diffuse aux
+  navigateurs en SSE.
+- **Recherche** : l'API HTTP traduit les filtres de l'interface en requêtes LogsQL envoyées à
+  VictoriaLogs.
+- **Frise** : `/api/histogram` (`histogram.go`) compte les logs par intervalle et par sévérité,
+  alignés sur l'heure locale ; les bornes du zoom (`from`/`to`) s'appliquent aussi à la liste
+  et à l'export.
+- **État** : les tags et les réglages des sources sont dans `/data` (volume `logstream-data`) ;
+  les logs eux-mêmes dans le volume `vlogs-data`.
+
+La source modifiable du schéma est
+[`docs/architecture.excalidraw`](docs/architecture.excalidraw)
+(à ouvrir sur [excalidraw.com](https://excalidraw.com)).
+
+## Démarrage
+
+```bash
+cp .env.example .env      # optional
+docker compose up -d --build
+./tools/send-test-logs.sh # sends 100 test messages
+```
+
+Ouvrez ensuite <http://localhost:8080>.
+
+## Envoyer des logs
+
+- **rsyslog** (Linux) : ajoutez `*.* @SERVER_IP:514` (UDP) ou `*.* @@SERVER_IP:514` (TCP)
+  dans `/etc/rsyslog.d/90-logstream.conf`, puis lancez `systemctl restart rsyslog`.
+- **Équipements réseau, NAS, pare-feu** : indiquez l'IP du serveur et le port 514 dans leurs
+  réglages « syslog distant » (remote syslog).
+- **Test manuel** : `logger -n 127.0.0.1 -P 514 -d "hello error"` (util-linux) ou
+  `echo "<14>test ok" | nc -u -w1 127.0.0.1 514`.
+
+**Paramètres > Sources > Syslog** active ou désactive la réception syslog et choisit les
+protocoles (UDP, TCP) sans redémarrage ; le choix est enregistré dans `/data/syslog.json`. Cet
+onglet affiche aussi l'état d'écoute (et l'erreur si le port est déjà utilisé). Le port
+lui-même est publié par docker-compose : modifiez `SYSLOG_PORT` dans `.env`, puis lancez
+`docker compose up -d`.
+
+Formats pris en charge : RFC 3164 (BSD) et RFC 5424. En TCP, les deux découpages sont acceptés :
+« un message par ligne » et « octet counting » (RFC 6587).
+
+## Recherche
+
+**Mode simple** (par défaut) : chaque mot est cherché comme sous-chaîne, sans tenir compte de
+la casse, dans le message, l'hôte et l'application. Les mots sont combinés avec ET.
+
+| Saisie | Signification |
+|---|---|
+| `error disk` | contient « error » **et** « disk » |
+| `"disk full"` | contient la phrase exacte |
+| `error -timeout` | contient « error » mais pas « timeout » |
+
+**Mode LogsQL** (bouton `Simple` / `LogsQL`) : le langage de requête complet de VictoriaLogs,
+par exemple `error AND host:web-01`, `app:~"ssh|nginx"` ou `* | stats by (host) count()`.
+Le direct est désactivé dans ce mode.
+
+Chaque ligne affiche, de gauche à droite : l'**heure de réception** (horloge du serveur),
+l'horodatage trouvé dans le message lui-même (`msg_time`), la sévérité, l'hôte, l'application,
+les codes des tags trouvés et le message. Un clic sur un hôte ou une application filtre dessus.
+
+Les logs sont indexés, recherchés et triés par **heure de réception** : les équipements dont
+l'horloge est fausse (par exemple des points d'accès dont le NTP échoue) apparaissent quand même
+dans la bonne plage de temps. Les logs stockés par les versions antérieures à ce changement sont
+indexés par l'horodatage de leur message ; purgez-les depuis les Paramètres pour repartir sur
+une base propre.
+
+Les badges de sévérité `err`/`crit` et `warning` reprennent les couleurs des tags `error` et
+`warning`.
+
+Raccourcis : `/` place le curseur dans la recherche, `Esc` la vide. Un clic sur une ligne
+affiche tous ses champs.
+
+Les en-têtes de colonnes restent visibles pendant le défilement. Faites glisser le bord d'un
+en-tête (réception, heure message, sévérité, hôte, app) pour redimensionner la colonne, et
+double-cliquez dessus pour revenir à la largeur automatique ; le message occupe l'espace
+restant. Les largeurs sont mémorisées par le navigateur (**Paramètres > Interface >
+Réinitialiser les colonnes** les rétablit toutes). Sur téléphone, la liste garde sa
+présentation sur deux lignes, sans colonnes.
+
+## Modes Flux et Période
+
+Le premier contrôle de la barre de filtres bascule entre deux modes d'affichage :
+
+- **Flux** : les derniers logs sur une durée glissante (5 min à 30 jours, ou tout), avec le direct.
+- **Période** : les logs entre une date de début et une date de fin, saisies dans le fuseau choisi
+  dans les Paramètres. ◀ et ▶ passent à la période précédente ou suivante de même durée, la loupe
+  la double autour de son milieu. Le direct se met en pause ; le mode et la période sont conservés
+  au rechargement.
+
+## Frise
+
+La frise au-dessus de la liste montre le volume de logs par intervalle, compté selon l'**heure
+de réception** sur l'horloge du serveur (elle correspond donc aux heures affichées dans les
+lignes).
+
+- Au survol d'un intervalle : ses bornes, son total et le détail par sévérité.
+- Un clic sur une barre affiche cet intervalle en mode Période ; un glisser sur plusieurs barres
+  affiche la sélection. « × Revenir au flux » repasse en mode Flux. La liste, les compteurs et
+  l'export CSV suivent la période.
+- En direct, le dernier intervalle grandit à l'arrivée des messages, et la frise se recharge à
+  chaque nouvel intervalle. Rien n'est rafraîchi tant que l'onglet du navigateur est masqué ;
+  la frise se met à jour dès qu'il redevient visible.
+
+**Paramètres > Interface > Frise** (mémorisé par navigateur) : échelle (linéaire, √ par défaut,
+log), hauteur (S/M/L : 40/80/120 px), couleur (empilement par sévérité, intensité comparée à
+la médiane de la fenêtre : calme, rafale au-delà de 3×, anomalie au-delà de 10×, ou aucune),
+barres ou aire, division (automatique, environ 100 intervalles, ou fixe : 1 s, 10 s, 1 min,
+5 min, 1 h, 1 jour) et rafraîchissement (désactivé, 5 s, 15 s, 30 s, 1 min, ou à chaque nouvel
+intervalle). Une division fixe qui dépasserait 300 intervalles sur la plage est élargie
+(signalé par « élargi »). Les intervalles sont alignés sur l'heure locale du fuseau horaire
+choisi dans les Paramètres (les jours commencent à minuit, heure locale).
+
+API : `curl 'localhost:8080/api/histogram?range=24h&step=auto&tz=Europe/Paris'` renvoie
+`step` (ms), `start` (ms), `count`, `now` (horloge du serveur) et les `buckets` non vides
+(`i` = numéro d'intervalle, `n` = total, `sev` = nombre par sévérité). Toutes les routes de
+logs acceptent aussi `from` / `to` (millisecondes Unix) à la place de `range`.
+
+## Logs des conteneurs Docker
+
+Logstream collecte aussi les logs des conteneurs Docker qui tournent sur la machine où il est
+installé (`DOCKER_LOGS=on`, le défaut dans `docker-compose.yml`). Ils sont recherchés, filtrés,
+colorés et exportés comme les messages syslog :
+
+- **host** est le nom de l'hôte Docker, **app** le service compose (ou le nom du conteneur), et
+  chaque log porte aussi `container`, `container_id`, `image`, `compose_project`,
+  `compose_service` et `stream` (stdout/stderr), visibles dans le détail de la ligne.
+- Le filtre **Source** n'affiche que les logs syslog ou que les logs Docker ; les lignes Docker
+  ont un petit cube devant le nom de l'application, de la couleur de son projet compose.
+- La sévérité vient de la ligne elle-même quand l'application l'écrit : JSON
+  (`"level":"error"`), logfmt (`level=warn`), `[ERROR]`, ou un niveau en majuscules au début de
+  la ligne (`ERROR`, `WARN`…). Sinon, elle vaut `info`. Les codes de couleur du terminal sont
+  supprimés.
+- **Paramètres > Sources** affiche une étiquette par conteneur (`project/service`) dans un seul
+  champ : les conteneurs suivis en couleur, puis les non suivis en gris ; un clic bascule une
+  étiquette. La couleur identifie le projet compose, et les noms d'applications Docker de la
+  liste utilisent la même couleur. Les conteneurs arrêtés sont masqués par défaut (« Afficher
+  les conteneurs arrêtés » les montre, en pointillés, et ils restent modifiables). Une zone de
+  filtre et « Tout activer » / « Tout désactiver » (appliqués aux étiquettes affichées) aident
+  quand les conteneurs sont nombreux. Les nouveaux conteneurs sont suivis automatiquement, sauf
+  si cette option est désactivée. Les choix sont enregistrés par service compose (ou nom de
+  conteneur) dans `/data/docker.json` : ils survivent aux recréations.
+- Logstream mémorise la position de la dernière ligne stockée pour chaque conteneur
+  (`/data/docker-state.json`) : après un redémarrage, il reprend sans perdre de lignes. La
+  position n'avance qu'une fois la ligne dans VictoriaLogs ou dans le tampon disque, et une file
+  pleine ralentit la lecture au lieu de perdre des lignes. Un conteneur vu pour la
+  première fois est lu à partir de `DOCKER_BACKFILL` en arrière (1 heure par défaut).
+- Logstream lui-même et le proxy ci-dessous ne sont jamais collectés ; ajoutez l'étiquette
+  `logstream.exclude=true` à tout autre conteneur pour l'exclure définitivement.
+
+**Sécurité** : l'accès au socket Docker équivaut à un accès root sur la machine. Logstream passe
+donc par [docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy), qui ne laisse
+passer que la liste des conteneurs, la lecture des logs, les événements et les informations du
+moteur (`GET` uniquement).
+
+## Logs système de l'hôte
+
+Logstream peut aussi collecter les logs système de la machine qui héberge la stack, sans rien
+configurer sur l'hôte. La source est **désactivée par défaut** : activez-la dans
+**Paramètres > Sources > Logs système de l'hôte** (enregistré dans `/data/hostlogs.json`).
+
+- `docker-compose.yml` monte `/var/log` et `/run/log/journal` en lecture seule sous `/host`.
+- Si l'hôte utilise systemd, Logstream lit directement les fichiers du **journal systemd**
+  (pas besoin de `journalctl` dans l'image) : **host** est le nom de la machine, **app** le
+  programme (`SYSLOG_IDENTIFIER`), la sévérité et la facility viennent du journal, et l'unité
+  systemd est conservée dans `unit`. Sinon, il suit les fichiers texte de `/var/log` (`syslog`,
+  `messages`, `*.log`), analysés comme des lignes syslog, avec le nom du fichier dans `log_file`.
+- Ces logs ont la source `host` : le filtre **Source** permet de les afficher seuls.
+- À l'activation, la dernière heure est lue d'abord (`HOST_LOGS_BACKFILL`) ; la position
+  atteinte est enregistrée dans `/data/hostlogs-state.json`, un redémarrage ne perd donc ni ne
+  duplique d'entrées.
+- **Droits** : le conteneur tourne avec un utilisateur sans privilège et reçoit le groupe `adm`
+  (gid 4), qui peut lire le journal et `/var/log` sur Debian et Ubuntu. Sur d'autres systèmes,
+  réglez `HOST_LOGS_GID` dans `.env` sur le gid de `systemd-journal`
+  (`getent group systemd-journal | cut -d: -f3`). Paramètres > Sources affiche un message clair
+  si l'accès est refusé.
+- Limites : les champs du journal compressés par journald (messages de plus de 512 octets,
+  compressés en zstd/lz4/xz) ne peuvent pas être décodés sans bibliothèque supplémentaire ; ils
+  sont comptés dans les Paramètres et affichés comme « (compressed journal entry) ». Les fichiers
+  texte tournés ou compressés (`*.1`, `*.gz`) ne sont pas lus.
+
+## Export CSV
+
+Le bouton **Exporter** (à côté du nombre de logs) télécharge tous les logs stockés qui
+correspondent aux filtres en cours (recherche, plage de temps, sévérité, hôte, application),
+du plus récent au plus ancien, jusqu'à `EXPORT_MAX` lignes (100 000 par défaut), et pas
+seulement les lignes à l'écran. Deux variantes :
+
+- **CSV** : séparateur virgule, UTF-8.
+- **CSV pour Excel** : séparateur point-virgule avec un BOM UTF-8, pour qu'Excel en français
+  l'ouvre directement avec les accents. Les cellules qui commencent par `=`, `+`, `-` ou `@`
+  sont préfixées par `'` pour qu'un message de log forgé ne puisse pas s'exécuter comme une
+  formule.
+
+Colonnes : `received`, `message_time` (toutes deux au format `YYYY-MM-DD HH:MM:SS.mmm` dans le
+fuseau horaire choisi dans les Paramètres), `severity`, `facility`, `host`, `host_ip`, `app`,
+`pid`, `source_ip`, `proto`, `message`. En mode LogsQL, seule la partie filtre est prise en
+charge (pas de `| pipes`).
+
+En ligne de commande : `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`.
+
+## Paramètres
+
+L'icône en forme d'engrenage ouvre les paramètres, organisés en onglets. Tout, sauf les tags de
+couleur, est mémorisé par navigateur.
+
+- **Localisation**
+  - *Langue* : anglais ou français.
+  - *Date et heure* : fuseau horaire (celui du navigateur, UTC ou environ 80 fuseaux courants)
+    et format d'affichage de l'heure de réception : `DD/MM/YYYY HH:MM:SS` (par défaut,
+    l'affichage français habituel), avec millisecondes, `YYYY-MM-DD`, horloge sur 12 heures,
+    ISO 8601 ou epoch Unix. Le fuseau horaire s'applique à toutes les dates affichées.
+- **Filtres** : tags de couleur. Chaque tag a un mot-clé, une couleur de fond (le texte passe
+  automatiquement en noir ou en blanc pour rester lisible) et des options : mot entier, respect
+  de la casse, expression régulière, actif. Les tags sont stockés sur le serveur dans
+  `/data/tags.json` (volume `logstream-data`) : ils sont donc partagés par tous les navigateurs.
+  Tags par défaut (pastel) : `warning` (orange) et `error` (rouge) ; `ok` (vert) est dans le
+  préréglage *Niveaux de log*. Les tags par
+  défaut qui utilisent encore les couleurs des versions précédentes passent automatiquement aux
+  couleurs pastel.
+  Le menu *+ Préréglage…* ajoute des tags tout faits : logs d'accès HTTP/HTTPS (codes de statut,
+  méthodes, sondes, robots, erreurs TLS et proxy), logs système (SSH, sudo, noyau, systemd,
+  pare-feu), applications (Docker, bases de données) et motifs généraux (niveaux de log,
+  adresses IPv4). Les tags déjà présents ne sont pas ajoutés en double, et les tags ajoutés se
+  modifient comme les autres. La liste vient d'un fichier texte modifiable (`PRESETS_FILE`) :
+  voir [docs/presets.fr.md](docs/presets.fr.md) pour le détail de chaque préréglage et le
+  format du fichier. Dans une expression
+  régulière, un groupe nommé `hl` (`(?<hl>…)`) ne colore que cette partie de la correspondance :
+  les préréglages s'en servent pour colorer le code de statut ou la méthode, pas le texte autour.
+  Chaque tag reçoit un code à deux chiffres (`01`, `02`…) attribué par le serveur : il reste
+  attaché au tag jusqu'à sa suppression (les tags créés par les versions précédentes en reçoivent
+  un aussi). La colonne *Filtres* de la liste affiche, en badges gris, les codes des tags actifs
+  trouvés dans chaque message ; elle a la place pour 3, au-delà elle en affiche 2 et `+N`, et
+  l'infobulle les liste tous.
+- **Interface**
+  - *Thème* : Système (suit la préférence de l'ordinateur ou du téléphone), Clair ou Sombre. Le
+    bouton soleil/lune de l'en-tête bascule entre clair et sombre.
+  - *Affichage des logs* : taille du texte (très petite, petite, moyenne, grande), densité
+    (normale, ou compacte pour afficher environ 50 % de lignes en plus à l'écran) et police :
+    la police monospace du système, l'une des 3 polices étroites intégrées, servies par
+    LogStream lui-même et utilisables hors ligne (Inconsolata Condensed, la plus étroite,
+    Iosevka et Ubuntu Mono), ou l'une des 11 polices libres conçues pour le texte dense
+    (JetBrains Mono, Fira Code, Source Code Pro, IBM Plex Mono, Cascadia Code, Roboto Mono,
+    Inconsolata, Red Hat Mono, Noto Sans Mono, Victor Mono, DM Mono). Ces 11 polices sont
+    chargées par le navigateur depuis [Bunny Fonts](https://fonts.bunny.net), un service
+    européen de polices respectueux de la vie privée ; sans accès à internet, la police du
+    système est utilisée. Les ligatures sont désactivées pour que `->` ou `!=` s'affichent tels
+    quels. Le réglage le plus dense est Très petite + Compacte + Inconsolata Condensed.
+- **Données** : chiffres de la base lus dans VictoriaLogs (actualisés au plus toutes les 30 s) :
+  lignes stockées, taille sur disque (index compris), taille brute et taux de compression,
+  période couverte avec la rétention, lignes des dernières 24 h et de la dernière heure, hôtes
+  et applications distincts, espace disque libre. Les tailles sont en Ko/Mo/Go (KB/MB/GB en
+  anglais). « Supprimer tous les logs » efface définitivement tous les logs stockés (il faut
+  taper `PURGE` pour confirmer). Les tags et les paramètres sont conservés. VictoriaLogs doit
+  être lancé avec `-delete.enable` (déjà présent dans `docker-compose.yml`). La fonction est
+  désactivée par défaut : mettez `ALLOW_PURGE=true` pour l'autoriser. Tout administrateur peut
+  alors purger : activez l'[authentification](#authentification) si l'interface est accessible
+  à d'autres.
+
+## Authentification
+
+`AUTH_MODE` choisit comment l'interface et l'API sont protégées (`/healthz` reste toujours ouvert) :
+
+- **`local`** (par défaut) : une page de connexion avec le compte `AUTH_USER` / `AUTH_PASS` ; laissez-les
+  vides pour n'avoir aucune authentification (par exemple derrière un reverse proxy qui contrôle déjà).
+  L'interface affiche alors un bandeau d'avertissement, que l'on peut fermer.
+- **`oidc`** : connexion par un fournisseur OpenID Connect (Keycloak, Authentik, Authelia, Zitadel…),
+  flux « authorization code » avec PKCE.
+
+En mode `local`, la page de connexion suit le thème et la langue de l'interface. La session dure
+`SESSION_TTL` (12 h par défaut), survit aux redémarrages (sa clé de signature est dans
+`/data/session.key`) et se termine quand `AUTH_USER` ou `AUTH_PASS` change ; le bouton de
+déconnexion (en haut à droite) y met fin. Les échecs de connexion sont écrits dans les logs avec
+l'adresse du client (`auth: failed login for "bob" from 192.0.2.7`). Les scripts peuvent toujours
+appeler l'API avec des identifiants HTTP Basic (`curl -u utilisateur:motdepasse`).
+
+Un **compte en lecture seule** optionnel, `AUTH_VIEWER_USER` / `AUTH_VIEWER_PASS`, peut chercher,
+suivre le direct et exporter, mais pas modifier les tags, les sources ni purger : ces réglages
+sont grisés dans son interface et l'API répond `403`.
+
+Pour afficher votre logo sur la page de connexion, montez un PNG dans le conteneur et indiquez
+son chemin dans `LOGIN_LOGO` :
+
+```yaml
+# docker-compose.yml, service logstream
+    volumes:
+      - ./logo.png:/config/logo.png:ro
+```
+```bash
+# .env
+LOGIN_LOGO=/config/logo.png
+```
+
+Pour utiliser OIDC :
+
+1. Dans le fournisseur, créez un client **confidentiel** (avec secret) pour logstream et déclarez
+   l'URL de retour `https://logs.example.org/auth/callback` (votre adresse).
+2. Dans `.env` :
+   ```bash
+   AUTH_MODE=oidc
+   OIDC_ISSUER=https://sso.example.org/realms/maison    # exactement l'« issuer » du fournisseur
+   OIDC_CLIENT_ID=logstream
+   OIDC_CLIENT_SECRET=...
+   OIDC_REDIRECT_URL=https://logs.example.org/auth/callback
+   ```
+3. `docker compose up -d`. Les logs affichent `oidc authentication enabled`, ou la raison pour
+   laquelle le fournisseur n'a pas pu être lu (issuer incorrect, injoignable…).
+
+Ouvrir l'interface renvoie vers la page de connexion du fournisseur, puis revient sur logstream.
+La session dure `SESSION_TTL` (12 h par défaut) et survit aux redémarrages (sa clé de
+signature est dans `/data/session.key`) ; à son expiration, la page repasse par la connexion. Le
+bouton de déconnexion (en haut à droite) termine la session logstream, puis ouvre la page de
+déconnexion du fournisseur s'il en a une.
+
+Tout utilisateur accepté par le fournisseur pour ce client peut se connecter : restreignez l'accès
+dans le fournisseur (Keycloak : rôles du client ou realm dédié ; Authentik : liaisons de
+l'application). Les connexions sont écrites dans les logs de logstream (`oidc: alice logged in`).
+Avec une URL de retour en `https`, les cookies ne sont envoyés qu'en HTTPS : logstream doit être
+joint à travers un reverse proxy TLS.
+
+Pour donner un accès en lecture seule à certains utilisateurs, définissez `OIDC_ADMIN_GROUP` (par
+exemple `logstream-admins`) : seuls ses membres sont administrateurs, les autres sont en lecture
+seule. Les groupes sont lus dans la revendication `groups` du jeton d'identité
+(`OIDC_GROUPS_CLAIM` pour en utiliser une autre) ; dans Keycloak, ajoutez au client un mapper
+« Group Membership » (le `/` initial est ignoré).
+
+Quel que soit le mode, chaque réponse porte des en-têtes de sécurité (Content-Security-Policy,
+X-Frame-Options…), et l'API refuse les modifications envoyées depuis un autre site (requêtes
+intersites).
+
+## Noms d'hôtes (DNS inverse)
+
+Quand un équipement envoie son adresse IP comme nom d'hôte (ou pas de nom d'hôte du tout),
+Logstream cherche son nom DNS (enregistrement PTR) et stocke le nom dans `host` et l'IP dans
+`host_ip`. Les résultats sont mis en cache (1 heure, 10 minutes quand il n'y a pas de nom). Les
+logs stockés auparavant avec une IP sont résolus à l'affichage, et le filtre d'hôte affiche
+`name (IP)`.
+
+Le conteneur utilise le DNS de Docker, qui relaie vers les résolveurs de l'hôte. Si vos noms
+locaux ne sont connus que de votre routeur ou d'un DNS local (Pi-hole, AdGuard, Unbound…),
+définissez `DNS_SERVER=192.168.1.1` (son adresse). Mettez `RDNS=off` pour désactiver les
+résolutions.
+
+## Configuration
+
+| Variable | Défaut | Rôle |
+|---|---|---|
+| `SYSLOG_PORT` | `514` | port syslog publié sur l'hôte |
+| `HTTP_PORT` | `8080` | port de l'interface web |
+| `RETENTION` | `30d` | durée de conservation des logs dans VictoriaLogs |
+| `AUTH_MODE` | `local` | `local` (page de connexion) ou `oidc`, voir [Authentification](#authentification) |
+| `AUTH_USER` / `AUTH_PASS` | vide | compte de la page de connexion (mode `local`) ; vide = pas d'authentification |
+| `AUTH_VIEWER_USER` / `AUTH_VIEWER_PASS` | vide | compte optionnel en lecture seule (mode `local`) |
+| `LOGIN_LOGO` | vide | PNG affiché sur la page de connexion, chemin dans le conteneur (mode `local`) |
+| `SESSION_TTL` | `12h` | durée de la session (les deux modes ; `OIDC_SESSION_TTL` fonctionne toujours) |
+| `OIDC_ISSUER` | vide | URL de l'issuer du fournisseur OpenID Connect (mode `oidc`) |
+| `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` | vide | client déclaré dans le fournisseur |
+| `OIDC_REDIRECT_URL` | vide | URL de retour de logstream, ex. `https://logs.example.org/auth/callback` |
+| `OIDC_SCOPES` | `openid profile email` | scopes demandés |
+| `OIDC_ADMIN_GROUP` | vide | seuls les membres de ce groupe sont administrateurs, les autres en lecture seule (vide = tout le monde est administrateur) |
+| `OIDC_GROUPS_CLAIM` | `groups` | revendication du jeton d'identité qui liste les groupes |
+| `RDNS` | `on` | résoudre les hôtes donnés par leur IP en noms DNS |
+| `DNS_SERVER` | vide | serveur DNS pour les résolutions inverses (`ip` ou `ip:port`) |
+| `ALLOW_PURGE` | `false` | autoriser « Supprimer tous les logs » dans les Paramètres |
+| `SYSLOG_TCP_MAX_CONNS` | `512` | connexions syslog TCP ouvertes en même temps ; au-delà, elles sont refusées |
+| `SYSLOG_TCP_IDLE` | `30m` | une connexion syslog TCP silencieuse pendant cette durée est fermée (les émetteurs se reconnectent) |
+| `EXPORT_MAX` | `100000` | nombre maximal de lignes dans un export CSV |
+| `PRESETS_FILE` | `/data/presets.json` | fichier des préréglages de tags ; liste intégrée s'il est absent (voir [docs/presets.fr.md](docs/presets.fr.md)) |
+| `DOCKER_LOGS` | `on` dans compose | collecter les logs des conteneurs Docker locaux |
+| `DOCKER_HOST` | `tcp://docker-proxy:2375` dans compose | adresse de l'API Docker (`unix:///var/run/docker.sock` hors compose) |
+| `DOCKER_BACKFILL` | `1h` | historique lu pour un conteneur vu pour la première fois |
+| `HOST_LOGS_GID` | `4` (adm) dans compose | groupe donné au conteneur pour lire les logs de l'hôte |
+| `HOST_LOGS_BACKFILL` | `1h` | historique lu à l'activation de la source « logs système de l'hôte » |
+| `HOST_LOGS_ROOT` | `/host` | emplacement de montage des répertoires de l'hôte |
+| `TZ` | `Europe/Paris` | fuseau horaire des horodatages RFC 3164 (qui n'en portent pas) |
+| `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | réglage de l'ingestion |
+| `SPOOL_MAX_MB` | `1024` | taille du tampon disque des lots refusés par VictoriaLogs (`0` = pas de tampon : 15 s de tentatives, puis perte) |
+
+## Débogage
+
+- `docker compose logs -f logstream` : erreurs de réception et erreurs d'envoi vers
+  VictoriaLogs.
+- La barre du bas affiche les compteurs reçus / stockés / perdus, les messages en attente dans
+  le tampon disque et la dernière erreur de stockage.
+- <http://localhost:9428/select/vmui> : l'interface de VictoriaLogs, pour essayer des requêtes
+  LogsQL.
+- API :
+  ```bash
+  curl 'localhost:8080/api/logs?q=error&range=1h&limit=5'   # the response includes the generated LogsQL query
+  curl localhost:8080/api/stats
+  curl localhost:8080/api/tags
+  ```
+- Lancement hors Docker (Go 1.22+) : `VLOGS_URL=http://localhost:9428 DATA_DIR=./data SYSLOG_ADDR=:5514 go run .`
+
+## Mise à jour
+
+Toutes les versions d'images sont épinglées : un `docker compose pull` ou une reconstruction ne
+change jamais un composant à votre insu.
+
+| Emplacement | Image | Version |
+|---|---|---|
+| `docker-compose.yml` | `victoriametrics/victoria-logs` | `v1.52.0` |
+| `docker-compose.yml` | `tecnativa/docker-socket-proxy` | `v0.5.0` |
+| `Dockerfile` (build) | `golang` | `1.27.1-alpine3.24` |
+| `Dockerfile` (exécution) | `alpine` | `3.24.2` |
+
+Pour mettre à jour l'une d'elles :
+
+1. Lisez les notes de version : [VictoriaLogs](https://docs.victoriametrics.com/victorialogs/changelog/),
+   [docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy/releases),
+   [Go](https://go.dev/doc/devel/release), [Alpine](https://alpinelinux.org/releases/).
+   VictoriaLogs conserve son format de stockage entre versions mineures ; lisez le changelog
+   avant un changement de version majeure.
+2. Modifiez la version dans le fichier indiqué ci-dessus, puis lancez `docker compose up -d --build`.
+3. Vérifiez la barre du bas (reçus / stockés / perdus) et **Paramètres > Sources**. Pour revenir
+   en arrière, remettez la version précédente et lancez la même commande.
+
+## Organisation du code
+
+| Fichier | Contenu |
+|---|---|
+| `main.go` | configuration, démarrage |
+| `auth.go` | authentification : OpenID Connect (découverte, PKCE, contrôle de l'ID token) et cookie de session signé |
+| `auth_local.go` | mode `local` : page de connexion (`web/login.html`), cookie de session, `LOGIN_LOGO` |
+| `syslog.go` | écoute UDP/TCP et analyse RFC 3164 / 5424 |
+| `store.go` | insertions par lots dans VictoriaLogs et requêtes LogsQL |
+| `query.go` | traduit les filtres de l'interface en LogsQL ; filtre du direct |
+| `histogram.go` | frise : division, alignement des intervalles, `/api/histogram` |
+| `hub.go` | envoie les nouveaux messages aux navigateurs (SSE) |
+| `rdns.go` | résolutions DNS inverses avec cache |
+| `export.go` | export CSV en flux |
+| `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) |
+| `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources |
+| `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` |
+| `tags.go` | stockage des tags de couleur |
+| `presets.go`, `presets.json` | préréglages de tags (`/api/presets`), liste intégrée |
+| `api.go` | routes HTTP `/api/*` |
+| `web/` | interface (HTML, CSS, JavaScript simple, sans étape de build), embarquée dans le binaire ; les traductions sont dans `web/app.js` (`I18N`), et dans `web/login.html` pour la page de connexion |
+
+## Remarque
+
+Avec Docker Desktop (macOS/Windows), l'IP source vue par le conteneur pour les paquets UDP est
+la passerelle Docker. Le champ `host` vient toujours de l'en-tête syslog, qui porte normalement
+le vrai nom de l'émetteur.

README.md

@@ -1,3 +1,5 @@
+English | [Français](README.fr.md)
+
 # Logstream
 
 A simple syslog sink: receives logs over UDP/TCP on port 514, stores them in
@@ -10,6 +12,26 @@ devices ──514 udp/tcp──▶ logstream (Go) ──HTTP batches──▶ Vi
                               └──── API / UI ◀─────────┘
 ```
 
+## Architecture
+
+![Schéma logique de Logstream](docs/architecture.png)
+
+- **Ingestion**: syslog (UDP/TCP) and Docker container logs both go through `sink()` (reverse DNS
+  on IP hosts, without holding up the listeners), then the `Store` queue, which sends them in
+  batches to VictoriaLogs. When VictoriaLogs is unreachable, batches are kept on disk
+  (`/data/spool`, up to `SPOOL_MAX_MB`) and sent again, oldest first, once it is back.
+- **Live view**: `sink()` also publishes each message to the `Hub`, which streams it to the
+  browsers over SSE.
+- **Search**: the HTTP API turns the UI filters into LogsQL queries sent to VictoriaLogs.
+- **Timeline**: `/api/histogram` (`histogram.go`) counts the logs per interval and severity,
+  aligned on the local time; the zoom bounds (`from`/`to`) also apply to the list and the export.
+- **State**: tags and source settings live in `/data` (`logstream-data` volume); the logs
+  themselves in the `vlogs-data` volume.
+
+The editable source of the diagram is
+[`docs/architecture.excalidraw`](docs/architecture.excalidraw)
+(open it on [excalidraw.com](https://excalidraw.com)).
+
 ## Getting started
 
 ```bash
@@ -52,7 +74,7 @@ message, host and app. Words are combined with AND.
 The live view is disabled in this mode.
 
 Each row shows, from left to right: the **reception time** (server clock), the timestamp
-found in the message itself (`msg_time`), severity, host, app and message. Click a host or an
+found in the message itself (`msg_time`), severity, host, app, codes of the tags found and message. Click a host or an
 app to filter on it.
 
 Logs are indexed, searched and sorted by **reception time**: devices with a wrong clock
@@ -64,15 +86,31 @@ Severity badges `err`/`crit` and `warning` use the colors of the `error` and `wa
 
 Shortcuts: `/` focuses the search box, `Esc` clears it. Clicking a row shows all its fields.
 
+The column headers stay visible while scrolling. Drag the edge of a header (received, message
+time, severity, host, app) to resize the column, double-click it to go back to the automatic
+width; the message takes the remaining space. Widths are remembered by the browser
+(**Settings > Interface > Reset column widths** restores them all). On phones the list keeps its
+two-line layout without columns.
+
+## Stream and Time range modes
+
+The first control of the filter bar switches between two display modes:
+
+- **Stream**: the latest logs over a sliding duration (5 min to 30 days, or all), with the live
+  view.
+- **Time range**: the logs between a start and an end date, typed in the time zone chosen in
+  Settings. ◀ and ▶ move to the previous or next range of the same length, the magnifier doubles
+  it around its middle. The live view pauses; the mode and the range are kept across reloads.
+
 ## Timeline
 
 The timeline above the list shows the volume of logs per interval, counted by **reception
 time** on the server clock (so it matches the times shown in the rows).
 
 - Hover an interval: its bounds, total and detail per severity.
-- Click a bar to zoom on that interval, or drag across several bars to zoom on the selection.
+- Click a bar to show that interval in Time range mode, or drag across several bars to show the
-  The time range then shows the zoomed period ("× Reset zoom" or any other range leaves it);
+  selection; "× Back to stream" returns to Stream mode. The list, the counters and the CSV export
-  the list, the counters and the CSV export follow the zoom, and the live view pauses.
+  follow the range.
 - In live mode the last interval grows as messages arrive, and the timeline reloads at each new
   interval. Nothing is refreshed while the browser tab is hidden; it catches up when shown again.
 
@@ -111,8 +149,10 @@ colored and exported like syslog messages:
   to the labels shown) help with many containers. New containers are followed automatically
   unless that option is turned off. Choices are saved per compose service (or container name)
   in `/data/docker.json`, so they survive re-creations.
-- Logstream remembers the position read in each container (`/data/docker-state.json`): after a
+- Logstream remembers the position of the last line stored for each container
-  restart it resumes without losing or duplicating lines. A container seen for the first time
+  (`/data/docker-state.json`): after a restart it resumes without losing lines. The position
+  only moves once a line is in VictoriaLogs or in the disk buffer, and a full queue slows the
+  reading down instead of dropping lines. A container seen for the first time
   is read from `DOCKER_BACKFILL` ago (1 hour by default).
 - Logstream itself and the proxy below are never collected; add the label
   `logstream.exclude=true` to any other container to exclude it for good.
@@ -121,6 +161,32 @@ colored and exported like syslog messages:
 therefore goes through [docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy),
 which only lets through listing containers, reading logs, events and engine info (`GET` only).
 
+## Host system logs
+
+Logstream can also collect the system logs of the machine hosting the stack, without
+configuring anything on the host. The source is **off by default**: turn it on in
+**Settings > Sources > Host system logs** (saved in `/data/hostlogs.json`).
+
+- `docker-compose.yml` mounts `/var/log` and `/run/log/journal` read-only under `/host`.
+- When the host runs systemd, Logstream reads the **systemd journal** files directly (no
+  `journalctl` needed in the image): **host** is the machine name, **app** the program
+  (`SYSLOG_IDENTIFIER`), severity and facility come from the journal, and the systemd unit is
+  kept in `unit`. Otherwise it follows the text files of `/var/log` (`syslog`, `messages`,
+  `*.log`), parsed like syslog lines, with the file name in `log_file`.
+- These logs have the `host` source: the **Source** filter shows them alone.
+- When the source is turned on, the last hour is read first (`HOST_LOGS_BACKFILL`); the
+  position reached is saved in `/data/hostlogs-state.json`, so a restart neither loses nor
+  duplicates entries.
+- **Permissions**: the container runs as an unprivileged user and gets the `adm` group
+  (gid 4), which can read the journal and `/var/log` on Debian and Ubuntu. On other systems,
+  set `HOST_LOGS_GID` in `.env` to the gid of `systemd-journal`
+  (`getent group systemd-journal | cut -d: -f3`). Settings > Sources shows a clear message when
+  access is denied.
+- Limits: journal fields compressed by journald (messages longer than 512 bytes, compressed
+  with zstd/lz4/xz) cannot be decoded without extra libraries; they are counted in Settings and
+  shown as "(compressed journal entry)". Rotated or compressed text files (`*.1`, `*.gz`) are
+  not read.
+
 ## CSV export
 
 The **Export** button (next to the log count) downloads every stored log matching the
@@ -153,22 +219,107 @@ remembered per browser.
   switches to black or white to stay readable) and options: whole word, match case,
   regular expression, active. Tags are stored on the server in `/data/tags.json`
   (`logstream-data` volume), so they are shared by every browser. Default tags (pastel):
-  `warning` (orange), `error` (red), `ok` (green). Default tags still using the colors of
+  `warning` (orange) and `error` (red); `ok` (green) is in the *Log levels* preset. Default
-  earlier versions are switched to the pastel ones automatically.
+  tags still using the colors of earlier versions are switched to the pastel ones
+  automatically.
+  The *+ Preset…* menu adds ready-made tags: HTTP/HTTPS access logs (status codes, methods,
+  probes, bots, TLS and proxy errors), system logs (SSH, sudo, kernel, systemd, firewall),
+  applications (Docker, databases) and general patterns (log levels, IPv4 addresses). Tags
+  already in the list are skipped, and the added tags can be edited like any other. The list
+  comes from a text file you can edit (`PRESETS_FILE`): see [docs/presets.md](docs/presets.md)
+  for each preset and the file format. In a regular expression, a group named `hl`
+  (`(?<hl>…)`) colors only that part of the match: the presets use it to color the status code
+  or the method, not the text around it.
+  Each tag gets a two-digit code (`01`, `02`…) assigned by the server: it stays with the tag
+  until the tag is deleted (codes are also given to tags created by earlier versions). The
+  *Filters* column of the log list shows, as grey badges, the codes of the active tags found in
+  each message; it has room for 3, beyond that it shows 2 and `+N`, and the tooltip lists them
+  all.
 - **Interface**
   - *Theme*: System (follows the computer/phone preference), Light or Dark. The sun/moon
     button in the header switches between light and dark.
-  - *Log display*: font size (tiny, small, medium, large) and font: the system monospace
+  - *Log display*: font size (tiny, small, medium, large), density (normal, or compact to
-    font, or one of 12 free fonts made for dense text (JetBrains Mono, Fira Code, Source
+    fit about 50% more lines on screen) and font: the system monospace font, one of 3 narrow
-    Code Pro, IBM Plex Mono, Cascadia Code, Roboto Mono, Ubuntu Mono, Inconsolata, Red Hat
+    built-in fonts served by LogStream itself, which work offline (Inconsolata Condensed, the
-    Mono, Noto Sans Mono, Victor Mono, DM Mono). They are loaded by the browser from
+    narrowest, Iosevka and Ubuntu Mono), or one of 11 free fonts made for dense text (JetBrains
+    Mono, Fira Code, Source Code Pro, IBM Plex Mono, Cascadia Code, Roboto Mono, Inconsolata,
+    Red Hat Mono, Noto Sans Mono, Victor Mono, DM Mono). These 11 are loaded by the browser from
     [Bunny Fonts](https://fonts.bunny.net), a privacy-friendly European font service; without
-    internet access, the system font is used. Ligatures are disabled so `->` or `!=` show as typed.
+    internet access, the system font is used. Ligatures are disabled so `->` or `!=` show as
-- **Data**: "Delete all logs" permanently erases every stored log (you must type
+    typed. The densest setting is Tiny + Compact + Inconsolata Condensed.
+- **Data**: database figures read from VictoriaLogs (refreshed at most every 30 s): stored
+  lines, size on disk (index included), raw size and compression ratio, period covered with
+  the retention, lines of the last 24 h and last hour, distinct hosts and apps, free disk
+  space. Sizes use KB/MB/GB (Ko/Mo/Go in French). "Delete all logs" permanently erases every stored log (you must type
   `PURGE` to confirm). Tags and settings are kept. VictoriaLogs needs `-delete.enable`
-  (already set in `docker-compose.yml`); set `ALLOW_PURGE=false` to disable the feature.
+  (already set in `docker-compose.yml`). The feature is off by default: set `ALLOW_PURGE=true`
-  Anyone who can open the UI can purge: set `AUTH_USER` / `AUTH_PASS` if the UI is reachable
+  to allow it. Any admin can then purge: turn on [authentication](#authentication) if the UI
-  by others.
+  is reachable by others.
+
+## Authentication
+
+`AUTH_MODE` picks how the UI and the API are protected (`/healthz` always stays open):
+
+- **`local`** (default): a login page with the account `AUTH_USER` / `AUTH_PASS`; leave them
+  empty to have no authentication (for instance behind a reverse proxy that already checks). The
+  UI then shows a warning banner, which can be closed.
+- **`oidc`**: login through an OpenID Connect provider (Keycloak, Authentik, Authelia, Zitadel…),
+  authorization code flow with PKCE.
+
+In `local` mode the login page follows the theme and language of the UI. The session lasts
+`SESSION_TTL` (12 h by default), survives restarts (its signing key is in `/data/session.key`) and
+ends when `AUTH_USER` or `AUTH_PASS` changes; the log out button (top right) ends it. Failed logins
+are written in the logs with the client address (`auth: failed login for "bob" from 192.0.2.7`).
+Scripts can still call the API with HTTP Basic credentials (`curl -u user:pass`).
+
+An optional **read-only account**, `AUTH_VIEWER_USER` / `AUTH_VIEWER_PASS`, can search, follow
+the live view and export, but cannot change tags, sources or purge: those settings are greyed
+out in its UI and the API answers `403`.
+
+To show your logo on the login page, mount a PNG in the container and point `LOGIN_LOGO` to it:
+
+```yaml
+# docker-compose.yml, logstream service
+    volumes:
+      - ./logo.png:/config/logo.png:ro
+```
+```bash
+# .env
+LOGIN_LOGO=/config/logo.png
+```
+
+To use OIDC:
+
+1. In the provider, create a **confidential** client (with a secret) for logstream and register
+   the redirect URL `https://logs.example.org/auth/callback` (your address).
+2. In `.env`:
+   ```bash
+   AUTH_MODE=oidc
+   OIDC_ISSUER=https://sso.example.org/realms/home      # exactly the "issuer" of the provider
+   OIDC_CLIENT_ID=logstream
+   OIDC_CLIENT_SECRET=...
+   OIDC_REDIRECT_URL=https://logs.example.org/auth/callback
+   ```
+3. `docker compose up -d`. The logs show `oidc authentication enabled`, or the reason the
+   provider could not be read (wrong issuer, unreachable…).
+
+Opening the UI sends you to the provider's login page, then back to logstream. The session
+lasts `SESSION_TTL` (12 h by default) and survives restarts (its signing key is in
+`/data/session.key`); when it ends, the page goes through the login again. The log out button
+(top right) ends the logstream session, then opens the provider's log out page if it has one.
+
+Every user the provider accepts for this client can log in: restrict access in the provider
+(Keycloak: client roles or a dedicated realm; Authentik: application bindings). Logins are written
+in the logstream logs (`oidc: alice logged in`). With an `https` redirect URL, the cookies are
+only sent over HTTPS: logstream must be reached through a TLS reverse proxy.
+
+To give read-only access to some users, set `OIDC_ADMIN_GROUP` (for instance
+`logstream-admins`): only its members are admins, the others are read-only. The groups are read
+from the `groups` claim of the ID token (`OIDC_GROUPS_CLAIM` to use another one); in Keycloak,
+add a "Group Membership" mapper to the client (a leading `/` is ignored).
+
+Whatever the mode, every answer carries security headers (Content-Security-Policy,
+X-Frame-Options…), and the API refuses changes sent from another site (cross-site requests).
 
 ## Host names (reverse DNS)
 
@@ -180,11 +331,6 @@ on display, and the host filter shows `name (IP)`.
 The container uses Docker's DNS, which forwards to the host's resolvers. If your local names
 are only known by your router or a local DNS (Pi-hole, AdGuard, Unbound…), set
 `DNS_SERVER=192.168.1.1` (its address). Set `RDNS=off` to disable lookups.
-- **Color tags**: each tag has a keyword, a background color (the text automatically
-  switches to black or white to stay readable) and options: whole word, match case,
-  regular expression, active. Tags are stored in `/data/tags.json` (`logstream-data` volume).
-  Default tags (pastel): `warning` (orange), `error` (red), `ok` (green). Default tags
-  still using the colors of earlier versions are switched to the pastel ones automatically.
 
 ## Configuration
 
@@ -193,21 +339,39 @@ are only known by your router or a local DNS (Pi-hole, AdGuard, Unbound…), set
 | `SYSLOG_PORT` | `514` | syslog port published on the host |
 | `HTTP_PORT` | `8080` | web UI port |
 | `RETENTION` | `30d` | how long VictoriaLogs keeps logs |
-| `AUTH_USER` / `AUTH_PASS` | empty | HTTP Basic authentication for the UI |
+| `AUTH_MODE` | `local` | `local` (login page) or `oidc`, see [Authentication](#authentication) |
+| `AUTH_USER` / `AUTH_PASS` | empty | account of the login page (`local` mode); empty = no authentication |
+| `AUTH_VIEWER_USER` / `AUTH_VIEWER_PASS` | empty | optional read-only account (`local` mode) |
+| `LOGIN_LOGO` | empty | PNG shown on the login page, path inside the container (`local` mode) |
+| `SESSION_TTL` | `12h` | session lifetime (both modes; `OIDC_SESSION_TTL` still works) |
+| `OIDC_ISSUER` | empty | issuer URL of the OpenID Connect provider (`oidc` mode) |
+| `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` | empty | client registered in the provider |
+| `OIDC_REDIRECT_URL` | empty | callback URL of logstream, e.g. `https://logs.example.org/auth/callback` |
+| `OIDC_SCOPES` | `openid profile email` | requested scopes |
+| `OIDC_ADMIN_GROUP` | empty | only members of this group are admins, the others read-only (empty = everyone is admin) |
+| `OIDC_GROUPS_CLAIM` | `groups` | ID token claim that lists the groups |
 | `RDNS` | `on` | resolve IP hosts to DNS names |
 | `DNS_SERVER` | empty | DNS server for reverse lookups (`ip` or `ip:port`) |
-| `ALLOW_PURGE` | `true` | allow "Delete all logs" in Settings |
+| `ALLOW_PURGE` | `false` | allow "Delete all logs" in Settings |
+| `SYSLOG_TCP_MAX_CONNS` | `512` | syslog TCP connections open at once; more are refused |
+| `SYSLOG_TCP_IDLE` | `30m` | a syslog TCP connection silent this long is closed (senders reconnect) |
 | `EXPORT_MAX` | `100000` | maximum number of rows in a CSV export |
+| `PRESETS_FILE` | `/data/presets.json` | color tag presets file; the built-in list when missing (see [docs/presets.md](docs/presets.md)) |
 | `DOCKER_LOGS` | `on` in compose | collect the logs of the local Docker containers |
 | `DOCKER_HOST` | `tcp://docker-proxy:2375` in compose | Docker API address (`unix:///var/run/docker.sock` outside compose) |
 | `DOCKER_BACKFILL` | `1h` | history read from a container seen for the first time |
+| `HOST_LOGS_GID` | `4` (adm) in compose | group given to the container to read the host logs |
+| `HOST_LOGS_BACKFILL` | `1h` | history read when the host system logs source is turned on |
+| `HOST_LOGS_ROOT` | `/host` | where the host directories are mounted |
 | `TZ` | `Europe/Paris` | time zone for RFC 3164 timestamps (which carry none) |
 | `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | ingestion tuning |
+| `SPOOL_MAX_MB` | `1024` | disk buffer size for batches VictoriaLogs could not take (`0` = no buffer: retried for 15 s, then dropped) |
 
 ## Debugging
 
 - `docker compose logs -f logstream`: receive errors and errors sending to VictoriaLogs.
-- The bottom bar shows received / stored / dropped counters and the last storage error.
+- The bottom bar shows received / stored / dropped counters, the messages waiting in the disk
+  buffer, and the last storage error.
 - <http://localhost:9428/select/vmui>: VictoriaLogs' own UI to try LogsQL queries.
 - API:
   ```bash
@@ -244,7 +408,9 @@ To update one of them:
 
 | File | Contents |
 |---|---|
-| `main.go` | configuration, startup, authentication |
+| `main.go` | configuration, startup |
+| `auth.go` | authentication: OpenID Connect (discovery, PKCE, ID token checks) and the signed session cookie |
+| `auth_local.go` | `local` mode: login page (`web/login.html`), session cookie, `LOGIN_LOGO` |
 | `syslog.go` | UDP/TCP listeners and RFC 3164 / 5424 parsing |
 | `store.go` | batched inserts into VictoriaLogs and LogsQL queries |
 | `query.go` | turns UI filters into LogsQL; live-view filter |
@@ -254,9 +420,11 @@ To update one of them:
 | `export.go` | streamed CSV export |
 | `docker.go` | Docker container logs (API, followers, positions, level detection) |
 | `syslogserver.go` | syslog listeners opened and closed from Settings > Sources |
+| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower |
 | `tags.go` | color tag storage |
+| `presets.go`, `presets.json` | color tag presets (`/api/presets`), built-in list |
 | `api.go` | `/api/*` HTTP routes |
-| `web/` | UI (HTML, CSS, plain JavaScript, no build step), embedded in the binary; translations live in `web/app.js` (`I18N`) |
+| `web/` | UI (HTML, CSS, plain JavaScript, no build step), embedded in the binary; translations live in `web/app.js` (`I18N`), and in `web/login.html` for the login page |
 
 ## Note
 

api.go

@@ -16,11 +16,14 @@ type API struct {
 	store      *Store
 	hub        *Hub
 	tags       *TagStore
+	presets    string // presets file, built-in presets when missing
 	rdns       *ReverseDNS
 	allowPurge bool
 	exportMax  int
 	docker     *DockerManager // nil when DOCKER_LOGS is off
 	syslog     *SyslogServer
+	host       *HostLogs
+	dbstats    dbStatsCache
 }
 
 func (a *API) Routes(mux *http.ServeMux) {
@@ -30,9 +33,11 @@ func (a *API) Routes(mux *http.ServeMux) {
 	mux.HandleFunc("GET /api/facets", a.facets)
 	mux.HandleFunc("GET /api/stream", a.stream)
 	mux.HandleFunc("GET /api/stats", a.stats)
+	mux.HandleFunc("GET /api/dbstats", a.dbStats)
 	mux.HandleFunc("GET /api/tags", a.listTags)
 	mux.HandleFunc("POST /api/tags", a.createTag)
 	mux.HandleFunc("POST /api/tags/reset", a.resetTags)
+	mux.HandleFunc("GET /api/presets", a.listPresets)
 	mux.HandleFunc("PUT /api/tags/{id}", a.updateTag)
 	mux.HandleFunc("DELETE /api/tags/{id}", a.deleteTag)
 	mux.HandleFunc("GET /api/purge", a.purgeStatus)
@@ -42,6 +47,28 @@ func (a *API) Routes(mux *http.ServeMux) {
 	mux.HandleFunc("PUT /api/syslog", a.syslogConfigure)
 	mux.HandleFunc("GET /api/docker", a.dockerStatus)
 	mux.HandleFunc("PUT /api/docker", a.dockerConfigure)
+	mux.HandleFunc("GET /api/hostlogs", a.hostLogsStatus)
+	mux.HandleFunc("PUT /api/hostlogs", a.hostLogsConfigure)
+}
+
+// GET /api/hostlogs: state of the host system logs source (Settings > Sources).
+func (a *API) hostLogsStatus(w http.ResponseWriter, r *http.Request) {
+	writeJSON(w, http.StatusOK, a.host.Status())
+}
+
+// PUT /api/hostlogs {"enabled": bool}
+func (a *API) hostLogsConfigure(w http.ResponseWriter, r *http.Request) {
+	var cfg hostLogsConfig
+	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&cfg); err != nil {
+		writeErr(w, http.StatusBadRequest, err)
+		return
+	}
+	if err := a.host.Configure(cfg); err != nil {
+		writeErr(w, http.StatusInternalServerError, err)
+		return
+	}
+	log.Printf("host system logs changed from %s: %+v", r.RemoteAddr, cfg)
+	writeJSON(w, http.StatusOK, a.host.Status())
 }
 
 // GET /api/syslog: syslog reception state (Settings > Sources).
@@ -289,10 +316,25 @@ func (a *API) stats(w http.ResponseWriter, r *http.Request) {
 	writeJSON(w, http.StatusOK, s)
 }
 
+// GET /api/dbstats[?refresh=1]: size and content of the VictoriaLogs base
+// (Settings > Data), cached for 30 s.
+func (a *API) dbStats(w http.ResponseWriter, r *http.Request) {
+	st, err := a.dbstats.get(r.Context(), a.store, r.URL.Query().Get("refresh") == "1")
+	if err != nil {
+		writeErr(w, http.StatusBadGateway, err)
+		return
+	}
+	writeJSON(w, http.StatusOK, st)
+}
+
 func (a *API) listTags(w http.ResponseWriter, r *http.Request) {
 	writeJSON(w, http.StatusOK, a.tags.List())
 }
 
+func (a *API) listPresets(w http.ResponseWriter, r *http.Request) {
+	writeJSON(w, http.StatusOK, loadPresets(a.presets))
+}
+
 func decodeTag(w http.ResponseWriter, r *http.Request) (Tag, error) {
 	var t Tag
 	err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64*1024)).Decode(&t)

auth.go

@@ -0,0 +1,658 @@
+package main
+
+import (
+	"bytes"
+	"crypto"
+	"crypto/ecdsa"
+	"crypto/elliptic"
+	"crypto/hmac"
+	"crypto/rand"
+	"crypto/rsa"
+	"crypto/sha256"
+	_ "crypto/sha512" // SHA-384/512 for RS384, ES384, RS512…
+	"crypto/subtle"
+	"encoding/base64"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"log"
+	"math/big"
+	"net/http"
+	"net/url"
+	"os"
+	"path/filepath"
+	"strings"
+	"sync"
+	"time"
+)
+
+// Web UI authentication. AUTH_MODE=local (default) shows a login page when AUTH_USER /
+// AUTH_PASS are set (auth_local.go); AUTH_MODE=oidc delegates the login to an
+// OpenID Connect provider (Keycloak, Authentik, Authelia…) with the authorization code
+// flow and PKCE. Only the standard library is used.
+
+const (
+	sessionCookie = "logstream_session"
+	loginCookie   = "logstream_login_" // + state: one cookie per login in progress
+	loginTTL      = 10 * time.Minute
+	clockSkew     = time.Minute
+)
+
+type authConfig struct {
+	mode         string
+	user, pass   string // local mode
+	viewerUser   string // local mode: optional read-only account
+	viewerPass   string
+	adminGroup   string // oidc: only members of this group are admins (empty: everyone)
+	groupsClaim  string // oidc: ID token claim listing the groups
+	issuer       string
+	clientID     string
+	clientSecret string
+	redirectURL  string
+	scopes       string
+	sessionTTL   time.Duration
+	dataDir      string
+	loginLogo    string // local mode: PNG shown on the login page
+
+}
+
+// newAuth returns the middleware that protects the UI and the API (except /healthz).
+func newAuth(c authConfig, next http.Handler) (http.Handler, error) {
+	if c.sessionTTL <= 0 {
+		c.sessionTTL = 12 * time.Hour
+	}
+	switch strings.ToLower(c.mode) {
+	case "", "local":
+		if c.user == "" {
+			return next, nil
+		}
+		l := newLocal(c)
+		l.next = next
+		return l, nil
+	case "oidc":
+		o, err := newOIDC(c)
+		if err != nil {
+			return nil, err
+		}
+		o.next = next
+		log.Printf("oidc authentication enabled (issuer %s)", c.issuer)
+		return o, nil
+	}
+	return nil, fmt.Errorf("AUTH_MODE=%q: expected local or oidc", c.mode)
+}
+
+type oidcMeta struct {
+	Issuer           string   `json:"issuer"`
+	AuthEndpoint     string   `json:"authorization_endpoint"`
+	TokenEndpoint    string   `json:"token_endpoint"`
+	JWKSURI          string   `json:"jwks_uri"`
+	EndSession       string   `json:"end_session_endpoint"`
+	TokenAuthMethods []string `json:"token_endpoint_auth_methods_supported"`
+}
+
+type OIDC struct {
+	cfg      authConfig
+	callback string // path of OIDC_REDIRECT_URL
+	secure   bool   // cookies only sent over HTTPS
+	key      []byte // signs the session and login cookies
+	client   *http.Client
+	next     http.Handler
+
+	mu     sync.Mutex
+	meta   *oidcMeta
+	keys   map[string]crypto.PublicKey
+	keysAt time.Time
+}
+
+func newOIDC(c authConfig) (*OIDC, error) {
+	var missing []string
+	for _, v := range [][2]string{
+		{"OIDC_ISSUER", c.issuer}, {"OIDC_CLIENT_ID", c.clientID},
+		{"OIDC_CLIENT_SECRET", c.clientSecret}, {"OIDC_REDIRECT_URL", c.redirectURL},
+	} {
+		if v[1] == "" {
+			missing = append(missing, v[0])
+		}
+	}
+	if len(missing) > 0 {
+		return nil, fmt.Errorf("AUTH_MODE=oidc: missing %s", strings.Join(missing, ", "))
+	}
+	ru, err := url.Parse(c.redirectURL)
+	if err != nil || ru.Host == "" || ru.Path == "" || ru.Path == "/" {
+		return nil, fmt.Errorf("OIDC_REDIRECT_URL=%q: expected a full URL such as https://logs.example.org/auth/callback", c.redirectURL)
+	}
+	if c.scopes == "" {
+		c.scopes = "openid profile email"
+	}
+	if !strings.Contains(" "+c.scopes+" ", " openid ") {
+		c.scopes = "openid " + c.scopes
+	}
+	if c.groupsClaim == "" {
+		c.groupsClaim = "groups"
+	}
+	return &OIDC{
+		cfg:      c,
+		callback: ru.Path,
+		secure:   ru.Scheme == "https",
+		key:      sessionKey(c.dataDir),
+		client:   &http.Client{Timeout: 10 * time.Second},
+	}, nil
+}
+
+// checkProvider reads the provider configuration at startup so a mistake shows in the logs.
+func (o *OIDC) checkProvider() {
+	if _, err := o.discover(); err != nil {
+		log.Printf("oidc: %v", err)
+	}
+}
+
+// sessionKey is kept in DATA_DIR so sessions survive a restart.
+func sessionKey(dir string) []byte {
+	path := filepath.Join(dir, "session.key")
+	if k, err := os.ReadFile(path); err == nil && len(k) >= 32 {
+		return k
+	}
+	k := make([]byte, 32)
+	if _, err := rand.Read(k); err != nil {
+		log.Fatalf("session key: %v", err)
+	}
+	if err := os.WriteFile(path, k, 0o600); err != nil {
+		log.Printf("auth: cannot save %s (%v): sessions end when logstream restarts", path, err)
+	}
+	return k
+}
+
+type session struct {
+	User   string `json:"u"`
+	Exp    int64  `json:"e"`
+	Viewer bool   `json:"v,omitempty"` // read-only user
+}
+
+// writeAuthRequired answers API calls without a session; the UI turns it into a reload
+// (and so into a new login).
+func writeAuthRequired(w http.ResponseWriter) {
+	w.Header().Set("Content-Type", "application/json")
+	w.WriteHeader(http.StatusUnauthorized)
+	_, _ = w.Write([]byte(`{"error":"authentication required","code":"auth"}` + "\n"))
+}
+
+type loginState struct {
+	Nonce    string `json:"n"`
+	Verifier string `json:"v"`
+	Return   string `json:"r"`
+	Exp      int64  `json:"e"`
+}
+
+func (o *OIDC) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+	switch r.URL.Path {
+	case "/healthz":
+		o.next.ServeHTTP(w, r)
+		return
+	case o.callback:
+		o.handleCallback(w, r)
+		return
+	case "/auth/logout":
+		o.handleLogout(w, r)
+		return
+	}
+	var s session
+	if c, err := r.Cookie(sessionCookie); err == nil && verifyCookie(o.key, c.Value, &s) && time.Now().Unix() < s.Exp {
+		if r.URL.Path == "/auth/me" {
+			writeJSON(w, http.StatusOK, map[string]string{"mode": "oidc", "user": s.User, "role": roleName(s.Viewer)})
+			return
+		}
+		if s.Viewer {
+			r = asViewer(r)
+		}
+		o.next.ServeHTTP(w, r)
+		return
+	}
+	// Not logged in: pages go to the provider, API calls get a 401 that the UI turns
+	// into a reload (and so into a new login).
+	if r.Method == http.MethodGet && !strings.HasPrefix(r.URL.Path, "/api/") && r.URL.Path != "/auth/me" {
+		o.startLogin(w, r)
+		return
+	}
+	writeAuthRequired(w)
+}
+
+func (o *OIDC) startLogin(w http.ResponseWriter, r *http.Request) {
+	meta, err := o.discover()
+	if err != nil {
+		log.Printf("oidc: %v", err)
+		http.Error(w, "identity provider unreachable, try again later", http.StatusBadGateway)
+		return
+	}
+	state, nonce, verifier := randomString(), randomString(), randomString()+randomString()
+	ret := r.URL.RequestURI()
+	if !strings.HasPrefix(ret, "/") || strings.HasPrefix(ret, "//") {
+		ret = "/"
+	}
+	http.SetCookie(w, &http.Cookie{
+		Name:     loginCookie + state,
+		Value:    signCookie(o.key, loginState{Nonce: nonce, Verifier: verifier, Return: ret, Exp: time.Now().Add(loginTTL).Unix()}),
+		Path:     "/",
+		MaxAge:   int(loginTTL.Seconds()),
+		HttpOnly: true,
+		Secure:   o.secure,
+		SameSite: http.SameSiteLaxMode, // sent back on the redirect from the provider
+	})
+	challenge := sha256.Sum256([]byte(verifier))
+	q := url.Values{
+		"response_type":         {"code"},
+		"client_id":             {o.cfg.clientID},
+		"redirect_uri":          {o.cfg.redirectURL},
+		"scope":                 {o.cfg.scopes},
+		"state":                 {state},
+		"nonce":                 {nonce},
+		"code_challenge":        {base64.RawURLEncoding.EncodeToString(challenge[:])},
+		"code_challenge_method": {"S256"},
+	}
+	http.Redirect(w, r, addQuery(meta.AuthEndpoint, q), http.StatusFound)
+}
+
+func (o *OIDC) handleCallback(w http.ResponseWriter, r *http.Request) {
+	q := r.URL.Query()
+	if e := q.Get("error"); e != "" {
+		log.Printf("oidc: login refused by the provider: %s %s", e, q.Get("error_description"))
+		http.Error(w, "login refused by the identity provider: "+e, http.StatusForbidden)
+		return
+	}
+	state := q.Get("state")
+	var ls loginState
+	c, err := r.Cookie(loginCookie + state)
+	if state == "" || err != nil || !verifyCookie(o.key, c.Value, &ls) || time.Now().Unix() > ls.Exp {
+		http.Error(w, "login expired or started in another browser: open LogStream again", http.StatusBadRequest)
+		return
+	}
+	http.SetCookie(w, &http.Cookie{Name: loginCookie + state, Path: "/", MaxAge: -1, HttpOnly: true, Secure: o.secure})
+
+	user, viewer, err := o.exchange(r, q.Get("code"), ls)
+	if err != nil {
+		log.Printf("oidc: login failed: %v", err)
+		http.Error(w, "login failed, see the LogStream logs", http.StatusForbidden)
+		return
+	}
+	log.Printf("oidc: %s logged in (%s)", user, roleName(viewer))
+	http.SetCookie(w, &http.Cookie{
+		Name:     sessionCookie,
+		Value:    signCookie(o.key, session{User: user, Exp: time.Now().Add(o.cfg.sessionTTL).Unix(), Viewer: viewer}),
+		Path:     "/",
+		MaxAge:   int(o.cfg.sessionTTL.Seconds()),
+		HttpOnly: true,
+		Secure:   o.secure,
+		SameSite: http.SameSiteLaxMode,
+	})
+	http.Redirect(w, r, ls.Return, http.StatusFound)
+}
+
+// The session ends here; the provider's own session ends on its logout page if it has one.
+func (o *OIDC) handleLogout(w http.ResponseWriter, r *http.Request) {
+	http.SetCookie(w, &http.Cookie{Name: sessionCookie, Path: "/", MaxAge: -1, HttpOnly: true, Secure: o.secure})
+	if meta, err := o.discover(); err == nil && meta.EndSession != "" {
+		http.Redirect(w, r, addQuery(meta.EndSession, url.Values{"client_id": {o.cfg.clientID}}), http.StatusFound)
+		return
+	}
+	http.Redirect(w, r, "/", http.StatusFound)
+}
+
+// exchange trades the code for tokens and returns the user name from the verified ID
+// token, and whether the user is read-only (not in OIDC_ADMIN_GROUP).
+func (o *OIDC) exchange(r *http.Request, code string, ls loginState) (string, bool, error) {
+	if code == "" {
+		return "", false, errors.New("no code in the callback")
+	}
+	meta, err := o.discover()
+	if err != nil {
+		return "", false, err
+	}
+	form := url.Values{
+		"grant_type":    {"authorization_code"},
+		"code":          {code},
+		"redirect_uri":  {o.cfg.redirectURL},
+		"code_verifier": {ls.Verifier},
+	}
+	// client_secret_basic is the default; some providers only accept client_secret_post.
+	post := len(meta.TokenAuthMethods) > 0 && !contains(meta.TokenAuthMethods, "client_secret_basic") && contains(meta.TokenAuthMethods, "client_secret_post")
+	if post {
+		form.Set("client_id", o.cfg.clientID)
+		form.Set("client_secret", o.cfg.clientSecret)
+	}
+	req, err := http.NewRequestWithContext(r.Context(), http.MethodPost, meta.TokenEndpoint, strings.NewReader(form.Encode()))
+	if err != nil {
+		return "", false, err
+	}
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+	req.Header.Set("Accept", "application/json")
+	if !post {
+		req.SetBasicAuth(url.QueryEscape(o.cfg.clientID), url.QueryEscape(o.cfg.clientSecret))
+	}
+	res, err := o.client.Do(req)
+	if err != nil {
+		return "", false, fmt.Errorf("token endpoint: %w", err)
+	}
+	defer res.Body.Close()
+	body, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
+	if res.StatusCode != http.StatusOK {
+		return "", false, fmt.Errorf("token endpoint: %s: %s", res.Status, bytes.TrimSpace(body))
+	}
+	var tok struct {
+		IDToken string `json:"id_token"`
+	}
+	if err := json.Unmarshal(body, &tok); err != nil || tok.IDToken == "" {
+		return "", false, errors.New("token endpoint: no id_token in the response")
+	}
+	claims, err := o.verifyIDToken(tok.IDToken, ls.Nonce)
+	if err != nil {
+		return "", false, err
+	}
+	viewer := o.cfg.adminGroup != "" && !hasGroup(claims[o.cfg.groupsClaim], o.cfg.adminGroup)
+	for _, k := range []string{"preferred_username", "email", "name", "sub"} {
+		if v, _ := claims[k].(string); v != "" {
+			return v, viewer, nil
+		}
+	}
+	return "", false, errors.New("id_token: no sub")
+}
+
+// hasGroup tells whether the groups claim (a list, or a single string) holds
+// group; a leading "/" (Keycloak group paths) is ignored.
+func hasGroup(claim any, group string) bool {
+	group = strings.TrimPrefix(group, "/")
+	var groups []string
+	switch v := claim.(type) {
+	case string:
+		groups = strings.Fields(strings.ReplaceAll(v, ",", " "))
+	case []any:
+		for _, g := range v {
+			if s, ok := g.(string); ok {
+				groups = append(groups, s)
+			}
+		}
+	}
+	for _, g := range groups {
+		if strings.TrimPrefix(g, "/") == group {
+			return true
+		}
+	}
+	return false
+}
+
+// verifyIDToken checks the signature (keys from jwks_uri) and the claims of an ID token.
+func (o *OIDC) verifyIDToken(raw, nonce string) (map[string]any, error) {
+	parts := strings.Split(raw, ".")
+	if len(parts) != 3 {
+		return nil, errors.New("id_token: not a JWT")
+	}
+	var hdr struct {
+		Alg string `json:"alg"`
+		Kid string `json:"kid"`
+	}
+	if err := decodeSegment(parts[0], &hdr); err != nil {
+		return nil, fmt.Errorf("id_token header: %w", err)
+	}
+	sig, err := base64.RawURLEncoding.DecodeString(parts[2])
+	if err != nil {
+		return nil, errors.New("id_token: bad signature encoding")
+	}
+	key, err := o.keyFor(hdr.Kid)
+	if err != nil {
+		return nil, err
+	}
+	if err := verifySignature(hdr.Alg, key, []byte(parts[0]+"."+parts[1]), sig); err != nil {
+		return nil, fmt.Errorf("id_token: %w", err)
+	}
+	var claims map[string]any
+	if err := decodeSegment(parts[1], &claims); err != nil {
+		return nil, fmt.Errorf("id_token claims: %w", err)
+	}
+	if iss, _ := claims["iss"].(string); iss != o.cfg.issuer {
+		return nil, fmt.Errorf("id_token: issuer %q, expected %q", iss, o.cfg.issuer)
+	}
+	var aud []string
+	switch v := claims["aud"].(type) {
+	case string:
+		aud = []string{v}
+	case []any:
+		for _, a := range v {
+			if s, ok := a.(string); ok {
+				aud = append(aud, s)
+			}
+		}
+	}
+	if !contains(aud, o.cfg.clientID) {
+		return nil, fmt.Errorf("id_token: audience %v does not include %q", aud, o.cfg.clientID)
+	}
+	if azp, ok := claims["azp"].(string); ok && len(aud) > 1 && azp != o.cfg.clientID {
+		return nil, fmt.Errorf("id_token: azp %q", azp)
+	}
+	now := time.Now()
+	exp, _ := claims["exp"].(float64)
+	if exp == 0 || now.After(time.Unix(int64(exp), 0).Add(clockSkew)) {
+		return nil, errors.New("id_token: expired (check the clocks)")
+	}
+	if iat, ok := claims["iat"].(float64); ok && time.Unix(int64(iat), 0).After(now.Add(clockSkew)) {
+		return nil, errors.New("id_token: issued in the future (check the clocks)")
+	}
+	if n, _ := claims["nonce"].(string); subtle.ConstantTimeCompare([]byte(n), []byte(nonce)) != 1 {
+		return nil, errors.New("id_token: wrong nonce")
+	}
+	return claims, nil
+}
+
+func verifySignature(alg string, key crypto.PublicKey, signed, sig []byte) error {
+	if len(alg) != 5 {
+		return fmt.Errorf("unsupported algorithm %q", alg)
+	}
+	var h crypto.Hash
+	switch alg[2:] {
+	case "256":
+		h = crypto.SHA256
+	case "384":
+		h = crypto.SHA384
+	case "512":
+		h = crypto.SHA512
+	}
+	if h == 0 {
+		return fmt.Errorf("unsupported algorithm %q", alg)
+	}
+	hh := h.New()
+	hh.Write(signed)
+	digest := hh.Sum(nil)
+	switch k := key.(type) {
+	case *rsa.PublicKey:
+		switch alg[:2] {
+		case "RS":
+			return rsa.VerifyPKCS1v15(k, h, digest, sig)
+		case "PS":
+			return rsa.VerifyPSS(k, h, digest, sig, &rsa.PSSOptions{SaltLength: rsa.PSSSaltLengthEqualsHash})
+		}
+	case *ecdsa.PublicKey:
+		size := (k.Curve.Params().BitSize + 7) / 8
+		if alg[:2] != "ES" || len(sig) != 2*size {
+			break
+		}
+		r, s := new(big.Int).SetBytes(sig[:size]), new(big.Int).SetBytes(sig[size:])
+		if ecdsa.Verify(k, digest, r, s) {
+			return nil
+		}
+		return errors.New("bad signature")
+	}
+	return fmt.Errorf("algorithm %q does not match the key", alg)
+}
+
+// discover reads the provider configuration once (and again after a failure).
+func (o *OIDC) discover() (*oidcMeta, error) {
+	o.mu.Lock()
+	defer o.mu.Unlock()
+	if o.meta != nil {
+		return o.meta, nil
+	}
+	u := strings.TrimSuffix(o.cfg.issuer, "/") + "/.well-known/openid-configuration"
+	var m oidcMeta
+	if err := o.getJSON(u, &m); err != nil {
+		return nil, fmt.Errorf("discovery: %w", err)
+	}
+	if m.Issuer != o.cfg.issuer {
+		return nil, fmt.Errorf("discovery: the provider says its issuer is %q, set OIDC_ISSUER to that exact value", m.Issuer)
+	}
+	if m.AuthEndpoint == "" || m.TokenEndpoint == "" || m.JWKSURI == "" {
+		return nil, errors.New("discovery: incomplete provider configuration")
+	}
+	o.meta = &m
+	return o.meta, nil
+}
+
+// keyFor returns the signing key kid; the key set is reloaded when the provider rotates its keys.
+func (o *OIDC) keyFor(kid string) (crypto.PublicKey, error) {
+	meta, err := o.discover()
+	if err != nil {
+		return nil, err
+	}
+	o.mu.Lock()
+	defer o.mu.Unlock()
+	pick := func() crypto.PublicKey {
+		if k, ok := o.keys[kid]; ok {
+			return k
+		}
+		if kid == "" && len(o.keys) == 1 {
+			for _, k := range o.keys {
+				return k
+			}
+		}
+		return nil
+	}
+	if k := pick(); k != nil {
+		return k, nil
+	}
+	if time.Since(o.keysAt) < 10*time.Second {
+		return nil, fmt.Errorf("id_token: unknown key %q", kid)
+	}
+	var set struct {
+		Keys []struct {
+			Kty string `json:"kty"`
+			Kid string `json:"kid"`
+			Use string `json:"use"`
+			N   string `json:"n"`
+			E   string `json:"e"`
+			Crv string `json:"crv"`
+			X   string `json:"x"`
+			Y   string `json:"y"`
+		} `json:"keys"`
+	}
+	if err := o.getJSON(meta.JWKSURI, &set); err != nil {
+		return nil, fmt.Errorf("jwks: %w", err)
+	}
+	keys := map[string]crypto.PublicKey{}
+	for _, k := range set.Keys {
+		if k.Use != "" && k.Use != "sig" {
+			continue
+		}
+		switch k.Kty {
+		case "RSA":
+			n, e := decodeBig(k.N), decodeBig(k.E)
+			if n != nil && e != nil && e.IsInt64() {
+				keys[k.Kid] = &rsa.PublicKey{N: n, E: int(e.Int64())}
+			}
+		case "EC":
+			var c elliptic.Curve
+			switch k.Crv {
+			case "P-256":
+				c = elliptic.P256()
+			case "P-384":
+				c = elliptic.P384()
+			case "P-521":
+				c = elliptic.P521()
+			}
+			x, y := decodeBig(k.X), decodeBig(k.Y)
+			if c != nil && x != nil && y != nil && c.IsOnCurve(x, y) {
+				keys[k.Kid] = &ecdsa.PublicKey{Curve: c, X: x, Y: y}
+			}
+		}
+	}
+	o.keys, o.keysAt = keys, time.Now()
+	if k := pick(); k != nil {
+		return k, nil
+	}
+	return nil, fmt.Errorf("id_token: unknown key %q", kid)
+}
+
+func (o *OIDC) getJSON(u string, v any) error {
+	res, err := o.client.Get(u)
+	if err != nil {
+		return err
+	}
+	defer res.Body.Close()
+	if res.StatusCode != http.StatusOK {
+		return fmt.Errorf("%s: %s", u, res.Status)
+	}
+	return json.NewDecoder(io.LimitReader(res.Body, 1<<20)).Decode(v)
+}
+
+// Cookies are base64url(JSON) + "." + base64url(HMAC-SHA256).
+func signCookie(key []byte, v any) string {
+	b, _ := json.Marshal(v)
+	p := base64.RawURLEncoding.EncodeToString(b)
+	m := hmac.New(sha256.New, key)
+	m.Write([]byte(p))
+	return p + "." + base64.RawURLEncoding.EncodeToString(m.Sum(nil))
+}
+
+func verifyCookie(key []byte, s string, v any) bool {
+	p, sig, ok := strings.Cut(s, ".")
+	if !ok {
+		return false
+	}
+	got, err := base64.RawURLEncoding.DecodeString(sig)
+	if err != nil {
+		return false
+	}
+	m := hmac.New(sha256.New, key)
+	m.Write([]byte(p))
+	if !hmac.Equal(got, m.Sum(nil)) {
+		return false
+	}
+	return decodeSegment(p, v) == nil
+}
+
+func decodeSegment(s string, v any) error {
+	b, err := base64.RawURLEncoding.DecodeString(s)
+	if err != nil {
+		return err
+	}
+	return json.Unmarshal(b, v)
+}
+
+func decodeBig(s string) *big.Int {
+	b, err := base64.RawURLEncoding.DecodeString(s)
+	if err != nil || len(b) == 0 {
+		return nil
+	}
+	return new(big.Int).SetBytes(b)
+}
+
+func randomString() string {
+	b := make([]byte, 16)
+	if _, err := rand.Read(b); err != nil {
+		panic(err)
+	}
+	return base64.RawURLEncoding.EncodeToString(b)
+}
+
+func addQuery(endpoint string, q url.Values) string {
+	sep := "?"
+	if strings.Contains(endpoint, "?") {
+		sep = "&"
+	}
+	return endpoint + sep + q.Encode()
+}
+
+func contains(list []string, s string) bool {
+	for _, v := range list {
+		if v == s {
+			return true
+		}
+	}
+	return false
+}

auth_local.go

@@ -0,0 +1,189 @@
+package main
+
+import (
+	"crypto/hmac"
+	"crypto/sha256"
+	"crypto/subtle"
+	"log"
+	"net"
+	"net/http"
+	"net/url"
+	"os"
+	"strings"
+	"time"
+)
+
+// Local mode with AUTH_USER set: a login page (web/login.html) in the colors of the UI and a
+// session kept in a signed cookie, like the OIDC mode. HTTP Basic credentials are still
+// accepted so scripts calling the API keep working, but the browser popup is gone.
+
+const loginPage = "/login.html"
+
+var loginFailDelay = time.Second // slows down password guessing
+
+type Local struct {
+	user, pass string
+	viewerUser string // optional read-only account
+	viewerPass string
+	ttl        time.Duration
+	logo       string // LOGIN_LOGO, served at /auth/logo
+	key        []byte
+	next       http.Handler
+}
+
+func newLocal(c authConfig) *Local {
+	// The key depends on the credentials: changing AUTH_USER or AUTH_PASS ends every session.
+	m := hmac.New(sha256.New, sessionKey(c.dataDir))
+	m.Write([]byte("local\x00" + c.user + "\x00" + c.pass + "\x00" + c.viewerUser + "\x00" + c.viewerPass))
+	if c.loginLogo != "" {
+		if _, err := os.Stat(c.loginLogo); err != nil {
+			log.Printf("auth: LOGIN_LOGO: %v", err)
+		}
+	}
+	log.Printf("local authentication enabled (user %s)", c.user)
+	if c.viewerUser != "" {
+		log.Printf("local read-only account enabled (user %s)", c.viewerUser)
+	}
+	return &Local{user: c.user, pass: c.pass, viewerUser: c.viewerUser, viewerPass: c.viewerPass, ttl: c.sessionTTL, logo: c.loginLogo, key: m.Sum(nil)}
+}
+
+func (l *Local) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+	switch r.URL.Path {
+	case "/healthz", "/style.css":
+		l.next.ServeHTTP(w, r)
+		return
+	case "/auth/logo":
+		l.serveLogo(w, r)
+		return
+	case "/auth/login":
+		l.handleLogin(w, r)
+		return
+	case "/auth/logout":
+		http.SetCookie(w, &http.Cookie{Name: sessionCookie, Path: "/", MaxAge: -1, HttpOnly: true, Secure: isHTTPS(r)})
+		http.Redirect(w, r, loginPage, http.StatusFound)
+		return
+	}
+	s, ok := l.sessionUser(r)
+	if !ok {
+		if u, p, basic := r.BasicAuth(); basic {
+			if viewer, valid := l.check(u, p); valid {
+				s, ok = session{User: u, Viewer: viewer}, true
+			}
+		}
+	}
+	if ok && s.Viewer {
+		r = asViewer(r)
+	}
+	switch {
+	case r.URL.Path == loginPage:
+		if ok {
+			http.Redirect(w, r, safeReturn(r.URL.Query().Get("r")), http.StatusFound)
+			return
+		}
+		w.Header().Set("Cache-Control", "no-store")
+		l.next.ServeHTTP(w, r)
+	case ok && r.URL.Path == "/auth/me":
+		writeJSON(w, http.StatusOK, map[string]string{"mode": "local", "user": s.User, "role": roleName(s.Viewer)})
+	case ok:
+		l.next.ServeHTTP(w, r)
+	case r.Method == http.MethodGet && !strings.HasPrefix(r.URL.Path, "/api/") && r.URL.Path != "/auth/me":
+		target := loginPage
+		if ret := r.URL.RequestURI(); ret != "/" {
+			target += "?" + url.Values{"r": {ret}}.Encode()
+		}
+		http.Redirect(w, r, target, http.StatusFound)
+	default:
+		writeAuthRequired(w)
+	}
+}
+
+func (l *Local) handleLogin(w http.ResponseWriter, r *http.Request) {
+	if r.Method != http.MethodPost {
+		http.Redirect(w, r, loginPage, http.StatusFound)
+		return
+	}
+	user, pass := r.PostFormValue("user"), r.PostFormValue("pass")
+	ret := safeReturn(r.PostFormValue("r"))
+	viewer, valid := l.check(user, pass)
+	if !valid {
+		log.Printf("auth: failed login for %q from %s", user, clientIP(r))
+		time.Sleep(loginFailDelay)
+		q := url.Values{"e": {"1"}}
+		if ret != "/" {
+			q.Set("r", ret)
+		}
+		http.Redirect(w, r, loginPage+"?"+q.Encode(), http.StatusSeeOther)
+		return
+	}
+	log.Printf("auth: %s logged in from %s (%s)", user, clientIP(r), roleName(viewer))
+	http.SetCookie(w, &http.Cookie{
+		Name:     sessionCookie,
+		Value:    signCookie(l.key, session{User: user, Exp: time.Now().Add(l.ttl).Unix(), Viewer: viewer}),
+		Path:     "/",
+		MaxAge:   int(l.ttl.Seconds()),
+		HttpOnly: true,
+		Secure:   isHTTPS(r),
+		SameSite: http.SameSiteLaxMode,
+	})
+	http.Redirect(w, r, ret, http.StatusSeeOther)
+}
+
+func (l *Local) sessionUser(r *http.Request) (session, bool) {
+	var s session
+	c, err := r.Cookie(sessionCookie)
+	if err != nil || !verifyCookie(l.key, c.Value, &s) || time.Now().Unix() >= s.Exp {
+		return session{}, false
+	}
+	return s, true
+}
+
+// check validates a user and password: the admin account, or the read-only one
+// (viewer=true) when AUTH_VIEWER_USER is set.
+func (l *Local) check(user, pass string) (viewer, ok bool) {
+	if same(user, l.user) && same(pass, l.pass) {
+		return false, true
+	}
+	if l.viewerUser != "" && same(user, l.viewerUser) && same(pass, l.viewerPass) {
+		return true, true
+	}
+	return false, false
+}
+
+// same compares in constant time, so the answer time says nothing of the secret.
+func same(a, b string) bool {
+	return subtle.ConstantTimeCompare([]byte(a), []byte(b)) == 1
+}
+
+// serveLogo sends LOGIN_LOGO; without it the login page hides the image.
+func (l *Local) serveLogo(w http.ResponseWriter, r *http.Request) {
+	if l.logo == "" {
+		http.NotFound(w, r)
+		return
+	}
+	w.Header().Set("Cache-Control", "no-cache")
+	http.ServeFile(w, r, l.logo)
+}
+
+// safeReturn keeps the page to open after login inside logstream.
+func safeReturn(ret string) string {
+	if !strings.HasPrefix(ret, "/") || strings.HasPrefix(ret, "//") || strings.HasPrefix(ret, "/\\") || ret == loginPage {
+		return "/"
+	}
+	return ret
+}
+
+// isHTTPS is true when the browser talks HTTPS, directly or through a reverse proxy.
+func isHTTPS(r *http.Request) bool {
+	return r.TLS != nil || strings.EqualFold(r.Header.Get("X-Forwarded-Proto"), "https")
+}
+
+func clientIP(r *http.Request) string {
+	if f := r.Header.Get("X-Forwarded-For"); f != "" {
+		return strings.TrimSpace(strings.Split(f, ",")[0])
+	}
+	host, _, err := net.SplitHostPort(r.RemoteAddr)
+	if err != nil {
+		return r.RemoteAddr
+	}
+	return host
+}

auth_local_test.go

@@ -0,0 +1,142 @@
+package main
+
+import (
+	"net/http"
+	"net/http/cookiejar"
+	"net/url"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// newLocalApp puts the local login in front of a handler that echoes "app" and returns a
+// browser (client with cookies) that does not follow redirects.
+func newLocalApp(t *testing.T, c authConfig) (string, *http.Client) {
+	t.Helper()
+	loginFailDelay = 0
+	c.mode, c.dataDir = "local", t.TempDir()
+	h, err := newAuth(c, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _, _ = w.Write([]byte("app " + r.URL.Path)) }))
+	if err != nil {
+		t.Fatal(err)
+	}
+	jar, _ := cookiejar.New(nil)
+	return "http://app.test", &http.Client{
+		Jar:           jar,
+		Transport:     hosts{"app.test": h},
+		CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
+	}
+}
+
+func login(t *testing.T, c *http.Client, app, user, pass, ret string) *http.Response {
+	t.Helper()
+	res, err := c.PostForm(app+"/auth/login", url.Values{"user": {user}, "pass": {pass}, "r": {ret}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	res.Body.Close()
+	return res
+}
+
+func TestLocalLoginFlow(t *testing.T) {
+	app, c := newLocalApp(t, authConfig{user: "admin", pass: "pw"})
+
+	res, _ := c.Get(app + "/api/logs?q=x")
+	if res.StatusCode != http.StatusUnauthorized || res.Header.Get("WWW-Authenticate") != "" {
+		t.Fatalf("API without session: %d %q", res.StatusCode, res.Header.Get("WWW-Authenticate"))
+	}
+	res, _ = c.Get(app + "/?q=disk")
+	if loc := res.Header.Get("Location"); res.StatusCode != http.StatusFound || loc != "/login.html?r=%2F%3Fq%3Ddisk" {
+		t.Fatalf("page without session: %d %q", res.StatusCode, loc)
+	}
+	for _, p := range []string{"/login.html", "/style.css", "/healthz"} {
+		if code, body := get(t, c, app+p); code != http.StatusOK || body != "app "+p {
+			t.Errorf("%s without session: %d %q", p, code, body)
+		}
+	}
+
+	res = login(t, c, app, "admin", "wrong", "/?q=disk")
+	if loc := res.Header.Get("Location"); res.StatusCode != http.StatusSeeOther || !strings.Contains(loc, "e=1") {
+		t.Fatalf("wrong password: %d %q", res.StatusCode, loc)
+	}
+	if code, _ := get(t, c, app+"/api/logs"); code != http.StatusUnauthorized {
+		t.Fatal("session created by a wrong password")
+	}
+
+	res = login(t, c, app, "admin", "pw", "//evil.example/")
+	if loc := res.Header.Get("Location"); loc != "/" {
+		t.Fatalf("open redirect: %q", loc)
+	}
+	res = login(t, c, app, "admin", "pw", "/?q=disk")
+	if loc := res.Header.Get("Location"); res.StatusCode != http.StatusSeeOther || loc != "/?q=disk" {
+		t.Fatalf("login: %d %q", res.StatusCode, loc)
+	}
+	if code, body := get(t, c, app+"/api/logs"); code != http.StatusOK || body != "app /api/logs" {
+		t.Fatalf("API with session: %d %q", code, body)
+	}
+	if code, body := get(t, c, app+"/auth/me"); code != http.StatusOK || !strings.Contains(body, `"user":"admin"`) || !strings.Contains(body, `"mode":"local"`) {
+		t.Fatalf("/auth/me: %d %s", code, body)
+	}
+	if res, _ := c.Get(app + "/login.html"); res.StatusCode != http.StatusFound {
+		t.Errorf("login page while logged in: %d", res.StatusCode)
+	}
+
+	if res, _ := c.Get(app + "/auth/logout"); res.Header.Get("Location") != "/login.html" {
+		t.Fatalf("logout: %q", res.Header.Get("Location"))
+	}
+	if code, _ := get(t, c, app+"/api/logs"); code != http.StatusUnauthorized {
+		t.Fatal("session still valid after logout")
+	}
+}
+
+func TestLocalBasicAuthForScripts(t *testing.T) {
+	app, c := newLocalApp(t, authConfig{user: "admin", pass: "pw"})
+	req, _ := http.NewRequest(http.MethodGet, app+"/api/logs", nil)
+	req.SetBasicAuth("admin", "pw")
+	if res, _ := c.Do(req); res.StatusCode != http.StatusOK {
+		t.Fatalf("basic auth: %d", res.StatusCode)
+	}
+	req.SetBasicAuth("admin", "nope")
+	if res, _ := c.Do(req); res.StatusCode != http.StatusUnauthorized {
+		t.Fatalf("wrong basic auth: %d", res.StatusCode)
+	}
+}
+
+func TestLocalPasswordChangeEndsSessions(t *testing.T) {
+	dir := t.TempDir()
+	loginFailDelay = 0
+	echo := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {})
+	h1, _ := newAuth(authConfig{mode: "local", user: "admin", pass: "old", dataDir: dir}, echo)
+	h2, _ := newAuth(authConfig{mode: "local", user: "admin", pass: "new", dataDir: dir}, echo)
+	cookie := signCookie(h1.(*Local).key, session{User: "admin", Exp: 9999999999})
+	r, _ := http.NewRequest(http.MethodGet, "/", nil)
+	r.AddCookie(&http.Cookie{Name: sessionCookie, Value: cookie})
+	if _, ok := h1.(*Local).sessionUser(r); !ok {
+		t.Fatal("session refused with the same password")
+	}
+	if _, ok := h2.(*Local).sessionUser(r); ok {
+		t.Fatal("session kept after a password change")
+	}
+}
+
+func TestLocalLogo(t *testing.T) {
+	app, c := newLocalApp(t, authConfig{user: "admin", pass: "pw"})
+	if code, _ := get(t, c, app+"/auth/logo"); code != http.StatusNotFound {
+		t.Errorf("no LOGIN_LOGO: %d", code)
+	}
+	png := filepath.Join(t.TempDir(), "logo.png")
+	_ = os.WriteFile(png, []byte("\x89PNG\r\n\x1a\nfake"), 0o644)
+	app, c = newLocalApp(t, authConfig{user: "admin", pass: "pw", loginLogo: png})
+	res, _ := c.Get(app + "/auth/logo")
+	if res.StatusCode != http.StatusOK || res.Header.Get("Content-Type") != "image/png" {
+		t.Errorf("LOGIN_LOGO: %d %q", res.StatusCode, res.Header.Get("Content-Type"))
+	}
+}
+
+func TestLocalWithoutUserIsOpen(t *testing.T) {
+	if h, _ := newAuth(authConfig{mode: "local"}, http.NotFoundHandler()); h == nil || isLocal(h) {
+		t.Error("local mode without AUTH_USER should not protect anything")
+	}
+}
+
+func isLocal(h http.Handler) bool { _, ok := h.(*Local); return ok }

auth_test.go

@@ -0,0 +1,230 @@
+package main
+
+import (
+	"crypto"
+	"crypto/ecdsa"
+	"crypto/elliptic"
+	"crypto/rand"
+	"crypto/rsa"
+	"crypto/sha256"
+	"encoding/base64"
+	"encoding/json"
+	"math/big"
+	"net/http"
+	"net/http/cookiejar"
+	"net/http/httptest"
+	"net/url"
+	"strings"
+	"testing"
+	"time"
+)
+
+// hosts routes requests to in-memory handlers (no listening socket needed).
+type hosts map[string]http.Handler
+
+func (h hosts) RoundTrip(r *http.Request) (*http.Response, error) {
+	rec := httptest.NewRecorder()
+	h[r.URL.Host].ServeHTTP(rec, r)
+	res := rec.Result()
+	res.Request = r
+	return res, nil
+}
+
+// fakeIdP is a minimal OpenID provider: it logs in "alice" without asking.
+type fakeIdP struct {
+	mux      *http.ServeMux
+	rsaKey   *rsa.PrivateKey
+	ecKey    *ecdsa.PrivateKey
+	useEC    bool
+	codes    map[string]url.Values // code -> authorize request
+	claims   func(map[string]any)  // last-minute changes to the ID token
+	tokenErr bool
+}
+
+func newFakeIdP(t *testing.T) *fakeIdP {
+	rk, _ := rsa.GenerateKey(rand.Reader, 2048)
+	ek, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	p := &fakeIdP{rsaKey: rk, ecKey: ek, codes: map[string]url.Values{}}
+	mux := http.NewServeMux()
+	p.mux = mux
+	iss := "http://idp.test/realm"
+	mux.HandleFunc("/realm/.well-known/openid-configuration", func(w http.ResponseWriter, r *http.Request) {
+		_ = json.NewEncoder(w).Encode(map[string]any{
+			"issuer": iss, "authorization_endpoint": iss + "/auth", "token_endpoint": iss + "/token",
+			"jwks_uri": iss + "/jwks", "end_session_endpoint": iss + "/logout",
+		})
+	})
+	mux.HandleFunc("/realm/jwks", func(w http.ResponseWriter, r *http.Request) {
+		b := func(i *big.Int) string { return base64.RawURLEncoding.EncodeToString(i.Bytes()) }
+		_ = json.NewEncoder(w).Encode(map[string]any{"keys": []any{
+			map[string]string{"kty": "RSA", "kid": "r1", "use": "sig", "n": b(rk.N), "e": "AQAB"},
+			map[string]string{"kty": "EC", "kid": "e1", "crv": "P-256", "x": b(ek.X), "y": b(ek.Y)},
+		}})
+	})
+	mux.HandleFunc("/realm/auth", func(w http.ResponseWriter, r *http.Request) {
+		q := r.URL.Query()
+		code := randomString()
+		p.codes[code] = q
+		http.Redirect(w, r, q.Get("redirect_uri")+"?code="+code+"&state="+q.Get("state"), http.StatusFound)
+	})
+	mux.HandleFunc("/realm/token", func(w http.ResponseWriter, r *http.Request) {
+		_ = r.ParseForm()
+		id, secret, _ := r.BasicAuth()
+		authz, ok := p.codes[r.Form.Get("code")]
+		sum := sha256.Sum256([]byte(r.Form.Get("code_verifier")))
+		if p.tokenErr || !ok || id != "logstream" || secret != "s3cret" ||
+			base64.RawURLEncoding.EncodeToString(sum[:]) != authz.Get("code_challenge") ||
+			r.Form.Get("redirect_uri") != authz.Get("redirect_uri") {
+			http.Error(w, `{"error":"invalid_grant"}`, http.StatusBadRequest)
+			return
+		}
+		delete(p.codes, r.Form.Get("code"))
+		c := map[string]any{
+			"iss": iss, "aud": "logstream", "sub": "123", "preferred_username": "alice",
+			"exp": time.Now().Add(5 * time.Minute).Unix(), "iat": time.Now().Unix(), "nonce": authz.Get("nonce"),
+		}
+		if p.claims != nil {
+			p.claims(c)
+		}
+		_ = json.NewEncoder(w).Encode(map[string]string{"access_token": "x", "id_token": p.sign(c)})
+	})
+	return p
+}
+
+func (p *fakeIdP) sign(claims map[string]any) string {
+	alg, kid := "RS256", "r1"
+	if p.useEC {
+		alg, kid = "ES256", "e1"
+	}
+	h, _ := json.Marshal(map[string]string{"alg": alg, "kid": kid, "typ": "JWT"})
+	c, _ := json.Marshal(claims)
+	in := base64.RawURLEncoding.EncodeToString(h) + "." + base64.RawURLEncoding.EncodeToString(c)
+	d := sha256.Sum256([]byte(in))
+	var sig []byte
+	if p.useEC {
+		r, s, _ := ecdsa.Sign(rand.Reader, p.ecKey, d[:])
+		sig = make([]byte, 64)
+		r.FillBytes(sig[:32])
+		s.FillBytes(sig[32:])
+	} else {
+		sig, _ = rsa.SignPKCS1v15(rand.Reader, p.rsaKey, crypto.SHA256, d[:])
+	}
+	return in + "." + base64.RawURLEncoding.EncodeToString(sig)
+}
+
+// newOIDCApp puts logstream's auth in front of a handler that echoes "app" and returns
+// a browser (client with cookies) that reaches both the app and the provider.
+func newOIDCApp(t *testing.T, idp *fakeIdP) (string, *http.Client) {
+	h, err := newAuth(authConfig{
+		mode: "oidc", issuer: "http://idp.test/realm", clientID: "logstream", clientSecret: "s3cret",
+		redirectURL: "http://app.test/auth/callback", dataDir: t.TempDir(),
+	}, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _, _ = w.Write([]byte("app " + r.URL.Path)) }))
+	if err != nil {
+		t.Fatal(err)
+	}
+	net := hosts{"app.test": h, "idp.test": idp.mux}
+	h.(*OIDC).client.Transport = net
+	jar, _ := cookiejar.New(nil)
+	return "http://app.test", &http.Client{Jar: jar, Transport: net}
+}
+
+func get(t *testing.T, c *http.Client, u string) (int, string) {
+	t.Helper()
+	res, err := c.Get(u)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer res.Body.Close()
+	var b strings.Builder
+	buf := make([]byte, 4096)
+	for {
+		n, err := res.Body.Read(buf)
+		b.Write(buf[:n])
+		if err != nil {
+			break
+		}
+	}
+	return res.StatusCode, b.String()
+}
+
+func TestOIDCLoginFlow(t *testing.T) {
+	for _, ec := range []bool{false, true} {
+		idp := newFakeIdP(t)
+		idp.useEC = ec
+		app, c := newOIDCApp(t, idp)
+
+		if code, _ := get(t, c, app+"/api/logs"); code != http.StatusUnauthorized {
+			t.Fatalf("api without session: %d", code)
+		}
+		if code, body := get(t, c, app+"/healthz"); code != 200 || body != "app /healthz" {
+			t.Fatalf("healthz: %d %q", code, body)
+		}
+		// A page goes through the provider and comes back to the page asked for.
+		if code, body := get(t, c, app+"/index.html?x=1"); code != 200 || body != "app /index.html" {
+			t.Fatalf("login (ec=%v): %d %q", ec, code, body)
+		}
+		if code, body := get(t, c, app+"/api/logs"); code != 200 || body != "app /api/logs" {
+			t.Fatalf("api with session: %d %q", code, body)
+		}
+		if code, body := get(t, c, app+"/auth/me"); code != 200 || !strings.Contains(body, `"user":"alice"`) {
+			t.Fatalf("me: %d %q", code, body)
+		}
+		// Logout drops the session (the fake provider has no logout page: 404).
+		get(t, c, app+"/auth/logout")
+		if code, _ := get(t, c, app+"/api/logs"); code != http.StatusUnauthorized {
+			t.Fatalf("api after logout: %d", code)
+		}
+	}
+}
+
+func TestOIDCRejectsBadTokens(t *testing.T) {
+	cases := map[string]func(map[string]any){
+		"wrong nonce":    func(c map[string]any) { c["nonce"] = "x" },
+		"wrong audience": func(c map[string]any) { c["aud"] = "other" },
+		"wrong issuer":   func(c map[string]any) { c["iss"] = "https://evil" },
+		"expired":        func(c map[string]any) { c["exp"] = time.Now().Add(-time.Hour).Unix() },
+	}
+	for name, change := range cases {
+		idp := newFakeIdP(t)
+		idp.claims = change
+		app, c := newOIDCApp(t, idp)
+		if code, _ := get(t, c, app+"/"); code != http.StatusForbidden {
+			t.Errorf("%s: login gave %d, expected 403", name, code)
+		}
+		if code, _ := get(t, c, app+"/api/logs"); code != http.StatusUnauthorized {
+			t.Errorf("%s: session created", name)
+		}
+	}
+}
+
+func TestOIDCCallbackNeedsLoginCookie(t *testing.T) {
+	idp := newFakeIdP(t)
+	app, c := newOIDCApp(t, idp)
+	if code, _ := get(t, c, app+"/auth/callback?code=abc&state=forged"); code != http.StatusBadRequest {
+		t.Fatalf("forged callback: %d", code)
+	}
+}
+
+func TestOIDCForgedSessionCookie(t *testing.T) {
+	idp := newFakeIdP(t)
+	app, c := newOIDCApp(t, idp)
+	u, _ := url.Parse(app)
+	payload := base64.RawURLEncoding.EncodeToString([]byte(`{"u":"mallory","e":9999999999}`))
+	c.Jar.SetCookies(u, []*http.Cookie{{Name: sessionCookie, Value: payload + ".AAAA"}})
+	if code, _ := get(t, c, app+"/api/logs"); code != http.StatusUnauthorized {
+		t.Fatalf("forged session accepted: %d", code)
+	}
+}
+
+func TestAuthModeConfig(t *testing.T) {
+	next := http.NotFoundHandler()
+	if _, err := newAuth(authConfig{mode: "oidc"}, next); err == nil || !strings.Contains(err.Error(), "OIDC_CLIENT_ID") {
+		t.Errorf("missing variables not reported: %v", err)
+	}
+	if _, err := newAuth(authConfig{mode: "ldap"}, next); err == nil {
+		t.Error("unknown mode accepted")
+	}
+	if h, err := newAuth(authConfig{mode: "local"}, next); err != nil || h == nil {
+		t.Errorf("local mode: %v", err)
+	}
+}

dbstats.go

@@ -0,0 +1,167 @@
+package main
+
+import (
+	"bufio"
+	"context"
+	"fmt"
+	"io"
+	"net/http"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+)
+
+// DBStats describes what VictoriaLogs holds (Settings > Data). Sizes are in
+// bytes; a field VictoriaLogs did not give stays at its zero value.
+type DBStats struct {
+	Rows       int64  `json:"rows"`       // stored log lines
+	DiskBytes  int64  `json:"diskBytes"`  // compressed data + index, on disk
+	IndexBytes int64  `json:"indexBytes"` // index part of DiskBytes
+	RawBytes   int64  `json:"rawBytes"`   // data before compression
+	FreeBytes  int64  `json:"freeBytes"`  // free space on the VictoriaLogs volume
+	Days       int64  `json:"days"`       // per-day partitions
+	Retention  string `json:"retention"`  // -retentionPeriod, e.g. "30d"
+	Oldest     string `json:"oldest,omitempty"`
+	Newest     string `json:"newest,omitempty"`
+	Last1h     int64  `json:"last1h"`
+	Last24h    int64  `json:"last24h"`
+	Hosts      int64  `json:"hosts"`
+	Apps       int64  `json:"apps"`
+	QueryError string `json:"queryError,omitempty"` // the LogsQL part failed, the metrics are still valid
+	Updated    string `json:"updated"`
+}
+
+// dbStatsCache keeps the last answer for a while: the LogsQL part reads the
+// whole base, which is too heavy to repeat each time the tab is opened.
+type dbStatsCache struct {
+	mu   sync.Mutex
+	at   time.Time
+	last *DBStats
+}
+
+const dbStatsTTL = 30 * time.Second
+
+func (c *dbStatsCache) get(ctx context.Context, s *Store, force bool) (*DBStats, error) {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	if c.last != nil && !force && time.Since(c.at) < dbStatsTTL {
+		return c.last, nil
+	}
+	st, err := s.DBStats(ctx)
+	if err != nil {
+		return nil, err
+	}
+	c.last, c.at = st, time.Now()
+	return st, nil
+}
+
+// DBStats reads the storage metrics of VictoriaLogs (/metrics), then counts
+// with LogsQL what the metrics do not tell (period, recent volume, sources).
+func (s *Store) DBStats(ctx context.Context) (*DBStats, error) {
+	st, err := s.storageMetrics(ctx)
+	if err != nil {
+		return nil, err
+	}
+	st.Updated = time.Now().UTC().Format(time.RFC3339)
+	if st.Rows == 0 {
+		return st, nil
+	}
+
+	qctx, cancel := context.WithTimeout(ctx, 20*time.Second)
+	defer cancel()
+	rows, err := s.Query(qctx, "* | stats min(_time) oldest, max(_time) newest, count_uniq(host) hosts, count_uniq(app) apps")
+	if err == nil && len(rows) > 0 {
+		r := rows[0]
+		st.Oldest, _ = r["oldest"].(string)
+		st.Newest, _ = r["newest"].(string)
+		st.Hosts, st.Apps = toInt(r["hosts"]), toInt(r["apps"])
+		rows, err = s.Query(qctx, "_time:24h | stats count() last24h, count() if (_time:1h) last1h")
+		if err == nil && len(rows) > 0 {
+			st.Last24h, st.Last1h = toInt(rows[0]["last24h"]), toInt(rows[0]["last1h"])
+		}
+	}
+	if err != nil {
+		st.QueryError = err.Error()
+	}
+	return st, nil
+}
+
+func (s *Store) storageMetrics(ctx context.Context) (*DBStats, error) {
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, s.base+"/metrics", nil)
+	if err != nil {
+		return nil, err
+	}
+	resp, err := s.client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != http.StatusOK {
+		msg, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
+		return nil, fmt.Errorf("VictoriaLogs /metrics: HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(msg)))
+	}
+	return parseStorageMetrics(resp.Body)
+}
+
+// parseStorageMetrics picks the storage figures out of the Prometheus text
+// format of VictoriaLogs' /metrics (app/vlstorage).
+func parseStorageMetrics(r io.Reader) (*DBStats, error) {
+	st := &DBStats{}
+	sc := bufio.NewScanner(r)
+	sc.Buffer(make([]byte, 64*1024), 1<<20)
+	for sc.Scan() {
+		line := sc.Text()
+		if line == "" || line[0] == '#' {
+			continue
+		}
+		name, labels, value := splitMetric(line)
+		switch name {
+		case "vl_storage_rows":
+			st.Rows += int64(value)
+		case "vl_data_size_bytes":
+			st.DiskBytes += int64(value)
+			if labels["type"] == "indexdb" {
+				st.IndexBytes += int64(value)
+			}
+		case "vl_uncompressed_data_size_bytes":
+			st.RawBytes += int64(value)
+		case "vl_free_disk_space_bytes":
+			st.FreeBytes = int64(value)
+		case "vl_partitions":
+			st.Days = int64(value)
+		case "flag":
+			if labels["name"] == "retentionPeriod" {
+				st.Retention = labels["value"]
+			}
+		}
+	}
+	return st, sc.Err()
+}
+
+// splitMetric splits `name{a="x",b="y"} 12` into its parts. Label values with
+// escaped quotes are not expected in the metrics read here.
+func splitMetric(line string) (string, map[string]string, float64) {
+	sp := strings.LastIndexByte(line, ' ')
+	if sp < 0 {
+		return "", nil, 0
+	}
+	value, _ := strconv.ParseFloat(line[sp+1:], 64)
+	head := line[:sp]
+	name, rest, ok := strings.Cut(head, "{")
+	if !ok {
+		return head, nil, value
+	}
+	labels := map[string]string{}
+	rest = strings.TrimSuffix(rest, "}")
+	for rest != "" {
+		k, v, ok := strings.Cut(rest, `="`)
+		if !ok {
+			break
+		}
+		val, after, _ := strings.Cut(v, `"`)
+		labels[strings.TrimSpace(k)] = val
+		rest = strings.TrimPrefix(after, ",")
+	}
+	return name, labels, value
+}

dbstats_test.go

@@ -0,0 +1,89 @@
+package main
+
+import (
+	"encoding/json"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+)
+
+// Excerpt of VictoriaLogs v1.52 /metrics.
+const vlMetrics = `# a comment
+vl_free_disk_space_bytes{path="/vlogs"} 52428800000
+vl_storage_rows{type="storage/inmemory"} 120
+vl_storage_rows{type="storage/small"} 3000
+vl_storage_rows{type="storage/big"} 1000000
+vl_partitions 12
+vl_data_size_bytes{type="indexdb"} 2048
+vl_data_size_bytes{type="storage"} 1048576
+vl_compressed_data_size_bytes{type="storage/big"} 1000000
+vl_uncompressed_data_size_bytes{type="storage/inmemory"} 4096
+vl_uncompressed_data_size_bytes{type="storage/small"} 100000
+vl_uncompressed_data_size_bytes{type="storage/big"} 20000000
+flag{name="retentionPeriod", value="30d", is_set="true"} 1
+flag{name="httpListenAddr", value=":9428", is_set="true"} 1
+`
+
+func TestParseStorageMetrics(t *testing.T) {
+	st, err := parseStorageMetrics(strings.NewReader(vlMetrics))
+	if err != nil {
+		t.Fatal(err)
+	}
+	want := DBStats{Rows: 1003120, DiskBytes: 1050624, IndexBytes: 2048, RawBytes: 20104096, FreeBytes: 52428800000, Days: 12, Retention: "30d"}
+	if *st != want {
+		t.Fatalf("got %+v\nwant %+v", *st, want)
+	}
+}
+
+// statsVL answers /metrics and the LogsQL queries of DBStats.
+type statsVL struct{ queries int }
+
+func (v *statsVL) RoundTrip(r *http.Request) (*http.Response, error) {
+	body := vlMetrics
+	if r.URL.Path == "/select/logsql/query" {
+		v.queries++
+		_ = r.ParseForm()
+		if strings.HasPrefix(r.PostForm.Get("query"), "_time:24h") {
+			body = `{"last24h":"5000","last1h":"300"}` + "\n"
+		} else {
+			body = `{"oldest":"2026-09-21T08:00:00Z","newest":"2026-10-03T15:00:00Z","hosts":"4","apps":"17"}` + "\n"
+		}
+	}
+	return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(body)), Header: http.Header{}}, nil
+}
+
+func TestDBStatsHandler(t *testing.T) {
+	vl := &statsVL{}
+	store := NewStore("http://vl", 10, 10, time.Second, nil)
+	store.client.Transport = vl
+	store.streamClient.Transport = vl
+	a := &API{store: store}
+
+	get := func(url string) DBStats {
+		rec := httptest.NewRecorder()
+		a.dbStats(rec, httptest.NewRequest("GET", url, nil))
+		if rec.Code != http.StatusOK {
+			t.Fatalf("status %d: %s", rec.Code, rec.Body)
+		}
+		var st DBStats
+		if err := json.Unmarshal(rec.Body.Bytes(), &st); err != nil {
+			t.Fatal(err)
+		}
+		return st
+	}
+	st := get("/api/dbstats")
+	if st.Rows != 1003120 || st.Oldest != "2026-09-21T08:00:00Z" || st.Hosts != 4 || st.Apps != 17 || st.Last24h != 5000 || st.Last1h != 300 || st.QueryError != "" {
+		t.Fatalf("unexpected stats: %+v", st)
+	}
+	get("/api/dbstats")
+	if vl.queries != 2 {
+		t.Fatalf("second call within the TTL should be cached, got %d queries", vl.queries)
+	}
+	get("/api/dbstats?refresh=1")
+	if vl.queries != 4 {
+		t.Fatalf("refresh=1 should query again, got %d queries", vl.queries)
+	}
+}

docker-compose.yml

@@ -12,24 +12,46 @@ services:
       - "${HTTP_PORT:-8080}:8080"
     environment:
       VLOGS_URL: http://victorialogs:9428
-      SYSLOG_PUBLIC_PORT: ${SYSLOG_PORT:-514}     # shown in Settings > Sources (the mapping is set above)
+      SYSLOG_PUBLIC_PORT: ${SYSLOG_PORT:-514}     # le port d'ecoute syslog par defaut (attention aux ports <1024)
       TZ: ${TZ:-Europe/Paris}
-      AUTH_USER: ${AUTH_USER:-}   # leave empty to disable authentication
+      AUTH_MODE: ${AUTH_MODE:-local}   # local (page de connexion, compte ci-dessous) ou oidc
+      AUTH_USER: ${AUTH_USER:-}   # vide = pas d'authentification, on delegue ca au reverse proxy traefik
       AUTH_PASS: ${AUTH_PASS:-}
-      RDNS: ${RDNS:-on}               # replace IP hosts with their DNS name (PTR)
+      AUTH_VIEWER_USER: ${AUTH_VIEWER_USER:-}   # compte en lecture seule (optionnel)
-      DNS_SERVER: ${DNS_SERVER:-}     # e.g. 192.168.1.1 to query your LAN DNS; empty = system resolver
+      AUTH_VIEWER_PASS: ${AUTH_VIEWER_PASS:-}
-      ALLOW_PURGE: ${ALLOW_PURGE:-true}
+      LOGIN_LOGO: ${LOGIN_LOGO:-}     # PNG affiche sur la page de connexion (chemin dans le conteneur, voir volumes)
+      PRESETS_FILE: ${PRESETS_FILE:-} # prereglages de tags (defaut /data/presets.json, voir docs/presets.fr.md)
+      OIDC_ISSUER: ${OIDC_ISSUER:-}
+      OIDC_CLIENT_ID: ${OIDC_CLIENT_ID:-}
+      OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-}
+      OIDC_REDIRECT_URL: ${OIDC_REDIRECT_URL:-}
+      OIDC_SCOPES: ${OIDC_SCOPES:-openid profile email}
+      OIDC_ADMIN_GROUP: ${OIDC_ADMIN_GROUP:-}     # vide = tout le monde est admin, sinon les autres sont en lecture seule
+      OIDC_GROUPS_CLAIM: ${OIDC_GROUPS_CLAIM:-groups}
+      SESSION_TTL: ${SESSION_TTL:-${OIDC_SESSION_TTL:-12h}}   # duree de la session (local et oidc)
+      RDNS: ${RDNS:-on}               # resol dns
+      DNS_SERVER: ${DNS_SERVER:-}     # si resolv directe
+      ALLOW_PURGE: ${ALLOW_PURGE:-false}   # true pour autoriser « Supprimer tous les logs »
       EXPORT_MAX: ${EXPORT_MAX:-100000}
-      DOCKER_LOGS: ${DOCKER_LOGS:-on}             # collect the logs of this machine's containers
+      DOCKER_LOGS: ${DOCKER_LOGS:-on}             # collecte des logs des conteneurs Docker
-      DOCKER_HOST: tcp://docker-proxy:2375        # read-only Docker API gateway (below)
+      DOCKER_HOST: tcp://docker-proxy:2375        # lecture seul de l'API Docker
-      DOCKER_BACKFILL: ${DOCKER_BACKFILL:-1h}     # history read from a container seen for the first time
+      DOCKER_BACKFILL: ${DOCKER_BACKFILL:-1h}
+      HOST_LOGS_BACKFILL: ${HOST_LOGS_BACKFILL:-1h}   # historique lu a l'activation des logs systeme de l'hote
+    group_add:
+      - "${HOST_LOGS_GID:-4}"        # groupe autorise a lire les logs de l'hote (4 = adm sur Debian/Ubuntu)
     volumes:
-      - logstream-data:/data         # tags.json, docker.json (container choices)
+      - logstream-data:/data         # tags.json, docker.json (les choix des conteneurs)
+      # logs systeme de l'hote, en lecture seule (source a activer dans Reglages > Sources)
+      - /var/log:/host/var/log:ro                     # journal systemd persistant et fichiers texte
+      - /run/log/journal:/host/run/log/journal:ro     # journal systemd volatile
+      # logo de la page de connexion, avec LOGIN_LOGO=/config/logo.png dans .env
+      # - ./logo.png:/config/logo.png:ro
+      # prereglages de tags personnalises, avec PRESETS_FILE=/config/presets.json dans .env
+      # - ./presets.json:/config/presets.json:ro
     labels:
-      logstream.exclude: "true"      # never collect Logstream's own logs
+      logstream.exclude: "true"      # pas de collect des logs logstream
 
   victorialogs:
-    # Pinned version: see "Updating" in the README before changing it
     image: victoriametrics/victoria-logs:v1.52.0
     container_name: logstream-victorialogs
     restart: unless-stopped
@@ -37,7 +59,7 @@ services:
       - -storageDataPath=/vlogs
       - -retentionPeriod=${RETENTION:-30d}
       - -httpListenAddr=:9428
-      - -delete.enable                # required by "Delete all logs" in Settings
+      - -delete.enable                # obliger pour autoriser la purge de la base via l'interface
     volumes:
       - vlogs-data:/vlogs
     ports:
@@ -45,9 +67,7 @@ services:
       - "127.0.0.1:9428:9428"
 
   docker-proxy:
-    # Read-only gateway to the Docker API: Logstream can only list containers,
+    # Docker proxy pour eviter de solliciter directement l'API docker
-    # read their logs, receive events and engine info. Anything else (start,
-    # stop, exec, images, volumes…) is refused.
     image: tecnativa/docker-socket-proxy:v0.5.0
     container_name: logstream-docker-proxy
     restart: unless-stopped
@@ -59,7 +79,7 @@ services:
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock:ro
     labels:
-      logstream.exclude: "true"      # its access log would only echo Logstream's own requests
+      logstream.exclude: "true"      # pour exclure les logs propres de logstream
 
 volumes:
   logstream-data:

docker.go

@@ -62,7 +62,7 @@ type DockerManager struct {
 	hostName   string
 	containers []DockerContainer
 	followers  map[string]*follower // container ID -> running follower
-	checkpoint map[string]time.Time // container ID -> timestamp of the last line read
+	checkpoint map[string]time.Time // container ID -> timestamp of the last line stored
 	dirty      bool
 	connected  bool
 	lastErr    string
@@ -389,16 +389,19 @@ func (m *DockerManager) follow(ctx context.Context, c DockerContainer) {
 	}
 }
 
-func sleepCtx(ctx context.Context, d time.Duration) {
+// sleepCtx waits for d, or returns false if the context ends first.
+func sleepCtx(ctx context.Context, d time.Duration) bool {
 	t := time.NewTimer(d)
 	defer t.Stop()
 	select {
 	case <-ctx.Done():
+		return false
 	case <-t.C:
+		return true
 	}
 }
 
-// since returns where to resume reading: just after the last line read, or
+// since returns where to resume reading: just after the last line stored, or
 // DOCKER_BACKFILL ago for a container seen for the first time.
 func (m *DockerManager) since(id string) time.Time {
 	m.mu.Lock()
@@ -530,8 +533,6 @@ func (m *DockerManager) emit(c DockerContainer, stream string, line []byte) {
 	} else if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
 		ts, s = t, ""
 	}
-	defer m.setCheckpoint(c.ID, ts)
-
 	s = ansiRe.ReplaceAllString(s, "")
 	if !utf8.ValidString(s) {
 		s = strings.ToValidUTF8(s, string(utf8.RuneError))
@@ -575,6 +576,10 @@ func (m *DockerManager) emit(c DockerContainer, stream string, line []byte) {
 			"compose_service": c.Service,
 			"stream":          stream,
 		},
+		// Docker keeps the logs: wait for room in the queue rather than drop the
+		// line, and resume after it only once it is stored.
+		Wait: true,
+		Done: func() { m.setCheckpoint(c.ID, ts) },
 	})
 }
 

docs/presets.fr.md

@@ -0,0 +1,114 @@
+[English](presets.md) · **Français**
+
+# Préréglages de tags de couleur
+
+Dans **Paramètres › Filtres**, le menu **+ Préréglage…** ajoute d'un clic un groupe de tags de
+couleur tout faits. Les tags ajoutés sont des tags ordinaires : vous pouvez changer leur couleur,
+leur motif ou leurs options, ou les supprimer. Un tag dont le motif est déjà dans la liste n'est
+pas ajouté en double.
+
+La liste vient d'un fichier texte, [`presets.json`](../presets.json), intégré à LogStream. Vous
+pouvez le remplacer par votre propre fichier (voir [Utiliser votre propre fichier](#utiliser-votre-propre-fichier)).
+
+## Préréglages intégrés
+
+### HTTP/HTTPS
+
+Ces préréglages lisent les logs d'accès de nginx et Apache (formats common et combined), Traefik
+(CLF et JSON), Caddy (JSON) et HAProxy (`option httplog`).
+
+| Préréglage | Tags | Ce qui est coloré |
+| --- | --- | --- |
+| Codes HTTP | `HTTP 2xx` vert, `HTTP 3xx` bleu, `HTTP 4xx` orange, `HTTP 5xx` rouge | seulement le code de statut, par exemple `404` dans `"GET /x HTTP/1.1" 404 153`, `"status":404` ou `"DownstreamStatus":404`. Les autres nombres de la ligne (taille, chemin) ne sont pas touchés. |
+| Méthodes HTTP | `GET/HEAD/OPTIONS` gris, `POST/PUT/PATCH` violet, `DELETE` rose | seulement la méthode dans `"GET /chemin` ou `"method":"GET"` (en majuscules uniquement) |
+| Sondes et attaques | `sondes / attaques` | `wp-login.php`, `xmlrpc.php`, `wp-admin`, `phpmyadmin`, `/.env`, `/.git`, `/.aws`, `/cgi-bin/`, `../`, `%2e%2e`, `/etc/passwd`, `<script`, `union select` |
+| Robots et scripts | `robots / scripts` | les mots finissant par `bot` (`Googlebot`, `bingbot`…), `crawler`, `spider`, `curl`, `wget`, `python-requests`, `Go-http-client`, `zgrab`, `masscan`, `nmap`, `sqlmap`, `nikto` |
+| Erreurs TLS/HTTPS et proxy | `erreurs TLS`, `erreurs proxy` | échecs de handshake TLS, certificats expirés ou refusés, `x509:` ; `upstream timed out`, `no live upstreams`, `connect() failed`, `connection refused`, `bad gateway`, `gateway timeout`, `service unavailable` |
+
+### Système
+
+| Préréglage | Tags | Ce qui est coloré |
+| --- | --- | --- |
+| SSH et connexions | `échecs de connexion` rouge, `connexions` vert | sshd/PAM : `Failed password`, `Invalid user`, `authentication failure`, `incorrect password attempts`, `NOT in sudoers`… ; `Accepted publickey`, `session opened for user`, `New session … of user` |
+| Commandes sudo | `commandes sudo` | la commande lancée, par exemple `COMMAND=/usr/bin/apt` |
+| Noyau : OOM, plantages, disques | `mémoire épuisée`, `erreurs noyau` | `Out of memory`, `oom-killer`, `Killed process 4242` ; `Kernel panic`, `BUG:`, `Oops`, `Call Trace`, `segfault at`, `I/O error`, `EXT4-fs error`, `blocked for more than 120 seconds`, `soft lockup` |
+| Services systemd | `services en échec` rouge, `démarrage/arrêt de service` vert | `Failed to start`, `Failed with result`, `Main process exited, code=killed`, `Start request repeated too quickly` ; `Started`, `Stopping`, `Reloaded`, `Reached target` |
+| Pare-feu et fail2ban | `pare-feu` | `[UFW BLOCK]`, `[UFW ALLOW]`, `DROP`, `REJECT`, `Ban 203.0.113.9`, `Unban …`, `Found …` |
+
+### Applications
+
+| Préréglage | Tags | Ce qui est coloré |
+| --- | --- | --- |
+| Docker et conteneurs | `problèmes de conteneur` | `exited with code 137` (codes non nuls seulement), `OOMKilled`, `unhealthy`, `Back-off restarting`, `CrashLoopBackOff`, `container die/kill/oom` |
+| Bases de données | `erreurs base de données` | PostgreSQL et MySQL/MariaDB : `deadlock detected`, `duplicate key`, `too many connections`, `lock wait timeout`, `slow query`, `server has gone away`, `Access denied for user`, `password authentication failed`… |
+
+### Général
+
+| Préréglage | Tags | Ce qui est coloré |
+| --- | --- | --- |
+| Niveaux de log | `fatal / critique` rouge, `info / notice` bleu, `debug / trace` gris, `ok` vert | ces mots en mots entiers, quelle que soit la casse (les tags par défaut `warning` et `error` couvrent le reste) |
+| Adresses IPv4 | `adresses IPv4` | `192.168.1.20`, `203.0.113.9`… Les numéros de version à quatre parties comme `1.2.3.4` sont aussi colorés. |
+
+Quand des tags se chevauchent, celui placé le plus haut dans la liste l'emporte : les
+préréglages ajoutés après les tags par défaut ne les masquent donc jamais.
+
+## Utiliser votre propre fichier
+
+LogStream lit le fichier indiqué par `PRESETS_FILE`, `/data/presets.json` par défaut (dans le
+volume `logstream-data`). S'il n'existe pas, la liste intégrée est utilisée. Le fichier est relu
+à chaque ouverture des Paramètres : pas besoin de redémarrer après une modification.
+
+Avec docker-compose, le plus simple est de garder le fichier à côté de `docker-compose.yml` :
+
+1. Copiez [`presets.json`](../presets.json) depuis ce dépôt et modifiez-le.
+2. Dans `docker-compose.yml`, décommentez la ligne `- ./presets.json:/config/presets.json:ro`.
+3. Dans `.env`, mettez `PRESETS_FILE=/config/presets.json`, puis lancez `docker compose up -d`.
+
+Si le fichier est invalide (erreur JSON, expression régulière ou couleur incorrecte, id en
+double), les Paramètres affichent l'erreur et la liste intégrée est utilisée jusqu'à correction.
+
+## Format du fichier
+
+Le fichier est une liste JSON de groupes. Chaque groupe a un nom et une liste de préréglages ;
+chaque préréglage a un `id`, un nom et ses tags.
+
+```json
+[
+  {
+    "group": { "en": "My apps", "fr": "Mes applis" },
+    "presets": [
+      {
+        "id": "monappli",
+        "name": "Mon appli",
+        "tags": [
+          { "label": "paiement refusé", "color": "#fca5a5", "pattern": "payment (?:failed|refused)" },
+          { "label": "commande", "color": "#86efac", "pattern": "order #\\d+" },
+          { "label": "lent", "color": "#fde68a", "pattern": "SLOW", "regex": false, "caseSensitive": true }
+        ]
+      }
+    ]
+  }
+]
+```
+
+| Champ | Obligatoire | Signification |
+| --- | --- | --- |
+| `group` | oui | nom du groupe dans le menu |
+| `id` | oui | identifiant unique du préréglage |
+| `name` | oui | nom du préréglage dans le menu |
+| `tags[].pattern` | oui | ce qu'il faut colorer : une expression régulière, ou du texte simple avec `"regex": false` |
+| `tags[].color` | oui | couleur de fond, `#rrggbb` |
+| `tags[].label` | non | nom affiché dans la liste des tags à la place du motif |
+| `tags[].regex` | non | `true` par défaut |
+| `tags[].wholeWord` | non | mots entiers seulement, `false` par défaut |
+| `tags[].caseSensitive` | non | respecter la casse, `false` par défaut |
+
+Les noms et libellés sont soit un seul texte pour toutes les langues (`"Mon appli"`), soit un
+texte par langue (`{ "en": "My app", "fr": "Mon appli" }`) ; une langue absente se rabat sur
+l'anglais.
+
+Les expressions régulières doivent fonctionner à la fois dans le navigateur (JavaScript) et en
+Go, qui les vérifie : évitez les assertions arrière `(?<=…)`, avant `(?=…)` et les références
+arrière `\1`. En JSON, chaque barre oblique inverse s'écrit deux fois : `\d` devient `"\\d"`. Un
+groupe nommé `hl`, `(?<hl>…)`, ne colore que cette partie de la correspondance, comme le font
+les préréglages HTTP avec `(?<hl>5\\d\\d)`.

docs/presets.md

@@ -0,0 +1,111 @@
+**English** · [Français](presets.fr.md)
+
+# Color tag presets
+
+In **Settings › Filters**, the **+ Preset…** menu adds a group of ready-made color tags in one
+click. Added tags are ordinary tags: you can change their color, pattern or options, or delete
+them. A tag whose pattern is already in the list is not added twice.
+
+The list comes from a text file, [`presets.json`](../presets.json), built into LogStream. You
+can replace it with your own file (see [Using your own file](#using-your-own-file)).
+
+## Built-in presets
+
+### HTTP/HTTPS
+
+These presets read access logs from nginx and Apache (common and combined formats), Traefik
+(CLF and JSON), Caddy (JSON) and HAProxy (`option httplog`).
+
+| Preset | Tags | What gets colored |
+| --- | --- | --- |
+| HTTP status codes | `HTTP 2xx` green, `HTTP 3xx` blue, `HTTP 4xx` orange, `HTTP 5xx` red | only the status code, e.g. `404` in `"GET /x HTTP/1.1" 404 153`, `"status":404` or `"DownstreamStatus":404`. Other numbers on the line (size, path) are left alone. |
+| HTTP methods | `GET/HEAD/OPTIONS` grey, `POST/PUT/PATCH` purple, `DELETE` pink | only the method in `"GET /path` or `"method":"GET"` (upper case only) |
+| Probes and attacks | `probes / attacks` | `wp-login.php`, `xmlrpc.php`, `wp-admin`, `phpmyadmin`, `/.env`, `/.git`, `/.aws`, `/cgi-bin/`, `../`, `%2e%2e`, `/etc/passwd`, `<script`, `union select` |
+| Bots and scripts | `bots / scripts` | words ending in `bot` (`Googlebot`, `bingbot`…), `crawler`, `spider`, `curl`, `wget`, `python-requests`, `Go-http-client`, `zgrab`, `masscan`, `nmap`, `sqlmap`, `nikto` |
+| TLS/HTTPS and proxy errors | `TLS errors`, `proxy errors` | TLS handshake failures, expired or rejected certificates, `x509:`; `upstream timed out`, `no live upstreams`, `connect() failed`, `connection refused`, `bad gateway`, `gateway timeout`, `service unavailable` |
+
+### System
+
+| Preset | Tags | What gets colored |
+| --- | --- | --- |
+| SSH and logins | `login failures` red, `logins` green | sshd/PAM: `Failed password`, `Invalid user`, `authentication failure`, `incorrect password attempts`, `NOT in sudoers`…; `Accepted publickey`, `session opened for user`, `New session … of user` |
+| sudo commands | `sudo commands` | the command run, e.g. `COMMAND=/usr/bin/apt` |
+| Kernel: OOM, crashes, disks | `out of memory`, `kernel errors` | `Out of memory`, `oom-killer`, `Killed process 4242`; `Kernel panic`, `BUG:`, `Oops`, `Call Trace`, `segfault at`, `I/O error`, `EXT4-fs error`, `blocked for more than 120 seconds`, `soft lockup` |
+| systemd services | `failed services` red, `service start/stop` green | `Failed to start`, `Failed with result`, `Main process exited, code=killed`, `Start request repeated too quickly`; `Started`, `Stopping`, `Reloaded`, `Reached target` |
+| Firewall and fail2ban | `firewall` | `[UFW BLOCK]`, `[UFW ALLOW]`, `DROP`, `REJECT`, `Ban 203.0.113.9`, `Unban …`, `Found …` |
+
+### Applications
+
+| Preset | Tags | What gets colored |
+| --- | --- | --- |
+| Docker and containers | `container problems` | `exited with code 137` (non-zero codes only), `OOMKilled`, `unhealthy`, `Back-off restarting`, `CrashLoopBackOff`, `container die/kill/oom` |
+| Databases | `database errors` | PostgreSQL and MySQL/MariaDB: `deadlock detected`, `duplicate key`, `too many connections`, `lock wait timeout`, `slow query`, `server has gone away`, `Access denied for user`, `password authentication failed`… |
+
+### General
+
+| Preset | Tags | What gets colored |
+| --- | --- | --- |
+| Log levels | `fatal / critical` red, `info / notice` blue, `debug / trace` grey, `ok` green | these words as whole words, any case (the default `warning` and `error` tags cover the rest) |
+| IPv4 addresses | `IPv4 addresses` | `192.168.1.20`, `203.0.113.9`… Four-part version numbers such as `1.2.3.4` are colored too. |
+
+When tags overlap, the one highest in the tag list wins, so presets added after the default
+tags never hide them.
+
+## Using your own file
+
+LogStream reads the file named by `PRESETS_FILE`, `/data/presets.json` by default (in the
+`logstream-data` volume). When the file does not exist, the built-in list is used. The file is
+read again each time Settings is opened: no restart is needed after an edit.
+
+With docker-compose, the simplest is to keep the file next to `docker-compose.yml`:
+
+1. Copy [`presets.json`](../presets.json) from this repository and edit it.
+2. In `docker-compose.yml`, uncomment the line `- ./presets.json:/config/presets.json:ro`.
+3. In `.env`, set `PRESETS_FILE=/config/presets.json`, then run `docker compose up -d`.
+
+If the file is invalid (JSON error, bad regular expression or color, duplicate id), Settings
+shows the error and the built-in list is used until the file is fixed.
+
+## File format
+
+The file is a JSON list of groups. Each group has a name and a list of presets; each preset has
+an `id`, a name and its tags.
+
+```json
+[
+  {
+    "group": { "en": "My apps", "fr": "Mes applis" },
+    "presets": [
+      {
+        "id": "myapp",
+        "name": "My app",
+        "tags": [
+          { "label": "payment failed", "color": "#fca5a5", "pattern": "payment (?:failed|refused)" },
+          { "label": "order", "color": "#86efac", "pattern": "order #\\d+" },
+          { "label": "slow", "color": "#fde68a", "pattern": "SLOW", "regex": false, "caseSensitive": true }
+        ]
+      }
+    ]
+  }
+]
+```
+
+| Field | Required | Meaning |
+| --- | --- | --- |
+| `group` | yes | name of the group in the menu |
+| `id` | yes | unique identifier of the preset |
+| `name` | yes | name of the preset in the menu |
+| `tags[].pattern` | yes | what to color: a regular expression, or plain text with `"regex": false` |
+| `tags[].color` | yes | background color, `#rrggbb` |
+| `tags[].label` | no | name shown in the tag list instead of the pattern |
+| `tags[].regex` | no | `true` by default |
+| `tags[].wholeWord` | no | only match whole words, `false` by default |
+| `tags[].caseSensitive` | no | match case, `false` by default |
+
+Names and labels are either one text for every language (`"My app"`) or one text per language
+(`{ "en": "My app", "fr": "Mon appli" }`); a missing language falls back to English.
+
+Regular expressions must work both in the browser (JavaScript) and in Go, which checks them:
+avoid look-behind `(?<=…)`, look-ahead `(?=…)` and back-references `\1`. In JSON, every
+backslash is written twice: `\d` becomes `"\\d"`. A group named `hl`, `(?<hl>…)`, colors only
+that part of the match, as the HTTP presets do with `(?<hl>5\\d\\d)`.

guard.go

@@ -0,0 +1,122 @@
+package main
+
+import (
+	"context"
+	"crypto/sha256"
+	"encoding/base64"
+	"io/fs"
+	"net/http"
+	"net/url"
+	"regexp"
+	"strings"
+)
+
+// Request guards shared by every auth mode: security headers, a cross-site
+// request check, and the read-only role.
+
+type viewerKey struct{}
+
+// asViewer marks the request as made by a read-only user.
+func asViewer(r *http.Request) *http.Request {
+	return r.WithContext(context.WithValue(r.Context(), viewerKey{}, true))
+}
+
+func isViewer(r *http.Request) bool {
+	v, _ := r.Context().Value(viewerKey{}).(bool)
+	return v
+}
+
+func roleName(viewer bool) string {
+	if viewer {
+		return "viewer"
+	}
+	return "admin"
+}
+
+func isSafeMethod(m string) bool {
+	return m == http.MethodGet || m == http.MethodHead || m == http.MethodOptions
+}
+
+// readOnly refuses the API calls that change something (tags, sources, purge)
+// to read-only users. Without authentication everyone is admin.
+func readOnly(next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if isViewer(r) && !isSafeMethod(r.Method) && strings.HasPrefix(r.URL.Path, "/api/") {
+			writeErr(w, http.StatusForbidden, &codedError{code: "read_only", msg: "read-only account: changes are reserved to administrators"})
+			return
+		}
+		next.ServeHTTP(w, r)
+	})
+}
+
+// crossSite tells whether a request that changes something comes from another
+// site (a form or script on a third-party page), using the headers browsers
+// add; tools such as curl send neither and are let through.
+func crossSite(r *http.Request) bool {
+	switch r.Header.Get("Sec-Fetch-Site") {
+	case "same-origin", "none":
+		return false
+	case "":
+	default: // same-site, cross-site
+		return true
+	}
+	o := r.Header.Get("Origin")
+	if o == "" {
+		return false
+	}
+	u, err := url.Parse(o)
+	return err != nil || !strings.EqualFold(u.Host, r.Host)
+}
+
+// secure adds the security headers to every answer and refuses cross-site
+// changes. Without authentication it also answers /auth/me, so the UI can
+// warn that anyone on the network has full access.
+func secure(next http.Handler, csp string, authOn bool) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		h := w.Header()
+		h.Set("X-Content-Type-Options", "nosniff")
+		h.Set("X-Frame-Options", "DENY")
+		h.Set("Referrer-Policy", "same-origin")
+		h.Set("Content-Security-Policy", csp)
+		if !isSafeMethod(r.Method) && crossSite(r) {
+			writeErr(w, http.StatusForbidden, &codedError{code: "cross_site", msg: "cross-site request refused"})
+			return
+		}
+		if !authOn && r.URL.Path == "/auth/me" {
+			writeJSON(w, http.StatusOK, map[string]string{"mode": "none", "role": "admin"})
+			return
+		}
+		next.ServeHTTP(w, r)
+	})
+}
+
+var inlineScript = regexp.MustCompile(`(?s)<script>(.*?)</script>`)
+
+// contentSecurityPolicy allows the UI's own files, the inline scripts of the
+// embedded pages (by hash) and the optional Bunny Fonts.
+func contentSecurityPolicy(static fs.FS) string {
+	scripts := []string{"'self'"}
+	for _, page := range []string{"index.html", "login.html"} {
+		b, err := fs.ReadFile(static, page)
+		if err != nil {
+			continue
+		}
+		for _, m := range inlineScript.FindAllSubmatch(b, -1) {
+			sum := sha256.Sum256(m[1])
+			scripts = append(scripts, "'sha256-"+base64.StdEncoding.EncodeToString(sum[:])+"'")
+		}
+	}
+	return strings.Join([]string{
+		"default-src 'self'",
+		"script-src " + strings.Join(scripts, " "),
+		// Inline style attributes carry the tag and project colors.
+		"style-src 'self' 'unsafe-inline' https://fonts.bunny.net",
+		"font-src 'self' https://fonts.bunny.net",
+		"img-src 'self' data:",
+		"connect-src 'self'",
+		"object-src 'none'",
+		"base-uri 'none'",
+		"form-action 'self'",
+		"frame-ancestors 'none'",
+	}, "; ")
+}

guard_test.go

@@ -0,0 +1,168 @@
+package main
+
+import (
+	"errors"
+	"io/fs"
+	"net"
+	"net/http"
+	"net/http/cookiejar"
+	"net/http/httptest"
+	"os"
+	"strings"
+	"testing"
+	"time"
+)
+
+var echo = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _, _ = w.Write([]byte("app " + r.URL.Path)) })
+
+func TestSecureHeadersAndCrossSite(t *testing.T) {
+	h := secure(echo, "default-src 'self'", false)
+	cases := []struct {
+		method string
+		hdr    map[string]string
+		want   int
+	}{
+		{"GET", map[string]string{"Sec-Fetch-Site": "cross-site"}, 200}, // reading is fine
+		{"POST", nil, 200}, // curl, scripts
+		{"POST", map[string]string{"Sec-Fetch-Site": "same-origin"}, 200},
+		{"POST", map[string]string{"Sec-Fetch-Site": "cross-site"}, 403},
+		{"DELETE", map[string]string{"Sec-Fetch-Site": "same-site"}, 403},
+		{"POST", map[string]string{"Origin": "http://logs.lan:8080"}, 200},
+		{"POST", map[string]string{"Origin": "https://evil.example"}, 403},
+		{"PUT", map[string]string{"Origin": "null"}, 403},
+	}
+	for _, c := range cases {
+		r := httptest.NewRequest(c.method, "http://logs.lan:8080/api/purge", nil)
+		for k, v := range c.hdr {
+			r.Header.Set(k, v)
+		}
+		rec := httptest.NewRecorder()
+		h.ServeHTTP(rec, r)
+		if rec.Code != c.want {
+			t.Errorf("%s %v: %d, want %d", c.method, c.hdr, rec.Code, c.want)
+		}
+		if rec.Header().Get("Content-Security-Policy") == "" || rec.Header().Get("X-Frame-Options") != "DENY" {
+			t.Errorf("%s %v: security headers missing", c.method, c.hdr)
+		}
+	}
+
+	rec := httptest.NewRecorder()
+	h.ServeHTTP(rec, httptest.NewRequest("GET", "/auth/me", nil))
+	if !strings.Contains(rec.Body.String(), `"mode":"none"`) || !strings.Contains(rec.Body.String(), `"role":"admin"`) {
+		t.Errorf("/auth/me without auth: %s", rec.Body)
+	}
+	rec = httptest.NewRecorder()
+	secure(echo, "", true).ServeHTTP(rec, httptest.NewRequest("GET", "/auth/me", nil))
+	if rec.Body.String() != "app /auth/me" {
+		t.Errorf("/auth/me with auth answered by the guard: %s", rec.Body)
+	}
+}
+
+func TestContentSecurityPolicyHashesInlineScripts(t *testing.T) {
+	static, _ := fs.Sub(webFS, "web")
+	csp := contentSecurityPolicy(static)
+	// index.html and login.html each have inline scripts.
+	if n := strings.Count(csp, "'sha256-"); n < 3 {
+		t.Errorf("%d script hashes in %q", n, csp)
+	}
+	for _, want := range []string{"frame-ancestors 'none'", "connect-src 'self'", "https://fonts.bunny.net"} {
+		if !strings.Contains(csp, want) {
+			t.Errorf("CSP lacks %q", want)
+		}
+	}
+}
+
+func TestLocalViewerIsReadOnly(t *testing.T) {
+	loginFailDelay = 0
+	h, err := newAuth(authConfig{mode: "local", user: "admin", pass: "pw", viewerUser: "guest", viewerPass: "ro", dataDir: t.TempDir()}, readOnly(echo))
+	if err != nil {
+		t.Fatal(err)
+	}
+	jar, _ := cookiejar.New(nil)
+	c := &http.Client{Jar: jar, Transport: hosts{"app.test": h}, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
+	app := "http://app.test"
+
+	login(t, c, app, "guest", "ro", "/")
+	if code, body := get(t, c, app+"/auth/me"); code != 200 || !strings.Contains(body, `"role":"viewer"`) {
+		t.Fatalf("me: %d %s", code, body)
+	}
+	if code, _ := get(t, c, app+"/api/logs"); code != 200 {
+		t.Errorf("viewer reading logs: %d", code)
+	}
+	res, err := c.Post(app+"/api/purge", "application/json", strings.NewReader(`{"confirm":"PURGE"}`))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if res.StatusCode != http.StatusForbidden {
+		t.Errorf("viewer purge: %d, want 403", res.StatusCode)
+	}
+
+	// The admin account still changes things, also through Basic auth.
+	req, _ := http.NewRequest("PUT", app+"/api/syslog", strings.NewReader("{}"))
+	req.SetBasicAuth("admin", "pw")
+	if res, _ := (&http.Client{Transport: hosts{"app.test": h}}).Do(req); res.StatusCode != 200 {
+		t.Errorf("admin change: %d", res.StatusCode)
+	}
+	req, _ = http.NewRequest("PUT", app+"/api/syslog", strings.NewReader("{}"))
+	req.SetBasicAuth("guest", "ro")
+	if res, _ := (&http.Client{Transport: hosts{"app.test": h}}).Do(req); res.StatusCode != http.StatusForbidden {
+		t.Errorf("viewer change through Basic auth: %d", res.StatusCode)
+	}
+}
+
+func TestOIDCAdminGroup(t *testing.T) {
+	for _, tc := range []struct {
+		groups any
+		role   string
+	}{
+		{[]any{"staff", "/logstream-admins"}, "admin"},
+		{[]any{"staff"}, "viewer"},
+		{nil, "viewer"},
+	} {
+		idp := newFakeIdP(t)
+		idp.claims = func(c map[string]any) {
+			if tc.groups != nil {
+				c["groups"] = tc.groups
+			}
+		}
+		h, err := newAuth(authConfig{
+			mode: "oidc", issuer: "http://idp.test/realm", clientID: "logstream", clientSecret: "s3cret",
+			redirectURL: "http://app.test/auth/callback", dataDir: t.TempDir(), adminGroup: "logstream-admins",
+		}, readOnly(echo))
+		if err != nil {
+			t.Fatal(err)
+		}
+		netw := hosts{"app.test": h, "idp.test": idp.mux}
+		h.(*OIDC).client.Transport = netw
+		jar, _ := cookiejar.New(nil)
+		c := &http.Client{Jar: jar, Transport: netw}
+		get(t, c, "http://app.test/")
+		if _, body := get(t, c, "http://app.test/auth/me"); !strings.Contains(body, `"role":"`+tc.role+`"`) {
+			t.Errorf("groups %v: %s, want role %s", tc.groups, body, tc.role)
+		}
+	}
+}
+
+func TestHasGroup(t *testing.T) {
+	if !hasGroup("ops logstream-admins", "/logstream-admins") || !hasGroup([]any{"a", "b"}, "b") || hasGroup(42, "b") {
+		t.Error("hasGroup")
+	}
+}
+
+func TestTCPIdleTimeout(t *testing.T) {
+	a, b := net.Pipe()
+	defer b.Close()
+	c := idleConn{a, 30 * time.Millisecond}
+	go func() { _, _ = b.Write([]byte("x")) }()
+	buf := make([]byte, 1)
+	if _, err := c.Read(buf); err != nil {
+		t.Fatal(err)
+	}
+	start := time.Now()
+	if _, err := c.Read(buf); !errors.Is(err, os.ErrDeadlineExceeded) {
+		t.Fatalf("silent connection: %v, want a deadline error", err)
+	}
+	if time.Since(start) > time.Second {
+		t.Error("deadline not applied")
+	}
+}

histogram_test.go

@@ -153,7 +153,7 @@ func TestHistogramHandler(t *testing.T) {
 {"_time":"2000-01-01T00:00:00Z","severity":"info","hits":"9"}
 `, recent.Format(time.RFC3339), recent.Format(time.RFC3339), older.Format(time.RFC3339))
 	}}
-	store := NewStore("http://vl", 10, 10, time.Second)
+	store := NewStore("http://vl", 10, 10, time.Second, nil)
 	store.streamClient.Transport = vl
 	a := &API{store: store}
 
@@ -207,7 +207,7 @@ func TestHistogramDayInParis(t *testing.T) {
 {"_time":"2026-09-21T20:00:00Z","severity":"info","hits":"100"}
 `
 	}}
-	store := NewStore("http://vl", 10, 10, time.Second)
+	store := NewStore("http://vl", 10, 10, time.Second, nil)
 	store.streamClient.Transport = vl
 	a := &API{store: store}
 	from := time.Date(2026, 9, 10, 0, 0, 0, 0, time.UTC)

hostlogs.go

@@ -0,0 +1,454 @@
+package main
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"io"
+	"io/fs"
+	"log"
+	"os"
+	"path/filepath"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"syscall"
+	"time"
+	"unicode/utf8"
+)
+
+// hostLogsConfig is saved in /data/hostlogs.json and edited in Settings > Sources.
+type hostLogsConfig struct {
+	Enabled bool `json:"enabled"`
+}
+
+// hostPos is where reading resumes in a journal file (by file ID) or a text
+// file (by name, with its inode to detect rotations). Off -1: archived file
+// read to the end.
+type hostPos struct {
+	Off int64  `json:"off"`
+	Ino uint64 `json:"ino,omitempty"`
+}
+
+// HostLogs collects the system logs of the machine hosting the stack: the
+// systemd journal when there is one, else the text files of /var/log. The
+// host directories are mounted read-only under root (/host by default).
+type HostLogs struct {
+	root      string
+	sink      func(*Entry)
+	backfill  time.Duration
+	cfgPath   string
+	statePath string
+	wake      chan struct{}
+
+	// Owned by the Run goroutine.
+	pos     map[string]hostPos
+	dirty   bool
+	started bool // a first scan was done since enabling: new files are read from their start
+
+	mu         sync.Mutex
+	cfg        hostLogsConfig
+	reset      bool
+	mode       string // "journal", "files" or "" (nothing found)
+	files      int
+	read       uint64
+	compressed uint64
+	errCode    string
+	errDetail  string
+}
+
+func NewHostLogs(root, dataDir string, backfill time.Duration, sink func(*Entry)) *HostLogs {
+	h := &HostLogs{
+		root:      root,
+		sink:      sink,
+		backfill:  backfill,
+		cfgPath:   filepath.Join(dataDir, "hostlogs.json"),
+		statePath: filepath.Join(dataDir, "hostlogs-state.json"),
+		wake:      make(chan struct{}, 1),
+		pos:       map[string]hostPos{},
+	}
+	if b, err := os.ReadFile(h.cfgPath); err == nil {
+		_ = json.Unmarshal(b, &h.cfg)
+	}
+	if b, err := os.ReadFile(h.statePath); err == nil {
+		_ = json.Unmarshal(b, &h.pos)
+		h.started = len(h.pos) > 0
+	}
+	return h
+}
+
+// Run polls the host logs every second while the source is enabled.
+func (h *HostLogs) Run(ctx context.Context) {
+	tick := time.NewTicker(time.Second)
+	defer tick.Stop()
+	n := 0
+	for {
+		select {
+		case <-ctx.Done():
+			h.saveState()
+			return
+		case <-tick.C:
+		case <-h.wake:
+		}
+		h.mu.Lock()
+		enabled, reset := h.cfg.Enabled, h.reset
+		h.reset = false
+		h.mu.Unlock()
+		if reset {
+			// Disabled then enabled again: start over from HOST_LOGS_BACKFILL ago
+			// rather than reading everything written in between.
+			h.pos, h.started, h.dirty = map[string]hostPos{}, false, true
+		}
+		if enabled {
+			h.scan(time.Now())
+		}
+		if n++; n%5 == 0 || reset {
+			h.saveState()
+		}
+	}
+}
+
+func (h *HostLogs) saveState() {
+	if !h.dirty {
+		return
+	}
+	h.dirty = false
+	if err := writeJSONFile(h.statePath, h.pos); err != nil {
+		log.Printf("host logs: saving positions: %v", err)
+	}
+}
+
+func (h *HostLogs) setStatus(mode string, files int, code, detail string) {
+	h.mu.Lock()
+	h.mode, h.files, h.errCode, h.errDetail = mode, files, code, detail
+	h.mu.Unlock()
+}
+
+// scan reads what was added since the previous poll.
+func (h *HostLogs) scan(now time.Time) {
+	notBefore := time.Time{}
+	if !h.started {
+		notBefore = now.Add(-h.backfill)
+	}
+	defer func() { h.started = true }()
+
+	var journals []string
+	for _, dir := range []string{"var/log/journal", "run/log/journal"} {
+		base := filepath.Join(h.root, dir)
+		for _, pat := range []string{"*.journal", "*/*.journal"} {
+			m, _ := filepath.Glob(filepath.Join(base, pat))
+			journals = append(journals, m...)
+		}
+	}
+	if len(journals) > 0 {
+		h.scanJournals(journals, notBefore)
+		return
+	}
+	h.scanFiles(notBefore.IsZero())
+}
+
+func (h *HostLogs) scanJournals(paths []string, notBefore time.Time) {
+	seen := map[string]bool{}
+	var firstErr error
+	for _, p := range paths {
+		f, err := os.Open(p)
+		if err != nil {
+			firstErr = keepFirst(firstErr, err)
+			continue
+		}
+		hdr, err := readJournalHeader(f)
+		f.Close()
+		if err != nil {
+			continue // file being created, or not a journal
+		}
+		key := "j:" + hdr.fileID
+		seen[key] = true
+		pos, known := h.pos[key]
+		if known && pos.Off < 0 {
+			continue
+		}
+		from, nb := pos.Off, time.Time{}
+		if !known {
+			if hdr.archived && !notBefore.IsZero() && time.UnixMicro(int64(hdr.tailRealtimeUS)).Before(notBefore) {
+				h.pos[key], h.dirty = hostPos{Off: -1}, true // archived before the backfill window
+				continue
+			}
+			nb = notBefore
+		}
+		next, hdr, err := readJournal(p, from, nb, h.emitJournal)
+		if err != nil {
+			firstErr = keepFirst(firstErr, err)
+			continue
+		}
+		if hdr.archived {
+			next = -1
+		}
+		if !known || next != pos.Off {
+			h.pos[key], h.dirty = hostPos{Off: next}, true
+		}
+	}
+	h.prune("j:", seen)
+	code, detail := "", ""
+	if firstErr != nil && len(seen) == 0 {
+		code, detail = errCode(firstErr), firstErr.Error()
+	}
+	h.setStatus("journal", len(seen), code, detail)
+}
+
+func keepFirst(first, err error) error {
+	if first != nil {
+		return first
+	}
+	return err
+}
+
+func errCode(err error) string {
+	if errors.Is(err, fs.ErrPermission) {
+		return "permission"
+	}
+	return "read"
+}
+
+func (h *HostLogs) prune(prefix string, seen map[string]bool) {
+	for k := range h.pos {
+		if strings.HasPrefix(k, prefix) && !seen[k] {
+			delete(h.pos, k)
+			h.dirty = true
+		}
+	}
+}
+
+// hostLogFile reports the classic text logs of /var/log (rotated and
+// compressed copies are left out).
+func hostLogFile(name string) bool {
+	return name == "syslog" || name == "messages" || strings.HasSuffix(name, ".log")
+}
+
+const maxHostRead = 4 << 20 // per file and per poll
+
+// scanFiles follows the text files of /var/log, for hosts without journald.
+// Files present at the first scan are read from their end (only new lines).
+func (h *HostLogs) scanFiles(fromStart bool) {
+	dir := filepath.Join(h.root, "var/log")
+	ents, err := os.ReadDir(dir)
+	if err != nil {
+		code := errCode(err)
+		if errors.Is(err, fs.ErrNotExist) {
+			code = "not_mounted"
+		}
+		h.setStatus("", 0, code, err.Error())
+		return
+	}
+	seen := map[string]bool{}
+	var firstErr error
+	for _, de := range ents {
+		if !de.Type().IsRegular() || !hostLogFile(de.Name()) {
+			continue
+		}
+		name := de.Name()
+		key := "f:" + name
+		fi, err := de.Info()
+		if err != nil {
+			continue
+		}
+		var ino uint64
+		if st, ok := fi.Sys().(*syscall.Stat_t); ok {
+			ino = uint64(st.Ino)
+		}
+		pos, known := h.pos[key]
+		switch {
+		case !known && !fromStart:
+			pos = hostPos{Off: fi.Size(), Ino: ino}
+		case !known || pos.Ino != ino || fi.Size() < pos.Off:
+			pos = hostPos{Off: 0, Ino: ino} // new, rotated or truncated file
+		}
+		if fi.Size() > pos.Off {
+			off, err := h.readFile(filepath.Join(dir, name), name, pos.Off)
+			if err != nil {
+				firstErr = keepFirst(firstErr, err)
+				continue // not readable: not counted as followed
+			}
+			pos.Off = off
+		}
+		seen[key] = true
+		if old, ok := h.pos[key]; !ok || old != pos {
+			h.pos[key], h.dirty = pos, true
+		}
+	}
+	h.prune("f:", seen)
+	code, detail := "", ""
+	if firstErr != nil && len(seen) == 0 {
+		code, detail = errCode(firstErr), firstErr.Error()
+	}
+	mode := "files"
+	if len(seen) == 0 && code == "" {
+		mode, code = "", "empty"
+	}
+	h.setStatus(mode, len(seen), code, detail)
+}
+
+// readFile sends the complete lines written after off and returns the new offset.
+func (h *HostLogs) readFile(path, name string, off int64) (int64, error) {
+	f, err := os.Open(path)
+	if err != nil {
+		return off, err
+	}
+	defer f.Close()
+	buf, err := io.ReadAll(io.NewSectionReader(f, off, maxHostRead))
+	if err != nil {
+		return off, err
+	}
+	end := bytes.LastIndexByte(buf, '\n')
+	if end < 0 {
+		if len(buf) < maxHostRead {
+			return off, nil // partial line: wait for its end
+		}
+		end = len(buf) - 1 // line longer than the read window: cut it
+	}
+	now := time.Now()
+	fac := fileFacility(name)
+	for _, line := range bytes.Split(buf[:end+1], []byte{'\n'}) {
+		if len(bytes.TrimSpace(line)) == 0 {
+			continue
+		}
+		e := ParseSyslog(line, "", "file", now)
+		if e.Host == "" {
+			e.Host = "localhost"
+		}
+		if n, ok := detectLevel(e.Message); ok {
+			e.SevNum, e.Severity = n, severityNames[n]
+		} else {
+			e.SevNum, e.Severity = 6, "info"
+		}
+		if fac >= 0 {
+			e.Facility = facilityNames[fac]
+		}
+		e.SourceType = "host"
+		e.Extra = map[string]string{"log_file": "/var/log/" + name}
+		e.Wait = true
+		h.sink(e)
+		h.count(1, 0)
+	}
+	return off + int64(end) + 1, nil
+}
+
+// fileFacility guesses the facility from the usual Debian/RHEL file names.
+func fileFacility(name string) int {
+	switch strings.TrimSuffix(name, ".log") {
+	case "kern":
+		return 0
+	case "mail", "maillog":
+		return 2
+	case "daemon":
+		return 3
+	case "auth", "secure":
+		return 4
+	case "cron":
+		return 9
+	}
+	return -1
+}
+
+func (h *HostLogs) count(read, compressed uint64) {
+	h.mu.Lock()
+	h.read += read
+	h.compressed += compressed
+	h.mu.Unlock()
+}
+
+// emitJournal turns a journal entry into an Entry.
+func (h *HostLogs) emitJournal(je *journalEntry) {
+	f := je.Fields
+	msg := f["MESSAGE"]
+	if msg == "" && je.Compressed > 0 {
+		msg = "(compressed journal entry: read it with journalctl)"
+	}
+	h.count(1, uint64(je.Compressed))
+	if strings.TrimSpace(msg) == "" {
+		return
+	}
+	if !utf8.ValidString(msg) {
+		msg = strings.ToValidUTF8(msg, "�")
+	}
+	sev := 6
+	if n, err := strconv.Atoi(f["PRIORITY"]); err == nil && n >= 0 && n <= 7 {
+		sev = n
+	}
+	fac := 1 // user
+	switch {
+	case f["_TRANSPORT"] == "kernel":
+		fac = 0
+	case f["_SYSTEMD_UNIT"] != "":
+		fac = 3 // daemon
+	}
+	if n, err := strconv.Atoi(f["SYSLOG_FACILITY"]); err == nil && n >= 0 && n < len(facilityNames) {
+		fac = n
+	}
+	app := firstNonEmpty(f["SYSLOG_IDENTIFIER"], f["_COMM"], strings.TrimSuffix(f["_SYSTEMD_UNIT"], ".service"))
+	host := firstNonEmpty(f["_HOSTNAME"], "localhost")
+	h.sink(&Entry{
+		Time:       je.Realtime,
+		Received:   time.Now(),
+		Host:       host,
+		App:        app,
+		ProcID:     firstNonEmpty(f["SYSLOG_PID"], f["_PID"]),
+		Facility:   facilityNames[fac],
+		Severity:   severityNames[sev],
+		SevNum:     sev,
+		Message:    strings.TrimRight(msg, "\n"),
+		Proto:      "journal",
+		SourceType: "host",
+		Extra:      map[string]string{"unit": f["_SYSTEMD_UNIT"]},
+		Wait:       true,
+	})
+}
+
+func firstNonEmpty(vals ...string) string {
+	for _, v := range vals {
+		if v != "" {
+			return v
+		}
+	}
+	return ""
+}
+
+// Configure saves and applies the configuration.
+func (h *HostLogs) Configure(cfg hostLogsConfig) error {
+	h.mu.Lock()
+	if h.cfg.Enabled && !cfg.Enabled {
+		h.reset = true
+		h.mode, h.files, h.errCode, h.errDetail = "", 0, "", ""
+	}
+	h.cfg = cfg
+	h.mu.Unlock()
+	select {
+	case h.wake <- struct{}{}:
+	default:
+	}
+	return writeJSONFile(h.cfgPath, cfg)
+}
+
+// Status is the state shown in Settings > Sources.
+func (h *HostLogs) Status() map[string]any {
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	dirs := []string{}
+	for _, d := range []string{"var/log/journal", "run/log/journal", "var/log"} {
+		if _, err := os.Stat(filepath.Join(h.root, d)); err == nil {
+			dirs = append(dirs, "/"+d)
+		}
+	}
+	sort.Strings(dirs)
+	return map[string]any{
+		"enabled":    h.cfg.Enabled,
+		"mode":       h.mode,
+		"files":      h.files,
+		"read":       h.read,
+		"compressed": h.compressed,
+		"code":       h.errCode,
+		"error":      h.errDetail,
+		"mounted":    dirs,
+	}
+}

hostlogs_test.go

@@ -0,0 +1,206 @@
+package main
+
+import (
+	"encoding/binary"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+)
+
+// fakeJournal builds a minimal journal file: header, data objects, entries.
+type fakeJournal struct {
+	compact bool
+	buf     []byte
+	seq     uint64
+	tail    uint64
+}
+
+func newFakeJournal(compact bool) *fakeJournal {
+	j := &fakeJournal{compact: compact, buf: make([]byte, 272)}
+	copy(j.buf, jSignature)
+	if compact {
+		binary.LittleEndian.PutUint32(j.buf[12:], jIncompatCompact)
+	}
+	j.buf[16] = 1 // online
+	copy(j.buf[24:40], "0123456789abcdef")
+	binary.LittleEndian.PutUint64(j.buf[88:], 272)
+	return j
+}
+
+func (j *fakeJournal) object(typ, flags byte, body []byte) uint64 {
+	for len(j.buf)%8 != 0 {
+		j.buf = append(j.buf, 0)
+	}
+	off := uint64(len(j.buf))
+	h := make([]byte, jObjHeaderSize)
+	h[0], h[1] = typ, flags
+	binary.LittleEndian.PutUint64(h[8:], uint64(jObjHeaderSize+len(body)))
+	j.buf = append(append(j.buf, h...), body...)
+	j.tail = off
+	binary.LittleEndian.PutUint64(j.buf[136:], off)
+	return off
+}
+
+func (j *fakeJournal) data(field string, flags byte) uint64 {
+	n := 48
+	if j.compact {
+		n = 56
+	}
+	return j.object(jObjData, flags, append(make([]byte, n), field...))
+}
+
+// entry appends an entry; linked=false leaves it unfinished (tail seqnum not updated).
+func (j *fakeJournal) entry(t time.Time, linked bool, fields ...string) {
+	var items []byte
+	for _, f := range fields {
+		flags := byte(0)
+		if strings.HasPrefix(f, "!") { // compressed data object
+			f, flags = f[1:], 4
+		}
+		off := j.data(f, flags)
+		if j.compact {
+			items = binary.LittleEndian.AppendUint32(items, uint32(off))
+		} else {
+			items = binary.LittleEndian.AppendUint64(items, off)
+			items = binary.LittleEndian.AppendUint64(items, 0)
+		}
+	}
+	j.seq++
+	body := make([]byte, 48)
+	binary.LittleEndian.PutUint64(body[0:], j.seq)
+	binary.LittleEndian.PutUint64(body[8:], uint64(t.UnixMicro()))
+	j.object(jObjEntry, 0, append(body, items...))
+	if linked {
+		binary.LittleEndian.PutUint64(j.buf[160:], j.seq)
+		binary.LittleEndian.PutUint64(j.buf[192:], uint64(t.UnixMicro()))
+	}
+}
+
+func (j *fakeJournal) write(t *testing.T, path string) {
+	t.Helper()
+	if err := os.WriteFile(path, j.buf, 0o644); err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestReadJournal(t *testing.T) {
+	for _, compact := range []bool{false, true} {
+		path := filepath.Join(t.TempDir(), "system.journal")
+		now := time.Now()
+		j := newFakeJournal(compact)
+		j.entry(now.Add(-2*time.Hour), true, "MESSAGE=too old", "PRIORITY=6")
+		j.entry(now.Add(-time.Minute), true, "MESSAGE=Started cron.", "PRIORITY=5", "SYSLOG_IDENTIFIER=systemd", "_HOSTNAME=srv1", "_PID=1")
+		j.write(t, path)
+
+		var got []*journalEntry
+		emit := func(e *journalEntry) { got = append(got, e) }
+		next, _, err := readJournal(path, 0, now.Add(-time.Hour), emit)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if len(got) != 1 || got[0].Fields["MESSAGE"] != "Started cron." || got[0].Fields["_HOSTNAME"] != "srv1" {
+			t.Fatalf("compact=%v: got %+v", compact, got)
+		}
+
+		// A new complete entry and one still being written.
+		j.entry(now, true, "MESSAGE=second", "!MESSAGE=big")
+		j.entry(now, false, "MESSAGE=unfinished")
+		j.write(t, path)
+		got = nil
+		next2, _, err := readJournal(path, next, time.Time{}, emit)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if len(got) != 1 || got[0].Fields["MESSAGE"] != "second" || got[0].Compressed != 1 {
+			t.Fatalf("compact=%v: resumed read got %d entries", compact, len(got))
+		}
+
+		// Once linked, the unfinished entry is read from where reading stopped.
+		binary.LittleEndian.PutUint64(j.buf[160:], j.seq)
+		j.write(t, path)
+		got = nil
+		if _, _, err := readJournal(path, next2, time.Time{}, emit); err != nil {
+			t.Fatal(err)
+		}
+		if len(got) != 1 || got[0].Fields["MESSAGE"] != "unfinished" {
+			t.Fatalf("compact=%v: unfinished entry got %+v", compact, got)
+		}
+	}
+}
+
+func TestHostLogsJournal(t *testing.T) {
+	root := t.TempDir()
+	dir := filepath.Join(root, "var/log/journal/machine")
+	if err := os.MkdirAll(dir, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	j := newFakeJournal(true)
+	j.entry(time.Now(), true, "MESSAGE=Accepted publickey", "PRIORITY=6", "SYSLOG_FACILITY=10", "SYSLOG_IDENTIFIER=sshd", "_PID=42", "_HOSTNAME=srv1", "_SYSTEMD_UNIT=ssh.service")
+	j.entry(time.Now(), true, "MESSAGE=oops", "PRIORITY=3", "_TRANSPORT=kernel", "_HOSTNAME=srv1")
+	j.write(t, filepath.Join(dir, "system.journal"))
+
+	var got []*Entry
+	h := NewHostLogs(root, t.TempDir(), time.Hour, func(e *Entry) { got = append(got, e) })
+	h.scan(time.Now())
+	if len(got) != 2 {
+		t.Fatalf("got %d entries", len(got))
+	}
+	e := got[0]
+	if e.App != "sshd" || e.ProcID != "42" || e.Facility != "authpriv" || e.Severity != "info" || e.Host != "srv1" || e.SourceType != "host" || e.Extra["unit"] != "ssh.service" {
+		t.Fatalf("entry %+v", e)
+	}
+	if got[1].Facility != "kern" || got[1].Severity != "err" {
+		t.Fatalf("kernel entry %+v", got[1])
+	}
+	h.scan(time.Now()) // nothing new
+	if len(got) != 2 {
+		t.Fatalf("re-read: %d entries", len(got))
+	}
+	if st := h.Status(); st["mode"] != "journal" || st["files"] != 1 {
+		t.Fatalf("status %+v", st)
+	}
+}
+
+func TestHostLogsFiles(t *testing.T) {
+	root := t.TempDir()
+	dir := filepath.Join(root, "var/log")
+	if err := os.MkdirAll(dir, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	auth := filepath.Join(dir, "auth.log")
+	if err := os.WriteFile(auth, []byte("Oct  3 07:00:00 srv1 sshd[1]: old line\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	_ = os.WriteFile(filepath.Join(dir, "auth.log.1"), []byte("rotated\n"), 0o644)
+
+	var got []*Entry
+	h := NewHostLogs(root, t.TempDir(), time.Hour, func(e *Entry) { got = append(got, e) })
+	h.scan(time.Now()) // existing content is skipped
+	if len(got) != 0 {
+		t.Fatalf("first scan read %d lines", len(got))
+	}
+	f, _ := os.OpenFile(auth, os.O_APPEND|os.O_WRONLY, 0)
+	_, _ = f.WriteString("Oct  3 08:00:00 srv1 sshd[7]: Failed password for root\nOct  3 08:00:01 srv1 sshd[7]: partial")
+	f.Close()
+	h.scan(time.Now())
+	if len(got) != 1 {
+		t.Fatalf("got %d lines", len(got))
+	}
+	e := got[0]
+	if e.Host != "srv1" || e.App != "sshd" || e.ProcID != "7" || e.Facility != "auth" || e.Extra["log_file"] != "/var/log/auth.log" || e.Proto != "file" {
+		t.Fatalf("entry %+v", e)
+	}
+	if st := h.Status(); st["mode"] != "files" || st["files"] != 1 {
+		t.Fatalf("status %+v", st)
+	}
+}
+
+func TestHostLogsNotMounted(t *testing.T) {
+	h := NewHostLogs(filepath.Join(t.TempDir(), "none"), t.TempDir(), time.Hour, func(*Entry) {})
+	h.scan(time.Now())
+	if st := h.Status(); st["code"] != "not_mounted" {
+		t.Fatalf("status %+v", st)
+	}
+}

journal.go

@@ -0,0 +1,174 @@
+package main
+
+import (
+	"bufio"
+	"bytes"
+	"encoding/binary"
+	"encoding/hex"
+	"errors"
+	"io"
+	"os"
+	"time"
+)
+
+// Minimal reader of systemd journal files (*.journal), enough to follow them
+// without journalctl: objects are walked in file order, which is the order
+// entries were written. Format: https://systemd.io/JOURNAL_FILE_FORMAT/
+
+const (
+	jHeaderMin     = 208 // header fields used below end at offset 208
+	jObjHeaderSize = 16
+
+	jObjData  = 1
+	jObjEntry = 3
+
+	jIncompatCompact = 1 << 4
+	jStateArchived   = 2
+	jObjCompressed   = 1<<0 | 1<<1 | 1<<2 // XZ, LZ4, ZSTD: not decoded (no stdlib codec)
+)
+
+var jSignature = []byte("LPKSHHRH")
+
+// jHeader holds the header fields the reader needs.
+type jHeader struct {
+	compact        bool
+	archived       bool
+	fileID         string
+	headerSize     uint64
+	tailObject     uint64 // offset of the last object
+	tailSeqnum     uint64 // seqnum of the last complete entry
+	tailRealtimeUS uint64 // realtime of the last entry (µs since the epoch)
+}
+
+func readJournalHeader(f io.ReaderAt) (jHeader, error) {
+	b := make([]byte, jHeaderMin)
+	if _, err := f.ReadAt(b, 0); err != nil {
+		return jHeader{}, err
+	}
+	if !bytes.Equal(b[:8], jSignature) {
+		return jHeader{}, errors.New("not a journal file")
+	}
+	le := binary.LittleEndian
+	h := jHeader{
+		compact:        le.Uint32(b[12:])&jIncompatCompact != 0,
+		archived:       b[16] == jStateArchived,
+		fileID:         hex.EncodeToString(b[24:40]),
+		headerSize:     le.Uint64(b[88:]),
+		tailObject:     le.Uint64(b[136:]),
+		tailSeqnum:     le.Uint64(b[160:]),
+		tailRealtimeUS: le.Uint64(b[192:]),
+	}
+	if h.headerSize < jHeaderMin {
+		return h, errors.New("journal header too small")
+	}
+	return h, nil
+}
+
+// journalEntry is one entry: its time and its "FIELD=value" pairs.
+type journalEntry struct {
+	Realtime   time.Time
+	Fields     map[string]string
+	Compressed int // fields that could not be read (compressed data)
+}
+
+// readJournal reads the entries complete after offset `from` (0 = start of
+// the file) and returns the offset to resume from. Entries older than
+// `notBefore` are skipped without decoding their fields.
+func readJournal(path string, from int64, notBefore time.Time, emit func(*journalEntry)) (next int64, h jHeader, err error) {
+	f, err := os.Open(path)
+	if err != nil {
+		return from, h, err
+	}
+	defer f.Close()
+	if h, err = readJournalHeader(f); err != nil {
+		return from, h, err
+	}
+	if from < int64(h.headerSize) {
+		from = int64(h.headerSize)
+	}
+	end := int64(h.tailObject)
+	r := bufio.NewReaderSize(io.NewSectionReader(f, from, 1<<62), 64*1024)
+	le := binary.LittleEndian
+	hdr := make([]byte, jObjHeaderSize)
+	off := from
+	for off <= end {
+		if _, err := io.ReadFull(r, hdr); err != nil {
+			return off, h, nil // object still being written
+		}
+		typ, size := hdr[0], int64(le.Uint64(hdr[8:]))
+		if size < jObjHeaderSize {
+			return off, h, nil
+		}
+		body := size - jObjHeaderSize
+		if typ == jObjEntry && body >= 48 && body < 1<<20 {
+			obj := make([]byte, body)
+			if _, err := io.ReadFull(r, obj); err != nil {
+				return off, h, nil
+			}
+			seq := le.Uint64(obj[0:])
+			if seq == 0 || seq > h.tailSeqnum {
+				return off, h, nil // not linked yet: retry at the next poll
+			}
+			rt := time.UnixMicro(int64(le.Uint64(obj[8:])))
+			if !rt.Before(notBefore) {
+				emit(readEntryFields(f, h.compact, rt, obj[48:]))
+			}
+		} else if _, err := r.Discard(int(body)); err != nil {
+			return off, h, nil
+		}
+		next := (off + size + 7) &^ 7 // objects are 8-byte aligned
+		if pad := next - off - size; pad > 0 {
+			if _, err := r.Discard(int(pad)); err != nil {
+				return next, h, nil // the object is complete: resume after it
+			}
+		}
+		off = next
+	}
+	return off, h, nil
+}
+
+// readEntryFields resolves the data objects referenced by an entry.
+func readEntryFields(f io.ReaderAt, compact bool, rt time.Time, items []byte) *journalEntry {
+	le := binary.LittleEndian
+	e := &journalEntry{Realtime: rt, Fields: make(map[string]string, 16)}
+	step := 16
+	if compact {
+		step = 4
+	}
+	payloadAt := int64(64)
+	if compact {
+		payloadAt = 72
+	}
+	hdr := make([]byte, jObjHeaderSize)
+	for i := 0; i+step <= len(items); i += step {
+		var off int64
+		if compact {
+			off = int64(le.Uint32(items[i:]))
+		} else {
+			off = int64(le.Uint64(items[i:]))
+		}
+		if off == 0 {
+			continue
+		}
+		if _, err := f.ReadAt(hdr, off); err != nil || hdr[0] != jObjData {
+			continue
+		}
+		if hdr[1]&jObjCompressed != 0 {
+			e.Compressed++
+			continue
+		}
+		size := int64(le.Uint64(hdr[8:]))
+		n := size - payloadAt
+		if n <= 0 || n > 1<<20 {
+			continue
+		}
+		p := make([]byte, n)
+		if _, err := f.ReadAt(p, off+payloadAt); err != nil {
+			continue
+		}
+		if k := bytes.IndexByte(p, '='); k > 0 {
+			e.Fields[string(p[:k])] = string(p[k+1:])
+		}
+	}
+	return e
+}

main.go

@@ -1,10 +1,9 @@
-// Logstream: a syslog (UDP/TCP) sink stored in VictoriaLogs,
+// Logstream: a syslog (UDP/TCP), Docker and host system logs sink stored in VictoriaLogs,
 // with a web interface to browse and search the logs.
 package main
 
 import (
 	"context"
-	"crypto/subtle"
 	"embed"
 	"errors"
 	"io/fs"
@@ -29,8 +28,7 @@ type config struct {
 	httpAddr   string
 	vlogsURL   string
 	dataDir    string
-	authUser   string
+	auth       authConfig
-	authPass   string
 	rdns       bool
 	dnsServer  string
 	allowPurge bool
@@ -38,9 +36,12 @@ type config struct {
 	dockerLogs bool
 	dockerHost string
 	backfill   time.Duration
+	hostRoot   string
+	hostFill   time.Duration
 	batchSize  int
 	queueSize  int
 	flushEvery time.Duration
+	spoolMax   int64
 }
 
 func getenv(key, def string) string {
@@ -57,6 +58,14 @@ func getenvInt(key string, def int) int {
 	return def
 }
 
+// getenvIntZero is getenvInt that also accepts 0 (to turn a feature off).
+func getenvIntZero(key string, def int) int {
+	if v, err := strconv.Atoi(os.Getenv(key)); err == nil && v >= 0 {
+		return v
+	}
+	return def
+}
+
 func getenvBool(key string, def bool) bool {
 	switch strings.ToLower(os.Getenv(key)) {
 	case "1", "true", "yes", "on":
@@ -80,20 +89,40 @@ func main() {
 		httpAddr:   getenv("HTTP_ADDR", ":8080"),
 		vlogsURL:   getenv("VLOGS_URL", "http://victorialogs:9428"),
 		dataDir:    getenv("DATA_DIR", "/data"),
-		authUser:   os.Getenv("AUTH_USER"),
+		auth: authConfig{
-		authPass:   os.Getenv("AUTH_PASS"),
+			mode:         getenv("AUTH_MODE", "local"),
+			user:         os.Getenv("AUTH_USER"),
+			pass:         os.Getenv("AUTH_PASS"),
+			viewerUser:   os.Getenv("AUTH_VIEWER_USER"),
+			viewerPass:   os.Getenv("AUTH_VIEWER_PASS"),
+			adminGroup:   os.Getenv("OIDC_ADMIN_GROUP"),
+			groupsClaim:  os.Getenv("OIDC_GROUPS_CLAIM"),
+			issuer:       os.Getenv("OIDC_ISSUER"),
+			clientID:     os.Getenv("OIDC_CLIENT_ID"),
+			clientSecret: os.Getenv("OIDC_CLIENT_SECRET"),
+			redirectURL:  os.Getenv("OIDC_REDIRECT_URL"),
+			scopes:       os.Getenv("OIDC_SCOPES"),
+			// SESSION_TTL applies to both modes; OIDC_SESSION_TTL is its former name.
+			sessionTTL: getenvDuration("SESSION_TTL", getenvDuration("OIDC_SESSION_TTL", 12*time.Hour)),
+			loginLogo:  os.Getenv("LOGIN_LOGO"),
+		},
 		rdns:       getenvBool("RDNS", true),
 		dnsServer:  os.Getenv("DNS_SERVER"),
-		allowPurge: getenvBool("ALLOW_PURGE", true),
+		allowPurge: getenvBool("ALLOW_PURGE", false),
 		exportMax:  getenvInt("EXPORT_MAX", 100000),
 		dockerLogs: getenvBool("DOCKER_LOGS", false),
 		dockerHost: getenv("DOCKER_HOST", "unix:///var/run/docker.sock"),
 		backfill:   getenvDuration("DOCKER_BACKFILL", time.Hour),
+		hostRoot:   getenv("HOST_LOGS_ROOT", "/host"),
+		hostFill:   getenvDuration("HOST_LOGS_BACKFILL", time.Hour),
 		batchSize:  getenvInt("BATCH_SIZE", 1000),
 		queueSize:  getenvInt("QUEUE_SIZE", 100000),
 		flushEvery: time.Duration(getenvInt("FLUSH_MS", 1000)) * time.Millisecond,
+		spoolMax:   int64(getenvIntZero("SPOOL_MAX_MB", 1024)) << 20,
 	}
 
+	cfg.auth.dataDir = cfg.dataDir
+
 	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
 	defer stop()
 
@@ -102,7 +131,14 @@ func main() {
 		log.Fatalf("tags: %v", err)
 	}
 
-	store := NewStore(cfg.vlogsURL, cfg.batchSize, cfg.queueSize, cfg.flushEvery)
+	var spool *Spool
+	if cfg.spoolMax > 0 {
+		if spool, err = OpenSpool(filepath.Join(cfg.dataDir, "spool"), cfg.spoolMax); err != nil {
+			log.Printf("disk buffer disabled: %v", err)
+			spool = nil
+		}
+	}
+	store := NewStore(cfg.vlogsURL, cfg.batchSize, cfg.queueSize, cfg.flushEvery, spool)
 	storeDone := make(chan struct{})
 	go func() {
 		store.Run(ctx)
@@ -113,12 +149,19 @@ func main() {
 	rdns := NewReverseDNS(cfg.rdns, cfg.dnsServer)
 	sink := func(e *Entry) {
 		// Host sent as an IP (or no host in the header): replace it with its DNS name.
-		// A new IP waits at most 300 ms; slower lookups finish in the background.
+		// A new IP waits at most 300 ms, without holding up the listener; slower
-		if name := rdns.Lookup(e.Host, 300*time.Millisecond); name != "" {
+		// lookups finish in the background.
+		rdns.Resolve(e.Host, 300*time.Millisecond, func(name string) {
+			if name != "" {
 				e.HostIP, e.Host = e.Host, name
 			}
 			store.Enqueue(e)
 			hub.Publish(e)
+		})
+	}
+	tcpMaxConns = getenvInt("SYSLOG_TCP_MAX_CONNS", tcpMaxConns)
+	if d := getenvDuration("SYSLOG_TCP_IDLE", tcpIdle); d > 0 {
+		tcpIdle = d
 	}
 	// Listening errors (port already used…) are shown in Settings > Sources.
 	syslogSrv := NewSyslogServer(ctx, cfg.syslogAddr, getenv("SYSLOG_PUBLIC_PORT", ""), cfg.dataDir, sink)
@@ -128,7 +171,8 @@ func main() {
 		log.Fatal(err)
 	}
 	mux := http.NewServeMux()
-	api := &API{store: store, hub: hub, tags: tags, rdns: rdns, allowPurge: cfg.allowPurge, exportMax: cfg.exportMax, syslog: syslogSrv}
+	presets := getenv("PRESETS_FILE", filepath.Join(cfg.dataDir, "presets.json"))
+	api := &API{store: store, hub: hub, tags: tags, presets: presets, rdns: rdns, allowPurge: cfg.allowPurge, exportMax: cfg.exportMax, syslog: syslogSrv}
 	if cfg.dockerLogs {
 		dm, err := NewDockerManager(cfg.dockerHost, cfg.dataDir, cfg.backfill, sink)
 		if err != nil {
@@ -139,13 +183,31 @@ func main() {
 			log.Printf("docker logs enabled through %s", cfg.dockerHost)
 		}
 	}
+	// System logs of the host (journal or /var/log), off until enabled in Settings > Sources.
+	api.host = NewHostLogs(cfg.hostRoot, cfg.dataDir, cfg.hostFill, sink)
+	go api.host.Run(ctx)
 	api.Routes(mux)
 	mux.Handle("GET /", http.FileServer(http.FS(static)))
 
+	handler, err := newAuth(cfg.auth, readOnly(mux))
+	if err != nil {
+		log.Fatalf("auth: %v", err)
+	}
+	if o, ok := handler.(*OIDC); ok {
+		go o.checkProvider()
+	}
+	_, local := handler.(*Local)
+	_, oidc := handler.(*OIDC)
+	authOn := local || oidc
+	if !authOn {
+		log.Printf("warning: no authentication (AUTH_USER is empty): anyone who can reach %s can read the logs and change the settings", cfg.httpAddr)
+	}
+	handler = secure(handler, contentSecurityPolicy(static), authOn)
 	srv := &http.Server{
 		Addr:              cfg.httpAddr,
-		Handler:           basicAuth(cfg.authUser, cfg.authPass, mux),
+		Handler:           handler,
 		ReadHeaderTimeout: 10 * time.Second,
+		IdleTimeout:       2 * time.Minute,
 		// Requests inherit the global context so SSE streams end on shutdown.
 		BaseContext: func(net.Listener) context.Context { return ctx },
 	}
@@ -163,25 +225,3 @@ func main() {
 	<-storeDone
 	log.Println("shutdown complete")
 }
-
-// basicAuth protects the UI when AUTH_USER is set (except /healthz).
-func basicAuth(user, pass string, next http.Handler) http.Handler {
-	if user == "" {
-		return next
-	}
-	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		if r.URL.Path == "/healthz" {
-			next.ServeHTTP(w, r)
-			return
-		}
-		u, p, ok := r.BasicAuth()
-		if !ok ||
-			subtle.ConstantTimeCompare([]byte(u), []byte(user)) != 1 ||
-			subtle.ConstantTimeCompare([]byte(p), []byte(pass)) != 1 {
-			w.Header().Set("WWW-Authenticate", `Basic realm="logstream"`)
-			http.Error(w, "authentication required", http.StatusUnauthorized)
-			return
-		}
-		next.ServeHTTP(w, r)
-	})
-}

presets.go

@@ -0,0 +1,110 @@
+package main
+
+import (
+	_ "embed"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"os"
+)
+
+// Built-in tag presets, used when no presets file is found (PRESETS_FILE,
+// /data/presets.json by default). See docs/presets.md.
+//
+//go:embed presets.json
+var builtinPresets []byte
+
+// i18nText is a text per language ({"en": "...", "fr": "..."}) or one plain
+// string for every language.
+type i18nText map[string]string
+
+func (t *i18nText) UnmarshalJSON(b []byte) error {
+	var s string
+	if json.Unmarshal(b, &s) == nil {
+		*t = i18nText{"en": s}
+		return nil
+	}
+	var m map[string]string
+	if err := json.Unmarshal(b, &m); err != nil {
+		return errors.New("expected a string or an object of strings per language")
+	}
+	*t = m
+	return nil
+}
+
+type PresetTag struct {
+	Label         i18nText `json:"label,omitempty"`
+	Pattern       string   `json:"pattern"`
+	Color         string   `json:"color"`
+	Regex         *bool    `json:"regex,omitempty"` // regular expression unless false
+	WholeWord     bool     `json:"wholeWord,omitempty"`
+	CaseSensitive bool     `json:"caseSensitive,omitempty"`
+}
+
+type Preset struct {
+	ID   string      `json:"id"`
+	Name i18nText    `json:"name"`
+	Tags []PresetTag `json:"tags"`
+}
+
+type PresetGroup struct {
+	Group   i18nText `json:"group"`
+	Presets []Preset `json:"presets"`
+}
+
+// parsePresets decodes and checks a presets file with the same rules as the
+// tags, so that every preset can be added as is.
+func parsePresets(b []byte) ([]PresetGroup, error) {
+	var groups []PresetGroup
+	if err := json.Unmarshal(b, &groups); err != nil {
+		return nil, err
+	}
+	seen := map[string]bool{}
+	for _, g := range groups {
+		for _, p := range g.Presets {
+			if p.ID == "" || seen[p.ID] {
+				return nil, fmt.Errorf("preset %q: missing or duplicate id", p.ID)
+			}
+			seen[p.ID] = true
+			if len(p.Tags) == 0 {
+				return nil, fmt.Errorf("preset %q: no tags", p.ID)
+			}
+			for i, pt := range p.Tags {
+				t := Tag{Pattern: pt.Pattern, Color: pt.Color, Regex: pt.Regex == nil || *pt.Regex}
+				if err := t.validate(); err != nil {
+					return nil, fmt.Errorf("preset %q, tag %d: %w", p.ID, i+1, err)
+				}
+			}
+		}
+	}
+	return groups, nil
+}
+
+// presetsResponse is what GET /api/presets returns.
+type presetsResponse struct {
+	Source string        `json:"source"` // "file" or "builtin"
+	File   string        `json:"file"`
+	Error  string        `json:"error,omitempty"` // the file is invalid: built-in presets are used
+	Groups []PresetGroup `json:"groups"`
+}
+
+// loadPresets reads the presets file on every call, so that edits apply
+// without a restart, and falls back to the built-in presets.
+func loadPresets(path string) presetsResponse {
+	res := presetsResponse{Source: "builtin", File: path}
+	if path != "" {
+		b, err := os.ReadFile(path)
+		switch {
+		case err == nil:
+			if res.Groups, err = parsePresets(b); err == nil {
+				res.Source = "file"
+				return res
+			}
+			res.Error = err.Error()
+		case !errors.Is(err, os.ErrNotExist):
+			res.Error = err.Error()
+		}
+	}
+	res.Groups, _ = parsePresets(builtinPresets) // checked by the tests
+	return res
+}

presets.json

@@ -0,0 +1,244 @@
+[
+  {
+    "group": "HTTP/HTTPS",
+    "presets": [
+      {
+        "id": "http_status",
+        "name": { "en": "HTTP status codes", "fr": "Codes HTTP" },
+        "tags": [
+          {
+            "label": "HTTP 2xx",
+            "color": "#86efac",
+            "pattern": "(?:\" |\"(?:status|DownstreamStatus|OriginStatus|status_code)\": ?|(?:-1|\\d+)/(?:-1|\\d+)/(?:-1|\\d+)/(?:-1|\\d+)/\\+?\\d+ )(?<hl>2\\d\\d)\\b"
+          },
+          {
+            "label": "HTTP 3xx",
+            "color": "#93c5fd",
+            "pattern": "(?:\" |\"(?:status|DownstreamStatus|OriginStatus|status_code)\": ?|(?:-1|\\d+)/(?:-1|\\d+)/(?:-1|\\d+)/(?:-1|\\d+)/\\+?\\d+ )(?<hl>3\\d\\d)\\b"
+          },
+          {
+            "label": "HTTP 4xx",
+            "color": "#fdba74",
+            "pattern": "(?:\" |\"(?:status|DownstreamStatus|OriginStatus|status_code)\": ?|(?:-1|\\d+)/(?:-1|\\d+)/(?:-1|\\d+)/(?:-1|\\d+)/\\+?\\d+ )(?<hl>4\\d\\d)\\b"
+          },
+          {
+            "label": "HTTP 5xx",
+            "color": "#f87171",
+            "pattern": "(?:\" |\"(?:status|DownstreamStatus|OriginStatus|status_code)\": ?|(?:-1|\\d+)/(?:-1|\\d+)/(?:-1|\\d+)/(?:-1|\\d+)/\\+?\\d+ )(?<hl>5\\d\\d)\\b"
+          }
+        ]
+      },
+      {
+        "id": "http_methods",
+        "name": { "en": "HTTP methods", "fr": "Méthodes HTTP" },
+        "tags": [
+          {
+            "label": "GET/HEAD/OPTIONS",
+            "color": "#cbd5e1",
+            "caseSensitive": true,
+            "pattern": "\"(?<hl>GET|HEAD|OPTIONS)[ \"]"
+          },
+          {
+            "label": "POST/PUT/PATCH",
+            "color": "#c4b5fd",
+            "caseSensitive": true,
+            "pattern": "\"(?<hl>POST|PUT|PATCH)[ \"]"
+          },
+          {
+            "label": "DELETE",
+            "color": "#f9a8d4",
+            "caseSensitive": true,
+            "pattern": "\"(?<hl>DELETE)[ \"]"
+          }
+        ]
+      },
+      {
+        "id": "http_probes",
+        "name": { "en": "Probes and attacks", "fr": "Sondes et attaques" },
+        "tags": [
+          {
+            "label": { "en": "probes / attacks", "fr": "sondes / attaques" },
+            "color": "#fda4af",
+            "pattern": "(?:wp-login\\.php|xmlrpc\\.php|wp-admin|phpmyadmin|/\\.env|/\\.git|/\\.aws|/cgi-bin/|\\.\\./|%2e%2e|/etc/passwd|<script|union(?:\\s|%20|\\+)+select)"
+          }
+        ]
+      },
+      {
+        "id": "http_bots",
+        "name": { "en": "Bots and scripts", "fr": "Robots et scripts" },
+        "tags": [
+          {
+            "label": { "en": "bots / scripts", "fr": "robots / scripts" },
+            "color": "#fde68a",
+            "pattern": "\\b(?:[a-z]*bot|crawler|spider|curl|wget|python-requests|Go-http-client|zgrab|masscan|nmap|sqlmap|nikto)\\b"
+          }
+        ]
+      },
+      {
+        "id": "http_errors",
+        "name": { "en": "TLS/HTTPS and proxy errors", "fr": "Erreurs TLS/HTTPS et proxy" },
+        "tags": [
+          {
+            "label": { "en": "TLS errors", "fr": "erreurs TLS" },
+            "color": "#f0abfc",
+            "pattern": "(?:TLS handshake error|SSL_do_handshake\\(\\) failed|SSL handshake|SSL routines|certificate (?:has )?expired|certificate verify failed|bad certificate|unknown certificate|x509:)"
+          },
+          {
+            "label": { "en": "proxy errors", "fr": "erreurs proxy" },
+            "color": "#fdba74",
+            "pattern": "(?:upstream timed out|upstream prematurely closed|no live upstreams|connect\\(\\) failed|connection refused|bad gateway|gateway time-?out|service unavailable)"
+          }
+        ]
+      }
+    ]
+  },
+  {
+    "group": { "en": "System", "fr": "Système" },
+    "presets": [
+      {
+        "id": "sys_auth",
+        "name": { "en": "SSH and logins", "fr": "SSH et connexions" },
+        "tags": [
+          {
+            "label": { "en": "login failures", "fr": "échecs de connexion" },
+            "color": "#fca5a5",
+            "pattern": "(?:Failed (?:password|publickey|none)|Invalid user|authentication failures?|Connection closed by (?:invalid|authenticating) user|maximum authentication attempts exceeded|FAILED (?:LOGIN|SU)|incorrect password attempts?|NOT in sudoers)"
+          },
+          {
+            "label": { "en": "logins", "fr": "connexions" },
+            "color": "#86efac",
+            "pattern": "(?:Accepted (?:password|publickey|keyboard-interactive(?:/pam)?)|session opened for user|New session \\S+ of user)"
+          }
+        ]
+      },
+      {
+        "id": "sys_sudo",
+        "name": { "en": "sudo commands", "fr": "Commandes sudo" },
+        "tags": [
+          {
+            "label": { "en": "sudo commands", "fr": "commandes sudo" },
+            "color": "#fde68a",
+            "caseSensitive": true,
+            "pattern": "\\bCOMMAND=\\S+"
+          }
+        ]
+      },
+      {
+        "id": "sys_kernel",
+        "name": { "en": "Kernel: OOM, crashes, disks", "fr": "Noyau : OOM, plantages, disques" },
+        "tags": [
+          {
+            "label": { "en": "out of memory", "fr": "mémoire épuisée" },
+            "color": "#f87171",
+            "pattern": "(?:Out of memory|oom-kill(?:er)?|oom_reaper|Killed process \\d+|invoked oom-killer)"
+          },
+          {
+            "label": { "en": "kernel errors", "fr": "erreurs noyau" },
+            "color": "#fda4af",
+            "pattern": "(?:Kernel panic|\\bBUG: |\\bOops\\b|Call Trace|segfault at|general protection fault|I/O error|EXT4-fs error|Buffer I/O error|blocked for more than \\d+ seconds|Hardware Error|soft lockup|hard LOCKUP)"
+          }
+        ]
+      },
+      {
+        "id": "sys_systemd",
+        "name": { "en": "systemd services", "fr": "Services systemd" },
+        "tags": [
+          {
+            "label": { "en": "failed services", "fr": "services en échec" },
+            "color": "#fca5a5",
+            "pattern": "(?:Failed to start|failed with result|Main process exited, code=(?:exited|killed|dumped)|entered failed state|Start request repeated too quickly|Dependency failed)"
+          },
+          {
+            "label": { "en": "service start/stop", "fr": "démarrage/arrêt de service" },
+            "color": "#bbf7d0",
+            "caseSensitive": true,
+            "pattern": "\\b(?:Started|Starting|Stopped|Stopping|Reloaded|Reloading|Reached target)\\b"
+          }
+        ]
+      },
+      {
+        "id": "sys_firewall",
+        "name": { "en": "Firewall and fail2ban", "fr": "Pare-feu et fail2ban" },
+        "tags": [
+          {
+            "label": { "en": "firewall", "fr": "pare-feu" },
+            "color": "#fdba74",
+            "caseSensitive": true,
+            "pattern": "(?:\\[UFW (?:BLOCK|ALLOW|AUDIT|LIMIT BLOCK)\\]|\\b(?:DROP|REJECT)\\b|\\b(?:Ban|Unban|Found) \\d{1,3}(?:\\.\\d{1,3}){3}\\b)"
+          }
+        ]
+      }
+    ]
+  },
+  {
+    "group": "Applications",
+    "presets": [
+      {
+        "id": "app_docker",
+        "name": { "en": "Docker and containers", "fr": "Docker et conteneurs" },
+        "tags": [
+          {
+            "label": { "en": "container problems", "fr": "problèmes de conteneur" },
+            "color": "#fcd34d",
+            "pattern": "(?:\\bOOMKilled\\b|exited with code [1-9]\\d*|exit code: [1-9]\\d*|health_status: unhealthy|\\bunhealthy\\b|Back-off restarting|CrashLoopBackOff|container (?:die|kill|oom)\\b|restarting \\(\\d+\\))"
+          }
+        ]
+      },
+      {
+        "id": "app_db",
+        "name": { "en": "Databases", "fr": "Bases de données" },
+        "tags": [
+          {
+            "label": { "en": "database errors", "fr": "erreurs base de données" },
+            "color": "#c4b5fd",
+            "pattern": "(?:\\bdeadlock(?: detected| found)?\\b|duplicate key|too many (?:connections|clients)|lock wait timeout|slow query|could not connect to server|server has gone away|out of shared memory|terminating connection|Access denied for user|password authentication failed)"
+          }
+        ]
+      }
+    ]
+  },
+  {
+    "group": { "en": "General", "fr": "Général" },
+    "presets": [
+      {
+        "id": "gen_levels",
+        "name": { "en": "Log levels", "fr": "Niveaux de log" },
+        "tags": [
+          {
+            "label": { "en": "fatal / critical", "fr": "fatal / critique" },
+            "color": "#ef4444",
+            "pattern": "\\b(?:fatal|crit(?:ical)?|panic|emerg(?:ency)?)\\b"
+          },
+          {
+            "label": "info / notice",
+            "color": "#bfdbfe",
+            "pattern": "\\b(?:info|notice)\\b"
+          },
+          {
+            "label": "debug / trace",
+            "color": "#e5e7eb",
+            "pattern": "\\b(?:debug|trace)\\b"
+          },
+          {
+            "label": "ok",
+            "color": "#86efac",
+            "regex": false,
+            "wholeWord": true,
+            "pattern": "ok"
+          }
+        ]
+      },
+      {
+        "id": "gen_ip",
+        "name": { "en": "IPv4 addresses", "fr": "Adresses IPv4" },
+        "tags": [
+          {
+            "label": { "en": "IPv4 addresses", "fr": "adresses IPv4" },
+            "color": "#a5f3fc",
+            "pattern": "\\b(?:(?:25[0-5]|2[0-4]\\d|1?\\d?\\d)\\.){3}(?:25[0-5]|2[0-4]\\d|1?\\d?\\d)\\b"
+          }
+        ]
+      }
+    ]
+  }
+]

presets_test.go

@@ -0,0 +1,46 @@
+package main
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+func TestBuiltinPresets(t *testing.T) {
+	groups, err := parsePresets(builtinPresets)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(groups) == 0 {
+		t.Fatal("no built-in presets")
+	}
+}
+
+func TestLoadPresetsFile(t *testing.T) {
+	dir := t.TempDir()
+	path := filepath.Join(dir, "presets.json")
+
+	if res := loadPresets(path); res.Source != "builtin" || res.Error != "" {
+		t.Fatalf("missing file: got %q, error %q", res.Source, res.Error)
+	}
+
+	os.WriteFile(path, []byte(`[{"group":"Mine","presets":[{"id":"a","name":{"en":"A","fr":"A fr"},
+		"tags":[{"label":"x","pattern":"foo|bar","color":"#112233"},{"pattern":"a.b","color":"#445566","regex":false}]}]}]`), 0o644)
+	res := loadPresets(path)
+	if res.Source != "file" || res.Error != "" || res.Groups[0].Presets[0].Name["fr"] != "A fr" || res.Groups[0].Group["en"] != "Mine" {
+		t.Fatalf("valid file: %+v", res)
+	}
+
+	for _, bad := range []string{
+		`not json`,
+		`[{"group":"G","presets":[{"id":"a","name":"A","tags":[{"pattern":"(","color":"#112233"}]}]}]`,
+		`[{"group":"G","presets":[{"id":"a","name":"A","tags":[{"pattern":"x","color":"red"}]}]}]`,
+		`[{"group":"G","presets":[{"id":"a","name":"A","tags":[]}]}]`,
+		`[{"group":"G","presets":[{"id":"a","name":"A","tags":[{"pattern":"x","color":"#112233"}]},{"id":"a","name":"B","tags":[{"pattern":"y","color":"#112233"}]}]}]`,
+	} {
+		os.WriteFile(path, []byte(bad), 0o644)
+		if res := loadPresets(path); res.Source != "builtin" || res.Error == "" || len(res.Groups) == 0 {
+			t.Errorf("%s: got %q, error %q", bad, res.Source, res.Error)
+		}
+	}
+}

query.go

@@ -17,7 +17,7 @@ type Filter struct {
 	Host     string
 	App      string
 	Severity int    // highest severity number included (0 = emerg … 7 = debug), -1 = all
-	Source   string // "syslog", "docker" or "" for all
+	Source   string // "syslog", "docker", "host" or "" for all
 }
 
 var rangeDurations = map[string]time.Duration{
@@ -119,10 +119,10 @@ func (f Filter) filterExpr() string {
 		parts = append(parts, "app:="+strconv.Quote(f.App))
 	}
 	switch f.Source {
-	case "docker":
+	case "docker", "host":
-		parts = append(parts, `source_type:="docker"`)
+		parts = append(parts, `source_type:=`+strconv.Quote(f.Source))
 	case "syslog": // also matches logs stored before source_type existed
-		parts = append(parts, `!(source_type:="docker")`)
+		parts = append(parts, `!(source_type:in("docker","host"))`)
 	}
 	if f.Severity >= 0 && f.Severity < 7 {
 		names := make([]string, 0, 8)
@@ -209,7 +209,7 @@ func (m *Matcher) Match(e *Entry) bool {
 	if m.f.App != "" && e.App != m.f.App {
 		return false
 	}
-	if (m.f.Source == "docker") != (e.SourceType == "docker") && m.f.Source != "" {
+	if m.f.Source != "" && m.f.Source != e.SourceType {
 		return false
 	}
 	if m.f.Severity >= 0 && e.SevNum > m.f.Severity {

rdns.go

@@ -1,6 +1,7 @@
 package main
 
 import (
+	"container/list"
 	"context"
 	"net"
 	"strings"
@@ -15,17 +16,26 @@ type ReverseDNS struct {
 	posTTL  time.Duration // cache duration of a found name
 	negTTL  time.Duration // cache duration of "no name"
 
+	lookups chan struct{} // limits the lookups running at once
+	waiters chan struct{} // limits the messages waiting for a lookup (Resolve)
+
 	mu    sync.Mutex
-	cache map[string]*rdnsEntry
+	cache map[string]*list.Element // ip -> element of lru
+	lru   *list.List               // *rdnsEntry, most recently used first
 }
 
 type rdnsEntry struct {
+	ip      string
 	name    string
 	expires time.Time
 	done    chan struct{} // closed once the lookup has finished
 }
 
-const rdnsMaxEntries = 10000
+const (
+	rdnsMaxEntries = 10000
+	rdnsMaxLookups = 64
+	rdnsMaxWaiters = 1024
+)
 
 // NewReverseDNS uses the system resolver, or `server` ("ip" or "ip:port") when set.
 func NewReverseDNS(enabled bool, server string) *ReverseDNS {
@@ -47,7 +57,10 @@ func NewReverseDNS(enabled bool, server string) *ReverseDNS {
 		r:       r,
 		posTTL:  time.Hour,
 		negTTL:  10 * time.Minute,
-		cache:   make(map[string]*rdnsEntry),
+		lookups: make(chan struct{}, rdnsMaxLookups),
+		waiters: make(chan struct{}, rdnsMaxWaiters),
+		cache:   make(map[string]*list.Element),
+		lru:     list.New(),
 	}
 }
 
@@ -60,6 +73,38 @@ func isClosed(ch chan struct{}) bool {
 	}
 }
 
+// entry returns the cache entry of ip, starting its lookup when it is missing
+// or expired, or nil when too many lookups are already running (a flood of
+// unknown addresses). The least recently used entry makes room for a new one.
+func (d *ReverseDNS) entry(ip string) *rdnsEntry {
+	d.mu.Lock()
+	defer d.mu.Unlock()
+	if el := d.cache[ip]; el != nil {
+		e := el.Value.(*rdnsEntry)
+		if !isClosed(e.done) || time.Now().Before(e.expires) {
+			d.lru.MoveToFront(el)
+			return e
+		}
+	}
+	select {
+	case d.lookups <- struct{}{}:
+	default:
+		return nil
+	}
+	if el := d.cache[ip]; el != nil {
+		d.lru.Remove(el)
+	}
+	for d.lru.Len() >= rdnsMaxEntries {
+		old := d.lru.Back()
+		d.lru.Remove(old)
+		delete(d.cache, old.Value.(*rdnsEntry).ip)
+	}
+	e := &rdnsEntry{ip: ip, done: make(chan struct{})}
+	d.cache[ip] = d.lru.PushFront(e)
+	go d.resolve(ip, e)
+	return e
+}
+
 // Lookup returns the name of ip, or "" when ip is not an IP address, has no
 // PTR record, or is not resolved within `wait`. A lookup that takes longer
 // keeps running in the background and fills the cache for later calls.
@@ -67,18 +112,10 @@ func (d *ReverseDNS) Lookup(ip string, wait time.Duration) string {
 	if !d.enabled || net.ParseIP(ip) == nil {
 		return ""
 	}
-	d.mu.Lock()
+	e := d.entry(ip)
-	e := d.cache[ip]
+	if e == nil {
-	if e == nil || (isClosed(e.done) && time.Now().After(e.expires)) {
+		return ""
-		if len(d.cache) >= rdnsMaxEntries {
-			d.cache = make(map[string]*rdnsEntry)
 	}
-		e = &rdnsEntry{done: make(chan struct{})}
-		d.cache[ip] = e
-		go d.resolve(ip, e)
-	}
-	d.mu.Unlock()
-
 	if !isClosed(e.done) {
 		timer := time.NewTimer(wait)
 		defer timer.Stop()
@@ -91,6 +128,43 @@ func (d *ReverseDNS) Lookup(ip string, wait time.Duration) string {
 	return e.name
 }
 
+// Resolve is Lookup without blocking the caller: fn gets the name (or "") at
+// once when it is known, otherwise from a goroutine after at most `wait`. The
+// syslog listeners use it so that a slow DNS server never delays the reading
+// of the next messages.
+func (d *ReverseDNS) Resolve(ip string, wait time.Duration, fn func(name string)) {
+	if !d.enabled || net.ParseIP(ip) == nil {
+		fn("")
+		return
+	}
+	e := d.entry(ip)
+	switch {
+	case e == nil:
+		fn("")
+		return
+	case isClosed(e.done):
+		fn(e.name)
+		return
+	}
+	select {
+	case d.waiters <- struct{}{}:
+	default:
+		fn("") // too many messages waiting already
+		return
+	}
+	go func() {
+		defer func() { <-d.waiters }()
+		timer := time.NewTimer(wait)
+		defer timer.Stop()
+		select {
+		case <-e.done:
+			fn(e.name)
+		case <-timer.C:
+			fn("")
+		}
+	}()
+}
+
 // LookupMany resolves several addresses in parallel; unresolved ones are absent.
 func (d *ReverseDNS) LookupMany(ips []string, wait time.Duration) map[string]string {
 	out := make(map[string]string)
@@ -112,6 +186,7 @@ func (d *ReverseDNS) LookupMany(ips []string, wait time.Duration) map[string]str
 }
 
 func (d *ReverseDNS) resolve(ip string, e *rdnsEntry) {
+	defer func() { <-d.lookups }()
 	ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
 	defer cancel()
 	ttl := d.negTTL

rdns_test.go

@@ -0,0 +1,77 @@
+package main
+
+import (
+	"context"
+	"errors"
+	"net"
+	"testing"
+	"time"
+)
+
+// slowDNS never answers before the context ends.
+func slowDNS() *ReverseDNS {
+	d := NewReverseDNS(true, "")
+	d.r = &net.Resolver{PreferGo: true, Dial: func(ctx context.Context, _, _ string) (net.Conn, error) {
+		<-ctx.Done()
+		return nil, errors.New("timeout")
+	}}
+	return d
+}
+
+func TestResolveDoesNotBlock(t *testing.T) {
+	d := slowDNS()
+	got := make(chan string, 1)
+	start := time.Now()
+	d.Resolve("192.0.2.1", 50*time.Millisecond, func(name string) { got <- name })
+	if time.Since(start) > 20*time.Millisecond {
+		t.Fatal("Resolve waited for the DNS server")
+	}
+	if name := <-got; name != "" {
+		t.Errorf("name %q, want none", name)
+	}
+
+	// Not an IP address: the callback runs at once.
+	called := false
+	d.Resolve("router", time.Second, func(name string) { called = name == "" })
+	if !called {
+		t.Error("callback not called synchronously for a host name")
+	}
+}
+
+func TestReverseDNSLimits(t *testing.T) {
+	d := slowDNS()
+	// Lookups beyond the limit are not started (and not cached).
+	for i := 0; i < rdnsMaxLookups+10; i++ {
+		d.Lookup(net.IPv4(10, 0, byte(i>>8), byte(i)).String(), 0)
+	}
+	if n := d.lru.Len(); n != rdnsMaxLookups {
+		t.Errorf("%d entries, want %d", n, rdnsMaxLookups)
+	}
+}
+
+func TestReverseDNSEvictsLeastRecentlyUsed(t *testing.T) {
+	d := slowDNS()
+	done := make(chan struct{})
+	close(done)
+	add := func(ip string) {
+		e := &rdnsEntry{ip: ip, name: ip + ".lan", expires: time.Now().Add(time.Hour), done: done}
+		d.cache[ip] = d.lru.PushFront(e)
+	}
+	for i := 0; i < rdnsMaxEntries; i++ {
+		add(net.IPv4(10, 1, byte(i>>8), byte(i)).String())
+	}
+	first := net.IPv4(10, 1, 0, 0).String()
+	if name := d.Lookup(first, 0); name != first+".lan" { // now the most recent
+		t.Fatalf("cached name %q", name)
+	}
+	d.entry("192.0.2.9")
+	if d.lru.Len() != rdnsMaxEntries {
+		t.Errorf("%d entries, want %d", d.lru.Len(), rdnsMaxEntries)
+	}
+	if d.cache[first] == nil {
+		t.Error("recently used entry evicted")
+	}
+	if d.cache[net.IPv4(10, 1, 0, 1).String()] != nil {
+		t.Error("least recently used entry kept")
+	}
+}

spool.go

@@ -0,0 +1,136 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+)
+
+// Spool keeps on disk the batches VictoriaLogs could not take, so that they
+// are sent later instead of being lost. Each batch is one NDJSON file named
+// <unix nanoseconds>-<lines>.ndjson; files are sent back oldest first.
+type Spool struct {
+	dir string
+	max int64 // maximum total size in bytes
+
+	mu    sync.Mutex
+	size  int64 // bytes on disk
+	lines int64 // messages on disk
+	seq   int64
+}
+
+// errSpoolFull is returned when a batch does not fit within SPOOL_MAX_MB.
+var errSpoolFull = fmt.Errorf("disk buffer full")
+
+// OpenSpool creates the directory if needed and counts the batches already
+// there (left by a previous run).
+func OpenSpool(dir string, max int64) (*Spool, error) {
+	if err := os.MkdirAll(dir, 0o755); err != nil {
+		return nil, err
+	}
+	s := &Spool{dir: dir, max: max}
+	files, err := s.files()
+	if err != nil {
+		return nil, err
+	}
+	for _, f := range files {
+		s.size += f.size
+		s.lines += f.lines
+	}
+	return s, nil
+}
+
+type spoolFile struct {
+	path  string
+	size  int64
+	lines int64
+}
+
+// files lists the batches on disk, oldest first. Unfinished writes (.tmp)
+// are removed.
+func (s *Spool) files() ([]spoolFile, error) {
+	entries, err := os.ReadDir(s.dir)
+	if err != nil {
+		return nil, err
+	}
+	var out []spoolFile
+	for _, e := range entries {
+		name := e.Name()
+		if strings.HasSuffix(name, ".tmp") {
+			_ = os.Remove(filepath.Join(s.dir, name))
+			continue
+		}
+		base, ok := strings.CutSuffix(name, ".ndjson")
+		if !ok {
+			continue
+		}
+		_, n, _ := strings.Cut(base, "-")
+		lines, _ := strconv.ParseInt(n, 10, 64)
+		info, err := e.Info()
+		if err != nil {
+			continue
+		}
+		out = append(out, spoolFile{path: filepath.Join(s.dir, name), size: info.Size(), lines: lines})
+	}
+	// The names start with a fixed-width timestamp: string order is time order.
+	sort.Slice(out, func(i, j int) bool { return out[i].path < out[j].path })
+	return out, nil
+}
+
+// Write saves one batch of `lines` messages.
+func (s *Spool) Write(body []byte, lines int) error {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	if s.size+int64(len(body)) > s.max {
+		return errSpoolFull
+	}
+	s.seq++
+	name := fmt.Sprintf("%020d%04d-%d.ndjson", time.Now().UnixNano(), s.seq%10000, lines)
+	path := filepath.Join(s.dir, name)
+	tmp := path + ".tmp"
+	if err := os.WriteFile(tmp, body, 0o644); err != nil {
+		_ = os.Remove(tmp)
+		return err
+	}
+	if err := os.Rename(tmp, path); err != nil {
+		_ = os.Remove(tmp)
+		return err
+	}
+	s.size += int64(len(body))
+	s.lines += int64(lines)
+	return nil
+}
+
+// Oldest returns the oldest batch, or ok=false when the spool is empty.
+func (s *Spool) Oldest() (f spoolFile, body []byte, ok bool, err error) {
+	files, err := s.files()
+	if err != nil || len(files) == 0 {
+		return f, nil, false, err
+	}
+	f = files[0]
+	body, err = os.ReadFile(f.path)
+	return f, body, err == nil, err
+}
+
+// Remove deletes a batch once VictoriaLogs has taken it.
+func (s *Spool) Remove(f spoolFile) {
+	if err := os.Remove(f.path); err != nil && !os.IsNotExist(err) {
+		return
+	}
+	s.mu.Lock()
+	s.size -= f.size
+	s.lines -= f.lines
+	s.mu.Unlock()
+}
+
+// Pending returns the number of messages and bytes waiting on disk.
+func (s *Spool) Pending() (lines, size int64) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	return s.lines, s.size
+}

store.go

@@ -17,13 +17,18 @@ import (
 )
 
 // Store sends messages to VictoriaLogs in batches and queries it with LogsQL.
+// Batches VictoriaLogs cannot take go to the disk spool (when enabled) and
+// are sent again, oldest first, once it answers.
 type Store struct {
 	base         string
 	client       *http.Client
 	streamClient *http.Client
 	in           chan *Entry
+	quit         chan struct{} // closed on shutdown: unblocks waiting producers
 	batchSize    int
 	flushEvery   time.Duration
+	spool        *Spool        // nil: no disk buffer
+	spooled      chan struct{} // wakes the replay loop up after a write to the spool
 
 	received atomic.Int64
 	ingested atomic.Int64
@@ -31,7 +36,7 @@ type Store struct {
 	lastErr  atomic.Value // string
 }
 
-func NewStore(base string, batchSize, queueSize int, flushEvery time.Duration) *Store {
+func NewStore(base string, batchSize, queueSize int, flushEvery time.Duration, spool *Spool) *Store {
 	s := &Store{
 		base:   strings.TrimRight(base, "/"),
 		client: &http.Client{Timeout: 60 * time.Second},
@@ -39,21 +44,34 @@ func NewStore(base string, batchSize, queueSize int, flushEvery time.Duration) *
 		// request context still cancels it when the browser goes away.
 		streamClient: &http.Client{},
 		in:           make(chan *Entry, queueSize),
+		quit:         make(chan struct{}),
 		batchSize:    batchSize,
 		flushEvery:   flushEvery,
+		spool:        spool,
+		spooled:      make(chan struct{}, 1),
 	}
 	s.lastErr.Store("")
 	return s
 }
 
-// Enqueue never blocks: when the queue is full, the message is counted as dropped.
+// Enqueue adds a message to the queue. When the queue is full, a message
+// whose producer can wait (Entry.Wait: Docker, host logs) blocks until there
+// is room; any other one (syslog) is counted as dropped.
 func (s *Store) Enqueue(e *Entry) {
 	s.received.Add(1)
 	select {
 	case s.in <- e:
+		return
 	default:
-		s.dropped.Add(1)
 	}
+	if e.Wait {
+		select {
+		case s.in <- e:
+			return
+		case <-s.quit:
+		}
+	}
+	s.dropped.Add(1)
 }
 
 // Run drains the queue into VictoriaLogs until the context is cancelled,
@@ -62,41 +80,41 @@ func (s *Store) Run(ctx context.Context) {
 	ticker := time.NewTicker(s.flushEvery)
 	defer ticker.Stop()
 	batch := make([]*Entry, 0, s.batchSize)
+	if s.spool != nil {
+		go s.replay(ctx)
+	}
 
-	flush := func() {
+	flush := func(ctx context.Context) {
-		if len(batch) == 0 {
+		if len(batch) > 0 {
-			return
+			s.store(ctx, batch)
-		}
+			clear(batch)
-		if err := s.insert(batch); err != nil {
-			s.dropped.Add(int64(len(batch)))
-			s.lastErr.Store(err.Error())
-			log.Printf("victorialogs: %d messages dropped: %v", len(batch), err)
-		} else {
-			s.ingested.Add(int64(len(batch)))
-			s.lastErr.Store("")
-		}
 			batch = batch[:0]
 		}
+	}
 
 	for {
 		select {
 		case e := <-s.in:
 			batch = append(batch, e)
 			if len(batch) >= s.batchSize {
-				flush()
+				flush(ctx)
 			}
 		case <-ticker.C:
-			flush()
+			flush(ctx)
 		case <-ctx.Done():
+			close(s.quit)
+			// The last batches get one short attempt, then go to the spool.
+			end, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+			defer cancel()
 			for {
 				select {
 				case e := <-s.in:
 					batch = append(batch, e)
 					if len(batch) >= s.batchSize {
-						flush()
+						flush(end)
 					}
 				default:
-					flush()
+					flush(end)
 					return
 				}
 			}
@@ -104,38 +122,158 @@ func (s *Store) Run(ctx context.Context) {
 	}
 }
 
-func (s *Store) insert(batch []*Entry) error {
+// store sends one batch to VictoriaLogs, or to the spool when VictoriaLogs
+// fails or older batches are still waiting there (to keep their order).
+// Entry.Done is called once the batch is stored or spooled.
+func (s *Store) store(ctx context.Context, batch []*Entry) {
+	body, err := encodeBatch(batch)
+	if err == nil {
+		err = s.save(ctx, body, len(batch))
+	}
+	if err != nil {
+		s.dropped.Add(int64(len(batch)))
+		s.lastErr.Store(err.Error())
+		log.Printf("victorialogs: %d messages dropped: %v", len(batch), err)
+		return
+	}
+	for _, e := range batch {
+		if e.Done != nil {
+			e.Done()
+		}
+	}
+}
+
+func (s *Store) save(ctx context.Context, body []byte, lines int) error {
+	if s.spool == nil {
+		// No disk buffer: retry a few times, then give up.
+		var err error
+		for attempt := 0; attempt < 5; attempt++ {
+			if attempt > 0 && !sleepCtx(ctx, time.Duration(1<<attempt)*500*time.Millisecond) { // 1s, 2s, 4s, 8s
+				break
+			}
+			if err = s.post(ctx, body); err == nil {
+				s.ingested.Add(int64(lines))
+				s.lastErr.Store("")
+				return nil
+			}
+		}
+		return err
+	}
+
+	var postErr error
+	if waiting, _ := s.spool.Pending(); waiting == 0 {
+		if postErr = s.post(ctx, body); postErr == nil {
+			s.ingested.Add(int64(lines))
+			s.lastErr.Store("")
+			return nil
+		}
+		s.lastErr.Store(postErr.Error())
+	}
+	err := s.spool.Write(body, lines)
+	if err == nil {
+		select {
+		case s.spooled <- struct{}{}:
+		default:
+		}
+		return nil
+	}
+	if postErr != nil {
+		return fmt.Errorf("%v; %v", postErr, err)
+	}
+	// Spool full while older batches wait: one direct attempt before dropping.
+	if postErr = s.post(ctx, body); postErr == nil {
+		s.ingested.Add(int64(lines))
+		return nil
+	}
+	return fmt.Errorf("%v; %v", err, postErr)
+}
+
+// replay sends the spooled batches back to VictoriaLogs, oldest first, with
+// an increasing pause (up to 30 s) while it keeps failing.
+func (s *Store) replay(ctx context.Context) {
+	if n, size := s.spool.Pending(); n > 0 {
+		log.Printf("spool: %d messages (%d bytes) waiting from a previous run", n, size)
+	}
+	backoff := time.Second
+	for {
+		f, body, ok, err := s.spool.Oldest()
+		if err != nil {
+			log.Printf("spool: %v", err)
+		}
+		if !ok {
+			select {
+			case <-ctx.Done():
+				return
+			case <-s.spooled:
+			case <-time.After(time.Minute):
+			}
+			continue
+		}
+		err = s.post(ctx, body)
+		switch {
+		case err == nil:
+			s.spool.Remove(f)
+			s.ingested.Add(f.lines)
+			s.lastErr.Store("")
+			backoff = time.Second
+			continue
+		case isRejected(err):
+			// VictoriaLogs refuses the data itself: sending it again would not help.
+			s.spool.Remove(f)
+			s.dropped.Add(f.lines)
+			log.Printf("spool: %d messages refused by victorialogs: %v", f.lines, err)
+			continue
+		}
+		s.lastErr.Store(err.Error())
+		if !sleepCtx(ctx, backoff) {
+			return
+		}
+		backoff = min(2*backoff, 30*time.Second)
+	}
+}
+
+func encodeBatch(batch []*Entry) ([]byte, error) {
 	var buf bytes.Buffer
 	enc := json.NewEncoder(&buf)
 	enc.SetEscapeHTML(false)
 	for _, e := range batch {
 		if err := enc.Encode(e.Record()); err != nil {
-			return err
+			return nil, err
 		}
 	}
-	u := s.base + "/insert/jsonline?_stream_fields=host,app&_msg_field=_msg&_time_field=_time"
+	return buf.Bytes(), nil
-
-	var err error
-	for attempt := 0; attempt < 5; attempt++ {
-		if attempt > 0 {
-			time.Sleep(time.Duration(1<<attempt) * 500 * time.Millisecond) // 1s, 2s, 4s, 8s
-		}
-		if err = s.post(u, buf.Bytes()); err == nil {
-			return nil
-		}
-	}
-	return err
 }
 
-func (s *Store) post(u string, body []byte) error {
+// insertError is an error status of VictoriaLogs to an insert.
-	resp, err := s.client.Post(u, "application/stream+json", bytes.NewReader(body))
+type insertError struct {
+	status int
+	msg    string
+}
+
+func (e *insertError) Error() string { return fmt.Sprintf("HTTP %d: %s", e.status, e.msg) }
+
+// isRejected tells whether VictoriaLogs refused the data itself (4xx other
+// than 429), as opposed to being unreachable or overloaded.
+func isRejected(err error) bool {
+	var ie *insertError
+	return errors.As(err, &ie) && ie.status >= 400 && ie.status < 500 && ie.status != http.StatusTooManyRequests
+}
+
+func (s *Store) post(ctx context.Context, body []byte) error {
+	u := s.base + "/insert/jsonline?_stream_fields=host,app&_msg_field=_msg&_time_field=_time"
+	req, err := http.NewRequestWithContext(ctx, http.MethodPost, u, bytes.NewReader(body))
+	if err != nil {
+		return err
+	}
+	req.Header.Set("Content-Type", "application/stream+json")
+	resp, err := s.client.Do(req)
 	if err != nil {
 		return err
 	}
 	defer resp.Body.Close()
 	if resp.StatusCode/100 != 2 {
 		msg, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
-		return fmt.Errorf("HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(msg)))
+		return &insertError{status: resp.StatusCode, msg: strings.TrimSpace(string(msg))}
 	}
 	_, _ = io.Copy(io.Discard, resp.Body)
 	return nil
@@ -265,11 +403,15 @@ func queryStatus(err error) int {
 }
 
 func (s *Store) Stats() map[string]any {
-	return map[string]any{
+	st := map[string]any{
 		"received":  s.received.Load(),
 		"ingested":  s.ingested.Load(),
 		"dropped":   s.dropped.Load(),
 		"queue":     len(s.in),
 		"lastError": s.lastErr.Load(),
 	}
+	if s.spool != nil {
+		st["spooled"], st["spoolBytes"] = s.spool.Pending()
+	}
+	return st
 }

store_test.go

@@ -0,0 +1,216 @@
+package main
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"io"
+	"net/http"
+	"os"
+	"path/filepath"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+// insertVL plays VictoriaLogs' insert endpoint: it answers `status` (0 = the
+// connection fails) and keeps the lines it accepted.
+type insertVL struct {
+	mu     sync.Mutex
+	status int
+	lines  []string
+}
+
+func (v *insertVL) set(status int) {
+	v.mu.Lock()
+	v.status = status
+	v.mu.Unlock()
+}
+
+func (v *insertVL) got() []string {
+	v.mu.Lock()
+	defer v.mu.Unlock()
+	return append([]string(nil), v.lines...)
+}
+
+func (v *insertVL) RoundTrip(r *http.Request) (*http.Response, error) {
+	body, _ := io.ReadAll(r.Body)
+	v.mu.Lock()
+	defer v.mu.Unlock()
+	if v.status == 0 {
+		return nil, errors.New("connection refused")
+	}
+	if v.status == http.StatusOK {
+		for _, l := range strings.Split(strings.TrimSpace(string(body)), "\n") {
+			v.lines = append(v.lines, l)
+		}
+	}
+	return &http.Response{StatusCode: v.status, Body: io.NopCloser(strings.NewReader("")), Header: http.Header{}}, nil
+}
+
+func testEntry(msg string, done *atomic.Int64) *Entry {
+	e := &Entry{Received: time.Now(), Time: time.Now(), Host: "h", App: "a", Message: msg}
+	if done != nil {
+		e.Done = func() { done.Add(1) }
+	}
+	return e
+}
+
+func waitFor(t *testing.T, what string, cond func() bool) {
+	t.Helper()
+	for deadline := time.Now().Add(5 * time.Second); time.Now().Before(deadline); time.Sleep(10 * time.Millisecond) {
+		if cond() {
+			return
+		}
+	}
+	t.Fatalf("timed out waiting for %s", what)
+}
+
+func TestSpoolFiles(t *testing.T) {
+	dir := t.TempDir()
+	sp, err := OpenSpool(dir, 100)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := sp.Write([]byte("a\nb\n"), 2); err != nil {
+		t.Fatal(err)
+	}
+	if err := sp.Write([]byte("c\n"), 1); err != nil {
+		t.Fatal(err)
+	}
+	if err := sp.Write(bytes.Repeat([]byte("x"), 100), 1); !errors.Is(err, errSpoolFull) {
+		t.Fatalf("write over the limit: %v, want errSpoolFull", err)
+	}
+	_ = os.WriteFile(filepath.Join(dir, "broken.ndjson.tmp"), []byte("z"), 0o644)
+
+	// A new run finds the batches left on disk and drops unfinished writes.
+	sp, err = OpenSpool(dir, 100)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if n, size := sp.Pending(); n != 3 || size != 6 {
+		t.Fatalf("pending %d lines %d bytes, want 3 and 6", n, size)
+	}
+	f, body, ok, err := sp.Oldest()
+	if !ok || err != nil || string(body) != "a\nb\n" || f.lines != 2 {
+		t.Fatalf("oldest: %q %+v %v %v", body, f, ok, err)
+	}
+	sp.Remove(f)
+	if _, body, _, _ := sp.Oldest(); string(body) != "c\n" {
+		t.Fatalf("next oldest %q", body)
+	}
+	if _, err := os.Stat(filepath.Join(dir, "broken.ndjson.tmp")); !os.IsNotExist(err) {
+		t.Error("unfinished write left in the spool")
+	}
+}
+
+func TestStoreSpoolsWhileVictoriaLogsIsDown(t *testing.T) {
+	sp, err := OpenSpool(t.TempDir(), 1<<20)
+	if err != nil {
+		t.Fatal(err)
+	}
+	vl := &insertVL{status: 0}
+	s := NewStore("http://vl", 2, 100, 20*time.Millisecond, sp)
+	s.client.Transport = vl
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	go s.Run(ctx)
+
+	var done atomic.Int64
+	for _, m := range []string{"one", "two", "three"} {
+		s.Enqueue(testEntry(m, &done))
+	}
+	// VictoriaLogs is down: the messages are kept on disk, and acknowledged.
+	waitFor(t, "3 spooled messages", func() bool { n, _ := sp.Pending(); return n == 3 })
+	if done.Load() != 3 || s.dropped.Load() != 0 {
+		t.Fatalf("done %d dropped %d, want 3 and 0", done.Load(), s.dropped.Load())
+	}
+	// While batches wait on disk, new ones queue behind them.
+	vl.set(http.StatusServiceUnavailable)
+	s.Enqueue(testEntry("four", &done))
+	waitFor(t, "4 spooled messages", func() bool { n, _ := sp.Pending(); return n == 4 })
+
+	vl.set(http.StatusOK)
+	waitFor(t, "the spool to drain", func() bool { n, _ := sp.Pending(); return n == 0 })
+	got := strings.Join(vl.got(), "\n")
+	for i, m := range []string{"one", "two", "three", "four"} {
+		if !strings.Contains(got, `"_msg":"`+m+`"`) {
+			t.Errorf("message %d %q not sent: %s", i, m, got)
+		}
+	}
+	if strings.Index(got, `"one"`) > strings.Index(got, `"four"`) {
+		t.Error("spooled batches sent out of order")
+	}
+	if s.ingested.Load() != 4 || s.lastErr.Load() != "" {
+		t.Errorf("ingested %d lastErr %q", s.ingested.Load(), s.lastErr.Load())
+	}
+
+	// Once the spool is empty, batches go straight to VictoriaLogs again.
+	s.Enqueue(testEntry("five", &done))
+	waitFor(t, "a direct insert", func() bool { return s.ingested.Load() == 5 })
+}
+
+func TestStoreDropsRefusedSpooledBatch(t *testing.T) {
+	sp, _ := OpenSpool(t.TempDir(), 1<<20)
+	_ = sp.Write([]byte("{bad json\n"), 1)
+	vl := &insertVL{status: http.StatusBadRequest}
+	s := NewStore("http://vl", 10, 10, time.Second, sp)
+	s.client.Transport = vl
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	go s.Run(ctx)
+	waitFor(t, "the refused batch to be dropped", func() bool { return s.dropped.Load() == 1 })
+	if n, _ := sp.Pending(); n != 0 {
+		t.Errorf("%d messages left in the spool", n)
+	}
+}
+
+func TestStoreWithoutSpoolDrops(t *testing.T) {
+	vl := &insertVL{status: 0}
+	s := NewStore("http://vl", 10, 10, time.Hour, nil)
+	s.client.Transport = vl
+	ctx, cancel := context.WithCancel(context.Background())
+	cancel() // shutdown: one attempt, no retry pauses
+	var done atomic.Int64
+	s.store(ctx, []*Entry{testEntry("x", &done)})
+	if done.Load() != 0 || s.dropped.Load() != 1 {
+		t.Errorf("done %d dropped %d, want 0 and 1", done.Load(), s.dropped.Load())
+	}
+}
+
+func TestEnqueueWait(t *testing.T) {
+	s := NewStore("http://vl", 10, 1, time.Hour, nil)
+	s.Enqueue(testEntry("fills the queue", nil))
+	s.Enqueue(testEntry("syslog: dropped", nil))
+	if s.dropped.Load() != 1 {
+		t.Fatalf("dropped %d, want 1", s.dropped.Load())
+	}
+	// A producer that can wait blocks until there is room…
+	queued := make(chan struct{})
+	go func() {
+		e := testEntry("docker: waits", nil)
+		e.Wait = true
+		s.Enqueue(e)
+		close(queued)
+	}()
+	select {
+	case <-queued:
+		t.Fatal("did not wait for room in the queue")
+	case <-time.After(50 * time.Millisecond):
+	}
+	<-s.in
+	<-queued
+	if s.dropped.Load() != 1 {
+		t.Errorf("dropped %d, want 1", s.dropped.Load())
+	}
+	// …or until shutdown.
+	close(s.quit)
+	e := testEntry("docker: shutdown", nil)
+	e.Wait = true
+	s.Enqueue(e)
+	if s.dropped.Load() != 2 {
+		t.Errorf("dropped %d after shutdown, want 2", s.dropped.Load())
+	}
+}

syslog.go

@@ -38,6 +38,12 @@ type Entry struct {
 
 	SourceType string            // "syslog" or "docker"
 	Extra      map[string]string // additional fields (docker: container, image, …)
+
+	// Not stored. Wait: the producer can be slowed down when the queue is full
+	// (Docker, host logs) instead of losing the message. Done: called once the
+	// message is stored in VictoriaLogs or in the disk spool.
+	Wait bool
+	Done func()
 }
 
 // Record returns the entry in the shape stored in VictoriaLogs and returned by the API.
@@ -241,7 +247,15 @@ func serveUDP(ctx context.Context, pc net.PacketConn, sink func(*Entry)) {
 	}
 }
 
+// TCP limits: connections open at once, and how long a connection may stay
+// silent before it is closed (senders reconnect on their own).
+var (
+	tcpMaxConns = 512
+	tcpIdle     = 30 * time.Minute
+)
+
 func serveTCP(ctx context.Context, ln net.Listener, sink func(*Entry)) {
+	slots := make(chan struct{}, tcpMaxConns)
 	for {
 		conn, err := ln.Accept()
 		if err != nil {
@@ -252,8 +266,30 @@ func serveTCP(ctx context.Context, ln net.Listener, sink func(*Entry)) {
 			time.Sleep(100 * time.Millisecond)
 			continue
 		}
-		go handleTCP(ctx, conn, sink)
+		select {
+		case slots <- struct{}{}:
+		default:
+			log.Printf("syslog tcp: %d connections already open, refusing %s", tcpMaxConns, conn.RemoteAddr())
+			conn.Close()
+			continue
 		}
+		go func() {
+			defer func() { <-slots }()
+			handleTCP(ctx, conn, sink)
+		}()
+	}
+}
+
+// idleConn pushes the read deadline back before each read, so only a
+// connection that stays silent for tcpIdle is closed.
+type idleConn struct {
+	net.Conn
+	idle time.Duration
+}
+
+func (c idleConn) Read(p []byte) (int, error) {
+	_ = c.Conn.SetReadDeadline(time.Now().Add(c.idle))
+	return c.Conn.Read(p)
 }
 
 const maxFrame = 1 << 20
@@ -266,7 +302,7 @@ func handleTCP(ctx context.Context, conn net.Conn, sink func(*Entry)) {
 	defer stop()
 
 	src := hostOf(conn.RemoteAddr())
-	r := bufio.NewReaderSize(conn, 64*1024)
+	r := bufio.NewReaderSize(idleConn{conn, tcpIdle}, 64*1024)
 	for {
 		c, err := r.ReadByte()
 		if err != nil {

tags.go

@@ -16,7 +16,9 @@ import (
 // Tag highlights a keyword in displayed messages.
 type Tag struct {
 	ID            string `json:"id"`
+	Code          string `json:"code"` // two digits, shown on matching log lines
 	Pattern       string `json:"pattern"`
+	Label         string `json:"label,omitempty"` // shown instead of the pattern (presets)
 	Color         string `json:"color"`
 	WholeWord     bool   `json:"wholeWord"`
 	CaseSensitive bool   `json:"caseSensitive"`
@@ -26,12 +28,46 @@ type Tag struct {
 
 func defaultTags() []Tag {
 	return []Tag{
-		{ID: "warning", Pattern: "warning", Color: "#fdba74", WholeWord: true, Enabled: true},
+		{ID: "warning", Code: "01", Pattern: "warning", Color: "#fdba74", WholeWord: true, Enabled: true},
-		{ID: "error", Pattern: "error", Color: "#fca5a5", WholeWord: true, Enabled: true},
+		{ID: "error", Code: "02", Pattern: "error", Color: "#fca5a5", WholeWord: true, Enabled: true},
-		{ID: "ok", Pattern: "ok", Color: "#86efac", WholeWord: true, Enabled: true},
 	}
 }
 
+var codeRe = regexp.MustCompile(`^[0-9]{2}$`)
+
+// freeCode returns the lowest code from 01 to 99 not used by tags, or "" when
+// all are taken. A code stays with its tag until the tag is deleted.
+func freeCode(tags []Tag) string {
+	used := map[string]bool{}
+	for _, t := range tags {
+		used[t.Code] = true
+	}
+	for n := 1; n <= 99; n++ {
+		if c := fmt.Sprintf("%02d", n); !used[c] {
+			return c
+		}
+	}
+	return ""
+}
+
+// assignCodes gives a code to the tags that have none (files written before
+// codes existed) or share one with an earlier tag.
+func assignCodes(tags []Tag) bool {
+	changed := false
+	seen := map[string]bool{}
+	for i := range tags {
+		if codeRe.MatchString(tags[i].Code) && !seen[tags[i].Code] {
+			seen[tags[i].Code] = true
+			continue
+		}
+		tags[i].Code = ""
+		tags[i].Code = freeCode(tags)
+		seen[tags[i].Code] = true
+		changed = true
+	}
+	return changed
+}
+
 // Colors of the default tags in earlier versions: still unchanged, they are
 // switched to the new pastel defaults when the file is loaded.
 var oldDefaultColors = map[string]string{"warning": "#f59e0b", "error": "#ef4444", "ok": "#22c55e"}
@@ -67,11 +103,13 @@ func (e *codedError) Error() string {
 
 var (
 	errTagNotFound = &codedError{code: "tag_not_found", msg: "tag not found"}
+	errTooManyTags = &codedError{code: "too_many_tags", msg: "too many tags (99 at most)"}
 	colorRe        = regexp.MustCompile(`^#[0-9a-fA-F]{6}$`)
 )
 
 func (t *Tag) validate() error {
 	t.Pattern = strings.TrimSpace(t.Pattern)
+	t.Label = strings.TrimSpace(t.Label)
 	if t.Pattern == "" {
 		return &codedError{code: "pattern_required", msg: "the keyword is required"}
 	}
@@ -106,7 +144,7 @@ func LoadTagStore(path string) (*TagStore, error) {
 	if err := json.Unmarshal(b, &s.tags); err != nil {
 		return nil, fmt.Errorf("%s: %w", path, err)
 	}
-	if migrateDefaultColors(s.tags) {
+	if c1, c2 := migrateDefaultColors(s.tags), assignCodes(s.tags); c1 || c2 {
 		if err := s.save(); err != nil {
 			return nil, err
 		}
@@ -143,6 +181,9 @@ func (s *TagStore) Create(t Tag) (Tag, error) {
 	t.ID = newID()
 	s.mu.Lock()
 	defer s.mu.Unlock()
+	if t.Code = freeCode(s.tags); t.Code == "" {
+		return t, errTooManyTags
+	}
 	s.tags = append(s.tags, t)
 	return t, s.save()
 }
@@ -156,6 +197,7 @@ func (s *TagStore) Update(id string, t Tag) (Tag, error) {
 	defer s.mu.Unlock()
 	for i := range s.tags {
 		if s.tags[i].ID == id {
+			t.Code = s.tags[i].Code // assigned by the server, never changed
 			s.tags[i] = t
 			return t, s.save()
 		}

tags_test.go

@@ -0,0 +1,54 @@
+package main
+
+import (
+	"os"
+	"path/filepath"
+	"slices"
+	"testing"
+)
+
+func TestTagCodes(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "tags.json")
+	// File written before codes existed, with a duplicate code.
+	old := `[{"id":"a","pattern":"x","color":"#000000"},{"id":"b","code":"07","pattern":"y","color":"#000000"},{"id":"c","code":"07","pattern":"z","color":"#000000"}]`
+	if err := os.WriteFile(path, []byte(old), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	s, err := LoadTagStore(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	got := []string{}
+	for _, tg := range s.List() {
+		got = append(got, tg.Code)
+	}
+	if want := []string{"01", "07", "02"}; !slices.Equal(got, want) {
+		t.Fatalf("migrated codes = %v, want %v", got, want)
+	}
+
+	n, err := s.Create(Tag{Pattern: "w", Color: "#000000"})
+	if err != nil || n.Code != "03" {
+		t.Fatalf("Create code = %q, %v; want 03", n.Code, err)
+	}
+	// The client cannot change a code.
+	u, err := s.Update("b", Tag{Code: "42", Pattern: "y2", Color: "#000000"})
+	if err != nil || u.Code != "07" {
+		t.Fatalf("Update code = %q, %v; want 07", u.Code, err)
+	}
+	// A deleted tag frees its code; the others keep theirs.
+	if err := s.Delete("a"); err != nil {
+		t.Fatal(err)
+	}
+	if n, _ := s.Create(Tag{Pattern: "v", Color: "#000000"}); n.Code != "01" {
+		t.Fatalf("code after delete = %q, want 01", n.Code)
+	}
+
+	// Codes are saved in the file.
+	s2, err := LoadTagStore(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got := s2.List()[0].Code; got != "07" {
+		t.Fatalf("reloaded code = %q, want 07", got)
+	}
+}

web/fonts/OFL-Inconsolata.txt

@@ -0,0 +1,93 @@
+Copyright 2006 The Inconsolata Project Authors (https://github.com/cyrealtype/Inconsolata)
+
+This Font Software is licensed under the SIL Open Font License, Version 1.1.
+This license is copied below, and is also available with a FAQ at:
+http://scripts.sil.org/OFL
+
+
+-----------------------------------------------------------
+SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
+-----------------------------------------------------------
+
+PREAMBLE
+The goals of the Open Font License (OFL) are to stimulate worldwide
+development of collaborative font projects, to support the font creation
+efforts of academic and linguistic communities, and to provide a free and
+open framework in which fonts may be shared and improved in partnership
+with others.
+
+The OFL allows the licensed fonts to be used, studied, modified and
+redistributed freely as long as they are not sold by themselves. The
+fonts, including any derivative works, can be bundled, embedded,
+redistributed and/or sold with any software provided that any reserved
+names are not used by derivative works. The fonts and derivatives,
+however, cannot be released under any other type of license. The
+requirement for fonts to remain under this license does not apply
+to any document created using the fonts or their derivatives.
+
+DEFINITIONS
+"Font Software" refers to the set of files released by the Copyright
+Holder(s) under this license and clearly marked as such. This may
+include source files, build scripts and documentation.
+
+"Reserved Font Name" refers to any names specified as such after the
+copyright statement(s).
+
+"Original Version" refers to the collection of Font Software components as
+distributed by the Copyright Holder(s).
+
+"Modified Version" refers to any derivative made by adding to, deleting,
+or substituting -- in part or in whole -- any of the components of the
+Original Version, by changing formats or by porting the Font Software to a
+new environment.
+
+"Author" refers to any designer, engineer, programmer, technical
+writer or other person who contributed to the Font Software.
+
+PERMISSION & CONDITIONS
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of the Font Software, to use, study, copy, merge, embed, modify,
+redistribute, and sell modified and unmodified copies of the Font
+Software, subject to the following conditions:
+
+1) Neither the Font Software nor any of its individual components,
+in Original or Modified Versions, may be sold by itself.
+
+2) Original or Modified Versions of the Font Software may be bundled,
+redistributed and/or sold with any software, provided that each copy
+contains the above copyright notice and this license. These can be
+included either as stand-alone text files, human-readable headers or
+in the appropriate machine-readable metadata fields within text or
+binary files as long as those fields can be easily viewed by the user.
+
+3) No Modified Version of the Font Software may use the Reserved Font
+Name(s) unless explicit written permission is granted by the corresponding
+Copyright Holder. This restriction only applies to the primary font name as
+presented to the users.
+
+4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
+Software shall not be used to promote, endorse or advertise any
+Modified Version, except to acknowledge the contribution(s) of the
+Copyright Holder(s) and the Author(s) or with their explicit written
+permission.
+
+5) The Font Software, modified or unmodified, in part or in whole,
+must be distributed entirely under this license, and must not be
+distributed under any other license. The requirement for fonts to
+remain under this license does not apply to any document created
+using the Font Software.
+
+TERMINATION
+This license becomes null and void if any of the above conditions are
+not met.
+
+DISCLAIMER
+THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
+OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
+DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
+OTHER DEALINGS IN THE FONT SOFTWARE.

web/fonts/OFL-Iosevka.txt

@@ -0,0 +1,110 @@
+Copyright (c) 2015-2023, Renzhi Li (aka. Belleve Invis, belleve@typeof.net)
+
+This Font Software is licensed under the SIL Open Font License, Version 1.1.
+
+This license is copied below, and is also available with a FAQ at:
+http://scripts.sil.org/OFL
+
+--------------------------
+
+
+SIL Open Font License v1.1
+====================================================
+
+
+Preamble
+----------
+
+The goals of the Open Font License (OFL) are to stimulate worldwide
+development of collaborative font projects, to support the font creation
+efforts of academic and linguistic communities, and to provide a free and
+open framework in which fonts may be shared and improved in partnership
+with others.
+
+The OFL allows the licensed fonts to be used, studied, modified and
+redistributed freely as long as they are not sold by themselves. The
+fonts, including any derivative works, can be bundled, embedded, 
+redistributed and/or sold with any software provided that any reserved
+names are not used by derivative works. The fonts and derivatives,
+however, cannot be released under any other type of license. The
+requirement for fonts to remain under this license does not apply
+to any document created using the fonts or their derivatives.
+
+
+Definitions
+-------------
+
+`"Font Software"` refers to the set of files released by the Copyright
+Holder(s) under this license and clearly marked as such. This may
+include source files, build scripts and documentation.
+
+`"Reserved Font Name"` refers to any names specified as such after the
+copyright statement(s).
+
+`"Original Version"` refers to the collection of Font Software components as
+distributed by the Copyright Holder(s).
+
+`"Modified Version"` refers to any derivative made by adding to, deleting,
+or substituting -- in part or in whole -- any of the components of the
+Original Version, by changing formats or by porting the Font Software to a
+new environment.
+
+`"Author"` refers to any designer, engineer, programmer, technical
+writer or other person who contributed to the Font Software.
+
+
+Permission & Conditions
+------------------------
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of the Font Software, to use, study, copy, merge, embed, modify,
+redistribute, and sell modified and unmodified copies of the Font
+Software, subject to the following conditions:
+
+1. Neither the Font Software nor any of its individual components,
+   in Original or Modified Versions, may be sold by itself.
+
+2. Original or Modified Versions of the Font Software may be bundled,
+   redistributed and/or sold with any software, provided that each copy
+   contains the above copyright notice and this license. These can be
+   included either as stand-alone text files, human-readable headers or
+   in the appropriate machine-readable metadata fields within text or
+   binary files as long as those fields can be easily viewed by the user.
+
+3. No Modified Version of the Font Software may use the Reserved Font
+   Name(s) unless explicit written permission is granted by the corresponding
+   Copyright Holder. This restriction only applies to the primary font name as
+   presented to the users.
+
+4. The name(s) of the Copyright Holder(s) or the Author(s) of the Font
+   Software shall not be used to promote, endorse or advertise any
+   Modified Version, except to acknowledge the contribution(s) of the
+   Copyright Holder(s) and the Author(s) or with their explicit written
+   permission.
+
+5. The Font Software, modified or unmodified, in part or in whole,
+   must be distributed entirely under this license, and must not be
+   distributed under any other license. The requirement for fonts to
+   remain under this license does not apply to any document created
+   using the Font Software.
+
+
+
+Termination
+-----------
+
+This license becomes null and void if any of the above conditions are
+not met.
+
+
+    DISCLAIMER
+    
+    THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+    EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
+    MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
+    OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
+    COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+    INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
+    DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+    FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
+    OTHER DEALINGS IN THE FONT SOFTWARE.

web/fonts/UFL-Ubuntu-Mono.txt

@@ -0,0 +1,96 @@
+-------------------------------
+UBUNTU FONT LICENCE Version 1.0
+-------------------------------
+
+PREAMBLE
+This licence allows the licensed fonts to be used, studied, modified and
+redistributed freely. The fonts, including any derivative works, can be
+bundled, embedded, and redistributed provided the terms of this licence
+are met. The fonts and derivatives, however, cannot be released under
+any other licence. The requirement for fonts to remain under this
+licence does not require any document created using the fonts or their
+derivatives to be published under this licence, as long as the primary
+purpose of the document is not to be a vehicle for the distribution of
+the fonts.
+
+DEFINITIONS
+"Font Software" refers to the set of files released by the Copyright
+Holder(s) under this licence and clearly marked as such. This may
+include source files, build scripts and documentation.
+
+"Original Version" refers to the collection of Font Software components
+as received under this licence.
+
+"Modified Version" refers to any derivative made by adding to, deleting,
+or substituting -- in part or in whole -- any of the components of the
+Original Version, by changing formats or by porting the Font Software to
+a new environment.
+
+"Copyright Holder(s)" refers to all individuals and companies who have a
+copyright ownership of the Font Software.
+
+"Substantially Changed" refers to Modified Versions which can be easily
+identified as dissimilar to the Font Software by users of the Font
+Software comparing the Original Version with the Modified Version.
+
+To "Propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification and with or without charging
+a redistribution fee), making available to the public, and in some
+countries other activities as well.
+
+PERMISSION & CONDITIONS
+This licence does not grant any rights under trademark law and all such
+rights are reserved.
+
+Permission is hereby granted, free of charge, to any person obtaining a
+copy of the Font Software, to propagate the Font Software, subject to
+the below conditions:
+
+1) Each copy of the Font Software must contain the above copyright
+notice and this licence. These can be included either as stand-alone
+text files, human-readable headers or in the appropriate machine-
+readable metadata fields within text or binary files as long as those
+fields can be easily viewed by the user.
+
+2) The font name complies with the following:
+(a) The Original Version must retain its name, unmodified.
+(b) Modified Versions which are Substantially Changed must be renamed to
+avoid use of the name of the Original Version or similar names entirely.
+(c) Modified Versions which are not Substantially Changed must be
+renamed to both (i) retain the name of the Original Version and (ii) add
+additional naming elements to distinguish the Modified Version from the
+Original Version. The name of such Modified Versions must be the name of
+the Original Version, with "derivative X" where X represents the name of
+the new work, appended to that name.
+
+3) The name(s) of the Copyright Holder(s) and any contributor to the
+Font Software shall not be used to promote, endorse or advertise any
+Modified Version, except (i) as required by this licence, (ii) to
+acknowledge the contribution(s) of the Copyright Holder(s) or (iii) with
+their explicit written permission.
+
+4) The Font Software, modified or unmodified, in part or in whole, must
+be distributed entirely under this licence, and must not be distributed
+under any other licence. The requirement for fonts to remain under this
+licence does not affect any document created using the Font Software,
+except any version of the Font Software extracted from a document
+created using the Font Software may only be distributed under this
+licence.
+
+TERMINATION
+This licence becomes null and void if any of the above conditions are
+not met.
+
+DISCLAIMER
+THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
+COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
+DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER
+DEALINGS IN THE FONT SOFTWARE.

web/index.html

@@ -3,7 +3,7 @@
 <head>
   <meta charset="utf-8">
   <meta name="viewport" content="width=device-width, initial-scale=1">
-  <title>Logstream</title>
+  <title>LogStream</title>
   <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='8' fill='%236366f1'/%3E%3Cpath d='M8 10h16M8 16h11M8 22h14' stroke='white' stroke-width='3' stroke-linecap='round'/%3E%3C/svg%3E">
   <link rel="stylesheet" href="style.css">
   <script>
@@ -20,7 +20,7 @@
   <header class="topbar">
     <div class="brand">
       <svg viewBox="0 0 32 32" aria-hidden="true"><rect width="32" height="32" rx="8"/><path d="M8 10h16M8 16h11M8 22h14"/></svg>
-      <span>Logstream</span>
+      <span>LogStream</span>
     </div>
 
     <div class="search">
@@ -41,11 +41,42 @@
         <svg class="sun" viewBox="0 0 24 24"><circle cx="12" cy="12" r="4"/><path d="M12 2v2M12 20v2M4.9 4.9l1.4 1.4M17.7 17.7l1.4 1.4M2 12h2M20 12h2M4.9 19.1l1.4-1.4M17.7 6.3l1.4-1.4"/></svg>
         <svg class="moon" viewBox="0 0 24 24"><path d="M21 12.8A9 9 0 1 1 11.2 3a7 7 0 0 0 9.8 9.8z"/></svg>
       </button>
+      <a id="logoutBtn" class="icon-btn" href="/auth/logout" hidden data-i18n-title="logout" data-i18n-aria="logout">
+        <!-- Log out icon (Lucide "log-out", ISC license) -->
+        <svg viewBox="0 0 24 24"><path d="M9 21H5a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h4"/><path d="m16 17 5-5-5-5"/><path d="M21 12H9"/></svg>
+      </a>
     </div>
     <div class="progress" aria-hidden="true"></div>
   </header>
 
   <section class="filters">
+    <div id="viewMode" class="seg view-mode" role="radiogroup" data-i18n-aria="viewAria">
+      <button type="button" role="radio" data-view="stream" data-i18n-title="viewStreamTitle">
+        <!-- Lucide "radio" (ISC license) -->
+        <svg viewBox="0 0 24 24" aria-hidden="true"><path d="M4.9 19.1C1 15.2 1 8.8 4.9 4.9"/><path d="M7.8 16.2c-2.3-2.3-2.3-6.1 0-8.5"/><circle cx="12" cy="12" r="2"/><path d="M16.2 7.8c2.3 2.3 2.3 6.1 0 8.5"/><path d="M19.1 4.9C23 8.8 23 15.1 19.1 19"/></svg>
+        <span data-i18n="viewStream">Stream</span>
+      </button>
+      <button type="button" role="radio" data-view="period" data-i18n-title="viewPeriodTitle">
+        <!-- Lucide "calendar-range" (ISC license) -->
+        <svg viewBox="0 0 24 24" aria-hidden="true"><rect width="18" height="18" x="3" y="4" rx="2"/><path d="M16 2v4M3 10h18M8 2v4M17 14h-6M13 18H7M7 14h.01M17 18h.01"/></svg>
+        <span data-i18n="viewPeriod">Time range</span>
+      </button>
+    </div>
+    <div id="period" class="period" hidden>
+      <button id="perPrev" class="btn tool" type="button" data-i18n-title="perPrev" data-i18n-aria="perPrev">
+        <svg viewBox="0 0 24 24" aria-hidden="true"><path d="m15 18-6-6 6-6"/></svg>
+      </button>
+      <input id="perFrom" type="datetime-local" data-i18n-title="perFrom" data-i18n-aria="perFrom">
+      <span class="muted" aria-hidden="true">→</span>
+      <input id="perTo" type="datetime-local" data-i18n-title="perTo" data-i18n-aria="perTo">
+      <button id="perNext" class="btn tool" type="button" data-i18n-title="perNext" data-i18n-aria="perNext">
+        <svg viewBox="0 0 24 24" aria-hidden="true"><path d="m9 18 6-6-6-6"/></svg>
+      </button>
+      <button id="perOut" class="btn tool" type="button" data-i18n-title="perOut" data-i18n-aria="perOut">
+        <!-- Lucide "zoom-out" (ISC license) -->
+        <svg viewBox="0 0 24 24" aria-hidden="true"><circle cx="11" cy="11" r="8"/><path d="m21 21-4.3-4.3M8 11h6"/></svg>
+      </button>
+    </div>
     <select id="range" data-i18n-aria="rangeAria">
       <option value="5m" data-i18n="r5m">5 min</option>
       <option value="15m" data-i18n="r15m">15 min</option>
@@ -70,6 +101,7 @@
       <option value="" data-i18n="srcAll">All sources</option>
       <option value="syslog">Syslog</option>
       <option value="docker">Docker</option>
+      <option value="host" data-i18n="srcHost">Host system</option>
     </select>
     <span class="spacer"></span>
     <span id="count" class="muted"></span>
@@ -92,10 +124,27 @@
 
   <section id="histo" class="histo" data-i18n-aria="histoAria" hidden></section>
 
+  <div id="authWarn" class="auth-warn" role="status" hidden>
+    <span data-i18n="authOff"></span>
+    <button id="authWarnClose" class="icon-btn" type="button" data-i18n-aria="close">
+      <svg viewBox="0 0 24 24"><path d="M6 6l12 12M18 6 6 18"/></svg>
+    </button>
+  </div>
   <div id="error" class="error-banner" hidden></div>
   <button id="newPill" class="pill" type="button" hidden></button>
 
+  <div id="table" class="table" hidden>
+    <div id="listHead" class="list-head" role="presentation">
+      <span data-col="rcv"><span class="lbl" data-i18n="colRcv">Received</span><i class="grip" data-i18n-title="colGrip"></i></span>
+      <span data-col="mt"><span class="lbl" data-i18n="colMt">Message time</span><i class="grip" data-i18n-title="colGrip"></i></span>
+      <span data-col="sev"><span class="lbl" data-i18n="colSev">Severity</span><i class="grip" data-i18n-title="colGrip"></i></span>
+      <span data-col="host"><span class="lbl" data-i18n="colHost">Host</span><i class="grip" data-i18n-title="colGrip"></i></span>
+      <span data-col="app"><span class="lbl" data-i18n="colApp">App</span><i class="grip" data-i18n-title="colGrip"></i></span>
+      <span data-col="codes"><span class="lbl" data-i18n="colCodes" data-i18n-title="codesTitle">Filters</span></span>
+      <span data-col="msg"><span class="lbl" data-i18n="colMsg">Message</span></span>
+    </div>
     <main id="list" class="list"></main>
+  </div>
   <div id="empty" class="empty" hidden>
     <svg viewBox="0 0 24 24" aria-hidden="true"><path d="M4 6h16M4 12h10M4 18h7"/></svg>
     <p><strong data-i18n="emptyTitle">No matching logs.</strong></p>
@@ -105,6 +154,9 @@
   <footer class="status">
     <span id="conn" class="conn"></span>
     <span id="stats" class="muted"></span>
+    <button id="toTop" class="icon-btn to-top" type="button" data-i18n-title="toTop" data-i18n-aria="toTop" hidden>
+      <svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 19V5M5 12l7-7 7 7"/></svg>
+    </button>
   </footer>
 
   <dialog id="settingsDlg" class="settings" aria-labelledby="settingsTitle">
@@ -164,15 +216,22 @@
           </div>
 
           <!-- Filters -->
-          <div class="set-panel" role="tabpanel" data-panel="filters" hidden>
+          <div class="set-panel wide" role="tabpanel" data-panel="filters" hidden>
-            <section class="set-section">
+            <section class="set-section tags-section">
+              <div class="sec-head">
                 <h3 data-i18n="tagsTitle">Color tags</h3>
+                <span id="tagCount" class="muted small"></span>
+              </div>
               <p class="muted small" data-i18n="tagsHelp"></p>
-              <div id="tagList" class="tag-list"></div>
+              <div class="tag-toolbar">
-              <footer>
                 <button id="addTag" class="btn primary" type="button" data-i18n="addTag">+ Add tag</button>
+                <select id="presetTags" class="field" aria-label="Presets" data-i18n-aria="presetAria">
+                  <option value="" data-i18n="presetPick">+ Preset…</option>
+                </select>
+                <input id="tagFilter" type="search" class="dk-filter" autocomplete="off" spellcheck="false" data-i18n-ph="tagFilter" data-i18n-aria="tagFilter">
                 <button id="resetTags" class="btn ghost" type="button" data-i18n="resetTags">Restore default tags</button>
-              </footer>
+              </div>
+              <div id="tagList" class="tag-list"></div>
             </section>
           </div>
 
@@ -197,6 +256,16 @@
             </section>
 
             <section class="set-section">
+              <h3 data-i18n="hostTitle">Host system logs</h3>
+              <p id="hostStatus" class="docker-status"></p>
+              <label class="switch-row">
+                <input id="hostEnabled" type="checkbox" class="switch">
+                <span data-i18n="hostEnabled">Collect the system logs of this machine</span>
+              </label>
+              <p class="muted small" data-i18n="hostHelp"></p>
+            </section>
+
+            <section class="set-section span">
               <h3 data-i18n="dockerTitle">Docker containers</h3>
               <p id="dockerStatus" class="docker-status"></p>
               <div id="dockerBody" hidden>
@@ -233,7 +302,7 @@
               </div>
               <p class="muted small hint" data-i18n="themeHelp"></p>
             </section>
-            <section class="set-section">
+            <section class="set-section span">
               <h3 data-i18n="logDisplay">Log display</h3>
               <div class="field-grid">
                 <span class="lbl" data-i18n="fontSize">Font size</span>
@@ -243,11 +312,20 @@
                   <button type="button" role="radio" data-size="medium" data-i18n="sizeMedium">Medium</button>
                   <button type="button" role="radio" data-size="large" data-i18n="sizeLarge">Large</button>
                 </div>
+                <span class="lbl" data-i18n="density">Density</span>
+                <div id="densitySwitch" class="seg" role="radiogroup" data-i18n-aria="density">
+                  <button type="button" role="radio" data-density="normal" data-i18n="densityNormal">Normal</button>
+                  <button type="button" role="radio" data-density="compact" data-i18n="densityCompact">Compact</button>
+                </div>
                 <label for="fontSelect" data-i18n="fontLabel">Font</label>
                 <select id="fontSelect" class="field"></select>
               </div>
               <div id="fontPreview" class="list preview-list" aria-hidden="true"></div>
               <p class="muted small" data-i18n="fontHelp"></p>
+              <div class="cols-row">
+                <button id="resetCols" class="btn ghost" type="button" data-i18n="resetCols">Reset column widths</button>
+                <span class="muted small" data-i18n="colsHelp"></span>
+              </div>
             </section>
             <section class="set-section">
               <h3 data-i18n="histoTitle">Timeline</h3>
@@ -285,9 +363,14 @@
           </div>
 
           <!-- Data -->
-          <div class="set-panel" role="tabpanel" data-panel="data" hidden>
+          <div class="set-panel wide" role="tabpanel" data-panel="data" hidden>
             <section class="set-section danger">
               <h3 data-i18n="dangerZone">Danger zone</h3>
+              <div class="db-head">
+                <span class="lbl" data-i18n="dbTitle">Database</span>
+                <button id="dbRefresh" class="link-btn" type="button" data-i18n="dbRefresh">Refresh</button>
+              </div>
+              <dl id="dbStats" class="db-stats"></dl>
               <p class="muted small" data-i18n="purgeHelp"></p>
               <div class="purge-row">
                 <button id="purgeBtn" class="btn danger" type="button" data-i18n="purgeBtn">Delete all logs…</button>

web/login.html

@@ -0,0 +1,95 @@
+<!doctype html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>LogStream</title>
+  <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='8' fill='%236366f1'/%3E%3Cpath d='M8 10h16M8 16h11M8 22h14' stroke='white' stroke-width='3' stroke-linecap='round'/%3E%3C/svg%3E">
+  <link rel="stylesheet" href="style.css">
+  <script>
+    // Same saved theme and language as the UI, applied before first paint.
+    try {
+      var t = localStorage.getItem('logstream.theme');
+      if (t === 'light' || t === 'dark') document.documentElement.dataset.theme = t;
+      var l = localStorage.getItem('logstream.lang');
+      if (l) document.documentElement.lang = l;
+    } catch (e) {}
+  </script>
+</head>
+<body class="login-page">
+  <div class="login-tools">
+    <div class="seg" role="radiogroup" id="langSwitch">
+      <button type="button" role="radio" data-lang="fr">FR</button>
+      <button type="button" role="radio" data-lang="en">EN</button>
+    </div>
+    <button id="themeBtn" class="icon-btn" type="button" data-i18n-title="theme" data-i18n-aria="theme">
+      <svg class="sun" viewBox="0 0 24 24"><circle cx="12" cy="12" r="4"/><path d="M12 2v2M12 20v2M4.9 4.9l1.4 1.4M17.7 17.7l1.4 1.4M2 12h2M20 12h2M4.9 19.1l1.4-1.4M17.7 6.3l1.4-1.4"/></svg>
+      <svg class="moon" viewBox="0 0 24 24"><path d="M21 12.8A9 9 0 1 1 11.2 3a7 7 0 0 0 9.8 9.8z"/></svg>
+    </button>
+  </div>
+
+  <main class="login-card">
+    <img id="logo" class="login-logo" src="auth/logo" alt="" hidden>
+    <div class="brand">
+      <svg viewBox="0 0 32 32" aria-hidden="true"><rect width="32" height="32" rx="8"/><path d="M8 10h16M8 16h11M8 22h14"/></svg>
+      <span>LogStream</span>
+    </div>
+    <p class="muted" data-i18n="intro">Sign in to view the logs.</p>
+
+    <form method="post" action="auth/login">
+      <input type="hidden" name="r" id="ret">
+      <label for="user" data-i18n="user">User</label>
+      <input id="user" name="user" type="text" autocomplete="username" autocapitalize="none" spellcheck="false" required autofocus>
+      <label for="pass" data-i18n="pass">Password</label>
+      <input id="pass" name="pass" type="password" autocomplete="current-password" required>
+      <p id="err" class="login-err" role="alert" data-i18n="error" hidden>Wrong user or password.</p>
+      <button class="btn primary" type="submit" data-i18n="submit">Sign in</button>
+    </form>
+  </main>
+
+  <script>
+    (function () {
+      var I18N = {
+        en: { intro: 'Sign in to view the logs.', user: 'User', pass: 'Password', submit: 'Sign in',
+              error: 'Wrong user or password.', theme: 'Light / dark theme' },
+        fr: { intro: 'Connectez-vous pour consulter les logs.', user: 'Utilisateur', pass: 'Mot de passe',
+              submit: 'Se connecter', error: 'Utilisateur ou mot de passe incorrect.', theme: 'Thème clair / sombre' },
+      };
+      function get(k) { try { return localStorage.getItem('logstream.' + k); } catch (e) { return null; } }
+      function set(k, v) { try { localStorage.setItem('logstream.' + k, v); } catch (e) {} }
+      var lang = get('lang');
+      if (!I18N[lang]) lang = /^fr\b/i.test(navigator.language || '') ? 'fr' : 'en';
+
+      function applyLang() {
+        var d = I18N[lang];
+        document.documentElement.lang = lang;
+        document.querySelectorAll('[data-i18n]').forEach(function (el) { el.textContent = d[el.dataset.i18n]; });
+        document.querySelectorAll('[data-i18n-title]').forEach(function (el) { el.title = d[el.dataset.i18nTitle]; });
+        document.querySelectorAll('[data-i18n-aria]').forEach(function (el) { el.setAttribute('aria-label', d[el.dataset.i18nAria]); });
+        document.querySelectorAll('#langSwitch [data-lang]').forEach(function (b) { b.setAttribute('aria-checked', String(b.dataset.lang === lang)); });
+      }
+      document.querySelectorAll('#langSwitch [data-lang]').forEach(function (b) {
+        b.addEventListener('click', function () { lang = b.dataset.lang; set('lang', lang); applyLang(); });
+      });
+
+      document.getElementById('themeBtn').addEventListener('click', function () {
+        var root = document.documentElement;
+        var dark = root.dataset.theme ? root.dataset.theme === 'dark' : matchMedia('(prefers-color-scheme: dark)').matches;
+        root.dataset.theme = dark ? 'light' : 'dark';
+        set('theme', root.dataset.theme);
+      });
+
+      var q = new URLSearchParams(location.search);
+      document.getElementById('ret').value = q.get('r') || '/';
+      document.getElementById('err').hidden = q.get('e') !== '1';
+
+      // LOGIN_LOGO: shown only when the server has one.
+      var logo = document.getElementById('logo');
+      logo.addEventListener('load', function () { logo.hidden = false; });
+      if (logo.complete && logo.naturalWidth) logo.hidden = false;
+
+      applyLang();
+    })();
+  </script>
+</body>
+</html>

web/style.css

@@ -1,3 +1,13 @@
+/* Narrow monospace fonts served by LogStream itself, so they also work offline (licences in
+   web/fonts). Latin subset only; other scripts fall back to --mono. Inconsolata Condensed is
+   Inconsolata's variable font pinned at width 75 (0.4em per character, others 0.5em). */
+@font-face { font-family: "Iosevka"; src: url("fonts/iosevka-400.woff2") format("woff2"); font-weight: 400 500; font-display: swap; }
+@font-face { font-family: "Iosevka"; src: url("fonts/iosevka-700.woff2") format("woff2"); font-weight: 600 800; font-display: swap; }
+@font-face { font-family: "Ubuntu Mono"; src: url("fonts/ubuntu-mono-400.woff2") format("woff2"); font-weight: 400 500; font-display: swap; }
+@font-face { font-family: "Ubuntu Mono"; src: url("fonts/ubuntu-mono-700.woff2") format("woff2"); font-weight: 600 800; font-display: swap; }
+@font-face { font-family: "Inconsolata Condensed"; src: url("fonts/inconsolata-condensed-400.woff2") format("woff2"); font-weight: 400 500; font-display: swap; }
+@font-face { font-family: "Inconsolata Condensed"; src: url("fonts/inconsolata-condensed-700.woff2") format("woff2"); font-weight: 600 800; font-display: swap; }
+
 /* ---------- Theme: everything goes through these variables ---------- */
 :root {
   --bg: #f6f7f9;
@@ -38,6 +48,8 @@
   --tag-warning-text: #111827;
 
   --log-size: 12.5px;   /* Settings > Interface > Font size */
+  --codes-w: 4.9rem;    /* tag codes column: room for 3 badges */
+  --code-bg: #858c97;   /* tag code badges */
   /* --log-font is set by Settings > Interface > Font (defaults to --mono) */
   --mono: ui-monospace, "SF Mono", "JetBrains Mono", "Cascadia Code", Menlo, Consolas, monospace;
   --sans: system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
@@ -228,6 +240,19 @@ body.busy .progress::after {
   font-size: 13px; max-width: 220px;
 }
 .filters select.set { border-color: var(--accent); background-color: var(--accent-soft); }
+/* Display mode: Stream or Time range, then the start → end fields of the range */
+.view-mode { padding: 2px; gap: 2px; border-radius: 9px; }
+.view-mode button { display: inline-flex; align-items: center; gap: 6px; height: 26px; padding: 0 10px; }
+.view-mode svg { width: 15px; height: 15px; }
+.period { display: inline-flex; align-items: center; gap: 6px; }
+.period .btn.tool { width: 32px; padding: 0; justify-content: center; }
+.period input {
+  height: 32px; padding: 0 8px;
+  border: 1px solid var(--accent); border-radius: 8px; background: var(--accent-soft);
+  font-size: 13px; font-variant-numeric: tabular-nums; color-scheme: light dark;
+}
+.period input:focus { outline: 0; box-shadow: 0 0 0 3px var(--accent-soft); }
+.period input.bad { border-color: var(--sev-err); background: color-mix(in srgb, var(--sev-err) 10%, transparent); }
 .spacer { flex: 1; }
 #count { font-size: 12.5px; font-variant-numeric: tabular-nums; }
 
@@ -296,9 +321,42 @@ body.busy .progress::after {
 .list { margin: 6px 20px 0; background: var(--panel); border: 1px solid var(--border); border-radius: var(--radius); overflow: hidden; }
 .list:empty { display: none; }
 
+/* Columns: received, message time, severity, host, app, tag codes, message. Widths come from
+   --col-* (set on #table when a column has been resized, see app.js), shared by
+   the header and every row through subgrid so that the columns line up. */
+.table {
+  display: grid;
+  grid-template-columns: var(--col-rcv, max-content) var(--col-mt, max-content) var(--col-sev, 4.6rem)
+    var(--col-host, minmax(5rem, 9rem)) var(--col-app, minmax(4rem, 8rem)) var(--codes-w) minmax(0, 1fr);
+  column-gap: 12px;
+  margin: 6px 20px 0; background: var(--panel); border: 1px solid var(--border); border-radius: var(--radius);
+  overflow: clip; /* rounded corners without breaking the sticky header (hidden would) */
+}
+.table[hidden] { display: none; }
+.table .list {
+  display: grid; grid-template-columns: subgrid; grid-column: 1 / -1;
+  margin: 0; border: 0; border-radius: 0; background: none; overflow: visible;
+}
+.table .row { grid-template-columns: subgrid; grid-column: 1 / -1; }
+.list-head {
+  display: grid; grid-template-columns: subgrid; grid-column: 1 / -1;
+  position: sticky; top: var(--topbar-h, 0px); z-index: 5;
+  padding: 0 14px 0 11px; border-left: 3px solid transparent;
+  background: var(--panel-2); border-bottom: 1px solid var(--border);
+  font-size: 10.5px; font-weight: 650; letter-spacing: .04em; text-transform: uppercase; color: var(--muted);
+}
+.list-head > span { position: relative; min-width: 0; padding: 4px 0; }
+/* Labels never widen a column (contain: inline-size), they are cut instead */
+.list-head .lbl { display: block; contain: inline-size; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font: inherit; }
+.list-head .grip { position: absolute; top: 0; bottom: 0; right: -9px; width: 6px; z-index: 1; cursor: col-resize; touch-action: none; }
+.list-head .grip::after { content: ""; position: absolute; left: 2px; top: 25%; bottom: 25%; border-left: 1px solid var(--border); }
+.list-head .grip:hover::after, .list-head .grip.active::after { top: 0; bottom: 0; border-left: 2px solid var(--accent); left: 2px; }
+body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-select: none; -webkit-user-select: none; }
+.cols-row { display: flex; flex-wrap: wrap; align-items: center; gap: 6px 12px; margin-top: 12px; }
+
 .row {
   display: grid;
-  grid-template-columns: max-content max-content 4.6rem minmax(5rem, 9rem) minmax(4rem, 8rem) 1fr;
+  grid-template-columns: max-content max-content 4.6rem minmax(5rem, 9rem) minmax(4rem, 8rem) var(--codes-w) 1fr;
   gap: 0 12px; align-items: baseline;
   padding: 5px 14px 5px 11px;
   border-left: 3px solid transparent;
@@ -308,12 +366,17 @@ body.busy .progress::after {
   cursor: pointer;
 }
 .row:last-child { border-bottom: 0; }
+/* Settings > Interface > Density: compact fits about 50% more lines on screen */
+:root[data-density="compact"] .row { padding-top: 1px; padding-bottom: 1px; line-height: 1.25; }
+:root[data-density="compact"] .row .sev { padding-top: 0; padding-bottom: 0; line-height: 1.3; }
+:root[data-density="compact"] .row .codes b { padding-top: 0; padding-bottom: 0; line-height: 1.2; }
 .row:hover { background: var(--row-hover); }
 .row:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; }
 .row.new { animation: flash 1.2s ease-out; }
 @keyframes flash { from { background: var(--accent-soft); } to { background: transparent; } }
 
-.row time { color: var(--muted); white-space: nowrap; font-variant-numeric: tabular-nums; }
+.row > * { min-width: 0; }
+.row time { color: var(--muted); white-space: nowrap; font-variant-numeric: tabular-nums; overflow: hidden; text-overflow: ellipsis; }
 .row time.rcv { color: var(--text); }
 .row time .ms { opacity: .6; }
 .row .host, .row .app { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
@@ -322,7 +385,7 @@ body.busy .progress::after {
 .row .msg { white-space: pre-wrap; word-break: break-word; min-width: 0; }
 
 .sev {
-  justify-self: start;
+  justify-self: start; max-width: 100%; overflow: hidden; text-overflow: ellipsis;
   font-size: 10.5px; font-weight: 700; text-transform: uppercase; letter-spacing: .03em;
   padding: 1px 6px; border-radius: 5px;
   color: var(--sev); background: color-mix(in srgb, var(--sev) 14%, transparent);
@@ -338,6 +401,17 @@ body.busy .progress::after {
 .row.sev-crit, .row.sev-err, .row.sev-warning { border-left-color: var(--sev); }
 .row.sev-crit, .row.sev-err { background: color-mix(in srgb, var(--sev) 7%, transparent); }
 .row .host[data-act], .row .app[data-act] { cursor: pointer; }
+/* Codes of the tags found in the message: grey badges, fixed size */
+.row .codes { white-space: nowrap; overflow: hidden; }
+.row .codes > span { display: inline-flex; gap: 3px; vertical-align: 1px; }
+.row .codes b, .tag-code {
+  min-width: 2.2ch; padding: 1px 4px; border-radius: 5px; text-align: center;
+  /* same font as the severity badges (the log font) */
+  font-family: var(--log-font, var(--mono)); font-size: 10.5px; font-weight: 700; letter-spacing: .03em;
+  line-height: 1.35; font-variant-numeric: tabular-nums;
+  background: var(--code-bg); color: #0b0f17;
+}
+.row .codes b.more { background: none; box-shadow: inset 0 0 0 1px var(--code-bg); color: var(--muted); }
 .row .host[data-act]:hover, .row .app[data-act]:hover { color: var(--accent); text-decoration: underline; text-underline-offset: 2px; }
 
 mark.tag {
@@ -372,6 +446,22 @@ mark.hit { background: var(--hit); color: inherit; border-radius: 3px; padding:
   font-family: var(--mono); font-size: 12.5px; white-space: pre-wrap;
 }
 
+.auth-warn {
+  display: flex; align-items: center; gap: 10px;
+  margin: 6px 20px 0; padding: 6px 8px 6px 14px;
+  border: 1px solid color-mix(in srgb, var(--sev-warning) 45%, transparent);
+  background: color-mix(in srgb, var(--sev-warning) 12%, transparent);
+  color: var(--text); border-radius: var(--radius); font-size: 12.5px;
+}
+.auth-warn[hidden] { display: none; }
+.auth-warn span { flex: 1; }
+.auth-warn .icon-btn { width: 26px; height: 26px; flex: none; }
+.ro-note {
+  margin: 0 0 12px; padding: 8px 12px; border-radius: var(--radius); font-size: 12.5px;
+  background: color-mix(in srgb, var(--accent) 10%, transparent); color: var(--text);
+}
+.read-only .set-panel .set-section[inert] { opacity: .55; }
+
 .pill {
   position: fixed; top: 70px; left: 50%; transform: translateX(-50%); z-index: 30;
   border: 0; border-radius: 999px; padding: 7px 16px;
@@ -391,6 +481,11 @@ mark.hit { background: var(--hit); color: inherit; border-radius: 3px; padding:
 .conn.ok::before { color: var(--sev-info); }
 .conn.ko::before { color: var(--sev-err); }
 #stats .bad { color: var(--sev-err); }
+#stats .warn { color: var(--sev-warning); }
+/* "Back to top", at the right end of the status bar: never over a log row */
+.to-top { width: 24px; height: 24px; margin: -5px -6px -5px auto; flex: none; } /* no taller bar */
+.to-top svg { width: 16px; height: 16px; }
+.to-top[hidden] { display: none; }
 
 .toast {
   position: fixed; bottom: 48px; left: 50%; transform: translateX(-50%); z-index: 50;
@@ -400,12 +495,12 @@ mark.hit { background: var(--hit); color: inherit; border-radius: 3px; padding:
 
 /* ---------- Settings dialog ---------- */
 dialog.settings {
-  width: min(900px, calc(100vw - 32px));
+  width: min(1280px, calc(100vw - 32px));
-  height: min(660px, calc(100vh - 64px)); max-height: none;
+  height: calc(100vh - 48px); max-height: 960px;
 }
 dialog.settings .dlg { display: flex; flex-direction: column; height: 100%; padding: 0; }
 dialog.settings .dlg > header { padding: 14px 16px 14px 22px; border-bottom: 1px solid var(--border); }
-.set-body { flex: 1; min-height: 0; display: grid; grid-template-columns: 200px minmax(0, 1fr); }
+.set-body { flex: 1; min-height: 0; display: grid; grid-template-columns: 176px minmax(0, 1fr); }
 .set-nav {
   display: flex; flex-direction: column; gap: 2px;
   padding: 12px 10px; border-right: 1px solid var(--border);
@@ -420,8 +515,19 @@ dialog.settings .dlg > header { padding: 14px 16px 14px 22px; border-bottom: 1px
 .set-nav button:hover { background: var(--panel-2); color: var(--text); }
 .set-nav button[aria-selected="true"] { background: var(--accent-soft); color: var(--accent); }
 .set-nav svg { width: 18px; height: 18px; flex: none; }
-.set-panels { overflow-y: auto; padding: 2px 24px 22px; }
+.set-panels { overflow-y: auto; padding: 16px 20px 20px; }
-.set-panel > .set-section:first-child { margin-top: 16px; }
+/* Sections are cards laid out in columns when there is room; .span ones take the full width. */
+.set-panel { display: grid; grid-template-columns: repeat(auto-fit, minmax(380px, 1fr)); gap: 14px; align-items: start; grid-auto-flow: row dense; }
+.set-panel[hidden] { display: none; }
+.set-panel.wide { grid-template-columns: minmax(0, 1fr); }
+.set-panel > .set-section, .set-panel > .set-section + .set-section {
+  margin: 0; padding: 14px 16px; min-width: 0;
+  border: 1px solid var(--border); border-radius: 12px;
+  background: color-mix(in srgb, var(--panel-2) 35%, var(--panel));
+}
+.set-panel > .set-section.span { grid-column: 1 / -1; }
+.set-panel > .ro-note { grid-column: 1 / -1; margin: 0; }
+.sec-head { display: flex; align-items: baseline; justify-content: space-between; gap: 12px; }
 .hint { margin-top: 8px !important; }
 .lbl { font-size: 13px; font-weight: 550; }
 .set-panel .preview-list { margin: 14px 0 10px; pointer-events: none; }
@@ -437,6 +543,7 @@ dialog.settings .dlg > header { padding: 14px 16px 14px 22px; border-bottom: 1px
 .preview-list .row .host { grid-area: host; }
 .preview-list .row .app { grid-area: app; display: block; }
 .preview-list .row .msg { grid-area: msg; }
+.preview-list .row .codes { display: none; }
 
 dialog {
   width: min(720px, calc(100vw - 32px)); max-height: calc(100vh - 64px);
@@ -465,6 +572,17 @@ select.field:focus { outline: 0; border-color: var(--accent); box-shadow: 0 0 0
   background: var(--panel-2); font-family: var(--mono); font-size: 12.5px;
 }
 .set-section.danger h3 { color: var(--sev-err); }
+.db-head { display: flex; align-items: baseline; justify-content: space-between; gap: 10px; margin: 4px 0 6px; }
+.db-stats {
+  display: grid; grid-template-columns: max-content minmax(0, 1fr); gap: 5px 14px;
+  margin: 0 0 14px; padding: 10px 12px; border-radius: 8px; background: var(--panel-2); font-size: 12.5px;
+}
+.db-stats dt { color: var(--muted); }
+.db-stats dd { margin: 0; min-width: 0; overflow-wrap: anywhere; }
+.db-stats dd b { font-weight: 600; font-variant-numeric: tabular-nums; }
+.db-msg { grid-column: 1 / -1; color: var(--muted); }
+.db-msg.bad { color: var(--sev-err); }
+.link-btn:disabled { opacity: .5; cursor: default; text-decoration: none; }
 .purge-row { display: flex; flex-wrap: wrap; align-items: center; gap: 10px; margin-top: 10px; }
 .btn.danger { color: var(--sev-err); border-color: color-mix(in srgb, var(--sev-err) 45%, var(--border)); }
 .btn.danger:hover:not(:disabled) { background: var(--sev-err); border-color: var(--sev-err); color: #fff; }
@@ -539,7 +657,19 @@ input.switch:checked::after { transform: translateX(14px); }
 input.switch:disabled { cursor: not-allowed; }
 input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
 
-.set-section footer { display: flex; flex-wrap: wrap; gap: 8px; justify-content: space-between; margin-top: 14px; }
+/* Settings > Filters: toolbar kept in view while the list scrolls, dense rows. */
+.tags-section { padding-top: 12px !important; }
+.tag-toolbar {
+  position: sticky; top: -16px; z-index: 2;
+  display: flex; flex-wrap: wrap; align-items: center; gap: 8px;
+  margin: 10px -16px 0; padding: 8px 16px;
+  background: color-mix(in srgb, var(--panel-2) 35%, var(--panel));
+  border-bottom: 1px solid var(--border);
+}
+.tag-toolbar .btn { height: 30px; }
+.tag-toolbar select.field { width: auto; height: 30px; }
+.tag-toolbar .dk-filter { flex: 1 1 220px; height: 30px; }
+.tag-toolbar #resetTags { margin-left: auto; }
 
 .seg { display: inline-flex; padding: 3px; gap: 3px; background: var(--panel-2); border: 1px solid var(--border); border-radius: 10px; }
 .seg button {
@@ -549,33 +679,38 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
 .seg button:hover { color: var(--text); }
 .seg button[aria-checked="true"] { background: var(--panel); color: var(--accent); box-shadow: 0 1px 3px rgba(0, 0, 0, .12); }
 
-.tag-list { display: flex; flex-direction: column; gap: 8px; margin-top: 12px; }
+/* One wide column: long regular expressions stay readable. */
+.tag-list { display: flex; flex-direction: column; gap: 3px; margin-top: 10px; }
 .tag-row {
-  display: grid; align-items: center; gap: 8px 10px;
+  display: grid; align-items: center; gap: 4px 6px;
-  grid-template-columns: 38px minmax(8rem, 1fr) 7.5rem auto 36px;
+  grid-template-columns: 2.4em 30px minmax(7rem, 1fr) 10rem auto 26px;
-  padding: 8px 10px; border: 1px solid var(--border); border-radius: 10px;
+  padding: 3px 4px 3px 6px; border: 1px solid var(--border); border-radius: 8px;
-  background: var(--panel-2);
+  background: var(--panel);
 }
+.tag-row[hidden] { display: none; }
 .tag-row.off { opacity: .55; }
+.tag-row .tag-code { font-size: 11px; padding: 2px 4px; cursor: default; text-align: center; }
 .tag-row input[type="color"] {
-  width: 38px; height: 32px; padding: 0; border: 1px solid var(--border); border-radius: 8px;
+  width: 30px; height: 26px; padding: 0; border: 1px solid var(--border); border-radius: 6px;
   background: none; cursor: pointer;
 }
-.tag-row input[type="color"]::-webkit-color-swatch-wrapper { padding: 3px; }
+.tag-row input[type="color"]::-webkit-color-swatch-wrapper { padding: 2px; }
 .tag-row input[type="color"]::-webkit-color-swatch { border: 0; border-radius: 5px; }
 .tag-row input[type="text"] {
-  height: 32px; padding: 0 10px; min-width: 0;
+  height: 26px; padding: 0 8px; min-width: 0;
-  border: 1px solid var(--border); border-radius: 8px; background: var(--panel);
+  border: 1px solid var(--border); border-radius: 6px; background: var(--panel);
-  font-family: var(--mono); font-size: 13px; outline: 0;
+  font-family: var(--mono); font-size: 12.5px; outline: 0;
 }
 .tag-row input[type="text"]:focus { border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); }
-.tag-row .preview { font-family: var(--mono); font-size: 12.5px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; text-align: center; }
+.tag-row .preview { font-family: var(--mono); font-size: 12px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; text-align: center; }
-.tag-row .opts { display: flex; gap: 4px; }
+.tag-row .opts { display: flex; gap: 3px; }
 .opt {
-  display: inline-flex; align-items: center; height: 28px; padding: 0 8px;
+  display: inline-flex; align-items: center; height: 24px; padding: 0 6px;
-  border: 1px solid var(--border); border-radius: 7px; background: var(--panel);
+  border: 1px solid var(--border); border-radius: 6px; background: var(--panel);
-  font-size: 11.5px; font-weight: 600; color: var(--muted); cursor: pointer; user-select: none;
+  font-size: 11px; font-weight: 600; color: var(--muted); cursor: pointer; user-select: none;
 }
+.tag-row .icon-btn { width: 26px; height: 26px; border-radius: 6px; }
+.tag-row .icon-btn svg { width: 15px; height: 15px; }
 .opt input { display: none; }
 .opt:has(input:checked) { color: var(--accent); border-color: var(--accent); background: var(--accent-soft); }
 .tag-row .err { grid-column: 1 / -1; color: var(--sev-err); font-size: 12px; }
@@ -589,14 +724,25 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
   .live .lbl { display: none; }
   .filters { padding: 10px 16px 6px; }
   .filters select { flex: 1 1 calc(50% - 8px); max-width: none; min-width: 0; }
+  .view-mode { flex: 1 1 100%; }
+  .view-mode button { flex: 1; justify-content: center; }
+  /* Phones: the two dates on one line, the ◀ ▶ − buttons below */
+  .period { flex: 1 1 100%; flex-wrap: wrap; }
+  .period > span { display: none; }
+  .period input { flex: 1 1 calc(50% - 3px); min-width: 0; }
+  .period .btn.tool { order: 1; flex: 1; }
   .spacer { display: none; }
   #count { flex: 1 1 50%; min-width: 0; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
   .histo { padding: 0 16px 6px; }
   .h-leg { display: none; }
-  .list, .error-banner { margin-left: 16px; margin-right: 16px; }
+  .list, .error-banner, .auth-warn, .table { margin-left: 16px; margin-right: 16px; }
-  .row {
+  /* Phones: no columns, two lines per log (layout below); the widths do not apply */
-    grid-template-columns: auto auto 1fr;
+  .table { display: block; }
-    grid-template-areas: "rcv sev host" "msg msg msg";
+  .table .list { display: block; margin: 0; }
+  .list-head { display: none; }
+  .row, .table .row {
+    grid-template-columns: auto auto 1fr auto;
+    grid-template-areas: "rcv sev host codes" "msg msg msg msg";
     gap: 3px 8px; padding: 8px 12px 8px 10px;
   }
   .row time.rcv { grid-area: rcv; }
@@ -604,6 +750,7 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
   .row .sev { grid-area: sev; }
   .row .host { grid-area: host; justify-self: end; max-width: 100%; }
   .row .app { display: none; }
+  .row .codes { grid-area: codes; }
   .row .msg { grid-area: msg; }
   .details { grid-column: 1 / -1; }
   .details dl { grid-template-columns: 1fr; }
@@ -620,13 +767,44 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
     font-size: 11.5px; text-align: center;
   }
   .set-nav button span { max-width: 100%; overflow: hidden; text-overflow: ellipsis; }
-  .set-panels { padding: 0 16px 18px; }
+  .set-panels { padding: 12px 12px 18px; }
-  #sizeSwitch { display: flex; }
+  .set-panel { grid-template-columns: minmax(0, 1fr); gap: 10px; }
-  #sizeSwitch button { flex: 1; padding: 0 4px; white-space: nowrap; font-size: 12px; }
+  .set-panel > .set-section, .set-panel > .set-section + .set-section { padding: 12px; }
+  .tag-toolbar { position: static; margin: 10px -12px 0; padding: 8px 12px; }
+  .tag-toolbar #resetTags { margin-left: 0; }
+  #sizeSwitch, #densitySwitch { display: flex; }
+  #sizeSwitch button, #densitySwitch button { flex: 1; padding: 0 4px; white-space: nowrap; font-size: 12px; }
+  :root[data-density="compact"] .row { padding-top: 3px; padding-bottom: 3px; }
   .field-grid select { margin-bottom: 6px; }
   .status { padding: 6px 16px; }
-  .tag-row { grid-template-columns: 38px 1fr 36px; }
+  .tag-row { grid-template-columns: auto 30px 1fr 26px; }
   .tag-row .preview { display: none; }
   .tag-row .opts { grid-column: 1 / -1; grid-row: 2; flex-wrap: wrap; }
-  .tag-row [data-del] { grid-column: 3; grid-row: 1; }
+  .tag-row [data-del] { grid-column: 4; grid-row: 1; }
 }
+
+/* ---------- Login page (AUTH_MODE=local) ---------- */
+body.login-page {
+  min-height: 100vh; padding: 16px;
+  display: grid; place-items: center;
+}
+.login-tools { position: fixed; top: 12px; right: 16px; display: flex; align-items: center; gap: 8px; }
+.login-card {
+  width: 100%; max-width: 360px;
+  display: flex; flex-direction: column; align-items: center; gap: 10px;
+  padding: 32px 28px 28px;
+  background: var(--panel); border: 1px solid var(--border); border-radius: 14px; box-shadow: var(--shadow);
+}
+.login-logo { max-width: 200px; max-height: 96px; object-fit: contain; margin-bottom: 6px; }
+.login-card .brand { font-size: 20px; }
+.login-card .brand svg { width: 32px; height: 32px; }
+.login-card > p { margin: 0 0 8px; text-align: center; }
+.login-card form { width: 100%; display: flex; flex-direction: column; gap: 6px; }
+.login-card label { font-size: 13px; font-weight: 550; }
+.login-card input {
+  height: 38px; padding: 0 11px; margin-bottom: 6px;
+  border: 1px solid var(--border); border-radius: 9px; background: var(--bg);
+}
+.login-card input:focus { outline: 0; border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); }
+.login-card .btn.primary { height: 38px; justify-content: center; margin-top: 6px; font-size: 14px; }
+.login-err { margin: 0; color: var(--sev-err); font-size: 13px; }