Compare commits

...
Author SHA1 Message Date
cedricandClaude Opus 5.5 c8f45044a3 Reading API /api/v1/logs with a stable cursor, and API tokens
GET /api/v1/logs?query=<LogsQL>&since=<cursor|time|duration>&limit=<n>
returns the logs after a cursor, oldest first, with next_cursor and
has_more. The cursor is a position on _time (the reception time) plus
the number of rows already returned among those sharing that exact
time, ordered by a hash of their content, so following the cursors
gives every row once even within groups cut by a page. Rows count once
older than the lag (30 s), and nothing is returned while the disk
spool still holds logs.

API tokens (Settings > API) give read-only access to /api/v1/ as
"Authorization: Bearer lst_...", in every auth mode; only their SHA-256
is kept in /data/api_tokens.json. README (EN/FR) documents the route and
an n8n workflow keeping the cursor in $getWorkflowStaticData.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 15:47:57 +01:00
cedric d61384ed2c Update docker-compose.yml 2026-10-07 18:25:26 +02:00
cedricandClaude Opus 5.5 dd8ca74cad Log list: tighter rows, badges no longer set the line height
Rows drop from ~29px to ~24px (normal density, medium font) and from
~18.6px to ~18px in compact; long messages still wrap.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 18:09:06 +02:00
claude BotandClaude Opus 5.5 2ca89b1c6d Source badge at the start of every log line
Each row now starts with a fixed-width badge naming its source, one color
per source: SYSLOG, DOCKER, KERNEL (dmesg), HOST (/var/log), AUDIT, FILE
and LS. The AUDIT/FILE/LS badges move there from before the app name; those
sources keep their colored edge and tint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 13:16:36 +02:00
cedricandClaude Opus 5.5 1a3b3ee23a Log files: find the history by date, count from the first poll
A file turned on was read from its start to find its last hour: with a
Traefik access log that holds months of lines (never rotated by
Traefik), LogStream spent minutes on old lines that were all left out,
and new lines waited behind them. Entries sent during the first poll
were also left out of the count shown in Settings.

- start of the FILES_BACKFILL history found by bisection on the dates
  of the lines; files without dates: only new lines
- up to 16 reads of 4 MB per file and per poll
- status "behind" (bytes left to read), shown as "Reading the history:
  N MB left"
- the status of a file exists before its first entries are counted

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:42:09 +02:00
cedricandClaude Opus 5.5 194edee6de Settings: Save and Cancel buttons instead of saving each change
Changes made in the Settings window show at once but are only kept with
the Save button, placed under the tab labels; it takes the accent color
while changes are waiting. Cancel goes back to the saved settings, and
closing the window (or the page) with unsaved changes asks first.

Tags are now sent as a whole list (PUT /api/tags, nothing saved when one
is invalid, the error gives its position) and "Restore default tags"
shows GET /api/tags/defaults until Save.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 11:03:01 +02:00
17 changed files with 1962 additions and 173 deletions

No files matched your search

+94 -5
View File
@@ -92,6 +92,10 @@ une base propre.
Les badges de sévérité `err`/`crit` et `warning` reprennent les couleurs des tags `error` et Les badges de sévérité `err`/`crit` et `warning` reprennent les couleurs des tags `error` et
`warning`. `warning`.
Chaque ligne commence par un badge de largeur fixe qui indique sa source, avec une couleur par
source : **SYSLOG**, **DOCKER**, **KERNEL** (dmesg), **HOST** (fichiers de `/var/log`), **AUDIT**,
**FILE** et **LS** (logs de LogStream lui-même).
Raccourcis : `/` place le curseur dans la recherche, `Esc` la vide. Un clic sur une ligne Raccourcis : `/` place le curseur dans la recherche, `Esc` la vide. Un clic sur une ligne
affiche tous ses champs. affiche tous ses champs.
@@ -278,7 +282,7 @@ source est **désactivée par défaut** : activez-la dans **Réglages > Sources
ne peut pas être résolu), `exe`, `res` (`success` / `failed`, d'après `res=` ou `success=`) et ne peut pas être résolu), `exe`, `res` (`success` / `failed`, d'après `res=` ou `success=`) et
`key` (clé de la règle). La sévérité est `warning` en cas d'échec, `info` sinon ; facility `key` (clé de la règle). La sévérité est `warning` en cas d'échec, `info` sinon ; facility
`authpriv` ; source `audit`. Dans la liste, les lignes d'audit ont un liseré gauche violet, `authpriv` ; source `audit`. Dans la liste, les lignes d'audit ont un liseré gauche violet,
un fond légèrement teinté et un badge **AUDIT** devant l'app ; le filtre **Source** permet de un fond légèrement teinté et le badge de source **AUDIT** ; le filtre **Source** permet de
les afficher seules. les afficher seules.
- **Noms** : `user` vient du format enrichi (`log_format = ENRICHED`, le défaut des versions - **Noms** : `user` vient du format enrichi (`log_format = ENRICHED`, le défaut des versions
récentes d'auditd), sinon du `/etc/passwd` de l'hôte s'il est monté en `/host/etc/passwd`. récentes d'auditd), sinon du `/etc/passwd` de l'hôte s'il est monté en `/host/etc/passwd`.
@@ -338,11 +342,13 @@ conteneur, un **format** (*Automatique* par défaut), un **hôte** facultatif (p
qui portent le nom d'un champ de LogStream (`host`, `app`…) sont renommés `field_host`, qui portent le nom d'un champ de LogStream (`host`, `app`…) sont renommés `field_host`,
`field_app`… `field_app`…
- **Affichage** : source `file`, le chemin dans `log_file`. Dans la liste, ces lignes ont un - **Affichage** : source `file`, le chemin dans `log_file`. Dans la liste, ces lignes ont un
liseré gauche vert, une légère teinte et un badge **FILE** avant l'app ; le filtre **Source** liseré gauche vert, une légère teinte et le badge de source **FILE** ; le filtre **Source**
les affiche seules. les affiche seules.
- À l'activation d'un fichier, les lignes de la dernière heure déjà présentes sont lues d'abord - À l'activation d'un fichier, les lignes de la dernière heure déjà présentes sont lues d'abord
(`FILES_BACKFILL`, `0` = seulement les nouvelles lignes ; les lignes sans horodatage sont alors (`FILES_BACKFILL`, `0` = seulement les nouvelles lignes ; les lignes sans horodatage sont alors
ignorées) ; un fichier créé ensuite est lu depuis son début. L'inode et une ignorées). Le début de cette heure est trouvé par dichotomie sur les dates des lignes : un gros
fichier qui contient des mois de logs n'est pas relu depuis son début ; un fichier créé ensuite
est lu depuis son début, jusqu'à 64 Mo par seconde, l'état indiquant ce qu'il reste à lire. L'inode et une
position sont enregistrés par fichier dans `/data/files-state.json` : après un redémarrage la position sont enregistrés par fichier dans `/data/files-state.json` : après un redémarrage la
lecture reprend là ; après une rotation (`access.log` renommé `access.log.1`), la fin de lecture reprend là ; après une rotation (`access.log` renommé `access.log.1`), la fin de
l'ancien fichier est lue avant le nouveau ; un fichier vidé sur place (`copytruncate`) est l'ancien fichier est lue avant le nouveau ; un fichier vidé sur place (`copytruncate`) est
@@ -384,10 +390,88 @@ charge (pas de `| pipes`).
En ligne de commande : `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`. En ligne de commande : `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`.
## API de lecture (n8n, scripts)
`GET /api/v1/logs` renvoie les logs stockés qui suivent un curseur, du plus ancien au plus
récent : un outil appelé à intervalle régulier (n8n, un script, un connecteur SIEM) reçoit ainsi
chaque nouveau log une fois, sans trou ni doublon, même quand beaucoup de logs ont le même
horodatage.
L'accès passe par un **jeton d'API**, créé dans *Paramètres > API* par un administrateur et
envoyé dans `Authorization: Bearer lst_…`. Un jeton est en lecture seule et n'ouvre que
`/api/v1/` ; il fonctionne quel que soit le mode d'[authentification](#authentification)
(`local`, `oidc` ou aucune). Il n'est affiché qu'une fois, à sa création : LogStream n'en garde
que l'empreinte SHA-256, dans `/data/api_tokens.json`. Le révoquer coupe l'accès immédiatement ;
la liste indique quand et d'où chaque jeton a servi pour la dernière fois. Une session ouverte
dans le navigateur peut aussi appeler la route.
| Paramètre | Rôle |
|---|---|
| `query` | filtre LogsQL (par défaut `*`), par exemple `severity:in(err,crit) app:sshd`. Les pipes (`\| ...`) sont refusés. |
| `since` | point de départ : le `next_cursor` de la réponse précédente, une date RFC 3339 (`2026-11-01T08:00:00Z`) ou une durée avant maintenant (`15m`, `6h`, `7d`). Par défaut : `1h`. |
| `limit` | lignes par réponse, de 1 à 10 000 (1 000 par défaut). |
| `lag` | seuls les logs reçus depuis au moins cette durée sont renvoyés (`30s` par défaut, `1h` au plus), pour ne pas sauter les logs encore dans la file ou en attente de leur nom DNS. |
```json
{
"rows": [{"_time": "2026-11-01T14:02:11.52Z", "_msg": "…", "host": "fw01", "app": "sshd", "severity": "err", …}],
"next_cursor": "eyJ0IjoiMjAyNi0xMS0wMVQxNDowMjoxMS41MjAwMDAwMDFaIiwicSI6IjJlYjE4YzNhIn0",
"has_more": false,
"until": "2026-11-01T14:05:30.1Z"
}
```
Les lignes contiennent les champs stockés (comme la vue JSON de l'interface), triées par
`_time`, l'heure de réception. Renvoyez toujours `next_cursor`, même quand `rows` est vide.
`has_more: true` signifie que d'autres logs sont prêts : rappelez tout de suite.
`waiting: true` signifie que des logs du [tampon disque](#configuration) sont encore en cours
d'envoi à VictoriaLogs : rien n'est renvoyé tant qu'ils ne sont pas stockés, pour n'en manquer
aucun. Le curseur est lié à sa `query` : changer la requête garde la position dans le temps. Les
erreurs arrivent sous la forme `{"error", "code"}` avec le statut 400 (paramètre ou LogsQL
invalide), 401 (jeton invalide ou révoqué) ou 502 (VictoriaLogs injoignable).
```bash
curl -H "Authorization: Bearer lst_…" \
"https://logs.example.org/api/v1/logs?query=severity:err&since=15m&limit=1000"
```
### Exemple : workflow n8n
Quatre nœuds, le curseur gardé dans les données statiques du workflow d'une exécution à l'autre :
1. **Schedule Trigger** : toutes les minutes.
2. **Code** (*Run Once for All Items*), lit le curseur :
```js
const data = $getWorkflowStaticData('global');
return [{ json: { since: data.cursor || '15m' } }];
```
3. **HTTP Request** : méthode `GET`, URL `https://logs.example.org/api/v1/logs`.
*Authentication* : Generic Credential Type > Header Auth, avec le nom `Authorization` et la
valeur `Bearer lst_…`. *Send Query Parameters* : `query` = `severity:in(emerg,alert,crit,err)`,
`since` = `{{ $json.since }}`, `limit` = `1000`.
4. **Code** (*Run Once for All Items*), enregistre le curseur et fait de chaque log un item :
```js
const data = $getWorkflowStaticData('global');
const page = $input.first().json;
data.cursor = page.next_cursor;
return page.rows.map((row) => ({ json: row }));
```
Branchez ensuite ce que les logs doivent alimenter (Slack, mail, ticket, base de données…). n8n ne
garde les données statiques que pour les exécutions du workflow actif, pas pour les tests
manuels : pendant les essais, chaque exécution repart de `15m`. Au-delà de 1 000 logs par minute,
`has_more` vaut true et les exécutions suivantes rattrapent le retard ; augmentez `limit` ou la
fréquence du déclencheur si besoin.
## Paramètres ## Paramètres
L'icône en forme d'engrenage ouvre les paramètres, organisés en onglets. Tout, sauf les tags de L'icône en forme d'engrenage ouvre les paramètres, organisés en onglets. Les modifications
couleur, est mémorisé par navigateur. s'affichent tout de suite mais ne sont conservées qu'avec le bouton **Valider**, sous les libellés
des onglets : il prend la couleur d'accent tant que des modifications sont en attente. **Annuler**
revient aux paramètres enregistrés, et fermer la fenêtre avec des modifications non validées
demande confirmation avant de les abandonner. Localisation et Affichage sont mémorisés par
navigateur ; les tags de couleur et les sources de logs sont enregistrés sur le serveur. La purge,
les jetons d'API et la réinitialisation des colonnes agissent immédiatement.
- **Localisation et langue** - **Localisation et langue**
- *Langue* : anglais ou français. - *Langue* : anglais ou français.
@@ -441,6 +525,9 @@ couleur, est mémorisé par navigateur.
désactivée par défaut : mettez `ALLOW_PURGE=true` pour l'autoriser. Tout administrateur peut désactivée par défaut : mettez `ALLOW_PURGE=true` pour l'autoriser. Tout administrateur peut
alors purger : activez l'[authentification](#authentification) si l'interface est accessible alors purger : activez l'[authentification](#authentification) si l'interface est accessible
à d'autres. à d'autres.
- **API** : jetons de l'[API de lecture](#api-de-lecture-n8n-scripts) : en créer un avec un nom
(affiché une seule fois, à copier tout de suite), voir quand et d'où chacun a servi pour la
dernière fois, le révoquer. Réservé aux administrateurs.
## Authentification ## Authentification
@@ -620,6 +707,8 @@ Pour mettre à jour l'une d'elles :
| `hub.go` | envoie les nouveaux messages aux navigateurs (SSE) | | `hub.go` | envoie les nouveaux messages aux navigateurs (SSE) |
| `rdns.go` | résolutions DNS inverses avec cache | | `rdns.go` | résolutions DNS inverses avec cache |
| `export.go` | export CSV en flux | | `export.go` | export CSV en flux |
| `logsapi.go` | API de lecture `/api/v1/logs` : curseur, groupes de logs de même heure |
| `apitokens.go` | jetons d'API : stockage (haché), contrôle Bearer, `/api/tokens` |
| `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) | | `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) |
| `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources de logs | | `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources de logs |
| `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` | | `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` |
+88 -6
View File
@@ -83,6 +83,9 @@ versions before this change are indexed by their message timestamp; purge them f
Settings to start clean. Settings to start clean.
Severity badges `err`/`crit` and `warning` use the colors of the `error` and `warning` tags. Severity badges `err`/`crit` and `warning` use the colors of the `error` and `warning` tags.
Each row starts with a fixed-width badge naming its source, one color per source: **SYSLOG**,
**DOCKER**, **KERNEL** (dmesg), **HOST** (files of `/var/log`), **AUDIT**, **FILE** and **LS**
(LogStream's own logs).
Shortcuts: `/` focuses the search box, `Esc` clears it. Clicking a row shows all its fields. Shortcuts: `/` focuses the search box, `Esc` clears it. Clicking a row shows all its fields.
@@ -254,7 +257,7 @@ also be changed (saved in `/data/audit.json`; `AUDIT_LOG` sets the default path)
`auid` (login user ID), `user` (its name, or the uid when it cannot be resolved), `exe`, `res` `auid` (login user ID), `user` (its name, or the uid when it cannot be resolved), `exe`, `res`
(`success` / `failed`, from `res=` or `success=`) and `key` (rule key). Severity is `warning` (`success` / `failed`, from `res=` or `success=`) and `key` (rule key). Severity is `warning`
when the result is a failure, `info` otherwise; facility `authpriv`; source `audit`. In the when the result is a failure, `info` otherwise; facility `authpriv`; source `audit`. In the
list, audit lines have a violet left edge, a light tint and an **AUDIT** badge before the app; list, audit lines have a violet left edge, a light tint and the **AUDIT** source badge;
the **Source** filter shows them alone. the **Source** filter shows them alone.
- **Names**: `user` comes from the enriched format (`log_format = ENRICHED`, the default of - **Names**: `user` comes from the enriched format (`log_format = ENRICHED`, the default of
recent auditd versions), else from the host's `/etc/passwd` when it is mounted as recent auditd versions), else from the host's `/etc/passwd` when it is mounted as
@@ -310,10 +313,12 @@ optional **host** (default: `/host/etc/hostname`, else `localhost`) and a switch
not fit the chosen format is kept as plain text. Fields named like LogStream's own ones not fit the chosen format is kept as plain text. Fields named like LogStream's own ones
(`host`, `app`…) are renamed `field_host`, `field_app`… (`host`, `app`…) are renamed `field_host`, `field_app`…
- **Display**: source `file`, the path in `log_file`. In the list these lines have a green left - **Display**: source `file`, the path in `log_file`. In the list these lines have a green left
edge, a light tint and a **FILE** badge before the app; the **Source** filter shows them alone. edge, a light tint and the **FILE** source badge; the **Source** filter shows them alone.
- When a file is turned on, the lines of the last hour already in it are read first - When a file is turned on, the lines of the last hour already in it are read first
(`FILES_BACKFILL`, `0` = only new lines; lines without a timestamp are then left out); a file (`FILES_BACKFILL`, `0` = only new lines; lines without a timestamp are then left out). The
created later is read from its start. The inode and an offset are saved per file in start of that hour is found by bisection on the dates of the lines, so a large file that holds
months of logs is not read from its start; a file created later is read from its start, up to
64 MB per second, the status showing what is left to read. The inode and an offset are saved per file in
`/data/files-state.json`: after a restart reading resumes there; after a rotation `/data/files-state.json`: after a restart reading resumes there; after a rotation
(`access.log` renamed `access.log.1`), the end of the old file is read before the new one; (`access.log` renamed `access.log.1`), the end of the old file is read before the new one;
a file truncated in place (`copytruncate`) is read again from its start. a file truncated in place (`copytruncate`) is read again from its start.
@@ -349,10 +354,82 @@ chosen in Settings), `severity`, `facility`, `host`, `host_ip`, `app`, `pid`, `s
From the command line: `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`. From the command line: `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`.
## Reading API (n8n, scripts)
`GET /api/v1/logs` returns the stored logs that follow a cursor, oldest first, so that a tool
called on a schedule (n8n, a script, a SIEM connector) gets every new log once, without gaps
or duplicates, even when many logs share the same timestamp.
Access goes through an **API token**, created in *Settings > API* by an administrator and sent
as `Authorization: Bearer lst_…`. A token is read-only and only opens `/api/v1/`; it works
whatever the [authentication](#authentication) mode (`local`, `oidc`, or none). It is shown once,
when it is created: LogStream only keeps its SHA-256 hash, in `/data/api_tokens.json`.
Revoking it cuts off access at once; the list shows when and from where each token was last
used. A logged-in browser session can call the route too.
| Parameter | Meaning |
|---|---|
| `query` | LogsQL filter (default `*`), e.g. `severity:in(err,crit) app:sshd`. Pipes (`\| ...`) are refused. |
| `since` | where to start: the `next_cursor` of the previous answer, an RFC 3339 time (`2026-11-01T08:00:00Z`) or a duration back from now (`15m`, `6h`, `7d`). Default: `1h`. |
| `limit` | rows per answer, 1 to 10,000 (default 1,000). |
| `lag` | only logs received at least this long ago are returned (default `30s`, at most `1h`), so that the logs still in the queue or waiting for their DNS name are not skipped. |
```json
{
"rows": [{"_time": "2026-11-01T14:02:11.52Z", "_msg": "…", "host": "fw01", "app": "sshd", "severity": "err", …}],
"next_cursor": "eyJ0IjoiMjAyNi0xMS0wMVQxNDowMjoxMS41MjAwMDAwMDFaIiwicSI6IjJlYjE4YzNhIn0",
"has_more": false,
"until": "2026-11-01T14:05:30.1Z"
}
```
Rows have the stored fields (as in the UI's JSON view), sorted by `_time`, the reception time.
Always send back `next_cursor`, even when `rows` is empty. `has_more: true` means more logs are
ready: call again at once. `waiting: true` means logs from the [disk buffer](#configuration) are
still being sent to VictoriaLogs: nothing is returned until they are stored, so none is missed.
The cursor belongs to its `query`: changing the query keeps the position in time. Errors come
as `{"error", "code"}` with status 400 (bad parameter or LogsQL), 401 (invalid or revoked token)
or 502 (VictoriaLogs unreachable).
```bash
curl -H "Authorization: Bearer lst_…" \
"https://logs.example.org/api/v1/logs?query=severity:err&since=15m&limit=1000"
```
### Example: n8n workflow
Four nodes, the cursor kept in the workflow's static data between runs:
1. **Schedule Trigger**: every minute.
2. **Code** (*Run Once for All Items*), reads the cursor:
```js
const data = $getWorkflowStaticData('global');
return [{ json: { since: data.cursor || '15m' } }];
```
3. **HTTP Request**: method `GET`, URL `https://logs.example.org/api/v1/logs`.
*Authentication*: Generic Credential Type > Header Auth, with name `Authorization` and value
`Bearer lst_…`. *Send Query Parameters*: `query` = `severity:in(emerg,alert,crit,err)`,
`since` = `{{ $json.since }}`, `limit` = `1000`.
4. **Code** (*Run Once for All Items*), saves the cursor and turns each log into an item:
```js
const data = $getWorkflowStaticData('global');
const page = $input.first().json;
data.cursor = page.next_cursor;
return page.rows.map((row) => ({ json: row }));
```
Then connect what the logs should feed (Slack, mail, ticket, database…). n8n only keeps static
data for runs of the active workflow, not for manual test runs: while testing, each run starts
again from `15m`. When more than 1,000 logs arrive per minute, `has_more` is true and the
following runs catch up; raise `limit` or run the trigger more often if needed.
## Settings ## Settings
The gear icon opens the settings, organized in tabs. Everything except color tags is The gear icon opens the settings, organized in tabs. Changes show at once but are only kept
remembered per browser. with the **Save** button, under the tab labels: it takes the accent color while changes are
waiting. **Cancel** goes back to the saved settings, and closing the window with unsaved changes
asks before dropping them. Localization and Appearance are remembered per browser; color tags
and log sources are saved on the server. Purging, API tokens and resetting column widths act at once.
- **Localization** - **Localization**
- *Language*: English or French. - *Language*: English or French.
@@ -400,6 +477,9 @@ remembered per browser.
(already set in `docker-compose.yml`). The feature is off by default: set `ALLOW_PURGE=true` (already set in `docker-compose.yml`). The feature is off by default: set `ALLOW_PURGE=true`
to allow it. Any admin can then purge: turn on [authentication](#authentication) if the UI to allow it. Any admin can then purge: turn on [authentication](#authentication) if the UI
is reachable by others. is reachable by others.
- **API**: API tokens for the [reading API](#reading-api-n8n-scripts): create one with a name
(shown once, copy it then), see when and from where each was last used, revoke it. Reserved
to administrators.
## Authentication ## Authentication
@@ -569,6 +649,8 @@ To update one of them:
| `hub.go` | pushes new messages to browsers (SSE) | | `hub.go` | pushes new messages to browsers (SSE) |
| `rdns.go` | cached reverse DNS lookups | | `rdns.go` | cached reverse DNS lookups |
| `export.go` | streamed CSV export | | `export.go` | streamed CSV export |
| `logsapi.go` | reading API `/api/v1/logs`: cursor, groups of logs with the same time |
| `apitokens.go` | API tokens: storage (hashed), Bearer check, `/api/tokens` |
| `docker.go` | Docker container logs (API, followers, positions, level detection) | | `docker.go` | Docker container logs (API, followers, positions, level detection) |
| `syslogserver.go` | syslog listeners opened and closed from Settings > Log sources | | `syslogserver.go` | syslog listeners opened and closed from Settings > Log sources |
| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower | | `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower |
+42
View File
@@ -28,11 +28,16 @@ type API struct {
files *Files files *Files
selfLog *SelfLog selfLog *SelfLog
dbstats dbStatsCache dbstats dbStatsCache
tokens *TokenStore
} }
func (a *API) Routes(mux *http.ServeMux) { func (a *API) Routes(mux *http.ServeMux) {
mux.HandleFunc("GET /healthz", a.health) mux.HandleFunc("GET /healthz", a.health)
mux.HandleFunc("GET /api/logs", a.logs) mux.HandleFunc("GET /api/logs", a.logs)
mux.HandleFunc("GET /api/v1/logs", a.logsV1)
mux.HandleFunc("GET /api/tokens", a.listTokens)
mux.HandleFunc("POST /api/tokens", a.createToken)
mux.HandleFunc("DELETE /api/tokens/{id}", a.deleteToken)
mux.HandleFunc("GET /api/histogram", a.histogram) mux.HandleFunc("GET /api/histogram", a.histogram)
mux.HandleFunc("GET /api/facets", a.facets) mux.HandleFunc("GET /api/facets", a.facets)
mux.HandleFunc("GET /api/stream", a.stream) mux.HandleFunc("GET /api/stream", a.stream)
@@ -40,6 +45,8 @@ func (a *API) Routes(mux *http.ServeMux) {
mux.HandleFunc("GET /api/dbstats", a.dbStats) mux.HandleFunc("GET /api/dbstats", a.dbStats)
mux.HandleFunc("GET /api/tags", a.listTags) mux.HandleFunc("GET /api/tags", a.listTags)
mux.HandleFunc("POST /api/tags", a.createTag) mux.HandleFunc("POST /api/tags", a.createTag)
mux.HandleFunc("PUT /api/tags", a.replaceTags)
mux.HandleFunc("GET /api/tags/defaults", a.defaultTags)
mux.HandleFunc("POST /api/tags/reset", a.resetTags) mux.HandleFunc("POST /api/tags/reset", a.resetTags)
mux.HandleFunc("GET /api/presets", a.listPresets) mux.HandleFunc("GET /api/presets", a.listPresets)
mux.HandleFunc("PUT /api/tags/{id}", a.updateTag) mux.HandleFunc("PUT /api/tags/{id}", a.updateTag)
@@ -531,6 +538,41 @@ func (a *API) purge(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"task_id": id}) writeJSON(w, http.StatusOK, map[string]string{"task_id": id})
} }
// PUT /api/tags [tags]: replaces the whole list. On an invalid tag, "index"
// gives its position and nothing is saved.
func (a *API) replaceTags(w http.ResponseWriter, r *http.Request) {
var in []Tag
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20)).Decode(&in); err != nil {
writeErr(w, http.StatusBadRequest, err)
return
}
tags, err := a.tags.Replace(in)
var le *tagListError
switch {
case errors.As(err, &le):
body := map[string]any{"error": err.Error(), "index": le.index}
var ce *codedError
if errors.As(err, &ce) {
body["code"] = ce.code
if ce.detail != "" {
body["detail"] = ce.detail
}
}
writeJSON(w, http.StatusBadRequest, body)
case errors.Is(err, errTooManyTags):
writeErr(w, http.StatusBadRequest, err)
case err != nil:
writeErr(w, http.StatusInternalServerError, err)
default:
writeJSON(w, http.StatusOK, tags)
}
}
// GET /api/tags/defaults: the tags "Restore default tags" shows before saving.
func (a *API) defaultTags(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, defaultTags())
}
func (a *API) resetTags(w http.ResponseWriter, r *http.Request) { func (a *API) resetTags(w http.ResponseWriter, r *http.Request) {
tags, err := a.tags.Reset() tags, err := a.tags.Reset()
if err != nil { if err != nil {
+254
View File
@@ -0,0 +1,254 @@
package main
import (
"crypto/rand"
"crypto/sha256"
"crypto/subtle"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"log"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"time"
"unicode/utf8"
)
// API tokens: read-only access to /api/v1/ for scripts and automation tools
// (n8n…), sent as "Authorization: Bearer lst_…". They are created and revoked
// in Settings > API by an administrator, and work whatever the AUTH_MODE,
// including without authentication. Only a SHA-256 of each token is kept, in
// DATA_DIR/api_tokens.json: a token is shown once, when it is created.
const (
tokenPrefix = "lst_"
maxTokens = 100
lastUsedPeriod = time.Minute // last use of a token is written to disk at most this often (or on a new IP)
)
var errTokenNotFound = errors.New("token not found")
type apiToken struct {
ID string `json:"id"`
Name string `json:"name"`
Hint string `json:"hint"` // start of the token, to recognize it
Hash string `json:"hash"` // hex SHA-256 of the whole token
Created time.Time `json:"created"`
LastUsed time.Time `json:"last_used,omitzero"`
LastIP string `json:"last_ip,omitempty"`
}
// apiTokenInfo is what the UI sees of a token (never its hash).
type apiTokenInfo struct {
ID string `json:"id"`
Name string `json:"name"`
Hint string `json:"hint"`
Created time.Time `json:"created"`
LastUsed time.Time `json:"last_used,omitzero"`
LastIP string `json:"last_ip,omitempty"`
}
type TokenStore struct {
path string
mu sync.Mutex
tokens []apiToken
}
func LoadTokenStore(path string) (*TokenStore, error) {
s := &TokenStore{path: path}
b, err := os.ReadFile(path)
if errors.Is(err, os.ErrNotExist) {
return s, nil
}
if err != nil {
return nil, err
}
if err := json.Unmarshal(b, &s.tokens); err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
return s, nil
}
// save writes the file atomically, readable by its owner only; the caller holds the lock.
func (s *TokenStore) save() error {
b, err := json.MarshalIndent(s.tokens, "", " ")
if err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(s.path), 0o755); err != nil {
return err
}
tmp := s.path + ".tmp"
if err := os.WriteFile(tmp, b, 0o600); err != nil {
return err
}
return os.Rename(tmp, s.path)
}
func hashToken(token string) string {
sum := sha256.Sum256([]byte(token))
return hex.EncodeToString(sum[:])
}
func (s *TokenStore) List() []apiTokenInfo {
s.mu.Lock()
defer s.mu.Unlock()
out := make([]apiTokenInfo, len(s.tokens))
for i, t := range s.tokens {
out[i] = apiTokenInfo{ID: t.ID, Name: t.Name, Hint: t.Hint, Created: t.Created, LastUsed: t.LastUsed, LastIP: t.LastIP}
}
return out
}
// Create makes a new token and returns it in clear, the only time it exists so.
func (s *TokenStore) Create(name string) (apiTokenInfo, string, error) {
name = strings.TrimSpace(name)
if name == "" || utf8.RuneCountInString(name) > 80 {
return apiTokenInfo{}, "", &codedError{code: "token_name", msg: "give the token a name of 1 to 80 characters"}
}
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {
return apiTokenInfo{}, "", err
}
secret := tokenPrefix + base64.RawURLEncoding.EncodeToString(b)
t := apiToken{ID: newID(), Name: name, Hint: secret[:len(tokenPrefix)+6], Hash: hashToken(secret), Created: time.Now().UTC()}
s.mu.Lock()
defer s.mu.Unlock()
if len(s.tokens) >= maxTokens {
return apiTokenInfo{}, "", &codedError{code: "token_max", msg: fmt.Sprintf("at most %d API tokens", maxTokens)}
}
s.tokens = append(s.tokens, t)
if err := s.save(); err != nil {
s.tokens = s.tokens[:len(s.tokens)-1]
return apiTokenInfo{}, "", err
}
return apiTokenInfo{ID: t.ID, Name: t.Name, Hint: t.Hint, Created: t.Created}, secret, nil
}
func (s *TokenStore) Delete(id string) error {
s.mu.Lock()
defer s.mu.Unlock()
for i, t := range s.tokens {
if t.ID == id {
s.tokens = append(s.tokens[:i:i], s.tokens[i+1:]...)
return s.save()
}
}
return errTokenNotFound
}
// Check tells whether token is a valid API token, and notes its use.
func (s *TokenStore) Check(token, ip string) (name string, ok bool) {
if !strings.HasPrefix(token, tokenPrefix) {
return "", false
}
h := []byte(hashToken(token))
s.mu.Lock()
defer s.mu.Unlock()
for i := range s.tokens {
t := &s.tokens[i]
if subtle.ConstantTimeCompare(h, []byte(t.Hash)) != 1 {
continue
}
now := time.Now().UTC()
stale := now.Sub(t.LastUsed) >= lastUsedPeriod || t.LastIP != ip
t.LastUsed, t.LastIP = now, ip
if stale {
if err := s.save(); err != nil {
log.Printf("api tokens: %v", err)
}
}
return t.Name, true
}
return "", false
}
// bearerToken returns the token of an "Authorization: Bearer …" header.
func bearerToken(r *http.Request) (string, bool) {
scheme, token, ok := strings.Cut(r.Header.Get("Authorization"), " ")
if !ok || !strings.EqualFold(scheme, "Bearer") {
return "", false
}
return strings.TrimSpace(token), true
}
// tokenAuth serves the requests carrying an API token itself, read-only and
// limited to /api/v1/, before the session login (auth); api is the handler
// behind the login.
func tokenAuth(tokens *TokenStore, auth, api http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token, ok := bearerToken(r)
if !ok {
auth.ServeHTTP(w, r)
return
}
ip := clientIP(r)
if _, valid := tokens.Check(token, ip); !valid {
log.Printf("api: invalid or revoked token from %s", ip)
time.Sleep(loginFailDelay)
w.Header().Set("WWW-Authenticate", `Bearer realm="logstream"`)
writeErr(w, http.StatusUnauthorized, &codedError{code: "token", msg: "invalid or revoked API token"})
return
}
if !strings.HasPrefix(r.URL.Path, "/api/v1/") || !isSafeMethod(r.Method) {
writeErr(w, http.StatusForbidden, &codedError{code: "token_scope", msg: "API tokens only give read access to /api/v1/"})
return
}
api.ServeHTTP(w, asViewer(r))
})
}
// GET /api/tokens: the API tokens (Settings > API), for administrators.
func (a *API) listTokens(w http.ResponseWriter, r *http.Request) {
if isViewer(r) {
writeErr(w, http.StatusForbidden, &codedError{code: "read_only", msg: "read-only account: changes are reserved to administrators"})
return
}
writeJSON(w, http.StatusOK, a.tokens.List())
}
// POST /api/tokens {"name": "n8n"}: returns the new token, with "token" in clear.
func (a *API) createToken(w http.ResponseWriter, r *http.Request) {
var body struct {
Name string `json:"name"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&body); err != nil {
writeErr(w, http.StatusBadRequest, err)
return
}
info, secret, err := a.tokens.Create(body.Name)
var ce *codedError
switch {
case errors.As(err, &ce):
writeErr(w, http.StatusBadRequest, err)
return
case err != nil:
writeErr(w, http.StatusInternalServerError, err)
return
}
log.Printf("api token %q created from %s", info.Name, clientIP(r))
writeJSON(w, http.StatusCreated, struct {
apiTokenInfo
Token string `json:"token"`
}{info, secret})
}
// DELETE /api/tokens/{id}: revokes a token at once.
func (a *API) deleteToken(w http.ResponseWriter, r *http.Request) {
err := a.tokens.Delete(r.PathValue("id"))
switch {
case errors.Is(err, errTokenNotFound):
writeErr(w, http.StatusNotFound, err)
case err != nil:
writeErr(w, http.StatusInternalServerError, err)
default:
log.Printf("api token %s revoked from %s", r.PathValue("id"), clientIP(r))
w.WriteHeader(http.StatusNoContent)
}
}
+176
View File
@@ -0,0 +1,176 @@
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestTokenStore(t *testing.T) {
path := filepath.Join(t.TempDir(), "api_tokens.json")
s, err := LoadTokenStore(path)
if err != nil {
t.Fatal(err)
}
if _, _, err := s.Create(" "); err == nil {
t.Fatal("a token without a name should be refused")
}
info, secret, err := s.Create("n8n")
if err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(secret, tokenPrefix) || len(secret) < 40 || !strings.HasPrefix(secret, info.Hint) {
t.Fatalf("token %q, hint %q", secret, info.Hint)
}
b, _ := os.ReadFile(path)
if strings.Contains(string(b), secret) || !strings.Contains(string(b), hashToken(secret)) {
t.Fatal("the file must hold the hash of the token, not the token")
}
if st, _ := os.Stat(path); st.Mode().Perm() != 0o600 {
t.Fatalf("file mode %v, want 0600", st.Mode().Perm())
}
if _, ok := s.Check(secret+"x", "10.0.0.1"); ok {
t.Fatal("a wrong token was accepted")
}
if name, ok := s.Check(secret, "10.0.0.1"); !ok || name != "n8n" {
t.Fatal("the token was refused")
}
// Reloaded from disk (restart): still valid, last use kept.
s2, err := LoadTokenStore(path)
if err != nil {
t.Fatal(err)
}
if l := s2.List(); len(l) != 1 || l[0].LastIP != "10.0.0.1" || l[0].LastUsed.IsZero() {
t.Fatalf("after reload: %+v", l)
}
if _, ok := s2.Check(secret, "10.0.0.2"); !ok {
t.Fatal("token refused after reload")
}
if err := s2.Delete(info.ID); err != nil {
t.Fatal(err)
}
if _, ok := s2.Check(secret, "10.0.0.2"); ok {
t.Fatal("a revoked token was accepted")
}
if err := s2.Delete(info.ID); err != errTokenNotFound {
t.Fatalf("second delete: %v", err)
}
}
func TestTokenAuth(t *testing.T) {
loginFailDelay = 0
tokens, _ := LoadTokenStore(filepath.Join(t.TempDir(), "api_tokens.json"))
_, secret, _ := tokens.Create("n8n")
var reached string
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
reached = r.URL.Path
if isViewer(r) {
reached += " viewer"
}
})
app := readOnly(mux)
login := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { writeAuthRequired(w) })
h := tokenAuth(tokens, login, app)
call := func(method, path, auth string) int {
reached = ""
req := httptest.NewRequest(method, path, nil)
if auth != "" {
req.Header.Set("Authorization", auth)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
return rec.Code
}
if code := call("GET", "/api/v1/logs", "Bearer "+secret); code != http.StatusOK || reached != "/api/v1/logs viewer" {
t.Fatalf("valid token: %d, reached %q", code, reached)
}
if code := call("GET", "/api/v1/logs", "bearer "+secret); code != http.StatusOK {
t.Fatalf("lower-case scheme: %d", code)
}
for _, tc := range []struct {
method, path, auth string
want int
}{
{"GET", "/api/v1/logs", "Bearer lst_wrong", http.StatusUnauthorized},
{"GET", "/api/v1/logs", "Bearer ", http.StatusUnauthorized},
{"GET", "/api/tags", "Bearer " + secret, http.StatusForbidden},
{"GET", "/api/tokens", "Bearer " + secret, http.StatusForbidden},
{"POST", "/api/v1/logs", "Bearer " + secret, http.StatusForbidden},
{"GET", "/api/v1/logs", "", http.StatusUnauthorized}, // no token: the login decides
{"GET", "/api/v1/logs", "Basic dTpw", http.StatusUnauthorized}, // Basic goes to the login
} {
if code := call(tc.method, tc.path, tc.auth); code != tc.want || reached != "" {
t.Errorf("%s %s %q: %d (reached %q), want %d", tc.method, tc.path, tc.auth, code, reached, tc.want)
}
}
}
// With local authentication, a token opens /api/v1/ without a session.
func TestTokenWithLocalAuth(t *testing.T) {
loginFailDelay = 0
dir := t.TempDir()
tokens, _ := LoadTokenStore(filepath.Join(dir, "api_tokens.json"))
_, secret, _ := tokens.Create("script")
app := readOnly(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _, _ = w.Write([]byte("ok")) }))
auth, err := newAuth(authConfig{mode: "local", user: "admin", pass: "pw", dataDir: dir, sessionTTL: time.Hour}, app)
if err != nil {
t.Fatal(err)
}
h := tokenAuth(tokens, auth, app)
req := httptest.NewRequest("GET", "/api/v1/logs", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("without token: %d", rec.Code)
}
req.Header.Set("Authorization", "Bearer "+secret)
rec = httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusOK || rec.Body.String() != "ok" {
t.Fatalf("with token: %d %s", rec.Code, rec.Body)
}
}
func TestTokenHandlers(t *testing.T) {
tokens, _ := LoadTokenStore(filepath.Join(t.TempDir(), "api_tokens.json"))
a := &API{tokens: tokens}
mux := http.NewServeMux()
a.Routes(mux)
do := func(r *http.Request) *httptest.ResponseRecorder {
rec := httptest.NewRecorder()
readOnly(mux).ServeHTTP(rec, r)
return rec
}
rec := do(httptest.NewRequest("POST", "/api/tokens", strings.NewReader(`{"name":"n8n"}`)))
var created struct {
ID, Token, Hint string
}
if err := json.Unmarshal(rec.Body.Bytes(), &created); rec.Code != http.StatusCreated || err != nil || created.Token == "" {
t.Fatalf("create: %d %s", rec.Code, rec.Body)
}
rec = do(httptest.NewRequest("GET", "/api/tokens", nil))
if rec.Code != http.StatusOK || strings.Contains(rec.Body.String(), created.Token) || strings.Contains(rec.Body.String(), "hash") {
t.Fatalf("list must not show the token or its hash: %s", rec.Body)
}
if rec := do(asViewer(httptest.NewRequest("GET", "/api/tokens", nil))); rec.Code != http.StatusForbidden {
t.Fatalf("viewer list: %d", rec.Code)
}
if rec := do(asViewer(httptest.NewRequest("DELETE", "/api/tokens/"+created.ID, nil))); rec.Code != http.StatusForbidden {
t.Fatalf("viewer delete: %d", rec.Code)
}
if rec := do(httptest.NewRequest("DELETE", "/api/tokens/"+created.ID, nil)); rec.Code != http.StatusNoContent {
t.Fatalf("delete: %d", rec.Code)
}
if rec := do(httptest.NewRequest("DELETE", "/api/tokens/"+created.ID, nil)); rec.Code != http.StatusNotFound {
t.Fatalf("delete again: %d", rec.Code)
}
}
+13 -8
View File
@@ -6,10 +6,10 @@ services:
depends_on: depends_on:
- victorialogs - victorialogs
- docker-proxy - docker-proxy
ports: # ports:
- "${SYSLOG_PORT:-5514}:5514/udp" # - "${SYSLOG_PORT:-5514}:5514/udp"
- "${SYSLOG_PORT:-5514}:5514/tcp" # - "${SYSLOG_PORT:-5514}:5514/tcp"
- "${HTTP_PORT:-8989}:8080" # - "${HTTP_PORT:-8989}:8080"
environment: environment:
VLOGS_URL: http://victorialogs:9428 VLOGS_URL: http://victorialogs:9428
SYSLOG_PUBLIC_PORT: ${SYSLOG_PORT:-514} # le port d'ecoute syslog par defaut (attention aux ports <1024) SYSLOG_PUBLIC_PORT: ${SYSLOG_PORT:-514} # le port d'ecoute syslog par defaut (attention aux ports <1024)
@@ -60,12 +60,12 @@ services:
- /run/log/journal:/host/run/log/journal:ro # journal systemd volatile - /run/log/journal:/host/run/log/journal:ro # journal systemd volatile
# audit Linux (auditd), source a activer dans Reglages > Sources ; le fichier doit etre # audit Linux (auditd), source a activer dans Reglages > Sources ; le fichier doit etre
# lisible par l'UID 10001 : log_group = adm dans /etc/audit/auditd.conf (voir le README) # lisible par l'UID 10001 : log_group = adm dans /etc/audit/auditd.conf (voir le README)
# - /var/log/audit:/var/log/audit:ro - /var/log/audit:/var/log/audit:ro
# noms des utilisateurs et de l'hote pour les evenements d'audit (optionnel) # noms des utilisateurs et de l'hote pour les evenements d'audit (optionnel)
# - /etc/passwd:/host/etc/passwd:ro - /etc/passwd:/host/etc/passwd:ro
# - /etc/hostname:/host/etc/hostname:ro - /etc/hostname:/host/etc/hostname:ro
# fichiers de logs d'autres programmes (Traefik, nginx...), a declarer dans Reglages > Sources > Fichiers de logs # fichiers de logs d'autres programmes (Traefik, nginx...), a declarer dans Reglages > Sources > Fichiers de logs
# - /var/log/traefik:/logs/traefik:ro - /home/traefik/log:/home/logs/traefik:ro
# logo de la page de connexion, avec LOGIN_LOGO=/config/logo.png dans .env # logo de la page de connexion, avec LOGIN_LOGO=/config/logo.png dans .env
# - ./logo.png:/config/logo.png:ro # - ./logo.png:/config/logo.png:ro
# prereglages de tags personnalises, avec PRESETS_FILE=/config/presets.json dans .env # prereglages de tags personnalises, avec PRESETS_FILE=/config/presets.json dans .env
@@ -102,7 +102,12 @@ services:
- /var/run/docker.sock:/var/run/docker.sock:ro - /var/run/docker.sock:/var/run/docker.sock:ro
labels: labels:
logstream.exclude: "true" # pour exclure les logs propres de logstream logstream.exclude: "true" # pour exclure les logs propres de logstream
networks:
- traefik-net
networks:
traefik-net:
external: true
volumes: volumes:
logstream-data: logstream-data:
vlogs-data: vlogs-data:
+81 -9
View File
@@ -53,6 +53,7 @@ type fileSourceStatus struct {
Open bool Open bool
Read uint64 Read uint64
Detected string // format found when the source is in auto Detected string // format found when the source is in auto
Behind int64 // bytes left to read (history being read)
Code string Code string
Error string Error string
} }
@@ -79,6 +80,7 @@ type fileTail struct {
known bool // readOff is set known bool // readOff is set
pending *pendingEntry pending *pendingEntry
auto *fileParser // format detected from the file, in auto auto *fileParser // format detected from the file, in auto
behind int64 // bytes left to read after the last poll
} }
// Files follows log files of other programs (Traefik, nginx, applications…) // Files follows log files of other programs (Traefik, nginx, applications…)
@@ -265,10 +267,11 @@ func (f *Files) Status() map[string]any {
for _, s := range f.cfg { for _, s := range f.cfg {
m := map[string]any{ m := map[string]any{
"id": s.ID, "name": s.Name, "path": s.Path, "format": s.Format, "regex": s.Regex, "id": s.ID, "name": s.Name, "path": s.Path, "format": s.Format, "regex": s.Regex,
"host": s.Host, "enabled": s.Enabled, "open": false, "read": 0, "detected": "", "code": "", "error": "", "host": s.Host, "enabled": s.Enabled, "open": false, "read": 0, "detected": "", "behind": 0, "code": "", "error": "",
} }
if st := f.status[s.ID]; st != nil && s.Enabled { if st := f.status[s.ID]; st != nil && s.Enabled {
m["open"], m["read"], m["detected"], m["code"], m["error"] = st.Open, st.Read, st.Detected, st.Code, st.Error m["open"], m["read"], m["detected"], m["code"], m["error"] = st.Open, st.Read, st.Detected, st.Code, st.Error
m["behind"] = st.Behind
} }
list = append(list, m) list = append(list, m)
} }
@@ -346,7 +349,8 @@ func (f *Files) poll(now time.Time) {
} }
const ( const (
maxFileRead = 4 << 20 // per file and per poll maxFileRead = 4 << 20 // per read
maxFileReads = 16 // reads per file and per poll (64 MB)
fileContinueDelay = 2 * time.Second // an entry waits this long for continuation lines fileContinueDelay = 2 * time.Second // an entry waits this long for continuation lines
) )
@@ -370,6 +374,10 @@ func (f *Files) setStatus(id string, err error) {
} }
changed := code != st.Code changed := code != st.Code
st.Open, st.Detected, st.Code, st.Error = err == nil, detected, code, detail st.Open, st.Detected, st.Code, st.Error = err == nil, detected, code, detail
st.Behind = 0
if tl := f.tails[id]; tl != nil && err == nil {
st.Behind = tl.behind
}
name := id name := id
for _, s := range f.cfg { for _, s := range f.cfg {
if s.ID == id { if s.ID == id {
@@ -384,9 +392,12 @@ func (f *Files) setStatus(id string, err error) {
func (f *Files) count(id string) { func (f *Files) count(id string) {
f.mu.Lock() f.mu.Lock()
if st := f.status[id]; st != nil { st := f.status[id]
st.Read++ if st == nil { // first poll of the file: its status comes after
st = &fileSourceStatus{}
f.status[id] = st
} }
st.Read++
f.mu.Unlock() f.mu.Unlock()
} }
@@ -430,11 +441,12 @@ func (f *Files) pollFile(s *fileSource, now time.Time) error {
pos = filePos{Ino: ino} pos = filePos{Ino: ino}
if first { if first {
// Turned on: the last FILES_BACKFILL of what is already there // Turned on: the last FILES_BACKFILL of what is already there
// (0: only new lines). // (0: only new lines), found by date rather than read from the
if f.backfill > 0 { // start of a file that can hold months of logs.
pos.Until = fi.Size()
} else {
pos.Off = fi.Size() pos.Off = fi.Size()
if p := f.parserOf(s, tl, path); f.backfill > 0 && p != nil {
pos.Off = backfillStart(path, p, fi.Size(), f.st.Started[s.ID].Add(-f.backfill), now)
pos.Until = fi.Size()
} }
} }
f.st.Pos[s.ID], f.dirty = pos, true f.st.Pos[s.ID], f.dirty = pos, true
@@ -469,18 +481,78 @@ func (f *Files) pollFile(s *fileSource, now time.Time) error {
if p == nil { if p == nil {
return nil // nothing complete to detect the format from yet return nil // nothing complete to detect the format from yet
} }
next, _, err := f.readFrom(s, tl, p, path, tl.readOff, pos.Until, now) for i := 0; i < maxFileReads; i++ {
next, more, err := f.readFrom(s, tl, p, path, tl.readOff, pos.Until, now)
if err != nil { if err != nil {
return err return err
} }
tl.readOff = next tl.readOff = next
if !more {
break
} }
}
}
tl.behind = max(fi.Size()-tl.readOff, 0)
if pe := tl.pending; pe != nil && now.Sub(pe.seen) >= fileContinueDelay { if pe := tl.pending; pe != nil && now.Sub(pe.seen) >= fileContinueDelay {
f.flushPending(s.ID, now) f.flushPending(s.ID, now)
} }
return nil return nil
} }
// backfillStart finds where the lines dated notBefore or later start, by
// bisection on the dates of the lines (logs are written in order). Undated
// lines are left out of the history anyway: then only new lines are read.
func backfillStart(path string, p *fileParser, size int64, notBefore, now time.Time) int64 {
fh, err := os.Open(path)
if err != nil {
return size
}
defer fh.Close()
buf := make([]byte, 64<<10)
// lineAt reads the first line starting at or after off (off itself when
// it is 0): its start and its date.
lineAt := func(off int64) (int64, time.Time, bool) {
n, _ := fh.ReadAt(buf, off)
b := buf[:n]
start := off
if off > 0 {
nl := bytes.IndexByte(b, '\n')
if nl < 0 {
return size, time.Time{}, false
}
b, start = b[nl+1:], off+int64(nl)+1
}
end := bytes.IndexByte(b, '\n')
if end < 0 {
return size, time.Time{}, false
}
r := p.parse(string(b[:end]), now)
return start, r.t, r.dated
}
if _, t, ok := lineAt(0); !ok || !t.Before(notBefore) {
if !ok {
// No date on the first line: try the last ones.
if _, _, ok := lineAt(max(size-int64(len(buf)), 0)); !ok {
return size
}
} else {
return 0 // all of it is recent
}
}
lo, hi := int64(0), size // line at lo: before notBefore; at hi: not
for hi-lo > int64(len(buf)) {
mid := lo + (hi-lo)/2
_, t, ok := lineAt(mid)
if ok && t.Before(notBefore) {
lo = mid
} else {
hi = mid
}
}
start, _, _ := lineAt(lo)
return min(start, size)
}
// readFrom parses the complete lines written after off and returns the new // readFrom parses the complete lines written after off and returns the new
// offset; more tells whether the read window was full. // offset; more tells whether the read window was full.
func (f *Files) readFrom(s *fileSource, tl *fileTail, p *fileParser, path string, off, until int64, now time.Time) (int64, bool, error) { func (f *Files) readFrom(s *fileSource, tl *fileTail, p *fileParser, path string, off, until int64, now time.Time) (int64, bool, error) {
+43
View File
@@ -1,6 +1,7 @@
package main package main
import ( import (
"fmt"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
@@ -382,3 +383,45 @@ func TestFilesAutoAndErrors(t *testing.T) {
t.Fatalf("re-enabled: got %q", msgs(got)) t.Fatalf("re-enabled: got %q", msgs(got))
} }
} }
// A large access log: the last hour is found by date, not read from the
// start, and every entry is counted from the first poll.
func TestFilesBackfillLargeFile(t *testing.T) {
dir, data := t.TempDir(), t.TempDir()
path := filepath.Join(dir, "access.log")
now := time.Now()
line := func(at time.Time, n int) string {
return fmt.Sprintf(`86.246.223.62 - - [%s] "GET /wp-content/x.png HTTP/2.0" 200 640365 "-" "-" %d "vblog-io-router@file" "http://wordpress" 3ms`+"\n",
at.UTC().Format("02/Jan/2006:15:04:05 -0700"), n)
}
var b strings.Builder
const n = 200000 // one line every 3 s for a week: ~32 MB
for i := 0; i < n; i++ {
b.WriteString(line(now.Add(-time.Duration(n-i)*3*time.Second), i))
}
appendFile(t, path, b.String())
sink := &fileSink{}
f := NewFiles(dir, data, time.Hour, sink.add)
f.Configure(filesConfig{Sources: []fileSourceDef{{ID: "a1", Name: "traefik-log", Path: path, Enabled: true}}})
f.poll(now)
if pos := f.st.Pos["a1"]; pos.Off < int64(b.Len())*99/100 {
t.Errorf("history read from offset %d of %d", pos.Off, b.Len())
}
appendFile(t, path, line(now, n))
f.poll(now.Add(time.Second))
got := sink.take()
// The last hour (1200 lines, the oldest one on the edge) and the new line.
if len(got) < 1200 || len(got) > 1201 || got[0].App != "traefik-log" || got[len(got)-1].Extra["http_status"] != "200" {
t.Fatalf("got %d entries", len(got))
}
if st := f.Status()["sources"].([]map[string]any)[0]; st["read"] != uint64(len(got)) || st["behind"] != int64(0) || st["detected"] != fmtCLF {
t.Errorf("status %+v", st)
}
// Undated plain text: only new lines.
txt := filepath.Join(dir, "app.log")
appendFile(t, txt, strings.Repeat("no date here\n", 1000))
p := mustParser(t, fmtText, "")
if off := backfillStart(txt, p, 13000, now.Add(-time.Hour), now); off != 13000 {
t.Errorf("undated: %d", off)
}
}
+281
View File
@@ -0,0 +1,281 @@
package main
import (
"context"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"net/http"
"sort"
"strconv"
"strings"
"time"
)
// GET /api/v1/logs: incremental reading of the stored logs for automation
// tools (n8n, scripts). Each answer gives the rows that follow a cursor,
// oldest first, and the cursor to send next time; following the cursors
// returns every row once, without gaps or duplicates.
//
// The cursor is a position on _time, which is the reception time given by
// LogStream (see Entry.Record), so new rows always come after it. Rows only
// count once they are older than the lag (30 s by default): a message can
// wait a little in the queue or for its DNS name before VictoriaLogs makes it
// visible. Several rows can share the same _time to the nanosecond; inside
// such a group the rows are ordered by a hash of their content, and the
// cursor remembers how many of them were already returned.
const (
logsAPIDefaultLimit = 1000
logsAPIMaxLimit = 10000
logsAPIDefaultLag = 30 * time.Second
logsAPIMaxLag = time.Hour
logsAPIDefaultSince = time.Hour
logsAPIMaxGroup = 100000 // rows with the very same _time
)
// logsCursor: every row with _time < T has been returned, plus the first K
// rows (in hash order) of those with _time == T. Q is a hash of the query:
// K only means something for the same query.
type logsCursor struct {
T time.Time `json:"t"`
K int `json:"k,omitempty"`
Q string `json:"q,omitempty"`
}
func (c logsCursor) encode() string {
b, _ := json.Marshal(c)
return base64.RawURLEncoding.EncodeToString(b)
}
func decodeCursor(s string) (logsCursor, bool) {
var c logsCursor
b, err := base64.RawURLEncoding.DecodeString(s)
if err != nil || json.Unmarshal(b, &c) != nil || c.T.IsZero() || c.K < 0 {
return logsCursor{}, false
}
return c, true
}
// parseSince reads the since parameter: a cursor from a previous answer, an
// RFC 3339 time, or a duration back from now (90s, 15m, 6h, 7d). Empty: the
// last hour.
func parseSince(s string, now time.Time) (logsCursor, error) {
s = strings.TrimSpace(s)
if s == "" {
return logsCursor{T: now.Add(-logsAPIDefaultSince)}, nil
}
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
return logsCursor{T: t}, nil
}
if d, ok := parseAge(s); ok {
return logsCursor{T: now.Add(-d)}, nil
}
if c, ok := decodeCursor(s); ok {
return c, nil
}
return logsCursor{}, &codedError{code: "bad_since", msg: "since: expected a next_cursor, an RFC 3339 time or a duration such as 15m, 6h or 7d"}
}
// parseAge reads a Go duration, plus "Nd" for days.
func parseAge(s string) (time.Duration, bool) {
if n, ok := strings.CutSuffix(s, "d"); ok {
days, err := strconv.Atoi(n)
if err != nil || days < 0 {
return 0, false
}
return time.Duration(days) * 24 * time.Hour, true
}
d, err := time.ParseDuration(s)
return d, err == nil && d >= 0
}
type logsPage struct {
Rows []map[string]any `json:"rows"`
NextCursor string `json:"next_cursor"`
HasMore bool `json:"has_more"` // call again right away: more rows are ready
Waiting bool `json:"waiting,omitempty"` // the disk buffer is still being sent to VictoriaLogs
Until string `json:"until"` // rows are returned up to this time (now - lag)
}
// GET /api/v1/logs?query=<LogsQL filter>&since=<cursor|time|duration>&limit=<n>&lag=<duration>
func (a *API) logsV1(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
filter := strings.TrimSpace(q.Get("query"))
if f, pipes := splitPipes(filter); strings.TrimSpace(pipes) != "" {
writeErr(w, http.StatusBadRequest, &codedError{code: "api_pipes", msg: "query: LogsQL pipes (| ...) are not supported, keep only the filter part"})
return
} else {
filter = strings.TrimSpace(f)
}
if filter == "" {
filter = "*"
}
limit := logsAPIDefaultLimit
if v := q.Get("limit"); v != "" {
n, err := strconv.Atoi(v)
if err != nil || n <= 0 || n > logsAPIMaxLimit {
writeErr(w, http.StatusBadRequest, &codedError{code: "bad_limit", msg: fmt.Sprintf("limit: expected 1 to %d", logsAPIMaxLimit)})
return
}
limit = n
}
lag := logsAPIDefaultLag
if v := q.Get("lag"); v != "" {
d, ok := parseAge(v)
if !ok || d > logsAPIMaxLag {
writeErr(w, http.StatusBadRequest, &codedError{code: "bad_lag", msg: "lag: expected a duration from 0s to 1h"})
return
}
lag = d
}
now := time.Now()
cur, err := parseSince(q.Get("since"), now)
if err != nil {
writeErr(w, http.StatusBadRequest, err)
return
}
qh := queryHash(filter)
if cur.Q != qh {
cur.K = 0 // the query changed: the rank inside the group no longer applies
}
cur.Q = qh
// Logs waiting in the disk buffer were received earlier: returning newer
// rows now would move the cursor past them.
if a.store.PendingSpool() > 0 {
writeJSON(w, http.StatusOK, logsPage{Rows: []map[string]any{}, NextCursor: cur.encode(), Waiting: true, Until: logsqlTime(cur.T)})
return
}
until := now.Add(-lag)
page, err := readLogsPage(r.Context(), a.store.Query, filter, cur, until, limit)
if err != nil {
writeErr(w, queryStatus(err), err)
return
}
a.annotateHosts(page.Rows)
writeJSON(w, http.StatusOK, page)
}
func queryHash(filter string) string {
sum := sha256.Sum256([]byte(filter))
return hex.EncodeToString(sum[:4])
}
type queryFunc func(ctx context.Context, query string) ([]map[string]any, error)
// readLogsPage returns at most limit rows after cur, up to until included.
func readLogsPage(ctx context.Context, run queryFunc, filter string, cur logsCursor, until time.Time, limit int) (logsPage, error) {
page := logsPage{Rows: []map[string]any{}, Until: logsqlTime(until)}
next := cur
start := cur.T
if cur.K > 0 {
// Rest of the group the previous page stopped in.
group, err := readGroup(ctx, run, filter, cur.T)
if err != nil {
return page, err
}
rest := group[min(cur.K, len(group)):]
if len(rest) >= limit {
page.Rows = rest[:limit]
next.K = cur.K + limit
page.HasMore = true
page.NextCursor = next.encode()
return page, nil
}
page.Rows = append(page.Rows, rest...)
start = cur.T.Add(time.Nanosecond)
}
left := limit - len(page.Rows)
var rows []map[string]any
if !start.After(until) {
q := fmt.Sprintf("_time:[%s, %s] (%s) | sort by (_time) | limit %d", logsqlTime(start), logsqlTime(until), filter, left+1)
var err error
if rows, err = run(ctx, q); err != nil {
return page, err
}
}
if len(rows) <= left {
// Everything up to until has been read.
page.Rows = append(page.Rows, rows...)
next = logsCursor{T: until.Add(time.Nanosecond), Q: cur.Q}
if next.T.Before(start) {
next.T = start
}
page.NextCursor = next.encode()
return page, nil
}
// One row too many: the group of the last time may be cut. Whole groups
// before it are returned; that last group starts the next page.
page.HasMore = true
last, err := rowTime(rows[left])
if err != nil {
return page, err
}
n := 0
for n < left {
t, err := rowTime(rows[n])
if err != nil {
return page, err
}
if !t.Before(last) {
break
}
n++
}
if n > 0 || len(page.Rows) > 0 {
page.Rows = append(page.Rows, rows[:n]...)
page.NextCursor = logsCursor{T: last, Q: cur.Q}.encode()
return page, nil
}
// The whole page is a single group: read it all to cut it in a stable order.
group, err := readGroup(ctx, run, filter, last)
if err != nil {
return page, err
}
page.Rows = group[:min(left, len(group))]
page.NextCursor = logsCursor{T: last, K: len(page.Rows), Q: cur.Q}.encode()
return page, nil
}
// readGroup returns the rows whose _time is exactly t, in a stable order.
func readGroup(ctx context.Context, run queryFunc, filter string, t time.Time) ([]map[string]any, error) {
ts := logsqlTime(t)
rows, err := run(ctx, fmt.Sprintf("_time:[%s, %s] (%s) | limit %d", ts, ts, filter, logsAPIMaxGroup+1))
if err != nil {
return nil, err
}
if len(rows) > logsAPIMaxGroup {
return nil, fmt.Errorf("more than %d logs share the time %s", logsAPIMaxGroup, ts)
}
keys := make([]string, len(rows))
for i, row := range rows {
b, _ := json.Marshal(row) // map keys are sorted: same row, same bytes
sum := sha256.Sum256(b)
keys[i] = string(sum[:])
}
idx := make([]int, len(rows))
for i := range idx {
idx[i] = i
}
sort.SliceStable(idx, func(i, j int) bool { return keys[idx[i]] < keys[idx[j]] })
out := make([]map[string]any, len(rows))
for i, k := range idx {
out[i] = rows[k]
}
return out, nil
}
func rowTime(row map[string]any) (time.Time, error) {
s, _ := row["_time"].(string)
t, err := time.Parse(time.RFC3339Nano, s)
if err != nil {
return time.Time{}, errors.New("VictoriaLogs returned a row without a valid _time")
}
return t, nil
}
+276
View File
@@ -0,0 +1,276 @@
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"math/rand"
"net/http"
"net/http/httptest"
"regexp"
"sort"
"strconv"
"strings"
"sync"
"testing"
"time"
)
// cursorVL plays VictoriaLogs for the two queries of /api/v1/logs: a time
// range sorted by _time, and an exact time. Like VictoriaLogs, it gives no
// order among rows of the same _time: they are shuffled on every query.
type cursorVL struct {
mu sync.Mutex
rows []map[string]any
rnd *rand.Rand
queries []string
}
var cursorQuery = regexp.MustCompile(`^_time:\[([^,]+), ([^\]]+)] \((.*?)\)( \| sort by \(_time\))? \| limit (\d+)$`)
func (v *cursorVL) add(t time.Time, msg string) {
v.mu.Lock()
defer v.mu.Unlock()
v.rows = append(v.rows, map[string]any{"_time": t.UTC().Format(time.RFC3339Nano), "_msg": msg, "host": "h1"})
}
func (v *cursorVL) RoundTrip(r *http.Request) (*http.Response, error) {
_ = r.ParseForm()
q := r.PostForm.Get("query")
m := cursorQuery.FindStringSubmatch(q)
if m == nil {
return &http.Response{StatusCode: http.StatusBadRequest, Body: io.NopCloser(strings.NewReader("unexpected query " + q)), Header: http.Header{}}, nil
}
from, _ := time.Parse(time.RFC3339Nano, m[1])
to, _ := time.Parse(time.RFC3339Nano, m[2])
limit, _ := strconv.Atoi(m[5])
v.mu.Lock()
defer v.mu.Unlock()
v.queries = append(v.queries, q)
var out []map[string]any
for _, row := range v.rows {
t, _ := time.Parse(time.RFC3339Nano, row["_time"].(string))
if !t.Before(from) && !t.After(to) {
out = append(out, row)
}
}
v.rnd.Shuffle(len(out), func(i, j int) { out[i], out[j] = out[j], out[i] })
if m[4] != "" {
sort.SliceStable(out, func(i, j int) bool {
a, _ := time.Parse(time.RFC3339Nano, out[i]["_time"].(string))
b, _ := time.Parse(time.RFC3339Nano, out[j]["_time"].(string))
return a.Before(b)
})
}
if len(out) > limit {
out = out[:limit]
}
var b strings.Builder
for _, row := range out {
line, _ := json.Marshal(row)
b.Write(line)
b.WriteByte('\n')
}
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(b.String())), Header: http.Header{}}, nil
}
func newCursorVL(seed int64) (*cursorVL, *Store) {
vl := &cursorVL{rnd: rand.New(rand.NewSource(seed))}
store := NewStore("http://vl", 10, 10, time.Second, nil)
store.client.Transport = vl
store.streamClient.Transport = vl
return vl, store
}
// fill adds rows by groups of the same _time, some groups larger than a
// page, some with identical rows, and returns the multiset of the rows.
func fill(vl *cursorVL, start time.Time, groups []int, tag string) map[string]int {
want := map[string]int{}
t := start
for g, size := range groups {
for i := 0; i < size; i++ {
msg := fmt.Sprintf("%s g%d r%d", tag, g, i%4) // r0..r3 repeat: identical rows
vl.add(t, msg)
want[rowKey(t, msg)]++
}
t = t.Add(time.Duration(1+g%3) * time.Microsecond)
}
return want
}
func rowKey(t time.Time, msg string) string { return t.UTC().Format(time.RFC3339Nano) + " " + msg }
func TestLogsCursorNoLossNoDuplicate(t *testing.T) {
base := time.Date(2026, 11, 1, 10, 0, 0, 0, time.UTC)
groups := []int{1, 3, 1, 1, 7, 2, 1, 12, 1, 4, 4, 1, 25, 1, 2}
for _, limit := range []int{1, 2, 3, 5, 8, 1000} {
for seed := int64(1); seed <= 5; seed++ {
vl, store := newCursorVL(seed)
want := fill(vl, base, groups, "a")
got := map[string]int{}
cur := logsCursor{T: base.Add(-time.Minute)}
until := base.Add(time.Second)
var last time.Time
for calls := 0; ; calls++ {
if calls > 500 {
t.Fatalf("limit %d: no end", limit)
}
page, err := readLogsPage(context.Background(), store.Query, "*", cur, until, limit)
if err != nil {
t.Fatal(err)
}
if len(page.Rows) > limit {
t.Fatalf("limit %d: page of %d rows", limit, len(page.Rows))
}
for _, row := range page.Rows {
rt, _ := rowTime(row)
if rt.Before(last) {
t.Fatalf("limit %d: rows out of order (%s after %s)", limit, rt, last)
}
last = rt
got[rowKey(rt, row["_msg"].(string))]++
}
var ok bool
if cur, ok = decodeCursor(page.NextCursor); !ok {
t.Fatalf("bad cursor %q", page.NextCursor)
}
if !page.HasMore {
break
}
if len(page.Rows) == 0 {
t.Fatalf("limit %d: has_more with an empty page", limit)
}
}
if fmt.Sprint(got) != fmt.Sprint(want) {
t.Fatalf("limit %d seed %d:\n got %v\nwant %v", limit, seed, got, want)
}
}
}
}
// New logs arrive between the calls, as with a scheduled n8n workflow.
func TestLogsCursorFollowsNewLogs(t *testing.T) {
vl, store := newCursorVL(7)
base := time.Date(2026, 11, 1, 10, 0, 0, 0, time.UTC)
want := map[string]int{}
got := map[string]int{}
cur := logsCursor{T: base}
for round := 0; round < 6; round++ {
start := base.Add(time.Duration(round) * time.Second)
for k, n := range fill(vl, start, []int{2, 5, 1, 3}, fmt.Sprintf("round%d", round)) {
want[k] += n
}
until := start.Add(500 * time.Millisecond) // after this round's rows
for {
page, err := readLogsPage(context.Background(), store.Query, "*", cur, until, 4)
if err != nil {
t.Fatal(err)
}
for _, row := range page.Rows {
rt, _ := rowTime(row)
got[rowKey(rt, row["_msg"].(string))]++
}
cur, _ = decodeCursor(page.NextCursor)
if !page.HasMore {
break
}
}
// Polling again with nothing new returns nothing.
page, err := readLogsPage(context.Background(), store.Query, "*", cur, until, 4)
if err != nil || len(page.Rows) != 0 || page.HasMore {
t.Fatalf("round %d: second poll returned %d rows (has_more %v, err %v)", round, len(page.Rows), page.HasMore, err)
}
}
if fmt.Sprint(got) != fmt.Sprint(want) {
t.Fatalf("got %v\nwant %v", got, want)
}
}
func TestParseSince(t *testing.T) {
now := time.Date(2026, 11, 1, 12, 0, 0, 0, time.UTC)
c := logsCursor{T: now.Add(-time.Minute), K: 3, Q: "abcd"}
for _, tc := range []struct {
in string
want logsCursor
}{
{"", logsCursor{T: now.Add(-time.Hour)}},
{"15m", logsCursor{T: now.Add(-15 * time.Minute)}},
{"7d", logsCursor{T: now.Add(-7 * 24 * time.Hour)}},
{"2026-11-01T08:30:00.5Z", logsCursor{T: time.Date(2026, 11, 1, 8, 30, 0, 5e8, time.UTC)}},
{c.encode(), c},
} {
got, err := parseSince(tc.in, now)
if err != nil || !got.T.Equal(tc.want.T) || got.K != tc.want.K || got.Q != tc.want.Q {
t.Errorf("parseSince(%q) = %+v, %v; want %+v", tc.in, got, err, tc.want)
}
}
for _, bad := range []string{"yesterday", "-5m", "e30"} {
if _, err := parseSince(bad, now); err == nil {
t.Errorf("parseSince(%q): expected an error", bad)
}
}
}
func TestLogsV1Handler(t *testing.T) {
vl, store := newCursorVL(3)
a := &API{store: store, rdns: NewReverseDNS(false, "")}
old := time.Now().Add(-10 * time.Minute)
fill(vl, old, []int{2, 1}, "x")
vl.add(time.Now().Add(time.Hour), "in the future: after until")
get := func(url string) (int, logsPage, map[string]string) {
rec := httptest.NewRecorder()
a.logsV1(rec, httptest.NewRequest("GET", url, nil))
var page logsPage
var e map[string]string
_ = json.Unmarshal(rec.Body.Bytes(), &page)
_ = json.Unmarshal(rec.Body.Bytes(), &e)
return rec.Code, page, e
}
code, page, _ := get("/api/v1/logs?query=host:h1&since=1h")
if code != http.StatusOK || len(page.Rows) != 3 || page.HasMore || page.NextCursor == "" {
t.Fatalf("status %d, %d rows, has_more %v", code, len(page.Rows), page.HasMore)
}
if q := vl.queries[len(vl.queries)-1]; !strings.Contains(q, "(host:h1)") {
t.Fatalf("query filter not applied: %s", q)
}
code, page, _ = get("/api/v1/logs?query=host:h1&since=" + page.NextCursor)
if code != http.StatusOK || len(page.Rows) != 0 {
t.Fatalf("second call: status %d, %d rows", code, len(page.Rows))
}
for url, wantCode := range map[string]string{
"/api/v1/logs?query=" + "*%20|%20stats%20count()": "api_pipes",
"/api/v1/logs?limit=0": "bad_limit",
"/api/v1/logs?limit=20000": "bad_limit",
"/api/v1/logs?lag=2h": "bad_lag",
"/api/v1/logs?since=nope": "bad_since",
} {
if code, _, e := get(url); code != http.StatusBadRequest || e["code"] != wantCode {
t.Errorf("%s: status %d, %v; want 400 %s", url, code, e, wantCode)
}
}
}
func TestLogsV1WaitsForSpool(t *testing.T) {
spool, err := OpenSpool(t.TempDir(), 1<<20)
if err != nil {
t.Fatal(err)
}
if err := spool.Write([]byte(`{"_msg":"x"}`+"\n"), 1); err != nil {
t.Fatal(err)
}
vl, _ := newCursorVL(1)
store := NewStore("http://vl", 10, 10, time.Second, spool)
store.streamClient.Transport = vl
a := &API{store: store}
rec := httptest.NewRecorder()
a.logsV1(rec, httptest.NewRequest("GET", "/api/v1/logs?since=2026-11-01T10:00:00Z", nil))
var page logsPage
_ = json.Unmarshal(rec.Body.Bytes(), &page)
c, _ := decodeCursor(page.NextCursor)
if rec.Code != http.StatusOK || !page.Waiting || len(page.Rows) != 0 || !c.T.Equal(time.Date(2026, 11, 1, 10, 0, 0, 0, time.UTC)) || len(vl.queries) != 0 {
t.Fatalf("status %d, waiting %v, %d rows, cursor %v, %d queries", rec.Code, page.Waiting, len(page.Rows), c.T, len(vl.queries))
}
}
+9 -3
View File
@@ -142,6 +142,10 @@ func main() {
if err != nil { if err != nil {
log.Fatalf("tags: %v", err) log.Fatalf("tags: %v", err)
} }
tokens, err := LoadTokenStore(filepath.Join(cfg.dataDir, "api_tokens.json"))
if err != nil {
log.Fatalf("api tokens: %v", err)
}
var spool *Spool var spool *Spool
if cfg.spoolMax > 0 { if cfg.spoolMax > 0 {
@@ -200,7 +204,7 @@ func main() {
} }
mux := http.NewServeMux() mux := http.NewServeMux()
presets := getenv("PRESETS_FILE", filepath.Join(cfg.dataDir, "presets.json")) presets := getenv("PRESETS_FILE", filepath.Join(cfg.dataDir, "presets.json"))
api := &API{store: store, hub: hub, tags: tags, presets: presets, rdns: rdns, allowPurge: cfg.allowPurge, exportMax: cfg.exportMax, syslog: syslogSrv} api := &API{store: store, hub: hub, tags: tags, presets: presets, rdns: rdns, allowPurge: cfg.allowPurge, exportMax: cfg.exportMax, syslog: syslogSrv, tokens: tokens}
if cfg.dockerLogs { if cfg.dockerLogs {
dm, err := NewDockerManager(cfg.dockerHost, cfg.dataDir, cfg.backfill, sink) dm, err := NewDockerManager(cfg.dockerHost, cfg.dataDir, cfg.backfill, sink)
if err != nil { if err != nil {
@@ -228,7 +232,8 @@ func main() {
api.Routes(mux) api.Routes(mux)
mux.Handle("GET /", http.FileServer(http.FS(static))) mux.Handle("GET /", http.FileServer(http.FS(static)))
handler, err := newAuth(cfg.auth, readOnly(mux)) app := readOnly(mux)
handler, err := newAuth(cfg.auth, app)
if err != nil { if err != nil {
log.Fatalf("auth: %v", err) log.Fatalf("auth: %v", err)
} }
@@ -241,7 +246,8 @@ func main() {
if !authOn { if !authOn {
log.Printf("warning: no authentication (AUTH_USER is empty): anyone who can reach %s can read the logs and change the settings", cfg.httpAddr) log.Printf("warning: no authentication (AUTH_USER is empty): anyone who can reach %s can read the logs and change the settings", cfg.httpAddr)
} }
handler = secure(handler, contentSecurityPolicy(static), authOn) // API tokens (Settings > API) are checked before the login, in every auth mode.
handler = secure(tokenAuth(tokens, handler, app), contentSecurityPolicy(static), authOn)
srv := &http.Server{ srv := &http.Server{
Addr: cfg.httpAddr, Addr: cfg.httpAddr,
Handler: handler, Handler: handler,
+9
View File
@@ -423,6 +423,15 @@ func queryStatus(err error) int {
return http.StatusBadGateway return http.StatusBadGateway
} }
// PendingSpool is the number of logs waiting in the disk spool.
func (s *Store) PendingSpool() int64 {
if s.spool == nil {
return 0
}
lines, _ := s.spool.Pending()
return lines
}
func (s *Store) Stats() map[string]any { func (s *Store) Stats() map[string]any {
st := map[string]any{ st := map[string]any{
"received": s.received.Load(), "received": s.received.Load(),
+45
View File
@@ -217,6 +217,51 @@ func (s *TagStore) Delete(id string) error {
return errTagNotFound return errTagNotFound
} }
// tagListError tells which tag of a list (PUT /api/tags) is invalid.
type tagListError struct {
index int
err error
}
func (e *tagListError) Error() string { return e.err.Error() }
func (e *tagListError) Unwrap() error { return e.err }
// Replace saves a whole list at once (the Save button of the settings). A tag
// whose id is known keeps its code; the others get a new id and a free code.
// Nothing is saved when a tag is invalid: the error gives its position.
func (s *TagStore) Replace(in []Tag) ([]Tag, error) {
if len(in) > 99 {
return nil, errTooManyTags
}
s.mu.Lock()
defer s.mu.Unlock()
codes := map[string]string{}
for _, t := range s.tags {
codes[t.ID] = t.Code
}
out := make([]Tag, len(in))
seen := map[string]bool{}
for i, t := range in {
if err := t.validate(); err != nil {
return nil, &tagListError{index: i, err: err}
}
if c, ok := codes[t.ID]; ok && !seen[t.ID] {
t.Code = c
} else {
t.ID, t.Code = newID(), ""
}
seen[t.ID] = true
out[i] = t
}
for i := range out {
if out[i].Code == "" {
out[i].Code = freeCode(out)
}
}
s.tags = out
return append([]Tag(nil), out...), s.save()
}
func (s *TagStore) Reset() ([]Tag, error) { func (s *TagStore) Reset() ([]Tag, error) {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
+35
View File
@@ -1,6 +1,7 @@
package main package main
import ( import (
"errors"
"os" "os"
"path/filepath" "path/filepath"
"slices" "slices"
@@ -52,3 +53,37 @@ func TestTagCodes(t *testing.T) {
t.Fatalf("reloaded code = %q, want 07", got) t.Fatalf("reloaded code = %q, want 07", got)
} }
} }
func TestTagReplace(t *testing.T) {
s, err := LoadTagStore(filepath.Join(t.TempDir(), "tags.json"))
if err != nil {
t.Fatal(err)
}
// Defaults: warning 01, error 02. Keep error, drop warning, add two tags
// (one with an unknown id, one repeating a known id).
got, err := s.Replace([]Tag{
{ID: "new1", Pattern: "a", Color: "#000000"},
{ID: "error", Code: "42", Pattern: "error", Color: "#111111", Enabled: true},
{ID: "error", Pattern: "b", Color: "#000000"},
})
if err != nil {
t.Fatal(err)
}
codes := []string{got[0].Code, got[1].Code, got[2].Code}
if want := []string{"01", "02", "03"}; !slices.Equal(codes, want) {
t.Fatalf("codes = %v, want %v", codes, want)
}
if got[1].ID != "error" || got[0].ID == "new1" || got[2].ID == "error" || got[1].Color != "#111111" {
t.Fatalf("ids/fields = %+v", got)
}
// An invalid tag saves nothing and gives its position.
_, err = s.Replace([]Tag{{Pattern: "ok", Color: "#000000"}, {Pattern: "(", Regex: true, Color: "#000000"}})
var le *tagListError
if !errors.As(err, &le) || le.index != 1 {
t.Fatalf("err = %v, want tagListError at 1", err)
}
if n := len(s.List()); n != 3 {
t.Fatalf("list after a failed Replace has %d tags, want 3", n)
}
}
+393 -115
View File
@@ -40,13 +40,16 @@ const I18N = {
copyMsg: 'Copy message', copyJson: 'Copy as JSON', copied: 'Copied', copyMsg: 'Copy message', copyJson: 'Copy as JSON', copied: 'Copied',
language: 'Language', language: 'Language',
tagsTitle: 'Color tags', tagsTitle: 'Color tags',
tagsHelp: 'Keywords are highlighted in messages. Changes are saved automatically. When tags overlap, the one highest in the list wins.', tagsHelp: 'Keywords are highlighted in messages. Changes apply with the Save button. When tags overlap, the one highest in the list wins.',
addTag: '+ Add tag', resetTags: 'Restore default tags', addTag: '+ Add tag', resetTags: 'Restore default tags',
optWhole: 'whole word', optWholeTitle: 'Only color whole words (avoids "ok" in "token")', optWhole: 'whole word', optWholeTitle: 'Only color whole words (avoids "ok" in "token")',
optCaseTitle: 'Match case', optRegexTitle: 'Regular expression', optCaseTitle: 'Match case', optRegexTitle: 'Regular expression',
optActive: 'active', optActiveTitle: 'Enable this tag', optActive: 'active', optActiveTitle: 'Enable this tag',
colorAria: 'Color', keyword: 'keyword', preview: 'preview', del: 'Delete', noTags: 'No tags.', colorAria: 'Color', keyword: 'keyword', preview: 'preview', del: 'Delete', noTags: 'No tags.',
newTag: 'new', newTag: 'new',
saveSettings: 'Save', cancelSettings: 'Cancel',
settingsClean: 'No pending changes', settingsDirty: 'Unsaved changes', settingsSaved: 'Settings saved',
unsavedConfirm: 'Some changes have not been saved. Discard them?',
confirmDelete: (p) => `Delete tag "${p}"?`, confirmDelete: (p) => `Delete tag "${p}"?`,
confirmReset: 'Replace all tags with the defaults (warning, error)?', confirmReset: 'Replace all tags with the defaults (warning, error)?',
presetAria: 'Add a preset', presetPick: '+ Preset…', presetAria: 'Add a preset', presetPick: '+ Preset…',
@@ -98,6 +101,7 @@ const I18N = {
fileDraft: 'Fill in the name and the path to save this file.', fileDraft: 'Fill in the name and the path to save this file.',
fileOk: (r) => `Followed: ${r} entries since startup.`, fileOk: (r) => `Followed: ${r} entries since startup.`,
fileDetected: (f) => ` Format detected: ${f}.`, fileDetected: (f) => ` Format detected: ${f}.`,
fileBehind: (mb) => ` Reading the history: ${mb} MB left.`,
probeFound: (f) => `File found, format detected: ${f}.`, probeFound: (f) => `File found, format detected: ${f}.`,
probeSample: 'Last line: ', probeSample: 'Last line: ',
file_empty: 'The file is empty for now: its format will be detected from its first lines.', file_empty: 'The file is empty for now: its format will be detected from its first lines.',
@@ -105,7 +109,7 @@ const I18N = {
file_not_found: 'No file at this path: mount it in the container (see docker-compose.yml) and check the path.', file_not_found: 'No file at this path: mount it in the container (see docker-compose.yml) and check the path.',
file_permission: 'Permission denied: the file must be readable by UID 10001 or the adm group (see the README). ', file_permission: 'Permission denied: the file must be readable by UID 10001 or the adm group (see the README). ',
file_read: 'Cannot read the file: ', file_read: 'Cannot read the file: ',
filesHelp: 'Follows the log file of another program, mounted read-only in the container (e.g. /var/log/traefik:/logs/traefik:ro, then /logs/traefik/access.log here). Once the path is filled in, the file is checked and its format detected from its last lines (JSON lines, logfmt, access logs of nginx, Apache or Traefik, else plain text); choose a format to force it, or a regular expression whose named groups time, level, msg, host and app fill these fields, the other groups giving extra fields. Each line gives an entry (badge FILE, app = the name); indented lines are added to the previous entry; for access logs 5xx are errors and 4xx warnings. When turned on, the last hour of dated lines is read first (FILES_BACKFILL); rotations are followed and reading resumes where it stopped.', filesHelp: 'Follows the log file of another program, mounted read-only in the container (e.g. /var/log/traefik:/logs/traefik:ro, then /logs/traefik/access.log here). Once the path is filled in, the file is checked and its format detected from its last lines (JSON lines, logfmt, access logs of nginx, Apache or Traefik, else plain text); choose a format to force it, or a regular expression whose named groups time, level, msg, host and app fill these fields, the other groups giving extra fields. Each line gives an entry (badge FILE at the start of the line, app = the name); indented lines are added to the previous entry; for access logs 5xx are errors and 4xx warnings. When turned on, the last hour of dated lines is read first (FILES_BACKFILL); rotations are followed and reading resumes where it stopped.',
auditTitle: 'Linux audit (auditd)', auditTitle: 'Linux audit (auditd)',
auditEnabled: 'Collect the audit events of this machine', auditEnabled: 'Collect the audit events of this machine',
auditPath: 'File', auditPath: 'File',
@@ -170,6 +174,25 @@ const I18N = {
exportNote: (n) => `Up to ${n} logs matching the current filters, newest first. Dates use the time zone chosen in Settings.`, exportNote: (n) => `Up to ${n} logs matching the current filters, newest first. Dates use the time zone chosen in Settings.`,
exporting: 'Exporting…', exportDone: (f) => `Downloaded: ${f}`, exportError: 'Export failed: ', exporting: 'Exporting…', exportDone: (f) => `Downloaded: ${f}`, exportError: 'Export failed: ',
err_export_pipes: 'Export does not support LogsQL pipes (| …): keep only the filter part', err_export_pipes: 'Export does not support LogsQL pipes (| …): keep only the filter part',
tabApi: 'API',
tokensTitle: 'API tokens',
tokensHelp: 'Tokens give scripts and automation tools (n8n…) read-only access to /api/v1/logs, whatever the authentication mode. Only a hash is kept: a token is shown once, when it is created. Revoking it cuts off access at once.',
tokenCount: (n) => `${n} token${n > 1 ? 's' : ''}`,
tokenNone: 'No API token yet.',
tokenNamePh: 'Token name (n8n, backup script…)',
tokenCreate: 'Create a token',
tokenOnce: 'Copy this token now: it will not be shown again.',
tokenCopy: 'Copy',
tokenRevoke: 'Revoke',
tokenRevokeAsk: (n) => `Revoke the token "${n}"? Tools using it lose access at once.`,
tokenRevoked: 'Token revoked',
tokenCreated: (d) => `created ${d}`,
tokenUsed: (p) => `last used ${p.d}${p.ip ? ' from ' + p.ip : ''}`,
tokenNever: 'never used',
tokensUse: 'Usage',
tokensUseHelp: 'Send the token in an Authorization header. Each answer has next_cursor: pass it as since on the next call to get the following logs, without gaps or duplicates (see the README for an n8n workflow).',
err_token_name: 'Give the token a name of 1 to 80 characters',
err_token_max: 'Too many API tokens',
tabLocale: 'Localization', tabFilters: 'Patterns', tabInterface: 'Appearance', tabData: 'Database', tabLocale: 'Localization', tabFilters: 'Patterns', tabInterface: 'Appearance', tabData: 'Database',
themeTitle: 'Theme', themeSystem: 'System', themeLight: 'Light', themeDark: 'Dark', themeTitle: 'Theme', themeSystem: 'System', themeLight: 'Light', themeDark: 'Dark',
themeHelp: 'System follows the light/dark preference of your computer or phone.', themeHelp: 'System follows the light/dark preference of your computer or phone.',
@@ -224,7 +247,7 @@ const I18N = {
hCappedTitle: 'The chosen division would exceed 300 intervals over this range: it has been enlarged.', hCappedTitle: 'The chosen division would exceed 300 intervals over this range: it has been enlarged.',
hUnzoom: '× Back to stream', unitDay: 'd', hUnzoom: '× Back to stream', unitDay: 'd',
toTop: 'Back to top', toTop: 'Back to top',
colRcv: 'Received', colMt: 'Message time', colSev: 'Severity', colHost: 'Host', colApp: 'App', colCodes: 'Pattern', colMsg: 'Message', colSrc: 'Source', colRcv: 'Received', colMt: 'Message time', colSev: 'Severity', colHost: 'Host', colApp: 'App', colCodes: 'Pattern', colMsg: 'Message',
codesTitle: 'Codes of the color tags found in the message', codesTitle: 'Codes of the color tags found in the message',
tagCodeTitle: 'Code shown on the log lines this tag matches', tagCodeTitle: 'Code shown on the log lines this tag matches',
colGrip: 'Drag to resize, double-click for the automatic width', colGrip: 'Drag to resize, double-click for the automatic width',
@@ -268,13 +291,16 @@ const I18N = {
copyMsg: 'Copier le message', copyJson: 'Copier en JSON', copied: 'Copié', copyMsg: 'Copier le message', copyJson: 'Copier en JSON', copied: 'Copié',
language: 'Langue', language: 'Langue',
tagsTitle: 'Tags de couleur', tagsTitle: 'Tags de couleur',
tagsHelp: 'Les mots-clés sont surlignés dans les messages. Les modifications sont enregistrées automatiquement. En cas de chevauchement, le tag le plus haut dans la liste l\'emporte.', tagsHelp: 'Les mots-clés sont surlignés dans les messages. Les modifications s\'appliquent avec le bouton Valider. En cas de chevauchement, le tag le plus haut dans la liste l\'emporte.',
addTag: '+ Ajouter un tag', resetTags: 'Rétablir les tags par défaut', addTag: '+ Ajouter un tag', resetTags: 'Rétablir les tags par défaut',
optWhole: 'mot entier', optWholeTitle: 'Ne colorer que les mots entiers (évite « ok » dans « token »)', optWhole: 'mot entier', optWholeTitle: 'Ne colorer que les mots entiers (évite « ok » dans « token »)',
optCaseTitle: 'Respecter la casse', optRegexTitle: 'Expression régulière', optCaseTitle: 'Respecter la casse', optRegexTitle: 'Expression régulière',
optActive: 'actif', optActiveTitle: 'Activer ce tag', optActive: 'actif', optActiveTitle: 'Activer ce tag',
colorAria: 'Couleur', keyword: 'mot-clé', preview: 'aperçu', del: 'Supprimer', noTags: 'Aucun tag.', colorAria: 'Couleur', keyword: 'mot-clé', preview: 'aperçu', del: 'Supprimer', noTags: 'Aucun tag.',
newTag: 'nouveau', newTag: 'nouveau',
saveSettings: 'Valider', cancelSettings: 'Annuler',
settingsClean: 'Aucune modification en attente', settingsDirty: 'Modifications non validées', settingsSaved: 'Paramètres enregistrés',
unsavedConfirm: 'Des modifications n\'ont pas été validées. Les abandonner ?',
confirmDelete: (p) => `Supprimer le tag « ${p} » ?`, confirmDelete: (p) => `Supprimer le tag « ${p} » ?`,
confirmReset: 'Remplacer tous les tags par les tags par défaut (warning, error) ?', confirmReset: 'Remplacer tous les tags par les tags par défaut (warning, error) ?',
presetAria: 'Ajouter un préréglage', presetPick: '+ Préréglage…', presetAria: 'Ajouter un préréglage', presetPick: '+ Préréglage…',
@@ -326,6 +352,7 @@ const I18N = {
fileDraft: 'Renseignez le nom et le chemin pour enregistrer ce fichier.', fileDraft: 'Renseignez le nom et le chemin pour enregistrer ce fichier.',
fileOk: (r) => `Suivi : ${r} entrées depuis le démarrage.`, fileOk: (r) => `Suivi : ${r} entrées depuis le démarrage.`,
fileDetected: (f) => ` Format détecté : ${f}.`, fileDetected: (f) => ` Format détecté : ${f}.`,
fileBehind: (mb) => ` Lecture de l\'historique : ${mb} Mo restants.`,
probeFound: (f) => `Fichier trouvé, format détecté : ${f}.`, probeFound: (f) => `Fichier trouvé, format détecté : ${f}.`,
probeSample: 'Dernière ligne : ', probeSample: 'Dernière ligne : ',
file_empty: 'Le fichier est vide pour l\'instant : son format sera détecté à partir de ses premières lignes.', file_empty: 'Le fichier est vide pour l\'instant : son format sera détecté à partir de ses premières lignes.',
@@ -333,7 +360,7 @@ const I18N = {
file_not_found: 'Aucun fichier à ce chemin : montez-le dans le conteneur (voir docker-compose.yml) et vérifiez le chemin.', file_not_found: 'Aucun fichier à ce chemin : montez-le dans le conteneur (voir docker-compose.yml) et vérifiez le chemin.',
file_permission: 'Accès refusé : le fichier doit être lisible par l\'UID 10001 ou le groupe adm (voir le README). ', file_permission: 'Accès refusé : le fichier doit être lisible par l\'UID 10001 ou le groupe adm (voir le README). ',
file_read: 'Lecture du fichier impossible : ', file_read: 'Lecture du fichier impossible : ',
filesHelp: 'Suit le fichier de logs d\'un autre programme, monté en lecture seule dans le conteneur (par ex. /var/log/traefik:/logs/traefik:ro, puis /logs/traefik/access.log ici). Une fois le chemin saisi, le fichier est vérifié et son format détecté à partir de ses dernières lignes (JSON lines, logfmt, journaux d\'accès de nginx, Apache ou Traefik, sinon texte brut) ; choisissez un format pour le forcer, ou une expression régulière dont les groupes nommés time, level, msg, host et app remplissent ces champs, les autres groupes donnant des champs supplémentaires. Chaque ligne donne une entrée (badge FILE, app = le nom) ; les lignes indentées sont rattachées à l\'entrée précédente ; pour les journaux d\'accès, 5xx en erreur et 4xx en warning. À l\'activation, la dernière heure de lignes datées est lue d\'abord (FILES_BACKFILL) ; les rotations sont suivies et la lecture reprend où elle s\'était arrêtée.', filesHelp: 'Suit le fichier de logs d\'un autre programme, monté en lecture seule dans le conteneur (par ex. /var/log/traefik:/logs/traefik:ro, puis /logs/traefik/access.log ici). Une fois le chemin saisi, le fichier est vérifié et son format détecté à partir de ses dernières lignes (JSON lines, logfmt, journaux d\'accès de nginx, Apache ou Traefik, sinon texte brut) ; choisissez un format pour le forcer, ou une expression régulière dont les groupes nommés time, level, msg, host et app remplissent ces champs, les autres groupes donnant des champs supplémentaires. Chaque ligne donne une entrée (badge FILE en début de ligne, app = le nom) ; les lignes indentées sont rattachées à l\'entrée précédente ; pour les journaux d\'accès, 5xx en erreur et 4xx en warning. À l\'activation, la dernière heure de lignes datées est lue d\'abord (FILES_BACKFILL) ; les rotations sont suivies et la lecture reprend où elle s\'était arrêtée.',
auditTitle: 'Audit Linux (auditd)', auditTitle: 'Audit Linux (auditd)',
auditEnabled: 'Collecter les événements d\'audit de cette machine', auditEnabled: 'Collecter les événements d\'audit de cette machine',
auditPath: 'Fichier', auditPath: 'Fichier',
@@ -398,6 +425,25 @@ const I18N = {
exportNote: (n) => `Jusqu'à ${n} logs correspondant aux filtres, du plus récent au plus ancien. Dates dans le fuseau choisi dans Paramètres.`, exportNote: (n) => `Jusqu'à ${n} logs correspondant aux filtres, du plus récent au plus ancien. Dates dans le fuseau choisi dans Paramètres.`,
exporting: 'Export…', exportDone: (f) => `Téléchargé : ${f}`, exportError: 'Échec de l\'export : ', exporting: 'Export…', exportDone: (f) => `Téléchargé : ${f}`, exportError: 'Échec de l\'export : ',
err_export_pipes: 'L\'export ne gère pas les pipes LogsQL (| …) : gardez seulement la partie filtre', err_export_pipes: 'L\'export ne gère pas les pipes LogsQL (| …) : gardez seulement la partie filtre',
tabApi: 'API',
tokensTitle: "Jetons d'API",
tokensHelp: "Les jetons donnent aux scripts et outils d'automatisation (n8n…) un accès en lecture seule à /api/v1/logs, quel que soit le mode d'authentification. Seule une empreinte est conservée : un jeton n'est affiché qu'une fois, à sa création. Le révoquer coupe l'accès immédiatement.",
tokenCount: (n) => `${n} jeton${n > 1 ? 's' : ''}`,
tokenNone: "Aucun jeton d'API pour l'instant.",
tokenNamePh: 'Nom du jeton (n8n, script de sauvegarde…)',
tokenCreate: 'Créer un jeton',
tokenOnce: "Copiez ce jeton maintenant : il ne sera plus affiché.",
tokenCopy: 'Copier',
tokenRevoke: 'Révoquer',
tokenRevokeAsk: (n) => `Révoquer le jeton « ${n} » ? Les outils qui l'utilisent perdent l'accès immédiatement.`,
tokenRevoked: 'Jeton révoqué',
tokenCreated: (d) => `créé le ${d}`,
tokenUsed: (p) => `dernière utilisation ${p.d}${p.ip ? ' depuis ' + p.ip : ''}`,
tokenNever: 'jamais utilisé',
tokensUse: 'Utilisation',
tokensUseHelp: "Envoyez le jeton dans un en-tête Authorization. Chaque réponse contient next_cursor : passez-le dans since à l'appel suivant pour obtenir les logs qui suivent, sans trou ni doublon (voir le README pour un workflow n8n).",
err_token_name: 'Donnez au jeton un nom de 1 à 80 caractères',
err_token_max: "Trop de jetons d'API",
tabLocale: 'Localisation et langue', tabFilters: 'Motifs de recherche', tabInterface: 'Affichage', tabData: 'Base de données', tabLocale: 'Localisation et langue', tabFilters: 'Motifs de recherche', tabInterface: 'Affichage', tabData: 'Base de données',
themeTitle: 'Thème', themeSystem: 'Système', themeLight: 'Clair', themeDark: 'Sombre', themeTitle: 'Thème', themeSystem: 'Système', themeLight: 'Clair', themeDark: 'Sombre',
themeHelp: 'Système suit la préférence clair/sombre de votre ordinateur ou de votre téléphone.', themeHelp: 'Système suit la préférence clair/sombre de votre ordinateur ou de votre téléphone.',
@@ -452,7 +498,7 @@ const I18N = {
hCappedTitle: 'La division choisie dépasserait 300 intervalles sur cette plage : elle a été élargie.', hCappedTitle: 'La division choisie dépasserait 300 intervalles sur cette plage : elle a été élargie.',
hUnzoom: '× Revenir au flux', unitDay: 'j', hUnzoom: '× Revenir au flux', unitDay: 'j',
toTop: 'Revenir en haut', toTop: 'Revenir en haut',
colRcv: 'Réception', colMt: 'Heure message', colSev: 'Sévérité', colHost: 'Hôte', colApp: 'App', colCodes: 'Motif de recherche', colMsg: 'Message', colSrc: 'Source', colRcv: 'Réception', colMt: 'Heure message', colSev: 'Sévérité', colHost: 'Hôte', colApp: 'App', colCodes: 'Motif de recherche', colMsg: 'Message',
codesTitle: 'Codes des tags de couleur trouvés dans le message', codesTitle: 'Codes des tags de couleur trouvés dans le message',
tagCodeTitle: 'Code affiché sur les lignes de log où ce tag est trouvé', tagCodeTitle: 'Code affiché sur les lignes de log où ce tag est trouvé',
colGrip: 'Glisser pour redimensionner, double-clic pour la largeur automatique', colGrip: 'Glisser pour redimensionner, double-clic pour la largeur automatique',
@@ -585,7 +631,7 @@ function apiError(res, text, data) {
if (data && data.code && I18N[lang]['err_' + data.code]) { if (data && data.code && I18N[lang]['err_' + data.code]) {
msg = t('err_' + data.code) + (data.detail ? (lang === 'fr' ? ' : ' : ': ') + data.detail : ''); msg = t('err_' + data.code) + (data.detail ? (lang === 'fr' ? ' : ' : ': ') + data.detail : '');
} }
return new Error(msg); return Object.assign(new Error(msg), { data });
} }
function toast(msg) { function toast(msg) {
@@ -618,6 +664,26 @@ const state = {
}; };
const recOf = new WeakMap(); // .row element -> record const recOf = new WeakMap(); // .row element -> record
// Settings window: changes are shown at once but only saved by the Save button.
const settings = {
saved: null, // browser preferences as saved (see localPrefs)
tags: null, // tags as saved on the server
pending: {}, // source changes waiting for Save, by source
orig: {}, // server values of the pending fields, to drop a change set back
saving: false,
};
// Records a source change for Save; a field set back to its saved value is dropped.
function pend(key, change, current) {
const orig = settings.orig[key] ??= { ...current };
const p = { ...settings.pending[key], ...change };
for (const k of Object.keys(p)) if (p[k] === orig[k]) delete p[k];
if (Object.keys(p).length) settings.pending[key] = p;
else { delete settings.pending[key]; delete settings.orig[key]; }
}
// A source state refreshed from the server, with the unsaved changes over it.
const withPending = (key, s) => (s && settings.pending[key] ? { ...s, ...settings.pending[key] } : s);
const list = $('#list'); const list = $('#list');
/* ================= Language ================= */ /* ================= Language ================= */
@@ -634,7 +700,6 @@ function applyLang() {
function setLang(next) { function setLang(next) {
if (next === lang || !I18N[next]) return; if (next === lang || !I18N[next]) return;
lang = next; lang = next;
store.set('lang', lang);
applyLang(); applyLang();
syncControls(); syncControls();
setConn(...state.conn); setConn(...state.conn);
@@ -670,11 +735,12 @@ function effectiveTheme() {
if (th) return th; if (th) return th;
return matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light'; return matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light';
} }
// mode: 'system' (follow the OS), 'light' or 'dark'. // mode: 'system' (follow the OS), 'light' or 'dark'. In Settings the theme is
function setTheme(mode) { // only stored by the Save button (persist = false).
function setTheme(mode, persist = true) {
if (mode === 'light' || mode === 'dark') document.documentElement.dataset.theme = mode; if (mode === 'light' || mode === 'dark') document.documentElement.dataset.theme = mode;
else { delete document.documentElement.dataset.theme; mode = 'system'; } else { delete document.documentElement.dataset.theme; mode = 'system'; }
store.set('theme', mode); if (persist) store.set('theme', mode);
renderThemeSwitch(); renderThemeSwitch();
} }
@@ -689,7 +755,7 @@ function renderThemeSwitch() {
$('#themeBtn').addEventListener('click', () => setTheme(effectiveTheme() === 'dark' ? 'light' : 'dark')); $('#themeBtn').addEventListener('click', () => setTheme(effectiveTheme() === 'dark' ? 'light' : 'dark'));
$('#themeSwitch').addEventListener('click', (ev) => { $('#themeSwitch').addEventListener('click', (ev) => {
const b = ev.target.closest('[data-theme-opt]'); const b = ev.target.closest('[data-theme-opt]');
if (b) setTheme(b.dataset.themeOpt); if (b) setTheme(b.dataset.themeOpt, false);
}); });
/* ================= Log font and size ================= */ /* ================= Log font and size ================= */
@@ -780,20 +846,15 @@ $('#sizeSwitch').addEventListener('click', (ev) => {
const b = ev.target.closest('[data-size]'); const b = ev.target.closest('[data-size]');
if (!b) return; if (!b) return;
applyLogSize(b.dataset.size); applyLogSize(b.dataset.size);
store.set('logSize', ui.size);
renderInterface(); renderInterface();
}); });
$('#densitySwitch').addEventListener('click', (ev) => { $('#densitySwitch').addEventListener('click', (ev) => {
const b = ev.target.closest('[data-density]'); const b = ev.target.closest('[data-density]');
if (!b) return; if (!b) return;
applyLogDensity(b.dataset.density); applyLogDensity(b.dataset.density);
store.set('logDensity', ui.density);
renderInterface(); renderInterface();
}); });
$('#fontSelect').addEventListener('change', (ev) => { $('#fontSelect').addEventListener('change', (ev) => applyLogFont(ev.target.value));
applyLogFont(ev.target.value);
store.set('logFont', ui.font);
});
/* ================= Settings tabs ================= */ /* ================= Settings tabs ================= */
@@ -962,15 +1023,18 @@ function fmtTime(s) {
return `${day} ${p.h}:${p.m}:${p.s}<span class="ms">.${p.ms}</span>`; return `${day} ${p.h}:${p.m}:${p.s}<span class="ms">.${p.ms}</span>`;
} }
// Sources with their own look in the list: a colored edge and a badge // Every row starts with a badge naming its source, one color per source
// before the app name (the severity badge stays the same). // (--src-<type> in style.css). Audit events, log files and LogStream's own
const SRC_BADGE = { audit: 'AUDIT', logstream: 'LS', file: 'FILE' }; // logs also get a colored edge and a light tint (the severity badge stays the same).
const SRC_BADGE = { syslog: 'SYSLOG', docker: 'DOCKER', kernel: 'KERNEL', host: 'HOST', audit: 'AUDIT', file: 'FILE', logstream: 'LS' };
const srcOf = (r) => (SRC_BADGE[r.source_type] ? r.source_type : 'syslog'); // older entries have no source_type
function rowHTML(r, isNew) { function rowHTML(r, isNew) {
const sev = r.severity || ''; const sev = r.severity || '';
const cls = SEV_CLASS[sev] || 'info'; const cls = SEV_CLASS[sev] || 'info';
const badge = SRC_BADGE[r.source_type]; const src = srcOf(r);
return `<article class="row sev-${cls}${badge ? ' src-' + r.source_type : ''}${isNew ? ' new' : ''}" tabindex="0">` return `<article class="row sev-${cls} src-${src}${isNew ? ' new' : ''}" tabindex="0">`
+ `<b class="src-badge">${SRC_BADGE[src]}</b>`
+ `<time class="rcv" title="${esc(t('rcvTitle'))}">${r.received ? esc(fmtFull(r.received)) : '—'}</time>` + `<time class="rcv" title="${esc(t('rcvTitle'))}">${r.received ? esc(fmtFull(r.received)) : '—'}</time>`
+ `<time class="mt" title="${esc(t('msgTimeTitle'))}">${fmtTime(r.msg_time || r._time)}</time>` + `<time class="mt" title="${esc(t('msgTimeTitle'))}">${fmtTime(r.msg_time || r._time)}</time>`
+ `<span class="sev">${esc(sev || '—')}</span>` + `<span class="sev">${esc(sev || '—')}</span>`
@@ -978,7 +1042,7 @@ function rowHTML(r, isNew) {
? `<span class="host" data-act="host" title="${esc(hostTitle(r) + ' · ' + t('clickHost'))}">${esc(r.host_name || r.host)}</span>` ? `<span class="host" data-act="host" title="${esc(hostTitle(r) + ' · ' + t('clickHost'))}">${esc(r.host_name || r.host)}</span>`
: '<span class="host"></span>') : '<span class="host"></span>')
+ (r.app + (r.app
? `<span class="app${r.source_type === 'docker' ? ' proj' : ''}" data-act="app"${r.source_type === 'docker' ? ` style="--h:${hueOf(r.compose_project || r.container || r.app)}"` : ''} title="${esc((r.compose_project ? r.compose_project + '/' : '') + (r.container ? r.container + ' · ' : '') + r.app + ' · ' + t('clickApp'))}">${r.source_type === 'docker' ? DOCKER_ICON : ''}${badge ? `<b class="src-badge">${badge}</b>` : ''}${esc(r.app)}</span>` ? `<span class="app${r.source_type === 'docker' ? ' proj' : ''}" data-act="app"${r.source_type === 'docker' ? ` style="--h:${hueOf(r.compose_project || r.container || r.app)}"` : ''} title="${esc((r.compose_project ? r.compose_project + '/' : '') + (r.container ? r.container + ' · ' : '') + r.app + ' · ' + t('clickApp'))}">${r.source_type === 'docker' ? DOCKER_ICON : ''}${esc(r.app)}</span>`
: '<span class="app"></span>') : '<span class="app"></span>')
+ `<span class="codes">${codesHTML(r._msg)}</span>` + `<span class="codes">${codesHTML(r._msg)}</span>`
+ `<div class="msg">${msgHTML(r)}</div>` + `<div class="msg">${msgHTML(r)}</div>`
@@ -1747,7 +1811,6 @@ function renderHistoSettings() {
function setHistoPref(k, v) { function setHistoPref(k, v) {
if (!HISTO_OPTS[k] || !HISTO_OPTS[k].includes(v)) return; if (!HISTO_OPTS[k] || !HISTO_OPTS[k].includes(v)) return;
histoPrefs[k] = v; histoPrefs[k] = v;
store.set('histo.' + k, v);
renderHistoSettings(); renderHistoSettings();
if (k === 'step') refreshHisto(); if (k === 'step') refreshHisto();
else if (k === 'refresh') { scheduleHisto(); if (v !== 'off') refreshHisto(); } else if (k === 'refresh') { scheduleHisto(); if (v !== 'off') refreshHisto(); }
@@ -1958,8 +2021,6 @@ function updateTimePreview() {
function onTimePrefsChange() { function onTimePrefsChange() {
timePrefs.tz = $('#tzSelect').value; timePrefs.tz = $('#tzSelect').value;
timePrefs.fmt = $('#fmtSelect').value; timePrefs.fmt = $('#fmtSelect').value;
store.set('tz', timePrefs.tz);
store.set('timefmt', timePrefs.fmt);
updateTimePreview(); updateTimePreview();
renderList(); renderList();
renderPeriod(); renderPeriod();
@@ -2027,7 +2088,7 @@ document.addEventListener('keydown', (ev) => { if (ev.key === 'Escape') setExpor
let syslogState = null; let syslogState = null;
async function loadSyslog() { async function loadSyslog() {
try { syslogState = await api('/api/syslog'); } catch { syslogState = null; } try { syslogState = withPending('syslog', await api('/api/syslog')); } catch { syslogState = null; }
renderSyslog(); renderSyslog();
} }
@@ -2066,12 +2127,10 @@ function renderSyslog() {
} }
} }
async function configureSyslog(change) { function configureSyslog(change) {
const s = syslogState || { enabled: true, udp: true, tcp: true }; const s = syslogState || { enabled: true, udp: true, tcp: true };
const body = { enabled: s.enabled, udp: s.udp, tcp: s.tcp, ...change }; pend('syslog', change, s);
syslogState = { ...s, ...body }; syslogState = { ...s, ...change };
renderSyslog();
try { syslogState = await api('/api/syslog', { method: 'PUT', body }); } catch (e) { toast(e.message); }
renderSyslog(); renderSyslog();
} }
@@ -2087,7 +2146,7 @@ $('#syslogProtos').addEventListener('click', (ev) => {
let hostState = null; let hostState = null;
async function loadHost() { async function loadHost() {
try { hostState = await api('/api/hostlogs'); } catch { hostState = null; } try { hostState = withPending('host', await api('/api/hostlogs')); } catch { hostState = null; }
renderHost(); renderHost();
} }
@@ -2114,12 +2173,10 @@ function renderHost() {
status.className = 'docker-status ' + cls; status.className = 'docker-status ' + cls;
} }
async function configureHost(enabled) { function configureHost(enabled) {
pend('host', { enabled }, hostState || {});
hostState = { ...(hostState || {}), enabled }; hostState = { ...(hostState || {}), enabled };
renderHost(); renderHost();
try { hostState = await api('/api/hostlogs', { method: 'PUT', body: { enabled } }); } catch (e) { toast(e.message); }
renderHost();
setTimeout(loadHost, 1500); // the first scan runs in the background
} }
$('#hostEnabled').addEventListener('change', (ev) => configureHost(ev.target.checked)); $('#hostEnabled').addEventListener('change', (ev) => configureHost(ev.target.checked));
@@ -2129,7 +2186,7 @@ $('#hostEnabled').addEventListener('change', (ev) => configureHost(ev.target.che
let dmesgState = null; let dmesgState = null;
async function loadDmesg() { async function loadDmesg() {
try { dmesgState = await api('/api/dmesg'); } catch { dmesgState = null; } try { dmesgState = withPending('dmesg', await api('/api/dmesg')); } catch { dmesgState = null; }
renderDmesg(); renderDmesg();
} }
@@ -2153,12 +2210,10 @@ function renderDmesg() {
status.className = 'docker-status ' + cls; status.className = 'docker-status ' + cls;
} }
async function configureDmesg(enabled) { function configureDmesg(enabled) {
pend('dmesg', { enabled }, dmesgState || {});
dmesgState = { ...(dmesgState || {}), enabled }; dmesgState = { ...(dmesgState || {}), enabled };
renderDmesg(); renderDmesg();
try { dmesgState = await api('/api/dmesg', { method: 'PUT', body: { enabled } }); } catch (e) { toast(e.message); }
renderDmesg();
setTimeout(loadDmesg, 1500); // the device is opened in the background
} }
$('#dmesgEnabled').addEventListener('change', (ev) => configureDmesg(ev.target.checked)); $('#dmesgEnabled').addEventListener('change', (ev) => configureDmesg(ev.target.checked));
@@ -2168,7 +2223,7 @@ $('#dmesgEnabled').addEventListener('change', (ev) => configureDmesg(ev.target.c
let auditState = null; let auditState = null;
async function loadAudit() { async function loadAudit() {
try { auditState = await api('/api/audit'); } catch { auditState = null; } try { auditState = withPending('audit', await api('/api/audit')); } catch { auditState = null; }
renderAudit(); renderAudit();
} }
@@ -2195,13 +2250,10 @@ function renderAudit() {
status.className = 'docker-status ' + cls; status.className = 'docker-status ' + cls;
} }
async function configureAudit(change) { function configureAudit(change) {
const body = { enabled: auditState?.enabled || false, path: auditState?.path || '', ...change }; pend('audit', change, { enabled: false, path: '', ...auditState });
auditState = { ...(auditState || {}), ...body }; auditState = { ...(auditState || {}), ...change };
renderAudit(); renderAudit();
try { auditState = await api('/api/audit', { method: 'PUT', body }); } catch (e) { toast(e.message); }
renderAudit();
setTimeout(loadAudit, 1500); // the file is read in the background
} }
$('#auditEnabled').addEventListener('change', (ev) => configureAudit({ enabled: ev.target.checked })); $('#auditEnabled').addEventListener('change', (ev) => configureAudit({ enabled: ev.target.checked }));
@@ -2212,7 +2264,7 @@ $('#auditPath').addEventListener('change', (ev) => configureAudit({ path: ev.tar
let selfState = null; let selfState = null;
async function loadSelfLog() { async function loadSelfLog() {
try { selfState = await api('/api/selflog'); } catch { selfState = null; } try { selfState = withPending('selflog', await api('/api/selflog')); } catch { selfState = null; }
renderSelfLog(); renderSelfLog();
} }
@@ -2228,11 +2280,9 @@ function renderSelfLog() {
status.className = 'docker-status ' + (s.enabled ? 'good' : 'muted'); status.className = 'docker-status ' + (s.enabled ? 'good' : 'muted');
} }
async function configureSelfLog(change) { function configureSelfLog(change) {
const body = { enabled: selfState?.enabled ?? true, min_level: selfState?.min_level || 'warning', ...change }; pend('selflog', change, { enabled: true, min_level: 'warning', ...selfState });
selfState = { ...(selfState || {}), ...body }; selfState = { ...(selfState || {}), ...change };
renderSelfLog();
try { selfState = await api('/api/selflog', { method: 'PUT', body }); } catch (e) { toast(e.message); }
renderSelfLog(); renderSelfLog();
} }
@@ -2247,9 +2297,18 @@ let fileDrafts = []; // rows added here, saved once they have a name and a pat
let fileSeq = 0; let fileSeq = 0;
const fileProbes = new Map(); // path -> result of /api/files/detect (format found, last line) const fileProbes = new Map(); // path -> result of /api/files/detect (format found, last line)
async function loadFiles() { const FILE_FIELDS = ['id', 'name', 'path', 'format', 'regex', 'host', 'enabled'];
try { filesState = await api('/api/files'); } catch { filesState = null; }
renderFiles(); // full: rebuild the rows (after Save or Cancel).
async function loadFiles(full) {
let fresh;
try { fresh = await api('/api/files'); } catch { filesState = null; renderFiles(full); return; }
if (settings.pending.files && filesState) {
// Unsaved edits: keep the rows as edited, with the server status of each file.
const byId = new Map(fresh.sources.map((x) => [x.id, x]));
filesState = { ...fresh, sources: filesState.sources.map((x) => ({ ...byId.get(x.id), ...Object.fromEntries(FILE_FIELDS.map((k) => [k, x[k]])) })) };
} else filesState = fresh;
renderFiles(full);
} }
// Status line of a row: the check of the path just typed, else the state // Status line of a row: the check of the path just typed, else the state
@@ -2263,7 +2322,10 @@ function fileStatus(s) {
if (s.draft) return [t('fileDraft'), 'muted']; if (s.draft) return [t('fileDraft'), 'muted'];
if (!s.enabled) return [t('fileOff'), 'muted']; if (!s.enabled) return [t('fileOff'), 'muted'];
if (s.code) return [t('file_' + s.code) + (s.code === 'not_found' ? '' : s.error || ''), 'bad']; if (s.code) return [t('file_' + s.code) + (s.code === 'not_found' ? '' : s.error || ''), 'bad'];
if (s.open) return [t('fileOk', fmtNum(s.read)) + (s.format === 'auto' && s.detected ? t('fileDetected', t('fmt_' + s.detected)) : ''), 'good']; if (s.open) {
return [t('fileOk', fmtNum(s.read)) + (s.format === 'auto' && s.detected ? t('fileDetected', t('fmt_' + s.detected)) : '')
+ (s.behind > 1 << 20 ? t('fileBehind', fmtNum(Math.round(s.behind / (1 << 20)))) : ''), 'good'];
}
return [t('hostWaiting'), 'muted']; return [t('hostWaiting'), 'muted'];
} }
@@ -2330,18 +2392,10 @@ function fileRowOf(key) {
async function saveFiles() { async function saveFiles() {
const ready = fileDrafts.filter((d) => d.name.trim() && d.path.trim()); const ready = fileDrafts.filter((d) => d.name.trim() && d.path.trim());
const sources = filesState.sources.concat(ready).map(({ id, name, path, format, regex, host, enabled }) => ({ id: id || '', name, path, format, regex, host, enabled })); const sources = filesState.sources.concat(ready).map(({ id, name, path, format, regex, host, enabled }) => ({ id: id || '', name, path, format, regex, host, enabled }));
try {
filesState = await api('/api/files', { method: 'PUT', body: { sources } }); filesState = await api('/api/files', { method: 'PUT', body: { sources } });
fileDrafts = fileDrafts.filter((d) => !ready.includes(d)); fileDrafts = fileDrafts.filter((d) => !ready.includes(d));
renderFiles(true);
} catch (e) {
toast(e.message);
await loadFiles();
return;
}
renderFiles();
setTimeout(loadFiles, 1500); // the files are read in the background setTimeout(loadFiles, 1500); // the files are read in the background
} }
@@ -2362,7 +2416,7 @@ $('#fileList').addEventListener('change', (ev) => {
if (f === 'format') row.querySelector('.fr-regex').hidden = s.format !== 'regex'; if (f === 'format') row.querySelector('.fr-regex').hidden = s.format !== 'regex';
if (f === 'format' && s.format === 'regex' && !s.regex) { row.querySelector('.fr-regex').focus(); return; } if (f === 'format' && s.format === 'regex' && !s.regex) { row.querySelector('.fr-regex').focus(); return; }
if (s.draft && !(s.name && s.path)) return; if (s.draft && !(s.name && s.path)) return;
saveFiles(); settings.pending.files = true;
}); });
$('#fileList').addEventListener('click', (ev) => { $('#fileList').addEventListener('click', (ev) => {
@@ -2375,7 +2429,8 @@ $('#fileList').addEventListener('click', (ev) => {
return; return;
} }
filesState.sources = filesState.sources.filter((s) => s.id !== key); filesState.sources = filesState.sources.filter((s) => s.id !== key);
saveFiles(); settings.pending.files = true;
renderFiles(true);
}); });
$('#addFile').addEventListener('click', () => { $('#addFile').addEventListener('click', () => {
@@ -2393,9 +2448,18 @@ const LOCK_ICON = '<svg viewBox="0 0 24 24" aria-hidden="true"><rect x="5" y="11
async function loadDocker() { async function loadDocker() {
try { dockerState = await api('/api/docker'); } catch (e) { dockerState = { enabled: true, connected: false, error: e.message }; } try { dockerState = await api('/api/docker'); } catch (e) { dockerState = { enabled: true, connected: false, error: e.message }; }
applyDockerPending();
renderDocker(); renderDocker();
} }
// Unsaved choices stay shown over the state refreshed from the server.
function applyDockerPending() {
const p = settings.pending.docker;
if (!p || !dockerState) return;
if ('defaultEnabled' in p) dockerState.defaultEnabled = p.defaultEnabled;
for (const c of dockerState.containers || []) if (c.key in p.containers) c.enabled = p.containers[c.key];
}
// One label per choice key ("project/service"): replicas of a service share it. // One label per choice key ("project/service"): replicas of a service share it.
function dockerChips() { function dockerChips() {
const byKey = new Map(); const byKey = new Map();
@@ -2458,12 +2522,28 @@ function renderDocker() {
: `<span class="chip-empty">${esc(t('dockerNoMatch'))}</span>`; : `<span class="chip-empty">${esc(t('dockerNoMatch'))}</span>`;
} }
async function configureDocker(body) { // Kept until Save; a choice set back to its saved value is dropped.
try { function configureDocker(change) {
dockerState = await api('/api/docker', { method: 'PUT', body }); if (!dockerState) return;
const orig = settings.orig.docker ??= {
defaultEnabled: !!dockerState.defaultEnabled,
containers: Object.fromEntries(dockerChips().map((c) => [c.key, !!c.enabled])),
};
const p = settings.pending.docker || { containers: {} };
if ('defaultEnabled' in change) {
if (change.defaultEnabled === orig.defaultEnabled) delete p.defaultEnabled;
else p.defaultEnabled = change.defaultEnabled;
}
for (const [k, on] of Object.entries(change.containers || {})) {
if (orig.containers[k] === on) delete p.containers[k];
else p.containers[k] = on;
}
if ('defaultEnabled' in p || Object.keys(p.containers).length) settings.pending.docker = p;
else { delete settings.pending.docker; delete settings.orig.docker; }
// Shown at once: the change itself, even when it brings back the saved value.
if ('defaultEnabled' in change) dockerState.defaultEnabled = change.defaultEnabled;
for (const c of dockerState.containers || []) if (c.key in (change.containers || {})) c.enabled = change.containers[c.key];
renderDocker(); renderDocker();
setTimeout(loadDocker, 1500); // followers start or stop in the background
} catch (e) { toast(e.message); loadDocker(); }
} }
$('#dockerDefault').addEventListener('change', (ev) => configureDocker({ defaultEnabled: ev.target.checked })); $('#dockerDefault').addEventListener('change', (ev) => configureDocker({ defaultEnabled: ev.target.checked }));
@@ -2476,12 +2556,7 @@ $('#dockerFilter').addEventListener('input', (ev) => { dockerView.filter = ev.ta
$('#dockerList').addEventListener('click', (ev) => { $('#dockerList').addEventListener('click', (ev) => {
const chip = ev.target.closest('.chip[data-key]'); const chip = ev.target.closest('.chip[data-key]');
if (!chip || chip.disabled) return; if (!chip || chip.disabled) return;
const key = chip.dataset.key; configureDocker({ containers: { [chip.dataset.key]: !chip.classList.contains('on') } });
const on = !chip.classList.contains('on');
// Immediate feedback, confirmed by the server answer.
for (const c of (dockerState && dockerState.containers) || []) if (c.key === key) c.enabled = on;
renderDocker();
configureDocker({ containers: { [key]: on } });
}); });
for (const [id, on] of [['#dockerAll', true], ['#dockerNone', false]]) { for (const [id, on] of [['#dockerAll', true], ['#dockerNone', false]]) {
$(id).addEventListener('click', () => { $(id).addEventListener('click', () => {
@@ -2508,6 +2583,7 @@ setInterval(() => {
} }
}, 4000); }, 4000);
/* ================= Database statistics ================= */ /* ================= Database statistics ================= */
// 1536 -> "1,5 Ko" / "1.5 KB" (binary multiples, the usual reading for disk sizes). // 1536 -> "1,5 Ko" / "1.5 KB" (binary multiples, the usual reading for disk sizes).
@@ -2626,6 +2702,74 @@ $('#purgeBtn').addEventListener('click', async () => {
} }
}); });
/* ================= API tokens ================= */
const tokens = { list: null, error: null };
function renderTokens() {
const el = $('#tokenList');
const day = (x) => fmtFull(x).replace(/[.,]\d{3}(?=\D*$)/, '');
$('#tokenCount').textContent = tokens.list ? t('tokenCount', tokens.list.length) : '';
$('#tokenExample').textContent = `curl -H "Authorization: Bearer lst_…" \\\n "${location.origin}/api/v1/logs?query=severity:err&since=15m&limit=1000"`;
if (!tokens.list) {
el.innerHTML = tokens.error ? `<div class="token-empty bad">${esc(tokens.error)}</div>` : '';
return;
}
if (!tokens.list.length) {
el.innerHTML = `<div class="token-empty">${esc(t('tokenNone'))}</div>`;
return;
}
el.innerHTML = tokens.list.map((k) => {
const used = k.last_used ? t('tokenUsed', { d: day(k.last_used), ip: k.last_ip }) : t('tokenNever');
return `<div class="token-row"><div><b>${esc(k.name)}</b> <code>${esc(k.hint)}…</code></div>`
+ `<button class="btn danger" type="button" data-revoke="${esc(k.id)}">${esc(t('tokenRevoke'))}</button>`
+ `<div class="muted">${esc(t('tokenCreated', day(k.created)))} · ${esc(used)}</div></div>`;
}).join('');
}
async function loadTokens() {
$('#tokenNew').hidden = true;
try {
tokens.list = await api('/api/tokens');
tokens.error = null;
} catch (e) {
tokens.list = null;
tokens.error = e.data?.code === 'read_only' ? '' : e.message;
}
renderTokens();
}
$('#tokenForm').addEventListener('submit', async (ev) => {
ev.preventDefault();
const name = $('#tokenName').value.trim();
if (!name) { $('#tokenName').focus(); return; }
try {
const res = await api('/api/tokens', { method: 'POST', body: { name } });
$('#tokenName').value = '';
await loadTokens();
$('#tokenSecret').textContent = res.token;
$('#tokenNew').hidden = false;
} catch (e) {
toast(e.message);
}
});
$('#tokenCopy').addEventListener('click', () => copy($('#tokenSecret').textContent));
$('#tokenList').addEventListener('click', async (ev) => {
const b = ev.target.closest('[data-revoke]');
if (!b) return;
const k = tokens.list?.find((x) => x.id === b.dataset.revoke);
if (!k || !confirm(t('tokenRevokeAsk', k.name))) return;
try {
await api('/api/tokens/' + encodeURIComponent(k.id), { method: 'DELETE' });
toast(t('tokenRevoked'));
} catch (e) {
toast(e.message);
}
loadTokens();
});
/* ================= Tag management ================= */ /* ================= Tag management ================= */
const PALETTE = ['#6366f1', '#0ea5e9', '#14b8a6', '#a855f7', '#ec4899', '#eab308', '#64748b', '#f97316']; const PALETTE = ['#6366f1', '#0ea5e9', '#14b8a6', '#a855f7', '#ec4899', '#eab308', '#64748b', '#f97316'];
@@ -2703,19 +2847,10 @@ function filterTagList() {
} }
$('#tagFilter').addEventListener('input', filterTagList); $('#tagFilter').addEventListener('input', filterTagList);
const saveTimers = {}; // Tags are edited here and sent as a whole list by Save (PUT /api/tags); a new
function scheduleSave(tag, rowEl) { // tag gets its id and code from the server then.
clearTimeout(saveTimers[tag.id]); let tagSeq = 0;
saveTimers[tag.id] = setTimeout(async () => { const newTag = (fields) => ({ code: '', label: '', wholeWord: false, caseSensitive: false, regex: false, enabled: true, ...fields, id: 'new-' + (++tagSeq) });
const errEl = rowEl.querySelector('.err');
try {
await api('/api/tags/' + encodeURIComponent(tag.id), { method: 'PUT', body: tag });
errEl.textContent = '';
} catch (e) {
errEl.textContent = e.message;
}
}, 400);
}
const applyTags = debounce(() => { compileMatchers(); rehighlight(); }, 150); const applyTags = debounce(() => { compileMatchers(); rehighlight(); }, 150);
@@ -2730,52 +2865,43 @@ $('#tagList').addEventListener('input', (ev) => {
const mark = rowEl.querySelector('.preview mark'); const mark = rowEl.querySelector('.preview mark');
mark.textContent = tag.label || tag.pattern || t('preview'); mark.textContent = tag.label || tag.pattern || t('preview');
mark.setAttribute('style', tagStyle(tag.color)); mark.setAttribute('style', tagStyle(tag.color));
scheduleSave(tag, rowEl); rowEl.querySelector('.err').textContent = '';
applyTags(); applyTags();
}); });
$('#tagList').addEventListener('click', async (ev) => { $('#tagList').addEventListener('click', (ev) => {
const del = ev.target.closest('[data-del]'); const del = ev.target.closest('[data-del]');
if (!del) return; if (!del) return;
const rowEl = del.closest('.tag-row'); const rowEl = del.closest('.tag-row');
const tag = state.tags.find((x) => x.id === rowEl.dataset.id); const tag = state.tags.find((x) => x.id === rowEl.dataset.id);
if (!tag || !confirm(t('confirmDelete', tag.pattern))) return; if (!tag || !confirm(t('confirmDelete', tag.pattern))) return;
try {
await api('/api/tags/' + encodeURIComponent(tag.id), { method: 'DELETE' });
state.tags = state.tags.filter((x) => x !== tag); state.tags = state.tags.filter((x) => x !== tag);
renderTagList(); renderTagList();
applyTags(); applyTags();
} catch (e) { toast(e.message); } renderSaveState();
}); });
$('#addTag').addEventListener('click', async () => { $('#addTag').addEventListener('click', () => {
const color = PALETTE[state.tags.length % PALETTE.length]; state.tags.push(newTag({ pattern: t('newTag'), color: PALETTE[state.tags.length % PALETTE.length], wholeWord: true }));
try {
const tag = await api('/api/tags', { method: 'POST', body: { pattern: t('newTag'), color, wholeWord: true, enabled: true } });
state.tags.push(tag);
$('#tagFilter').value = ''; $('#tagFilter').value = '';
renderTagList(); renderTagList();
applyTags(); applyTags();
const input = $('#tagList').lastElementChild.querySelector('input[type="text"]'); const input = $('#tagList').lastElementChild.querySelector('input[type="text"]');
input.focus(); input.focus();
input.select(); input.select();
} catch (e) { toast(e.message); }
}); });
// Adds a preset group, skipping tags whose pattern is already in the list. // Adds a preset group, skipping tags whose pattern is already in the list.
$('#presetTags').addEventListener('change', async (ev) => { $('#presetTags').addEventListener('change', (ev) => {
const preset = state.presets.flatMap((g) => g.presets).find((p) => p.id === ev.target.value); const preset = state.presets.flatMap((g) => g.presets).find((p) => p.id === ev.target.value);
ev.target.value = ''; ev.target.value = '';
if (!preset) return; if (!preset) return;
let added = 0; let added = 0;
try {
for (const p of preset.tags) { for (const p of preset.tags) {
if (state.tags.some((x) => x.pattern === p.pattern)) continue; if (state.tags.some((x) => x.pattern === p.pattern)) continue;
const tag = { ...p, label: pickText(p.label), regex: p.regex ?? true, enabled: true }; state.tags.push(newTag({ ...p, label: pickText(p.label), regex: p.regex ?? true, enabled: true }));
state.tags.push(await api('/api/tags', { method: 'POST', body: tag }));
added++; added++;
} }
} catch (e) { toast(e.message); }
renderTagList(); renderTagList();
applyTags(); applyTags();
toast(t('presetAdded', added)); toast(t('presetAdded', added));
@@ -2784,13 +2910,16 @@ $('#presetTags').addEventListener('change', async (ev) => {
$('#resetTags').addEventListener('click', async () => { $('#resetTags').addEventListener('click', async () => {
if (!confirm(t('confirmReset'))) return; if (!confirm(t('confirmReset'))) return;
try { try {
state.tags = await api('/api/tags/reset', { method: 'POST' }); state.tags = await api('/api/tags/defaults');
renderTagList(); renderTagList();
applyTags(); applyTags();
} catch (e) { toast(e.message); } } catch (e) { toast(e.message); }
renderSaveState();
}); });
$('#settingsBtn').addEventListener('click', () => { $('#settingsBtn').addEventListener('click', () => {
settingsBaseline();
renderSaveState();
renderTimeSettings(); renderTimeSettings();
renderTagList(); renderTagList();
loadPresets(); loadPresets();
@@ -2805,11 +2934,160 @@ $('#settingsBtn').addEventListener('click', () => {
loadDocker(); loadDocker();
renderDbStats(); renderDbStats();
loadDbStats(); loadDbStats();
loadTokens();
showSettingsTab(store.get('settingsTab', 'locale')); showSettingsTab(store.get('settingsTab', 'locale'));
$('#settingsDlg').showModal(); $('#settingsDlg').showModal();
}); });
$('#closeSettings').addEventListener('click', () => $('#settingsDlg').close()); /* ================= Settings: Save / Cancel ================= */
$('#settingsDlg').addEventListener('click', (ev) => { if (ev.target === ev.currentTarget) ev.currentTarget.close(); });
// Browser preferences of the Settings window (stored in localStorage by Save).
function localPrefs() {
return {
lang, theme: document.documentElement.dataset.theme || 'system',
font: ui.font, size: ui.size, density: ui.density, tz: timePrefs.tz, fmt: timePrefs.fmt, histo: { ...histoPrefs },
};
}
const tagFields = ({ id, pattern, label, color, wholeWord, caseSensitive, regex, enabled }) =>
({ id, pattern, label: label || '', color, wholeWord: !!wholeWord, caseSensitive: !!caseSensitive, regex: !!regex, enabled: !!enabled });
const tagsJSON = (tags) => JSON.stringify(tags.map(tagFields));
// What Cancel goes back to: the state when the window was opened or last saved.
function settingsBaseline() {
settings.saved = localPrefs();
settings.tags = structuredClone(state.tags);
settings.pending = {};
settings.orig = {};
}
function settingsDirty() {
return JSON.stringify(localPrefs()) !== JSON.stringify(settings.saved)
|| tagsJSON(state.tags) !== tagsJSON(settings.tags)
|| Object.keys(settings.pending).length > 0;
}
function renderSaveState() {
if (!settings.saved) return;
const dirty = settingsDirty();
$('#setActions').classList.toggle('dirty', dirty);
$('#setSave').disabled = !dirty || settings.saving;
$('#setCancel').disabled = !dirty || settings.saving;
$('#setState').textContent = t(dirty ? 'settingsDirty' : 'settingsClean');
}
// Each source sends its whole setting, as edited.
const SOURCE_SAVE = {
syslog: async () => { syslogState = await api('/api/syslog', { method: 'PUT', body: { enabled: syslogState.enabled, udp: syslogState.udp, tcp: syslogState.tcp } }); },
host: async () => { hostState = await api('/api/hostlogs', { method: 'PUT', body: { enabled: hostState.enabled } }); },
dmesg: async () => { dmesgState = await api('/api/dmesg', { method: 'PUT', body: { enabled: dmesgState.enabled } }); },
audit: async () => { auditState = await api('/api/audit', { method: 'PUT', body: { enabled: !!auditState.enabled, path: auditState.path || '' } }); },
selflog: async () => { selfState = await api('/api/selflog', { method: 'PUT', body: { enabled: selfState.enabled, min_level: selfState.min_level || 'warning' } }); },
docker: async () => { dockerState = await api('/api/docker', { method: 'PUT', body: settings.pending.docker }); },
files: saveFiles,
};
async function saveSettings() {
settings.saving = true;
renderSaveState();
const p = localPrefs();
for (const [k, v] of [['lang', p.lang], ['theme', p.theme], ['logFont', p.font], ['logSize', p.size], ['logDensity', p.density], ['tz', p.tz], ['timefmt', p.fmt]]) store.set(k, v);
for (const [k, v] of Object.entries(p.histo)) store.set('histo.' + k, v);
settings.saved = p;
const errors = [];
if (tagsJSON(state.tags) !== tagsJSON(settings.tags)) {
try {
state.tags = await api('/api/tags', { method: 'PUT', body: state.tags.map(tagFields) });
settings.tags = structuredClone(state.tags);
renderTagList();
applyTags();
} catch (e) {
errors.push(e.message);
const row = $('#tagList').querySelectorAll('.tag-row')[e.data?.index];
if (row) {
showSettingsTab('filters');
row.hidden = false;
row.querySelector('.err').textContent = e.message;
row.scrollIntoView({ block: 'nearest' });
}
}
}
for (const key of Object.keys(settings.pending)) {
try {
await SOURCE_SAVE[key]();
delete settings.pending[key];
delete settings.orig[key];
} catch (e) { errors.push(e.message); }
}
// Sources start or stop in the background: show their state a bit later.
setTimeout(reloadSources, 1500);
reloadSources();
settings.saving = false;
renderSaveState();
toast(errors.length ? errors.join(' · ') : t('settingsSaved'));
}
function reloadSources() {
loadSyslog();
loadHost();
loadDmesg();
loadAudit();
loadSelfLog();
loadFiles(true);
loadDocker();
}
function cancelSettings() {
const p = settings.saved;
setLang(p.lang);
setTheme(p.theme, false);
applyLogFont(p.font);
applyLogSize(p.size);
applyLogDensity(p.density);
if (p.tz !== timePrefs.tz || p.fmt !== timePrefs.fmt) {
Object.assign(timePrefs, { tz: p.tz, fmt: p.fmt });
renderList();
renderPeriod();
refreshHisto();
}
for (const [k, v] of Object.entries(p.histo)) if (histoPrefs[k] !== v) setHistoPref(k, v);
renderInterface();
renderTimeSettings();
state.tags = structuredClone(settings.tags);
renderTagList();
applyTags();
settings.pending = {};
settings.orig = {};
fileDrafts = [];
reloadSources();
renderSaveState();
}
// Closing with unsaved changes asks first, then drops them.
function closeSettings() {
if (settingsDirty()) {
if (!confirm(t('unsavedConfirm'))) return;
cancelSettings();
}
$('#settingsDlg').close();
}
$('#setSave').addEventListener('click', saveSettings);
$('#setCancel').addEventListener('click', cancelSettings);
$('#closeSettings').addEventListener('click', closeSettings);
$('#settingsDlg').addEventListener('click', (ev) => { if (ev.target === ev.currentTarget) closeSettings(); });
$('#settingsDlg').addEventListener('cancel', (ev) => { ev.preventDefault(); closeSettings(); }); // Escape
// Closed anyway (a second Escape in some browsers): the changes are not kept.
$('#settingsDlg').addEventListener('close', (ev) => { if (!ev.currentTarget.open && settingsDirty()) cancelSettings(); });
// Every control of the window: refresh the Save button once the change is applied.
for (const type of ['input', 'change', 'click']) {
$('#settingsDlg').addEventListener(type, () => setTimeout(renderSaveState));
}
addEventListener('beforeunload', (ev) => {
if ($('#settingsDlg').open && settingsDirty()) ev.preventDefault();
});
/* ================= Startup ================= */ /* ================= Startup ================= */
@@ -2858,7 +3136,7 @@ fetch('/auth/me').then((res) => (res.ok ? res.json() : null)).then((me) => {
} }
if (me.role === 'viewer') { if (me.role === 'viewer') {
document.body.classList.add('read-only'); document.body.classList.add('read-only');
for (const p of document.querySelectorAll('[data-panel="filters"], [data-panel="sources"], [data-panel="data"]')) { for (const p of document.querySelectorAll('[data-panel="filters"], [data-panel="sources"], [data-panel="data"], [data-panel="api"]')) {
for (const s of p.querySelectorAll('.set-section')) s.inert = true; for (const s of p.querySelectorAll('.set-section')) s.inert = true;
const note = document.createElement('p'); const note = document.createElement('p');
note.className = 'ro-note'; note.className = 'ro-note';
+43
View File
@@ -140,6 +140,7 @@
<div id="table" class="table" hidden> <div id="table" class="table" hidden>
<div id="listHead" class="list-head" role="presentation"> <div id="listHead" class="list-head" role="presentation">
<span data-col="src"><span class="lbl" data-i18n="colSrc">Source</span></span>
<span data-col="rcv"><span class="lbl" data-i18n="colRcv">Received</span><i class="grip" data-i18n-title="colGrip"></i></span> <span data-col="rcv"><span class="lbl" data-i18n="colRcv">Received</span><i class="grip" data-i18n-title="colGrip"></i></span>
<span data-col="mt"><span class="lbl" data-i18n="colMt">Message time</span><i class="grip" data-i18n-title="colGrip"></i></span> <span data-col="mt"><span class="lbl" data-i18n="colMt">Message time</span><i class="grip" data-i18n-title="colGrip"></i></span>
<span data-col="sev"><span class="lbl" data-i18n="colSev">Severity</span><i class="grip" data-i18n-title="colGrip"></i></span> <span data-col="sev"><span class="lbl" data-i18n="colSev">Severity</span><i class="grip" data-i18n-title="colGrip"></i></span>
@@ -174,6 +175,7 @@
</header> </header>
<div class="set-body"> <div class="set-body">
<div class="set-side">
<nav id="setNav" class="set-nav" role="tablist" aria-orientation="vertical"> <nav id="setNav" class="set-nav" role="tablist" aria-orientation="vertical">
<button type="button" role="tab" data-tab="locale"> <button type="button" role="tab" data-tab="locale">
<svg viewBox="0 0 24 24"><circle cx="12" cy="12" r="9"/><path d="M3 12h18M12 3a14 14 0 0 1 0 18M12 3a14 14 0 0 0 0 18"/></svg> <svg viewBox="0 0 24 24"><circle cx="12" cy="12" r="9"/><path d="M3 12h18M12 3a14 14 0 0 1 0 18M12 3a14 14 0 0 0 0 18"/></svg>
@@ -195,7 +197,20 @@
<svg viewBox="0 0 24 24"><ellipse cx="12" cy="5" rx="8" ry="3"/><path d="M4 5v14c0 1.7 3.6 3 8 3s8-1.3 8-3V5M4 12c0 1.7 3.6 3 8 3s8-1.3 8-3"/></svg> <svg viewBox="0 0 24 24"><ellipse cx="12" cy="5" rx="8" ry="3"/><path d="M4 5v14c0 1.7 3.6 3 8 3s8-1.3 8-3V5M4 12c0 1.7 3.6 3 8 3s8-1.3 8-3"/></svg>
<span data-i18n="tabData">Database</span> <span data-i18n="tabData">Database</span>
</button> </button>
<button type="button" role="tab" data-tab="api">
<svg viewBox="0 0 24 24"><circle cx="7.5" cy="15.5" r="4.5"/><path d="m10.7 12.3 9.3-9.3M16 7l3 3M18 5l2 2"/></svg>
<span data-i18n="tabApi">API</span>
</button>
</nav> </nav>
<div id="setActions" class="set-actions">
<button id="setSave" class="btn save" type="button" disabled>
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M5 12.5 10 17l9-10"/></svg>
<span data-i18n="saveSettings">Save</span>
</button>
<button id="setCancel" class="btn ghost" type="button" disabled data-i18n="cancelSettings">Cancel</button>
<p id="setState" class="set-state" role="status"></p>
</div>
</div>
<div class="set-panels"> <div class="set-panels">
<!-- Localization --> <!-- Localization -->
@@ -435,6 +450,34 @@
</div> </div>
</section> </section>
</div> </div>
<!-- API tokens -->
<div class="set-panel wide" role="tabpanel" data-panel="api" hidden>
<section class="set-section">
<div class="sec-head">
<h3 data-i18n="tokensTitle">API tokens</h3>
<span id="tokenCount" class="muted small"></span>
</div>
<p class="muted small" data-i18n="tokensHelp"></p>
<div id="tokenList" class="token-list"></div>
<form id="tokenForm" class="token-form">
<input id="tokenName" type="text" class="dk-filter" maxlength="80" autocomplete="off" spellcheck="false" data-i18n-ph="tokenNamePh" data-i18n-aria="tokenNamePh">
<button id="tokenCreate" class="btn" type="submit" data-i18n="tokenCreate">Create a token</button>
</form>
<div id="tokenNew" class="token-new" hidden>
<p class="small" data-i18n="tokenOnce"></p>
<div class="token-secret">
<code id="tokenSecret"></code>
<button id="tokenCopy" class="btn" type="button" data-i18n="tokenCopy">Copy</button>
</div>
</div>
</section>
<section class="set-section">
<h3 data-i18n="tokensUse">Usage</h3>
<p class="muted small" data-i18n="tokensUseHelp"></p>
<pre id="tokenExample" class="token-example"></pre>
</section>
</div>
</div> </div>
</div> </div>
</div> </div>
+80 -27
View File
@@ -22,6 +22,10 @@
--hit: rgba(99, 102, 241, .18); --hit: rgba(99, 102, 241, .18);
--shadow: 0 10px 30px rgba(15, 23, 42, .12); --shadow: 0 10px 30px rgba(15, 23, 42, .12);
--chip-l: 38%; /* lightness of project-colored text (labels, Docker apps) */ --chip-l: 38%; /* lightness of project-colored text (labels, Docker apps) */
--src-syslog: #64748b; /* source badges: syslog received over the network */
--src-docker: #1d6fd6; /* Docker containers */
--src-kernel: #b4540a; /* kernel ring buffer (dmesg) */
--src-host: #b0407a; /* /var/log files of the host system */
--src-audit: #7c4dbd; /* audit events (auditd) */ --src-audit: #7c4dbd; /* audit events (auditd) */
--src-ls: #1f9aa3; /* LogStream internal logs */ --src-ls: #1f9aa3; /* LogStream internal logs */
--src-file: #5f8a1f; /* log files (Settings > Sources > Log files) */ --src-file: #5f8a1f; /* log files (Settings > Sources > Log files) */
@@ -74,6 +78,10 @@
--hit: rgba(129, 140, 248, .28); --hit: rgba(129, 140, 248, .28);
--shadow: 0 10px 30px rgba(0, 0, 0, .5); --shadow: 0 10px 30px rgba(0, 0, 0, .5);
--chip-l: 70%; --chip-l: 70%;
--src-syslog: #94a3b8;
--src-docker: #5aa2f5;
--src-kernel: #e7a25a;
--src-host: #e58bbb;
--src-audit: #a983e6; --src-audit: #a983e6;
--src-ls: #3cc3cc; --src-ls: #3cc3cc;
--src-file: #9ccc54; --src-file: #9ccc54;
@@ -110,10 +118,13 @@
--hit: rgba(129, 140, 248, .28); --hit: rgba(129, 140, 248, .28);
--shadow: 0 10px 30px rgba(0, 0, 0, .5); --shadow: 0 10px 30px rgba(0, 0, 0, .5);
--chip-l: 70%; --chip-l: 70%;
--src-syslog: #94a3b8;
--src-docker: #5aa2f5;
--src-kernel: #e7a25a;
--src-host: #e58bbb;
--src-audit: #a983e6; --src-audit: #a983e6;
--src-ls: #3cc3cc; --src-ls: #3cc3cc;
--src-file: #9ccc54; --src-file: #9ccc54;
--src-file: #9ccc54;
--src-tint: 9%; --src-tint: 9%;
--sev-crit: #fb7185; --sev-crit: #fb7185;
--sev-err: #f87171; --sev-err: #f87171;
@@ -344,12 +355,12 @@ body.busy .progress::after {
.list { margin: 6px 20px 0; background: var(--panel); border: 1px solid var(--border); border-radius: var(--radius); overflow: hidden; } .list { margin: 6px 20px 0; background: var(--panel); border: 1px solid var(--border); border-radius: var(--radius); overflow: hidden; }
.list:empty { display: none; } .list:empty { display: none; }
/* Columns: received, message time, severity, host, app, tag codes, message. Widths come from /* Columns: source, received, message time, severity, host, app, tag codes, message. Widths come from
--col-* (set on #table when a column has been resized, see app.js), shared by --col-* (set on #table when a column has been resized, see app.js), shared by
the header and every row through subgrid so that the columns line up. */ the header and every row through subgrid so that the columns line up. */
.table { .table {
display: grid; display: grid;
grid-template-columns: var(--col-rcv, max-content) var(--col-mt, max-content) var(--col-sev, 4.6rem) grid-template-columns: minmax(3.9rem, max-content) var(--col-rcv, max-content) var(--col-mt, max-content) var(--col-sev, 4.6rem)
var(--col-host, minmax(5rem, 9rem)) var(--col-app, minmax(4rem, 8rem)) var(--codes-w) minmax(0, 1fr); var(--col-host, minmax(5rem, 9rem)) var(--col-app, minmax(4rem, 8rem)) var(--codes-w) minmax(0, 1fr);
column-gap: 12px; column-gap: 12px;
margin: 6px 20px 0; background: var(--panel); border: 1px solid var(--border); border-radius: var(--radius); margin: 6px 20px 0; background: var(--panel); border: 1px solid var(--border); border-radius: var(--radius);
@@ -379,20 +390,19 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
.row { .row {
display: grid; display: grid;
grid-template-columns: max-content max-content 4.6rem minmax(5rem, 9rem) minmax(4rem, 8rem) var(--codes-w) 1fr; grid-template-columns: max-content max-content max-content 4.6rem minmax(5rem, 9rem) minmax(4rem, 8rem) var(--codes-w) 1fr;
gap: 0 12px; align-items: baseline; gap: 0 12px; align-items: baseline;
padding: 5px 14px 5px 11px; padding: 3px 14px 3px 11px;
border-left: 3px solid transparent; border-left: 3px solid transparent;
border-bottom: 1px solid var(--border); border-bottom: 1px solid var(--border);
font-family: var(--log-font, var(--mono)); font-size: var(--log-size); font-family: var(--log-font, var(--mono)); font-size: var(--log-size); line-height: 1.35;
font-variant-ligatures: none; /* keep -> != etc. as typed */ font-variant-ligatures: none; /* keep -> != etc. as typed */
cursor: pointer; cursor: pointer;
} }
.row:last-child { border-bottom: 0; } .row:last-child { border-bottom: 0; }
/* Settings > Interface > Density: compact fits about 50% more lines on screen */ /* Settings > Interface > Density: compact fits about 50% more lines on screen */
:root[data-density="compact"] .row { padding-top: 1px; padding-bottom: 1px; line-height: 1.25; } :root[data-density="compact"] .row { padding-top: 1px; padding-bottom: 1px; line-height: 1.2; }
:root[data-density="compact"] .row .sev { padding-top: 0; padding-bottom: 0; line-height: 1.3; } :root[data-density="compact"] .row :is(.src-badge, .sev, .codes b) { line-height: 1.15; }
:root[data-density="compact"] .row .codes b { padding-top: 0; padding-bottom: 0; line-height: 1.2; }
.row:hover { background: var(--row-hover); } .row:hover { background: var(--row-hover); }
.row:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; } .row:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; }
.row.new { animation: flash 1.2s ease-out; } .row.new { animation: flash 1.2s ease-out; }
@@ -410,7 +420,7 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
.sev { .sev {
justify-self: start; max-width: 100%; overflow: hidden; text-overflow: ellipsis; justify-self: start; max-width: 100%; overflow: hidden; text-overflow: ellipsis;
font-size: 10.5px; font-weight: 700; text-transform: uppercase; letter-spacing: .03em; font-size: 10.5px; font-weight: 700; text-transform: uppercase; letter-spacing: .03em;
padding: 1px 6px; border-radius: 5px; padding: 0 6px; border-radius: 5px; line-height: 1.3;
color: var(--sev); background: color-mix(in srgb, var(--sev) 14%, transparent); color: var(--sev); background: color-mix(in srgb, var(--sev) 14%, transparent);
} }
.sev-crit { --sev: var(--tag-error); --sev-text: var(--tag-error-text); } .sev-crit { --sev: var(--tag-error); --sev-text: var(--tag-error-text); }
@@ -424,16 +434,21 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
.row.sev-crit, .row.sev-err, .row.sev-warning { border-left-color: var(--sev); } .row.sev-crit, .row.sev-err, .row.sev-warning { border-left-color: var(--sev); }
.row.sev-crit, .row.sev-err { background: color-mix(in srgb, var(--sev) 7%, transparent); } .row.sev-crit, .row.sev-err { background: color-mix(in srgb, var(--sev) 7%, transparent); }
.row .host[data-act], .row .app[data-act] { cursor: pointer; } .row .host[data-act], .row .app[data-act] { cursor: pointer; }
/* Audit events and LogStream internal logs: colored edge (over the severity /* Source badge at the start of every row, fixed width, one color per source.
one), light tint, badge before the app name. */ Audit events, log files and LogStream internal logs also get a colored edge
(over the severity one) and a light tint. */
.row.src-syslog { --src: var(--src-syslog); }
.row.src-docker { --src: var(--src-docker); }
.row.src-kernel { --src: var(--src-kernel); }
.row.src-host { --src: var(--src-host); }
.row.src-audit { --src: var(--src-audit); } .row.src-audit { --src: var(--src-audit); }
.row.src-logstream { --src: var(--src-ls); } .row.src-logstream { --src: var(--src-ls); }
.row.src-file { --src: var(--src-file); } .row.src-file { --src: var(--src-file); }
.row.src-audit, .row.src-logstream, .row.src-file { border-left-color: var(--src); background: color-mix(in srgb, var(--src) var(--src-tint), transparent); } .row.src-audit, .row.src-logstream, .row.src-file { border-left-color: var(--src); background: color-mix(in srgb, var(--src) var(--src-tint), transparent); }
.row.src-audit:hover, .row.src-logstream:hover, .row.src-file:hover { background: color-mix(in srgb, var(--src) 13%, var(--row-hover)); } .row.src-audit:hover, .row.src-logstream:hover, .row.src-file:hover { background: color-mix(in srgb, var(--src) 13%, var(--row-hover)); }
.src-badge { .src-badge {
display: inline-block; margin-right: 5px; padding: 0 4px; border-radius: 4px; vertical-align: 1px; justify-self: start; box-sizing: content-box; width: 6ch; padding: 0 4px; border-radius: 4px; text-align: center;
font: 700 9.5px/1.45 var(--sans); letter-spacing: .04em; font-family: var(--log-font, var(--mono)); font-size: 10px; font-weight: 700; letter-spacing: .02em; line-height: 1.3;
color: var(--src); background: color-mix(in srgb, var(--src) 16%, transparent); color: var(--src); background: color-mix(in srgb, var(--src) 16%, transparent);
box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--src) 40%, transparent); box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--src) 40%, transparent);
} }
@@ -448,10 +463,10 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
.row .codes { white-space: nowrap; overflow: hidden; } .row .codes { white-space: nowrap; overflow: hidden; }
.row .codes > span { display: inline-flex; gap: 3px; vertical-align: 1px; } .row .codes > span { display: inline-flex; gap: 3px; vertical-align: 1px; }
.row .codes b, .tag-code { .row .codes b, .tag-code {
min-width: 2.2ch; padding: 1px 4px; border-radius: 5px; text-align: center; min-width: 2.2ch; padding: 0 4px; border-radius: 5px; text-align: center;
/* same font as the severity badges (the log font) */ /* same font as the severity badges (the log font) */
font-family: var(--log-font, var(--mono)); font-size: 10.5px; font-weight: 700; letter-spacing: .03em; font-family: var(--log-font, var(--mono)); font-size: 10.5px; font-weight: 700; letter-spacing: .03em;
line-height: 1.35; font-variant-numeric: tabular-nums; line-height: 1.3; font-variant-numeric: tabular-nums;
background: var(--code-bg); color: #0b0f17; background: var(--code-bg); color: #0b0f17;
} }
.row .codes b.more { background: none; box-shadow: inset 0 0 0 1px var(--code-bg); color: var(--muted); } .row .codes b.more { background: none; box-shadow: inset 0 0 0 1px var(--code-bg); color: var(--muted); }
@@ -544,11 +559,24 @@ dialog.settings {
dialog.settings .dlg { display: flex; flex-direction: column; height: 100%; padding: 0; } dialog.settings .dlg { display: flex; flex-direction: column; height: 100%; padding: 0; }
dialog.settings .dlg > header { padding: 14px 16px 14px 22px; border-bottom: 1px solid var(--border); } dialog.settings .dlg > header { padding: 14px 16px 14px 22px; border-bottom: 1px solid var(--border); }
.set-body { flex: 1; min-height: 0; display: grid; grid-template-columns: minmax(176px, max-content) minmax(0, 1fr); } .set-body { flex: 1; min-height: 0; display: grid; grid-template-columns: minmax(176px, max-content) minmax(0, 1fr); }
.set-nav { .set-side {
display: flex; flex-direction: column; gap: 2px; display: flex; flex-direction: column; min-height: 0; overflow-y: auto;
padding: 12px 10px; border-right: 1px solid var(--border); border-right: 1px solid var(--border);
background: color-mix(in srgb, var(--panel-2) 50%, transparent); background: color-mix(in srgb, var(--panel-2) 50%, transparent);
} }
.set-nav { display: flex; flex-direction: column; gap: 2px; padding: 12px 10px; }
/* Save / Cancel, under the section labels: Save takes the accent color while changes wait. */
.set-actions {
display: flex; flex-wrap: wrap; gap: 6px; margin: 4px 10px 12px; padding-top: 12px;
border-top: 1px solid var(--border);
}
.set-actions .btn { flex: 1; justify-content: center; }
.set-actions .btn.save { color: var(--muted); }
.set-actions .btn svg { width: 16px; height: 16px; }
.set-actions.dirty .btn.save { background: var(--accent); border-color: var(--accent); color: #fff; box-shadow: 0 0 0 3px var(--accent-soft); }
.set-actions.dirty .btn.save:hover { filter: brightness(1.08); }
.set-state { flex-basis: 100%; margin: 2px 2px 0; font-size: 12px; color: var(--muted); }
.set-actions.dirty .set-state { color: var(--accent); font-weight: 550; }
.set-nav button { .set-nav button {
display: flex; align-items: center; gap: 10px; display: flex; align-items: center; gap: 10px;
height: 38px; padding: 0 12px; border: 0; border-radius: 9px; height: 38px; padding: 0 12px; border: 0; border-radius: 9px;
@@ -576,10 +604,11 @@ dialog.settings .dlg > header { padding: 14px 16px 14px 22px; border-bottom: 1px
.set-panel .preview-list { margin: 14px 0 10px; pointer-events: none; } .set-panel .preview-list { margin: 14px 0 10px; pointer-events: none; }
.preview-list .row { .preview-list .row {
cursor: default; cursor: default;
grid-template-columns: max-content max-content minmax(0, max-content) minmax(0, 1fr); grid-template-columns: max-content max-content max-content minmax(0, max-content) minmax(0, 1fr);
grid-template-areas: "rcv sev host app" "msg msg msg msg"; grid-template-areas: "src rcv sev host app" "msg msg msg msg msg";
gap: 3px 10px; gap: 3px 10px;
} }
.preview-list .row .src-badge { grid-area: src; }
.preview-list .row time.rcv { grid-area: rcv; } .preview-list .row time.rcv { grid-area: rcv; }
.preview-list .row time.mt { display: none; } .preview-list .row time.mt { display: none; }
.preview-list .row .sev { grid-area: sev; } .preview-list .row .sev { grid-area: sev; }
@@ -814,10 +843,11 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
.table .list { display: block; margin: 0; } .table .list { display: block; margin: 0; }
.list-head { display: none; } .list-head { display: none; }
.row, .table .row { .row, .table .row {
grid-template-columns: auto auto 1fr auto; grid-template-columns: auto auto auto 1fr auto;
grid-template-areas: "rcv sev host codes" "msg msg msg msg"; grid-template-areas: "src rcv sev host codes" "msg msg msg msg msg";
gap: 3px 8px; padding: 8px 12px 8px 10px; gap: 3px 8px; padding: 8px 12px 8px 10px;
} }
.row .src-badge { grid-area: src; }
.row time.rcv { grid-area: rcv; } .row time.rcv { grid-area: rcv; }
.row time.mt { display: none; } /* still shown in the details panel */ .row time.mt { display: none; } /* still shown in the details panel */
.row .sev { grid-area: sev; } .row .sev { grid-area: sev; }
@@ -831,10 +861,11 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
.field-grid { grid-template-columns: 1fr; gap: 4px; } .field-grid { grid-template-columns: 1fr; gap: 4px; }
dialog.settings { height: calc(100vh - 24px); width: calc(100vw - 16px); } dialog.settings { height: calc(100vh - 24px); width: calc(100vw - 16px); }
.set-body { grid-template-columns: 1fr; grid-template-rows: auto minmax(0, 1fr); } .set-body { grid-template-columns: 1fr; grid-template-rows: auto minmax(0, 1fr); }
.set-nav { .set-side { border-right: 0; border-bottom: 1px solid var(--border); overflow: visible; }
display: grid; grid-template-columns: repeat(5, minmax(0, 1fr)); padding: 6px; .set-nav { display: grid; grid-template-columns: repeat(5, minmax(0, 1fr)); padding: 6px; }
border-right: 0; border-bottom: 1px solid var(--border); .set-actions { margin: 0 6px 6px; padding-top: 6px; align-items: center; }
} .set-actions .btn { flex: 0 0 auto; height: 30px; }
.set-state { flex: 1; flex-basis: auto; margin: 0; text-align: right; }
.set-nav button { .set-nav button {
flex-direction: column; justify-content: center; height: auto; padding: 6px 2px; gap: 3px; flex-direction: column; justify-content: center; height: auto; padding: 6px 2px; gap: 3px;
font-size: 11.5px; text-align: center; font-size: 11.5px; text-align: center;
@@ -881,3 +912,25 @@ body.login-page {
.login-card input:focus { outline: 0; border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); } .login-card input:focus { outline: 0; border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); }
.login-card .btn.primary { height: 38px; justify-content: center; margin-top: 6px; font-size: 14px; } .login-card .btn.primary { height: 38px; justify-content: center; margin-top: 6px; font-size: 14px; }
.login-err { margin: 0; color: var(--sev-err); font-size: 13px; } .login-err { margin: 0; color: var(--sev-err); font-size: 13px; }
/* API tokens (Settings > API) */
.token-list { display: flex; flex-direction: column; gap: 4px; margin: 12px 0 10px; }
.token-row {
display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 2px 12px; align-items: center;
padding: 8px 10px; border-radius: 8px; background: var(--panel-2); font-size: 12.5px;
}
.token-row b { font-weight: 600; overflow-wrap: anywhere; }
.token-row code { font-family: var(--mono); font-size: 11.5px; color: var(--muted); }
.token-row .muted { grid-column: 1; font-size: 11.5px; }
.token-row .btn { grid-row: 1 / span 2; grid-column: 2; height: 28px; font-size: 12px; }
.token-empty { color: var(--muted); font-size: 12.5px; }
.token-form { display: flex; gap: 10px; align-items: center; }
.token-form .btn { height: 28px; font-size: 12px; }
.token-new { margin-top: 12px; padding: 10px 12px; border-radius: 8px; border: 1px solid var(--accent); background: var(--accent-soft); }
.token-secret { display: flex; gap: 10px; align-items: center; margin-top: 8px; }
.token-secret code { flex: 1; min-width: 0; font: 12px var(--mono); overflow-wrap: anywhere; user-select: all; }
.token-secret .btn { height: 28px; font-size: 12px; }
.token-example {
margin: 8px 0 0; padding: 8px 12px; border-radius: 8px; background: var(--panel-2);
font: 12px var(--mono); white-space: pre-wrap; overflow-wrap: anywhere;
}