Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a5b74facc0 |
No files matched your search
@@ -390,79 +390,6 @@ charge (pas de `| pipes`).
|
|||||||
|
|
||||||
En ligne de commande : `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`.
|
En ligne de commande : `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`.
|
||||||
|
|
||||||
## API de lecture (n8n, scripts)
|
|
||||||
|
|
||||||
`GET /api/v1/logs` renvoie les logs stockés qui suivent un curseur, du plus ancien au plus
|
|
||||||
récent : un outil appelé à intervalle régulier (n8n, un script, un connecteur SIEM) reçoit ainsi
|
|
||||||
chaque nouveau log une fois, sans trou ni doublon, même quand beaucoup de logs ont le même
|
|
||||||
horodatage.
|
|
||||||
|
|
||||||
L'accès passe par un **jeton d'API**, créé dans *Paramètres > API* par un administrateur et
|
|
||||||
envoyé dans `Authorization: Bearer lst_…`. Un jeton est en lecture seule et n'ouvre que
|
|
||||||
`/api/v1/` ; il fonctionne quel que soit le mode d'[authentification](#authentification)
|
|
||||||
(`local`, `oidc` ou aucune). Il n'est affiché qu'une fois, à sa création : LogStream n'en garde
|
|
||||||
que l'empreinte SHA-256, dans `/data/api_tokens.json`. Le révoquer (puis Valider) coupe l'accès immédiatement ;
|
|
||||||
la liste indique quand et d'où chaque jeton a servi pour la dernière fois. Une session ouverte
|
|
||||||
dans le navigateur peut aussi appeler la route.
|
|
||||||
|
|
||||||
| Paramètre | Rôle |
|
|
||||||
|---|---|
|
|
||||||
| `query` | filtre LogsQL (par défaut `*`), par exemple `severity:in(err,crit) app:sshd`. Les pipes (`\| ...`) sont refusés. |
|
|
||||||
| `since` | point de départ : le `next_cursor` de la réponse précédente, une date RFC 3339 (`2026-11-01T08:00:00Z`) ou une durée avant maintenant (`15m`, `6h`, `7d`). Par défaut : `1h`. |
|
|
||||||
| `limit` | lignes par réponse, de 1 à 10 000 (1 000 par défaut). |
|
|
||||||
| `lag` | seuls les logs reçus depuis au moins cette durée sont renvoyés (`30s` par défaut, `1h` au plus), pour ne pas sauter les logs encore dans la file ou en attente de leur nom DNS. |
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"rows": [{"_time": "2026-11-01T14:02:11.52Z", "_msg": "…", "host": "fw01", "app": "sshd", "severity": "err", …}],
|
|
||||||
"next_cursor": "eyJ0IjoiMjAyNi0xMS0wMVQxNDowMjoxMS41MjAwMDAwMDFaIiwicSI6IjJlYjE4YzNhIn0",
|
|
||||||
"has_more": false,
|
|
||||||
"until": "2026-11-01T14:05:30.1Z"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Les lignes contiennent les champs stockés (comme la vue JSON de l'interface), triées par
|
|
||||||
`_time`, l'heure de réception. Renvoyez toujours `next_cursor`, même quand `rows` est vide.
|
|
||||||
`has_more: true` signifie que d'autres logs sont prêts : rappelez tout de suite.
|
|
||||||
`waiting: true` signifie que des logs du [tampon disque](#configuration) sont encore en cours
|
|
||||||
d'envoi à VictoriaLogs : rien n'est renvoyé tant qu'ils ne sont pas stockés, pour n'en manquer
|
|
||||||
aucun. Le curseur est lié à sa `query` : changer la requête garde la position dans le temps. Les
|
|
||||||
erreurs arrivent sous la forme `{"error", "code"}` avec le statut 400 (paramètre ou LogsQL
|
|
||||||
invalide), 401 (jeton invalide ou révoqué) ou 502 (VictoriaLogs injoignable).
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -H "Authorization: Bearer lst_…" \
|
|
||||||
"https://logs.example.org/api/v1/logs?query=severity:err&since=15m&limit=1000"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Exemple : workflow n8n
|
|
||||||
|
|
||||||
Quatre nœuds, le curseur gardé dans les données statiques du workflow d'une exécution à l'autre :
|
|
||||||
|
|
||||||
1. **Schedule Trigger** : toutes les minutes.
|
|
||||||
2. **Code** (*Run Once for All Items*), lit le curseur :
|
|
||||||
```js
|
|
||||||
const data = $getWorkflowStaticData('global');
|
|
||||||
return [{ json: { since: data.cursor || '15m' } }];
|
|
||||||
```
|
|
||||||
3. **HTTP Request** : méthode `GET`, URL `https://logs.example.org/api/v1/logs`.
|
|
||||||
*Authentication* : Generic Credential Type > Header Auth, avec le nom `Authorization` et la
|
|
||||||
valeur `Bearer lst_…`. *Send Query Parameters* : `query` = `severity:in(emerg,alert,crit,err)`,
|
|
||||||
`since` = `{{ $json.since }}`, `limit` = `1000`.
|
|
||||||
4. **Code** (*Run Once for All Items*), enregistre le curseur et fait de chaque log un item :
|
|
||||||
```js
|
|
||||||
const data = $getWorkflowStaticData('global');
|
|
||||||
const page = $input.first().json;
|
|
||||||
data.cursor = page.next_cursor;
|
|
||||||
return page.rows.map((row) => ({ json: row }));
|
|
||||||
```
|
|
||||||
|
|
||||||
Branchez ensuite ce que les logs doivent alimenter (Slack, mail, ticket, base de données…). n8n ne
|
|
||||||
garde les données statiques que pour les exécutions du workflow actif, pas pour les tests
|
|
||||||
manuels : pendant les essais, chaque exécution repart de `15m`. Au-delà de 1 000 logs par minute,
|
|
||||||
`has_more` vaut true et les exécutions suivantes rattrapent le retard ; augmentez `limit` ou la
|
|
||||||
fréquence du déclencheur si besoin.
|
|
||||||
|
|
||||||
## Paramètres
|
## Paramètres
|
||||||
|
|
||||||
L'icône en forme d'engrenage ouvre les paramètres, organisés en onglets. Les modifications
|
L'icône en forme d'engrenage ouvre les paramètres, organisés en onglets. Les modifications
|
||||||
@@ -525,11 +452,6 @@ et la réinitialisation des colonnes agissent immédiatement.
|
|||||||
désactivée par défaut : mettez `ALLOW_PURGE=true` pour l'autoriser. Tout administrateur peut
|
désactivée par défaut : mettez `ALLOW_PURGE=true` pour l'autoriser. Tout administrateur peut
|
||||||
alors purger : activez l'[authentification](#authentification) si l'interface est accessible
|
alors purger : activez l'[authentification](#authentification) si l'interface est accessible
|
||||||
à d'autres.
|
à d'autres.
|
||||||
- **API** : jetons de l'[API de lecture](#api-de-lecture-n8n-scripts) : en ajouter un avec un
|
|
||||||
nom, en révoquer un, voir quand et d'où chacun a servi pour la dernière fois. Ajouts et
|
|
||||||
révocations attendent **Valider** comme les autres réglages (**Annuler** les abandonne). Après
|
|
||||||
Valider, chaque nouveau jeton est affiché une seule fois avec un bouton Copier, et l'exemple
|
|
||||||
curl en dessous l'utilise, surligné, avec son propre bouton Copier. Réservé aux administrateurs.
|
|
||||||
|
|
||||||
## Authentification
|
## Authentification
|
||||||
|
|
||||||
@@ -709,8 +631,6 @@ Pour mettre à jour l'une d'elles :
|
|||||||
| `hub.go` | envoie les nouveaux messages aux navigateurs (SSE) |
|
| `hub.go` | envoie les nouveaux messages aux navigateurs (SSE) |
|
||||||
| `rdns.go` | résolutions DNS inverses avec cache |
|
| `rdns.go` | résolutions DNS inverses avec cache |
|
||||||
| `export.go` | export CSV en flux |
|
| `export.go` | export CSV en flux |
|
||||||
| `logsapi.go` | API de lecture `/api/v1/logs` : curseur, groupes de logs de même heure |
|
|
||||||
| `apitokens.go` | jetons d'API : stockage (haché), contrôle Bearer, `/api/tokens` |
|
|
||||||
| `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) |
|
| `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) |
|
||||||
| `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources de logs |
|
| `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources de logs |
|
||||||
| `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` |
|
| `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` |
|
||||||
@@ -721,7 +641,6 @@ Pour mettre à jour l'une d'elles :
|
|||||||
| `tags.go` | stockage des tags de couleur |
|
| `tags.go` | stockage des tags de couleur |
|
||||||
| `presets.go`, `presets.json` | préréglages de tags (`/api/presets`), liste intégrée |
|
| `presets.go`, `presets.json` | préréglages de tags (`/api/presets`), liste intégrée |
|
||||||
| `api.go` | routes HTTP `/api/*` |
|
| `api.go` | routes HTTP `/api/*` |
|
||||||
| `static.go` | fichiers de l'interface embarqués : ETag par fichier, revérifiés à chaque chargement (`no-cache`, 304 s'ils n'ont pas changé) pour toujours prendre une mise à jour ; polices en cache 30 jours |
|
|
||||||
| `web/` | interface (HTML, CSS, JavaScript simple, sans étape de build), embarquée dans le binaire ; les traductions sont dans `web/app.js` (`I18N`), et dans `web/login.html` pour la page de connexion |
|
| `web/` | interface (HTML, CSS, JavaScript simple, sans étape de build), embarquée dans le binaire ; les traductions sont dans `web/app.js` (`I18N`), et dans `web/login.html` pour la page de connexion |
|
||||||
|
|
||||||
## Remarque
|
## Remarque
|
||||||
|
|||||||
@@ -354,75 +354,6 @@ chosen in Settings), `severity`, `facility`, `host`, `host_ip`, `app`, `pid`, `s
|
|||||||
|
|
||||||
From the command line: `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`.
|
From the command line: `curl -o logs.csv 'localhost:8080/api/export.csv?q=error&range=24h&tz=Europe/Paris'`.
|
||||||
|
|
||||||
## Reading API (n8n, scripts)
|
|
||||||
|
|
||||||
`GET /api/v1/logs` returns the stored logs that follow a cursor, oldest first, so that a tool
|
|
||||||
called on a schedule (n8n, a script, a SIEM connector) gets every new log once, without gaps
|
|
||||||
or duplicates, even when many logs share the same timestamp.
|
|
||||||
|
|
||||||
Access goes through an **API token**, created in *Settings > API* by an administrator and sent
|
|
||||||
as `Authorization: Bearer lst_…`. A token is read-only and only opens `/api/v1/`; it works
|
|
||||||
whatever the [authentication](#authentication) mode (`local`, `oidc`, or none). It is shown once,
|
|
||||||
when it is created: LogStream only keeps its SHA-256 hash, in `/data/api_tokens.json`.
|
|
||||||
Revoking it (then Save) cuts off access at once; the list shows when and from where each token was last
|
|
||||||
used. A logged-in browser session can call the route too.
|
|
||||||
|
|
||||||
| Parameter | Meaning |
|
|
||||||
|---|---|
|
|
||||||
| `query` | LogsQL filter (default `*`), e.g. `severity:in(err,crit) app:sshd`. Pipes (`\| ...`) are refused. |
|
|
||||||
| `since` | where to start: the `next_cursor` of the previous answer, an RFC 3339 time (`2026-11-01T08:00:00Z`) or a duration back from now (`15m`, `6h`, `7d`). Default: `1h`. |
|
|
||||||
| `limit` | rows per answer, 1 to 10,000 (default 1,000). |
|
|
||||||
| `lag` | only logs received at least this long ago are returned (default `30s`, at most `1h`), so that the logs still in the queue or waiting for their DNS name are not skipped. |
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"rows": [{"_time": "2026-11-01T14:02:11.52Z", "_msg": "…", "host": "fw01", "app": "sshd", "severity": "err", …}],
|
|
||||||
"next_cursor": "eyJ0IjoiMjAyNi0xMS0wMVQxNDowMjoxMS41MjAwMDAwMDFaIiwicSI6IjJlYjE4YzNhIn0",
|
|
||||||
"has_more": false,
|
|
||||||
"until": "2026-11-01T14:05:30.1Z"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Rows have the stored fields (as in the UI's JSON view), sorted by `_time`, the reception time.
|
|
||||||
Always send back `next_cursor`, even when `rows` is empty. `has_more: true` means more logs are
|
|
||||||
ready: call again at once. `waiting: true` means logs from the [disk buffer](#configuration) are
|
|
||||||
still being sent to VictoriaLogs: nothing is returned until they are stored, so none is missed.
|
|
||||||
The cursor belongs to its `query`: changing the query keeps the position in time. Errors come
|
|
||||||
as `{"error", "code"}` with status 400 (bad parameter or LogsQL), 401 (invalid or revoked token)
|
|
||||||
or 502 (VictoriaLogs unreachable).
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -H "Authorization: Bearer lst_…" \
|
|
||||||
"https://logs.example.org/api/v1/logs?query=severity:err&since=15m&limit=1000"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example: n8n workflow
|
|
||||||
|
|
||||||
Four nodes, the cursor kept in the workflow's static data between runs:
|
|
||||||
|
|
||||||
1. **Schedule Trigger**: every minute.
|
|
||||||
2. **Code** (*Run Once for All Items*), reads the cursor:
|
|
||||||
```js
|
|
||||||
const data = $getWorkflowStaticData('global');
|
|
||||||
return [{ json: { since: data.cursor || '15m' } }];
|
|
||||||
```
|
|
||||||
3. **HTTP Request**: method `GET`, URL `https://logs.example.org/api/v1/logs`.
|
|
||||||
*Authentication*: Generic Credential Type > Header Auth, with name `Authorization` and value
|
|
||||||
`Bearer lst_…`. *Send Query Parameters*: `query` = `severity:in(emerg,alert,crit,err)`,
|
|
||||||
`since` = `{{ $json.since }}`, `limit` = `1000`.
|
|
||||||
4. **Code** (*Run Once for All Items*), saves the cursor and turns each log into an item:
|
|
||||||
```js
|
|
||||||
const data = $getWorkflowStaticData('global');
|
|
||||||
const page = $input.first().json;
|
|
||||||
data.cursor = page.next_cursor;
|
|
||||||
return page.rows.map((row) => ({ json: row }));
|
|
||||||
```
|
|
||||||
|
|
||||||
Then connect what the logs should feed (Slack, mail, ticket, database…). n8n only keeps static
|
|
||||||
data for runs of the active workflow, not for manual test runs: while testing, each run starts
|
|
||||||
again from `15m`. When more than 1,000 logs arrive per minute, `has_more` is true and the
|
|
||||||
following runs catch up; raise `limit` or run the trigger more often if needed.
|
|
||||||
|
|
||||||
## Settings
|
## Settings
|
||||||
|
|
||||||
The gear icon opens the settings, organized in tabs. Changes show at once but are only kept
|
The gear icon opens the settings, organized in tabs. Changes show at once but are only kept
|
||||||
@@ -477,11 +408,6 @@ and log sources are saved on the server. Purging and resetting column widths act
|
|||||||
(already set in `docker-compose.yml`). The feature is off by default: set `ALLOW_PURGE=true`
|
(already set in `docker-compose.yml`). The feature is off by default: set `ALLOW_PURGE=true`
|
||||||
to allow it. Any admin can then purge: turn on [authentication](#authentication) if the UI
|
to allow it. Any admin can then purge: turn on [authentication](#authentication) if the UI
|
||||||
is reachable by others.
|
is reachable by others.
|
||||||
- **API**: API tokens for the [reading API](#reading-api-n8n-scripts): add one with a name,
|
|
||||||
revoke one, see when and from where each was last used. Additions and revocations wait for
|
|
||||||
**Save** like the other settings (**Cancel** drops them). After Save, each new token is shown
|
|
||||||
once with a copy button, and the curl example below uses it, highlighted, with its own copy
|
|
||||||
button. Reserved to administrators.
|
|
||||||
|
|
||||||
## Authentication
|
## Authentication
|
||||||
|
|
||||||
@@ -651,8 +577,6 @@ To update one of them:
|
|||||||
| `hub.go` | pushes new messages to browsers (SSE) |
|
| `hub.go` | pushes new messages to browsers (SSE) |
|
||||||
| `rdns.go` | cached reverse DNS lookups |
|
| `rdns.go` | cached reverse DNS lookups |
|
||||||
| `export.go` | streamed CSV export |
|
| `export.go` | streamed CSV export |
|
||||||
| `logsapi.go` | reading API `/api/v1/logs`: cursor, groups of logs with the same time |
|
|
||||||
| `apitokens.go` | API tokens: storage (hashed), Bearer check, `/api/tokens` |
|
|
||||||
| `docker.go` | Docker container logs (API, followers, positions, level detection) |
|
| `docker.go` | Docker container logs (API, followers, positions, level detection) |
|
||||||
| `syslogserver.go` | syslog listeners opened and closed from Settings > Log sources |
|
| `syslogserver.go` | syslog listeners opened and closed from Settings > Log sources |
|
||||||
| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower |
|
| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower |
|
||||||
@@ -663,7 +587,6 @@ To update one of them:
|
|||||||
| `tags.go` | color tag storage |
|
| `tags.go` | color tag storage |
|
||||||
| `presets.go`, `presets.json` | color tag presets (`/api/presets`), built-in list |
|
| `presets.go`, `presets.json` | color tag presets (`/api/presets`), built-in list |
|
||||||
| `api.go` | `/api/*` HTTP routes |
|
| `api.go` | `/api/*` HTTP routes |
|
||||||
| `static.go` | embedded UI files: ETag per file, revalidated on each load (`no-cache`, 304 when unchanged) so an update is always picked up; fonts cached 30 days |
|
|
||||||
| `web/` | UI (HTML, CSS, plain JavaScript, no build step), embedded in the binary; translations live in `web/app.js` (`I18N`), and in `web/login.html` for the login page |
|
| `web/` | UI (HTML, CSS, plain JavaScript, no build step), embedded in the binary; translations live in `web/app.js` (`I18N`), and in `web/login.html` for the login page |
|
||||||
|
|
||||||
## Note
|
## Note
|
||||||
|
|||||||
@@ -28,16 +28,11 @@ type API struct {
|
|||||||
files *Files
|
files *Files
|
||||||
selfLog *SelfLog
|
selfLog *SelfLog
|
||||||
dbstats dbStatsCache
|
dbstats dbStatsCache
|
||||||
tokens *TokenStore
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (a *API) Routes(mux *http.ServeMux) {
|
func (a *API) Routes(mux *http.ServeMux) {
|
||||||
mux.HandleFunc("GET /healthz", a.health)
|
mux.HandleFunc("GET /healthz", a.health)
|
||||||
mux.HandleFunc("GET /api/logs", a.logs)
|
mux.HandleFunc("GET /api/logs", a.logs)
|
||||||
mux.HandleFunc("GET /api/v1/logs", a.logsV1)
|
|
||||||
mux.HandleFunc("GET /api/tokens", a.listTokens)
|
|
||||||
mux.HandleFunc("POST /api/tokens", a.createToken)
|
|
||||||
mux.HandleFunc("DELETE /api/tokens/{id}", a.deleteToken)
|
|
||||||
mux.HandleFunc("GET /api/histogram", a.histogram)
|
mux.HandleFunc("GET /api/histogram", a.histogram)
|
||||||
mux.HandleFunc("GET /api/facets", a.facets)
|
mux.HandleFunc("GET /api/facets", a.facets)
|
||||||
mux.HandleFunc("GET /api/stream", a.stream)
|
mux.HandleFunc("GET /api/stream", a.stream)
|
||||||
|
|||||||
-254
@@ -1,254 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"crypto/rand"
|
|
||||||
"crypto/sha256"
|
|
||||||
"crypto/subtle"
|
|
||||||
"encoding/base64"
|
|
||||||
"encoding/hex"
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"fmt"
|
|
||||||
"log"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"sync"
|
|
||||||
"time"
|
|
||||||
"unicode/utf8"
|
|
||||||
)
|
|
||||||
|
|
||||||
// API tokens: read-only access to /api/v1/ for scripts and automation tools
|
|
||||||
// (n8n…), sent as "Authorization: Bearer lst_…". They are created and revoked
|
|
||||||
// in Settings > API by an administrator, and work whatever the AUTH_MODE,
|
|
||||||
// including without authentication. Only a SHA-256 of each token is kept, in
|
|
||||||
// DATA_DIR/api_tokens.json: a token is shown once, when it is created.
|
|
||||||
|
|
||||||
const (
|
|
||||||
tokenPrefix = "lst_"
|
|
||||||
maxTokens = 100
|
|
||||||
lastUsedPeriod = time.Minute // last use of a token is written to disk at most this often (or on a new IP)
|
|
||||||
)
|
|
||||||
|
|
||||||
var errTokenNotFound = errors.New("token not found")
|
|
||||||
|
|
||||||
type apiToken struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
Hint string `json:"hint"` // start of the token, to recognize it
|
|
||||||
Hash string `json:"hash"` // hex SHA-256 of the whole token
|
|
||||||
Created time.Time `json:"created"`
|
|
||||||
LastUsed time.Time `json:"last_used,omitzero"`
|
|
||||||
LastIP string `json:"last_ip,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// apiTokenInfo is what the UI sees of a token (never its hash).
|
|
||||||
type apiTokenInfo struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
Hint string `json:"hint"`
|
|
||||||
Created time.Time `json:"created"`
|
|
||||||
LastUsed time.Time `json:"last_used,omitzero"`
|
|
||||||
LastIP string `json:"last_ip,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type TokenStore struct {
|
|
||||||
path string
|
|
||||||
mu sync.Mutex
|
|
||||||
tokens []apiToken
|
|
||||||
}
|
|
||||||
|
|
||||||
func LoadTokenStore(path string) (*TokenStore, error) {
|
|
||||||
s := &TokenStore{path: path}
|
|
||||||
b, err := os.ReadFile(path)
|
|
||||||
if errors.Is(err, os.ErrNotExist) {
|
|
||||||
return s, nil
|
|
||||||
}
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if err := json.Unmarshal(b, &s.tokens); err != nil {
|
|
||||||
return nil, fmt.Errorf("%s: %w", path, err)
|
|
||||||
}
|
|
||||||
return s, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// save writes the file atomically, readable by its owner only; the caller holds the lock.
|
|
||||||
func (s *TokenStore) save() error {
|
|
||||||
b, err := json.MarshalIndent(s.tokens, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(filepath.Dir(s.path), 0o755); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
tmp := s.path + ".tmp"
|
|
||||||
if err := os.WriteFile(tmp, b, 0o600); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return os.Rename(tmp, s.path)
|
|
||||||
}
|
|
||||||
|
|
||||||
func hashToken(token string) string {
|
|
||||||
sum := sha256.Sum256([]byte(token))
|
|
||||||
return hex.EncodeToString(sum[:])
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *TokenStore) List() []apiTokenInfo {
|
|
||||||
s.mu.Lock()
|
|
||||||
defer s.mu.Unlock()
|
|
||||||
out := make([]apiTokenInfo, len(s.tokens))
|
|
||||||
for i, t := range s.tokens {
|
|
||||||
out[i] = apiTokenInfo{ID: t.ID, Name: t.Name, Hint: t.Hint, Created: t.Created, LastUsed: t.LastUsed, LastIP: t.LastIP}
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create makes a new token and returns it in clear, the only time it exists so.
|
|
||||||
func (s *TokenStore) Create(name string) (apiTokenInfo, string, error) {
|
|
||||||
name = strings.TrimSpace(name)
|
|
||||||
if name == "" || utf8.RuneCountInString(name) > 80 {
|
|
||||||
return apiTokenInfo{}, "", &codedError{code: "token_name", msg: "give the token a name of 1 to 80 characters"}
|
|
||||||
}
|
|
||||||
b := make([]byte, 32)
|
|
||||||
if _, err := rand.Read(b); err != nil {
|
|
||||||
return apiTokenInfo{}, "", err
|
|
||||||
}
|
|
||||||
secret := tokenPrefix + base64.RawURLEncoding.EncodeToString(b)
|
|
||||||
t := apiToken{ID: newID(), Name: name, Hint: secret[:len(tokenPrefix)+6], Hash: hashToken(secret), Created: time.Now().UTC()}
|
|
||||||
|
|
||||||
s.mu.Lock()
|
|
||||||
defer s.mu.Unlock()
|
|
||||||
if len(s.tokens) >= maxTokens {
|
|
||||||
return apiTokenInfo{}, "", &codedError{code: "token_max", msg: fmt.Sprintf("at most %d API tokens", maxTokens)}
|
|
||||||
}
|
|
||||||
s.tokens = append(s.tokens, t)
|
|
||||||
if err := s.save(); err != nil {
|
|
||||||
s.tokens = s.tokens[:len(s.tokens)-1]
|
|
||||||
return apiTokenInfo{}, "", err
|
|
||||||
}
|
|
||||||
return apiTokenInfo{ID: t.ID, Name: t.Name, Hint: t.Hint, Created: t.Created}, secret, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *TokenStore) Delete(id string) error {
|
|
||||||
s.mu.Lock()
|
|
||||||
defer s.mu.Unlock()
|
|
||||||
for i, t := range s.tokens {
|
|
||||||
if t.ID == id {
|
|
||||||
s.tokens = append(s.tokens[:i:i], s.tokens[i+1:]...)
|
|
||||||
return s.save()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return errTokenNotFound
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check tells whether token is a valid API token, and notes its use.
|
|
||||||
func (s *TokenStore) Check(token, ip string) (name string, ok bool) {
|
|
||||||
if !strings.HasPrefix(token, tokenPrefix) {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
h := []byte(hashToken(token))
|
|
||||||
s.mu.Lock()
|
|
||||||
defer s.mu.Unlock()
|
|
||||||
for i := range s.tokens {
|
|
||||||
t := &s.tokens[i]
|
|
||||||
if subtle.ConstantTimeCompare(h, []byte(t.Hash)) != 1 {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
now := time.Now().UTC()
|
|
||||||
stale := now.Sub(t.LastUsed) >= lastUsedPeriod || t.LastIP != ip
|
|
||||||
t.LastUsed, t.LastIP = now, ip
|
|
||||||
if stale {
|
|
||||||
if err := s.save(); err != nil {
|
|
||||||
log.Printf("api tokens: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return t.Name, true
|
|
||||||
}
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
|
|
||||||
// bearerToken returns the token of an "Authorization: Bearer …" header.
|
|
||||||
func bearerToken(r *http.Request) (string, bool) {
|
|
||||||
scheme, token, ok := strings.Cut(r.Header.Get("Authorization"), " ")
|
|
||||||
if !ok || !strings.EqualFold(scheme, "Bearer") {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
return strings.TrimSpace(token), true
|
|
||||||
}
|
|
||||||
|
|
||||||
// tokenAuth serves the requests carrying an API token itself, read-only and
|
|
||||||
// limited to /api/v1/, before the session login (auth); api is the handler
|
|
||||||
// behind the login.
|
|
||||||
func tokenAuth(tokens *TokenStore, auth, api http.Handler) http.Handler {
|
|
||||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
token, ok := bearerToken(r)
|
|
||||||
if !ok {
|
|
||||||
auth.ServeHTTP(w, r)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
ip := clientIP(r)
|
|
||||||
if _, valid := tokens.Check(token, ip); !valid {
|
|
||||||
log.Printf("api: invalid or revoked token from %s", ip)
|
|
||||||
time.Sleep(loginFailDelay)
|
|
||||||
w.Header().Set("WWW-Authenticate", `Bearer realm="logstream"`)
|
|
||||||
writeErr(w, http.StatusUnauthorized, &codedError{code: "token", msg: "invalid or revoked API token"})
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if !strings.HasPrefix(r.URL.Path, "/api/v1/") || !isSafeMethod(r.Method) {
|
|
||||||
writeErr(w, http.StatusForbidden, &codedError{code: "token_scope", msg: "API tokens only give read access to /api/v1/"})
|
|
||||||
return
|
|
||||||
}
|
|
||||||
api.ServeHTTP(w, asViewer(r))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// GET /api/tokens: the API tokens (Settings > API), for administrators.
|
|
||||||
func (a *API) listTokens(w http.ResponseWriter, r *http.Request) {
|
|
||||||
if isViewer(r) {
|
|
||||||
writeErr(w, http.StatusForbidden, &codedError{code: "read_only", msg: "read-only account: changes are reserved to administrators"})
|
|
||||||
return
|
|
||||||
}
|
|
||||||
writeJSON(w, http.StatusOK, a.tokens.List())
|
|
||||||
}
|
|
||||||
|
|
||||||
// POST /api/tokens {"name": "n8n"}: returns the new token, with "token" in clear.
|
|
||||||
func (a *API) createToken(w http.ResponseWriter, r *http.Request) {
|
|
||||||
var body struct {
|
|
||||||
Name string `json:"name"`
|
|
||||||
}
|
|
||||||
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&body); err != nil {
|
|
||||||
writeErr(w, http.StatusBadRequest, err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
info, secret, err := a.tokens.Create(body.Name)
|
|
||||||
var ce *codedError
|
|
||||||
switch {
|
|
||||||
case errors.As(err, &ce):
|
|
||||||
writeErr(w, http.StatusBadRequest, err)
|
|
||||||
return
|
|
||||||
case err != nil:
|
|
||||||
writeErr(w, http.StatusInternalServerError, err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
log.Printf("api token %q created from %s", info.Name, clientIP(r))
|
|
||||||
writeJSON(w, http.StatusCreated, struct {
|
|
||||||
apiTokenInfo
|
|
||||||
Token string `json:"token"`
|
|
||||||
}{info, secret})
|
|
||||||
}
|
|
||||||
|
|
||||||
// DELETE /api/tokens/{id}: revokes a token at once.
|
|
||||||
func (a *API) deleteToken(w http.ResponseWriter, r *http.Request) {
|
|
||||||
err := a.tokens.Delete(r.PathValue("id"))
|
|
||||||
switch {
|
|
||||||
case errors.Is(err, errTokenNotFound):
|
|
||||||
writeErr(w, http.StatusNotFound, err)
|
|
||||||
case err != nil:
|
|
||||||
writeErr(w, http.StatusInternalServerError, err)
|
|
||||||
default:
|
|
||||||
log.Printf("api token %s revoked from %s", r.PathValue("id"), clientIP(r))
|
|
||||||
w.WriteHeader(http.StatusNoContent)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,176 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestTokenStore(t *testing.T) {
|
|
||||||
path := filepath.Join(t.TempDir(), "api_tokens.json")
|
|
||||||
s, err := LoadTokenStore(path)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if _, _, err := s.Create(" "); err == nil {
|
|
||||||
t.Fatal("a token without a name should be refused")
|
|
||||||
}
|
|
||||||
info, secret, err := s.Create("n8n")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if !strings.HasPrefix(secret, tokenPrefix) || len(secret) < 40 || !strings.HasPrefix(secret, info.Hint) {
|
|
||||||
t.Fatalf("token %q, hint %q", secret, info.Hint)
|
|
||||||
}
|
|
||||||
b, _ := os.ReadFile(path)
|
|
||||||
if strings.Contains(string(b), secret) || !strings.Contains(string(b), hashToken(secret)) {
|
|
||||||
t.Fatal("the file must hold the hash of the token, not the token")
|
|
||||||
}
|
|
||||||
if st, _ := os.Stat(path); st.Mode().Perm() != 0o600 {
|
|
||||||
t.Fatalf("file mode %v, want 0600", st.Mode().Perm())
|
|
||||||
}
|
|
||||||
if _, ok := s.Check(secret+"x", "10.0.0.1"); ok {
|
|
||||||
t.Fatal("a wrong token was accepted")
|
|
||||||
}
|
|
||||||
if name, ok := s.Check(secret, "10.0.0.1"); !ok || name != "n8n" {
|
|
||||||
t.Fatal("the token was refused")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Reloaded from disk (restart): still valid, last use kept.
|
|
||||||
s2, err := LoadTokenStore(path)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if l := s2.List(); len(l) != 1 || l[0].LastIP != "10.0.0.1" || l[0].LastUsed.IsZero() {
|
|
||||||
t.Fatalf("after reload: %+v", l)
|
|
||||||
}
|
|
||||||
if _, ok := s2.Check(secret, "10.0.0.2"); !ok {
|
|
||||||
t.Fatal("token refused after reload")
|
|
||||||
}
|
|
||||||
if err := s2.Delete(info.ID); err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if _, ok := s2.Check(secret, "10.0.0.2"); ok {
|
|
||||||
t.Fatal("a revoked token was accepted")
|
|
||||||
}
|
|
||||||
if err := s2.Delete(info.ID); err != errTokenNotFound {
|
|
||||||
t.Fatalf("second delete: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestTokenAuth(t *testing.T) {
|
|
||||||
loginFailDelay = 0
|
|
||||||
tokens, _ := LoadTokenStore(filepath.Join(t.TempDir(), "api_tokens.json"))
|
|
||||||
_, secret, _ := tokens.Create("n8n")
|
|
||||||
|
|
||||||
var reached string
|
|
||||||
mux := http.NewServeMux()
|
|
||||||
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
reached = r.URL.Path
|
|
||||||
if isViewer(r) {
|
|
||||||
reached += " viewer"
|
|
||||||
}
|
|
||||||
})
|
|
||||||
app := readOnly(mux)
|
|
||||||
login := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { writeAuthRequired(w) })
|
|
||||||
h := tokenAuth(tokens, login, app)
|
|
||||||
|
|
||||||
call := func(method, path, auth string) int {
|
|
||||||
reached = ""
|
|
||||||
req := httptest.NewRequest(method, path, nil)
|
|
||||||
if auth != "" {
|
|
||||||
req.Header.Set("Authorization", auth)
|
|
||||||
}
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
h.ServeHTTP(rec, req)
|
|
||||||
return rec.Code
|
|
||||||
}
|
|
||||||
if code := call("GET", "/api/v1/logs", "Bearer "+secret); code != http.StatusOK || reached != "/api/v1/logs viewer" {
|
|
||||||
t.Fatalf("valid token: %d, reached %q", code, reached)
|
|
||||||
}
|
|
||||||
if code := call("GET", "/api/v1/logs", "bearer "+secret); code != http.StatusOK {
|
|
||||||
t.Fatalf("lower-case scheme: %d", code)
|
|
||||||
}
|
|
||||||
for _, tc := range []struct {
|
|
||||||
method, path, auth string
|
|
||||||
want int
|
|
||||||
}{
|
|
||||||
{"GET", "/api/v1/logs", "Bearer lst_wrong", http.StatusUnauthorized},
|
|
||||||
{"GET", "/api/v1/logs", "Bearer ", http.StatusUnauthorized},
|
|
||||||
{"GET", "/api/tags", "Bearer " + secret, http.StatusForbidden},
|
|
||||||
{"GET", "/api/tokens", "Bearer " + secret, http.StatusForbidden},
|
|
||||||
{"POST", "/api/v1/logs", "Bearer " + secret, http.StatusForbidden},
|
|
||||||
{"GET", "/api/v1/logs", "", http.StatusUnauthorized}, // no token: the login decides
|
|
||||||
{"GET", "/api/v1/logs", "Basic dTpw", http.StatusUnauthorized}, // Basic goes to the login
|
|
||||||
} {
|
|
||||||
if code := call(tc.method, tc.path, tc.auth); code != tc.want || reached != "" {
|
|
||||||
t.Errorf("%s %s %q: %d (reached %q), want %d", tc.method, tc.path, tc.auth, code, reached, tc.want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// With local authentication, a token opens /api/v1/ without a session.
|
|
||||||
func TestTokenWithLocalAuth(t *testing.T) {
|
|
||||||
loginFailDelay = 0
|
|
||||||
dir := t.TempDir()
|
|
||||||
tokens, _ := LoadTokenStore(filepath.Join(dir, "api_tokens.json"))
|
|
||||||
_, secret, _ := tokens.Create("script")
|
|
||||||
app := readOnly(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _, _ = w.Write([]byte("ok")) }))
|
|
||||||
auth, err := newAuth(authConfig{mode: "local", user: "admin", pass: "pw", dataDir: dir, sessionTTL: time.Hour}, app)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
h := tokenAuth(tokens, auth, app)
|
|
||||||
req := httptest.NewRequest("GET", "/api/v1/logs", nil)
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
h.ServeHTTP(rec, req)
|
|
||||||
if rec.Code != http.StatusUnauthorized {
|
|
||||||
t.Fatalf("without token: %d", rec.Code)
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+secret)
|
|
||||||
rec = httptest.NewRecorder()
|
|
||||||
h.ServeHTTP(rec, req)
|
|
||||||
if rec.Code != http.StatusOK || rec.Body.String() != "ok" {
|
|
||||||
t.Fatalf("with token: %d %s", rec.Code, rec.Body)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestTokenHandlers(t *testing.T) {
|
|
||||||
tokens, _ := LoadTokenStore(filepath.Join(t.TempDir(), "api_tokens.json"))
|
|
||||||
a := &API{tokens: tokens}
|
|
||||||
mux := http.NewServeMux()
|
|
||||||
a.Routes(mux)
|
|
||||||
do := func(r *http.Request) *httptest.ResponseRecorder {
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
readOnly(mux).ServeHTTP(rec, r)
|
|
||||||
return rec
|
|
||||||
}
|
|
||||||
rec := do(httptest.NewRequest("POST", "/api/tokens", strings.NewReader(`{"name":"n8n"}`)))
|
|
||||||
var created struct {
|
|
||||||
ID, Token, Hint string
|
|
||||||
}
|
|
||||||
if err := json.Unmarshal(rec.Body.Bytes(), &created); rec.Code != http.StatusCreated || err != nil || created.Token == "" {
|
|
||||||
t.Fatalf("create: %d %s", rec.Code, rec.Body)
|
|
||||||
}
|
|
||||||
rec = do(httptest.NewRequest("GET", "/api/tokens", nil))
|
|
||||||
if rec.Code != http.StatusOK || strings.Contains(rec.Body.String(), created.Token) || strings.Contains(rec.Body.String(), "hash") {
|
|
||||||
t.Fatalf("list must not show the token or its hash: %s", rec.Body)
|
|
||||||
}
|
|
||||||
if rec := do(asViewer(httptest.NewRequest("GET", "/api/tokens", nil))); rec.Code != http.StatusForbidden {
|
|
||||||
t.Fatalf("viewer list: %d", rec.Code)
|
|
||||||
}
|
|
||||||
if rec := do(asViewer(httptest.NewRequest("DELETE", "/api/tokens/"+created.ID, nil))); rec.Code != http.StatusForbidden {
|
|
||||||
t.Fatalf("viewer delete: %d", rec.Code)
|
|
||||||
}
|
|
||||||
if rec := do(httptest.NewRequest("DELETE", "/api/tokens/"+created.ID, nil)); rec.Code != http.StatusNoContent {
|
|
||||||
t.Fatalf("delete: %d", rec.Code)
|
|
||||||
}
|
|
||||||
if rec := do(httptest.NewRequest("DELETE", "/api/tokens/"+created.ID, nil)); rec.Code != http.StatusNotFound {
|
|
||||||
t.Fatalf("delete again: %d", rec.Code)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+8
-13
@@ -6,10 +6,10 @@ services:
|
|||||||
depends_on:
|
depends_on:
|
||||||
- victorialogs
|
- victorialogs
|
||||||
- docker-proxy
|
- docker-proxy
|
||||||
# ports:
|
ports:
|
||||||
# - "${SYSLOG_PORT:-5514}:5514/udp"
|
- "${SYSLOG_PORT:-5514}:5514/udp"
|
||||||
# - "${SYSLOG_PORT:-5514}:5514/tcp"
|
- "${SYSLOG_PORT:-5514}:5514/tcp"
|
||||||
# - "${HTTP_PORT:-8989}:8080"
|
- "${HTTP_PORT:-8989}:8080"
|
||||||
environment:
|
environment:
|
||||||
VLOGS_URL: http://victorialogs:9428
|
VLOGS_URL: http://victorialogs:9428
|
||||||
SYSLOG_PUBLIC_PORT: ${SYSLOG_PORT:-514} # le port d'ecoute syslog par defaut (attention aux ports <1024)
|
SYSLOG_PUBLIC_PORT: ${SYSLOG_PORT:-514} # le port d'ecoute syslog par defaut (attention aux ports <1024)
|
||||||
@@ -60,12 +60,12 @@ services:
|
|||||||
- /run/log/journal:/host/run/log/journal:ro # journal systemd volatile
|
- /run/log/journal:/host/run/log/journal:ro # journal systemd volatile
|
||||||
# audit Linux (auditd), source a activer dans Reglages > Sources ; le fichier doit etre
|
# audit Linux (auditd), source a activer dans Reglages > Sources ; le fichier doit etre
|
||||||
# lisible par l'UID 10001 : log_group = adm dans /etc/audit/auditd.conf (voir le README)
|
# lisible par l'UID 10001 : log_group = adm dans /etc/audit/auditd.conf (voir le README)
|
||||||
- /var/log/audit:/var/log/audit:ro
|
# - /var/log/audit:/var/log/audit:ro
|
||||||
# noms des utilisateurs et de l'hote pour les evenements d'audit (optionnel)
|
# noms des utilisateurs et de l'hote pour les evenements d'audit (optionnel)
|
||||||
- /etc/passwd:/host/etc/passwd:ro
|
# - /etc/passwd:/host/etc/passwd:ro
|
||||||
- /etc/hostname:/host/etc/hostname:ro
|
# - /etc/hostname:/host/etc/hostname:ro
|
||||||
# fichiers de logs d'autres programmes (Traefik, nginx...), a declarer dans Reglages > Sources > Fichiers de logs
|
# fichiers de logs d'autres programmes (Traefik, nginx...), a declarer dans Reglages > Sources > Fichiers de logs
|
||||||
- /home/traefik/log:/home/logs/traefik:ro
|
# - /var/log/traefik:/logs/traefik:ro
|
||||||
# logo de la page de connexion, avec LOGIN_LOGO=/config/logo.png dans .env
|
# logo de la page de connexion, avec LOGIN_LOGO=/config/logo.png dans .env
|
||||||
# - ./logo.png:/config/logo.png:ro
|
# - ./logo.png:/config/logo.png:ro
|
||||||
# prereglages de tags personnalises, avec PRESETS_FILE=/config/presets.json dans .env
|
# prereglages de tags personnalises, avec PRESETS_FILE=/config/presets.json dans .env
|
||||||
@@ -102,12 +102,7 @@ services:
|
|||||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
labels:
|
labels:
|
||||||
logstream.exclude: "true" # pour exclure les logs propres de logstream
|
logstream.exclude: "true" # pour exclure les logs propres de logstream
|
||||||
networks:
|
|
||||||
- traefik-net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
traefik-net:
|
|
||||||
external: true
|
|
||||||
volumes:
|
volumes:
|
||||||
logstream-data:
|
logstream-data:
|
||||||
vlogs-data:
|
vlogs-data:
|
||||||
-281
@@ -1,281 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"crypto/sha256"
|
|
||||||
"encoding/base64"
|
|
||||||
"encoding/hex"
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"sort"
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// GET /api/v1/logs: incremental reading of the stored logs for automation
|
|
||||||
// tools (n8n, scripts). Each answer gives the rows that follow a cursor,
|
|
||||||
// oldest first, and the cursor to send next time; following the cursors
|
|
||||||
// returns every row once, without gaps or duplicates.
|
|
||||||
//
|
|
||||||
// The cursor is a position on _time, which is the reception time given by
|
|
||||||
// LogStream (see Entry.Record), so new rows always come after it. Rows only
|
|
||||||
// count once they are older than the lag (30 s by default): a message can
|
|
||||||
// wait a little in the queue or for its DNS name before VictoriaLogs makes it
|
|
||||||
// visible. Several rows can share the same _time to the nanosecond; inside
|
|
||||||
// such a group the rows are ordered by a hash of their content, and the
|
|
||||||
// cursor remembers how many of them were already returned.
|
|
||||||
|
|
||||||
const (
|
|
||||||
logsAPIDefaultLimit = 1000
|
|
||||||
logsAPIMaxLimit = 10000
|
|
||||||
logsAPIDefaultLag = 30 * time.Second
|
|
||||||
logsAPIMaxLag = time.Hour
|
|
||||||
logsAPIDefaultSince = time.Hour
|
|
||||||
logsAPIMaxGroup = 100000 // rows with the very same _time
|
|
||||||
)
|
|
||||||
|
|
||||||
// logsCursor: every row with _time < T has been returned, plus the first K
|
|
||||||
// rows (in hash order) of those with _time == T. Q is a hash of the query:
|
|
||||||
// K only means something for the same query.
|
|
||||||
type logsCursor struct {
|
|
||||||
T time.Time `json:"t"`
|
|
||||||
K int `json:"k,omitempty"`
|
|
||||||
Q string `json:"q,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c logsCursor) encode() string {
|
|
||||||
b, _ := json.Marshal(c)
|
|
||||||
return base64.RawURLEncoding.EncodeToString(b)
|
|
||||||
}
|
|
||||||
|
|
||||||
func decodeCursor(s string) (logsCursor, bool) {
|
|
||||||
var c logsCursor
|
|
||||||
b, err := base64.RawURLEncoding.DecodeString(s)
|
|
||||||
if err != nil || json.Unmarshal(b, &c) != nil || c.T.IsZero() || c.K < 0 {
|
|
||||||
return logsCursor{}, false
|
|
||||||
}
|
|
||||||
return c, true
|
|
||||||
}
|
|
||||||
|
|
||||||
// parseSince reads the since parameter: a cursor from a previous answer, an
|
|
||||||
// RFC 3339 time, or a duration back from now (90s, 15m, 6h, 7d). Empty: the
|
|
||||||
// last hour.
|
|
||||||
func parseSince(s string, now time.Time) (logsCursor, error) {
|
|
||||||
s = strings.TrimSpace(s)
|
|
||||||
if s == "" {
|
|
||||||
return logsCursor{T: now.Add(-logsAPIDefaultSince)}, nil
|
|
||||||
}
|
|
||||||
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
|
|
||||||
return logsCursor{T: t}, nil
|
|
||||||
}
|
|
||||||
if d, ok := parseAge(s); ok {
|
|
||||||
return logsCursor{T: now.Add(-d)}, nil
|
|
||||||
}
|
|
||||||
if c, ok := decodeCursor(s); ok {
|
|
||||||
return c, nil
|
|
||||||
}
|
|
||||||
return logsCursor{}, &codedError{code: "bad_since", msg: "since: expected a next_cursor, an RFC 3339 time or a duration such as 15m, 6h or 7d"}
|
|
||||||
}
|
|
||||||
|
|
||||||
// parseAge reads a Go duration, plus "Nd" for days.
|
|
||||||
func parseAge(s string) (time.Duration, bool) {
|
|
||||||
if n, ok := strings.CutSuffix(s, "d"); ok {
|
|
||||||
days, err := strconv.Atoi(n)
|
|
||||||
if err != nil || days < 0 {
|
|
||||||
return 0, false
|
|
||||||
}
|
|
||||||
return time.Duration(days) * 24 * time.Hour, true
|
|
||||||
}
|
|
||||||
d, err := time.ParseDuration(s)
|
|
||||||
return d, err == nil && d >= 0
|
|
||||||
}
|
|
||||||
|
|
||||||
type logsPage struct {
|
|
||||||
Rows []map[string]any `json:"rows"`
|
|
||||||
NextCursor string `json:"next_cursor"`
|
|
||||||
HasMore bool `json:"has_more"` // call again right away: more rows are ready
|
|
||||||
Waiting bool `json:"waiting,omitempty"` // the disk buffer is still being sent to VictoriaLogs
|
|
||||||
Until string `json:"until"` // rows are returned up to this time (now - lag)
|
|
||||||
}
|
|
||||||
|
|
||||||
// GET /api/v1/logs?query=<LogsQL filter>&since=<cursor|time|duration>&limit=<n>&lag=<duration>
|
|
||||||
func (a *API) logsV1(w http.ResponseWriter, r *http.Request) {
|
|
||||||
q := r.URL.Query()
|
|
||||||
filter := strings.TrimSpace(q.Get("query"))
|
|
||||||
if f, pipes := splitPipes(filter); strings.TrimSpace(pipes) != "" {
|
|
||||||
writeErr(w, http.StatusBadRequest, &codedError{code: "api_pipes", msg: "query: LogsQL pipes (| ...) are not supported, keep only the filter part"})
|
|
||||||
return
|
|
||||||
} else {
|
|
||||||
filter = strings.TrimSpace(f)
|
|
||||||
}
|
|
||||||
if filter == "" {
|
|
||||||
filter = "*"
|
|
||||||
}
|
|
||||||
limit := logsAPIDefaultLimit
|
|
||||||
if v := q.Get("limit"); v != "" {
|
|
||||||
n, err := strconv.Atoi(v)
|
|
||||||
if err != nil || n <= 0 || n > logsAPIMaxLimit {
|
|
||||||
writeErr(w, http.StatusBadRequest, &codedError{code: "bad_limit", msg: fmt.Sprintf("limit: expected 1 to %d", logsAPIMaxLimit)})
|
|
||||||
return
|
|
||||||
}
|
|
||||||
limit = n
|
|
||||||
}
|
|
||||||
lag := logsAPIDefaultLag
|
|
||||||
if v := q.Get("lag"); v != "" {
|
|
||||||
d, ok := parseAge(v)
|
|
||||||
if !ok || d > logsAPIMaxLag {
|
|
||||||
writeErr(w, http.StatusBadRequest, &codedError{code: "bad_lag", msg: "lag: expected a duration from 0s to 1h"})
|
|
||||||
return
|
|
||||||
}
|
|
||||||
lag = d
|
|
||||||
}
|
|
||||||
now := time.Now()
|
|
||||||
cur, err := parseSince(q.Get("since"), now)
|
|
||||||
if err != nil {
|
|
||||||
writeErr(w, http.StatusBadRequest, err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
qh := queryHash(filter)
|
|
||||||
if cur.Q != qh {
|
|
||||||
cur.K = 0 // the query changed: the rank inside the group no longer applies
|
|
||||||
}
|
|
||||||
cur.Q = qh
|
|
||||||
|
|
||||||
// Logs waiting in the disk buffer were received earlier: returning newer
|
|
||||||
// rows now would move the cursor past them.
|
|
||||||
if a.store.PendingSpool() > 0 {
|
|
||||||
writeJSON(w, http.StatusOK, logsPage{Rows: []map[string]any{}, NextCursor: cur.encode(), Waiting: true, Until: logsqlTime(cur.T)})
|
|
||||||
return
|
|
||||||
}
|
|
||||||
until := now.Add(-lag)
|
|
||||||
page, err := readLogsPage(r.Context(), a.store.Query, filter, cur, until, limit)
|
|
||||||
if err != nil {
|
|
||||||
writeErr(w, queryStatus(err), err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
a.annotateHosts(page.Rows)
|
|
||||||
writeJSON(w, http.StatusOK, page)
|
|
||||||
}
|
|
||||||
|
|
||||||
func queryHash(filter string) string {
|
|
||||||
sum := sha256.Sum256([]byte(filter))
|
|
||||||
return hex.EncodeToString(sum[:4])
|
|
||||||
}
|
|
||||||
|
|
||||||
type queryFunc func(ctx context.Context, query string) ([]map[string]any, error)
|
|
||||||
|
|
||||||
// readLogsPage returns at most limit rows after cur, up to until included.
|
|
||||||
func readLogsPage(ctx context.Context, run queryFunc, filter string, cur logsCursor, until time.Time, limit int) (logsPage, error) {
|
|
||||||
page := logsPage{Rows: []map[string]any{}, Until: logsqlTime(until)}
|
|
||||||
next := cur
|
|
||||||
start := cur.T
|
|
||||||
if cur.K > 0 {
|
|
||||||
// Rest of the group the previous page stopped in.
|
|
||||||
group, err := readGroup(ctx, run, filter, cur.T)
|
|
||||||
if err != nil {
|
|
||||||
return page, err
|
|
||||||
}
|
|
||||||
rest := group[min(cur.K, len(group)):]
|
|
||||||
if len(rest) >= limit {
|
|
||||||
page.Rows = rest[:limit]
|
|
||||||
next.K = cur.K + limit
|
|
||||||
page.HasMore = true
|
|
||||||
page.NextCursor = next.encode()
|
|
||||||
return page, nil
|
|
||||||
}
|
|
||||||
page.Rows = append(page.Rows, rest...)
|
|
||||||
start = cur.T.Add(time.Nanosecond)
|
|
||||||
}
|
|
||||||
|
|
||||||
left := limit - len(page.Rows)
|
|
||||||
var rows []map[string]any
|
|
||||||
if !start.After(until) {
|
|
||||||
q := fmt.Sprintf("_time:[%s, %s] (%s) | sort by (_time) | limit %d", logsqlTime(start), logsqlTime(until), filter, left+1)
|
|
||||||
var err error
|
|
||||||
if rows, err = run(ctx, q); err != nil {
|
|
||||||
return page, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if len(rows) <= left {
|
|
||||||
// Everything up to until has been read.
|
|
||||||
page.Rows = append(page.Rows, rows...)
|
|
||||||
next = logsCursor{T: until.Add(time.Nanosecond), Q: cur.Q}
|
|
||||||
if next.T.Before(start) {
|
|
||||||
next.T = start
|
|
||||||
}
|
|
||||||
page.NextCursor = next.encode()
|
|
||||||
return page, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// One row too many: the group of the last time may be cut. Whole groups
|
|
||||||
// before it are returned; that last group starts the next page.
|
|
||||||
page.HasMore = true
|
|
||||||
last, err := rowTime(rows[left])
|
|
||||||
if err != nil {
|
|
||||||
return page, err
|
|
||||||
}
|
|
||||||
n := 0
|
|
||||||
for n < left {
|
|
||||||
t, err := rowTime(rows[n])
|
|
||||||
if err != nil {
|
|
||||||
return page, err
|
|
||||||
}
|
|
||||||
if !t.Before(last) {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
n++
|
|
||||||
}
|
|
||||||
if n > 0 || len(page.Rows) > 0 {
|
|
||||||
page.Rows = append(page.Rows, rows[:n]...)
|
|
||||||
page.NextCursor = logsCursor{T: last, Q: cur.Q}.encode()
|
|
||||||
return page, nil
|
|
||||||
}
|
|
||||||
// The whole page is a single group: read it all to cut it in a stable order.
|
|
||||||
group, err := readGroup(ctx, run, filter, last)
|
|
||||||
if err != nil {
|
|
||||||
return page, err
|
|
||||||
}
|
|
||||||
page.Rows = group[:min(left, len(group))]
|
|
||||||
page.NextCursor = logsCursor{T: last, K: len(page.Rows), Q: cur.Q}.encode()
|
|
||||||
return page, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// readGroup returns the rows whose _time is exactly t, in a stable order.
|
|
||||||
func readGroup(ctx context.Context, run queryFunc, filter string, t time.Time) ([]map[string]any, error) {
|
|
||||||
ts := logsqlTime(t)
|
|
||||||
rows, err := run(ctx, fmt.Sprintf("_time:[%s, %s] (%s) | limit %d", ts, ts, filter, logsAPIMaxGroup+1))
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if len(rows) > logsAPIMaxGroup {
|
|
||||||
return nil, fmt.Errorf("more than %d logs share the time %s", logsAPIMaxGroup, ts)
|
|
||||||
}
|
|
||||||
keys := make([]string, len(rows))
|
|
||||||
for i, row := range rows {
|
|
||||||
b, _ := json.Marshal(row) // map keys are sorted: same row, same bytes
|
|
||||||
sum := sha256.Sum256(b)
|
|
||||||
keys[i] = string(sum[:])
|
|
||||||
}
|
|
||||||
idx := make([]int, len(rows))
|
|
||||||
for i := range idx {
|
|
||||||
idx[i] = i
|
|
||||||
}
|
|
||||||
sort.SliceStable(idx, func(i, j int) bool { return keys[idx[i]] < keys[idx[j]] })
|
|
||||||
out := make([]map[string]any, len(rows))
|
|
||||||
for i, k := range idx {
|
|
||||||
out[i] = rows[k]
|
|
||||||
}
|
|
||||||
return out, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func rowTime(row map[string]any) (time.Time, error) {
|
|
||||||
s, _ := row["_time"].(string)
|
|
||||||
t, err := time.Parse(time.RFC3339Nano, s)
|
|
||||||
if err != nil {
|
|
||||||
return time.Time{}, errors.New("VictoriaLogs returned a row without a valid _time")
|
|
||||||
}
|
|
||||||
return t, nil
|
|
||||||
}
|
|
||||||
-276
@@ -1,276 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"math/rand"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"regexp"
|
|
||||||
"sort"
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
"sync"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// cursorVL plays VictoriaLogs for the two queries of /api/v1/logs: a time
|
|
||||||
// range sorted by _time, and an exact time. Like VictoriaLogs, it gives no
|
|
||||||
// order among rows of the same _time: they are shuffled on every query.
|
|
||||||
type cursorVL struct {
|
|
||||||
mu sync.Mutex
|
|
||||||
rows []map[string]any
|
|
||||||
rnd *rand.Rand
|
|
||||||
queries []string
|
|
||||||
}
|
|
||||||
|
|
||||||
var cursorQuery = regexp.MustCompile(`^_time:\[([^,]+), ([^\]]+)] \((.*?)\)( \| sort by \(_time\))? \| limit (\d+)$`)
|
|
||||||
|
|
||||||
func (v *cursorVL) add(t time.Time, msg string) {
|
|
||||||
v.mu.Lock()
|
|
||||||
defer v.mu.Unlock()
|
|
||||||
v.rows = append(v.rows, map[string]any{"_time": t.UTC().Format(time.RFC3339Nano), "_msg": msg, "host": "h1"})
|
|
||||||
}
|
|
||||||
|
|
||||||
func (v *cursorVL) RoundTrip(r *http.Request) (*http.Response, error) {
|
|
||||||
_ = r.ParseForm()
|
|
||||||
q := r.PostForm.Get("query")
|
|
||||||
m := cursorQuery.FindStringSubmatch(q)
|
|
||||||
if m == nil {
|
|
||||||
return &http.Response{StatusCode: http.StatusBadRequest, Body: io.NopCloser(strings.NewReader("unexpected query " + q)), Header: http.Header{}}, nil
|
|
||||||
}
|
|
||||||
from, _ := time.Parse(time.RFC3339Nano, m[1])
|
|
||||||
to, _ := time.Parse(time.RFC3339Nano, m[2])
|
|
||||||
limit, _ := strconv.Atoi(m[5])
|
|
||||||
|
|
||||||
v.mu.Lock()
|
|
||||||
defer v.mu.Unlock()
|
|
||||||
v.queries = append(v.queries, q)
|
|
||||||
var out []map[string]any
|
|
||||||
for _, row := range v.rows {
|
|
||||||
t, _ := time.Parse(time.RFC3339Nano, row["_time"].(string))
|
|
||||||
if !t.Before(from) && !t.After(to) {
|
|
||||||
out = append(out, row)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
v.rnd.Shuffle(len(out), func(i, j int) { out[i], out[j] = out[j], out[i] })
|
|
||||||
if m[4] != "" {
|
|
||||||
sort.SliceStable(out, func(i, j int) bool {
|
|
||||||
a, _ := time.Parse(time.RFC3339Nano, out[i]["_time"].(string))
|
|
||||||
b, _ := time.Parse(time.RFC3339Nano, out[j]["_time"].(string))
|
|
||||||
return a.Before(b)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
if len(out) > limit {
|
|
||||||
out = out[:limit]
|
|
||||||
}
|
|
||||||
var b strings.Builder
|
|
||||||
for _, row := range out {
|
|
||||||
line, _ := json.Marshal(row)
|
|
||||||
b.Write(line)
|
|
||||||
b.WriteByte('\n')
|
|
||||||
}
|
|
||||||
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(b.String())), Header: http.Header{}}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func newCursorVL(seed int64) (*cursorVL, *Store) {
|
|
||||||
vl := &cursorVL{rnd: rand.New(rand.NewSource(seed))}
|
|
||||||
store := NewStore("http://vl", 10, 10, time.Second, nil)
|
|
||||||
store.client.Transport = vl
|
|
||||||
store.streamClient.Transport = vl
|
|
||||||
return vl, store
|
|
||||||
}
|
|
||||||
|
|
||||||
// fill adds rows by groups of the same _time, some groups larger than a
|
|
||||||
// page, some with identical rows, and returns the multiset of the rows.
|
|
||||||
func fill(vl *cursorVL, start time.Time, groups []int, tag string) map[string]int {
|
|
||||||
want := map[string]int{}
|
|
||||||
t := start
|
|
||||||
for g, size := range groups {
|
|
||||||
for i := 0; i < size; i++ {
|
|
||||||
msg := fmt.Sprintf("%s g%d r%d", tag, g, i%4) // r0..r3 repeat: identical rows
|
|
||||||
vl.add(t, msg)
|
|
||||||
want[rowKey(t, msg)]++
|
|
||||||
}
|
|
||||||
t = t.Add(time.Duration(1+g%3) * time.Microsecond)
|
|
||||||
}
|
|
||||||
return want
|
|
||||||
}
|
|
||||||
|
|
||||||
func rowKey(t time.Time, msg string) string { return t.UTC().Format(time.RFC3339Nano) + " " + msg }
|
|
||||||
|
|
||||||
func TestLogsCursorNoLossNoDuplicate(t *testing.T) {
|
|
||||||
base := time.Date(2026, 11, 1, 10, 0, 0, 0, time.UTC)
|
|
||||||
groups := []int{1, 3, 1, 1, 7, 2, 1, 12, 1, 4, 4, 1, 25, 1, 2}
|
|
||||||
for _, limit := range []int{1, 2, 3, 5, 8, 1000} {
|
|
||||||
for seed := int64(1); seed <= 5; seed++ {
|
|
||||||
vl, store := newCursorVL(seed)
|
|
||||||
want := fill(vl, base, groups, "a")
|
|
||||||
got := map[string]int{}
|
|
||||||
cur := logsCursor{T: base.Add(-time.Minute)}
|
|
||||||
until := base.Add(time.Second)
|
|
||||||
var last time.Time
|
|
||||||
for calls := 0; ; calls++ {
|
|
||||||
if calls > 500 {
|
|
||||||
t.Fatalf("limit %d: no end", limit)
|
|
||||||
}
|
|
||||||
page, err := readLogsPage(context.Background(), store.Query, "*", cur, until, limit)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if len(page.Rows) > limit {
|
|
||||||
t.Fatalf("limit %d: page of %d rows", limit, len(page.Rows))
|
|
||||||
}
|
|
||||||
for _, row := range page.Rows {
|
|
||||||
rt, _ := rowTime(row)
|
|
||||||
if rt.Before(last) {
|
|
||||||
t.Fatalf("limit %d: rows out of order (%s after %s)", limit, rt, last)
|
|
||||||
}
|
|
||||||
last = rt
|
|
||||||
got[rowKey(rt, row["_msg"].(string))]++
|
|
||||||
}
|
|
||||||
var ok bool
|
|
||||||
if cur, ok = decodeCursor(page.NextCursor); !ok {
|
|
||||||
t.Fatalf("bad cursor %q", page.NextCursor)
|
|
||||||
}
|
|
||||||
if !page.HasMore {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
if len(page.Rows) == 0 {
|
|
||||||
t.Fatalf("limit %d: has_more with an empty page", limit)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if fmt.Sprint(got) != fmt.Sprint(want) {
|
|
||||||
t.Fatalf("limit %d seed %d:\n got %v\nwant %v", limit, seed, got, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// New logs arrive between the calls, as with a scheduled n8n workflow.
|
|
||||||
func TestLogsCursorFollowsNewLogs(t *testing.T) {
|
|
||||||
vl, store := newCursorVL(7)
|
|
||||||
base := time.Date(2026, 11, 1, 10, 0, 0, 0, time.UTC)
|
|
||||||
want := map[string]int{}
|
|
||||||
got := map[string]int{}
|
|
||||||
cur := logsCursor{T: base}
|
|
||||||
for round := 0; round < 6; round++ {
|
|
||||||
start := base.Add(time.Duration(round) * time.Second)
|
|
||||||
for k, n := range fill(vl, start, []int{2, 5, 1, 3}, fmt.Sprintf("round%d", round)) {
|
|
||||||
want[k] += n
|
|
||||||
}
|
|
||||||
until := start.Add(500 * time.Millisecond) // after this round's rows
|
|
||||||
for {
|
|
||||||
page, err := readLogsPage(context.Background(), store.Query, "*", cur, until, 4)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
for _, row := range page.Rows {
|
|
||||||
rt, _ := rowTime(row)
|
|
||||||
got[rowKey(rt, row["_msg"].(string))]++
|
|
||||||
}
|
|
||||||
cur, _ = decodeCursor(page.NextCursor)
|
|
||||||
if !page.HasMore {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Polling again with nothing new returns nothing.
|
|
||||||
page, err := readLogsPage(context.Background(), store.Query, "*", cur, until, 4)
|
|
||||||
if err != nil || len(page.Rows) != 0 || page.HasMore {
|
|
||||||
t.Fatalf("round %d: second poll returned %d rows (has_more %v, err %v)", round, len(page.Rows), page.HasMore, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if fmt.Sprint(got) != fmt.Sprint(want) {
|
|
||||||
t.Fatalf("got %v\nwant %v", got, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestParseSince(t *testing.T) {
|
|
||||||
now := time.Date(2026, 11, 1, 12, 0, 0, 0, time.UTC)
|
|
||||||
c := logsCursor{T: now.Add(-time.Minute), K: 3, Q: "abcd"}
|
|
||||||
for _, tc := range []struct {
|
|
||||||
in string
|
|
||||||
want logsCursor
|
|
||||||
}{
|
|
||||||
{"", logsCursor{T: now.Add(-time.Hour)}},
|
|
||||||
{"15m", logsCursor{T: now.Add(-15 * time.Minute)}},
|
|
||||||
{"7d", logsCursor{T: now.Add(-7 * 24 * time.Hour)}},
|
|
||||||
{"2026-11-01T08:30:00.5Z", logsCursor{T: time.Date(2026, 11, 1, 8, 30, 0, 5e8, time.UTC)}},
|
|
||||||
{c.encode(), c},
|
|
||||||
} {
|
|
||||||
got, err := parseSince(tc.in, now)
|
|
||||||
if err != nil || !got.T.Equal(tc.want.T) || got.K != tc.want.K || got.Q != tc.want.Q {
|
|
||||||
t.Errorf("parseSince(%q) = %+v, %v; want %+v", tc.in, got, err, tc.want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for _, bad := range []string{"yesterday", "-5m", "e30"} {
|
|
||||||
if _, err := parseSince(bad, now); err == nil {
|
|
||||||
t.Errorf("parseSince(%q): expected an error", bad)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestLogsV1Handler(t *testing.T) {
|
|
||||||
vl, store := newCursorVL(3)
|
|
||||||
a := &API{store: store, rdns: NewReverseDNS(false, "")}
|
|
||||||
old := time.Now().Add(-10 * time.Minute)
|
|
||||||
fill(vl, old, []int{2, 1}, "x")
|
|
||||||
vl.add(time.Now().Add(time.Hour), "in the future: after until")
|
|
||||||
|
|
||||||
get := func(url string) (int, logsPage, map[string]string) {
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
a.logsV1(rec, httptest.NewRequest("GET", url, nil))
|
|
||||||
var page logsPage
|
|
||||||
var e map[string]string
|
|
||||||
_ = json.Unmarshal(rec.Body.Bytes(), &page)
|
|
||||||
_ = json.Unmarshal(rec.Body.Bytes(), &e)
|
|
||||||
return rec.Code, page, e
|
|
||||||
}
|
|
||||||
code, page, _ := get("/api/v1/logs?query=host:h1&since=1h")
|
|
||||||
if code != http.StatusOK || len(page.Rows) != 3 || page.HasMore || page.NextCursor == "" {
|
|
||||||
t.Fatalf("status %d, %d rows, has_more %v", code, len(page.Rows), page.HasMore)
|
|
||||||
}
|
|
||||||
if q := vl.queries[len(vl.queries)-1]; !strings.Contains(q, "(host:h1)") {
|
|
||||||
t.Fatalf("query filter not applied: %s", q)
|
|
||||||
}
|
|
||||||
code, page, _ = get("/api/v1/logs?query=host:h1&since=" + page.NextCursor)
|
|
||||||
if code != http.StatusOK || len(page.Rows) != 0 {
|
|
||||||
t.Fatalf("second call: status %d, %d rows", code, len(page.Rows))
|
|
||||||
}
|
|
||||||
for url, wantCode := range map[string]string{
|
|
||||||
"/api/v1/logs?query=" + "*%20|%20stats%20count()": "api_pipes",
|
|
||||||
"/api/v1/logs?limit=0": "bad_limit",
|
|
||||||
"/api/v1/logs?limit=20000": "bad_limit",
|
|
||||||
"/api/v1/logs?lag=2h": "bad_lag",
|
|
||||||
"/api/v1/logs?since=nope": "bad_since",
|
|
||||||
} {
|
|
||||||
if code, _, e := get(url); code != http.StatusBadRequest || e["code"] != wantCode {
|
|
||||||
t.Errorf("%s: status %d, %v; want 400 %s", url, code, e, wantCode)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestLogsV1WaitsForSpool(t *testing.T) {
|
|
||||||
spool, err := OpenSpool(t.TempDir(), 1<<20)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if err := spool.Write([]byte(`{"_msg":"x"}`+"\n"), 1); err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
vl, _ := newCursorVL(1)
|
|
||||||
store := NewStore("http://vl", 10, 10, time.Second, spool)
|
|
||||||
store.streamClient.Transport = vl
|
|
||||||
a := &API{store: store}
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
a.logsV1(rec, httptest.NewRequest("GET", "/api/v1/logs?since=2026-11-01T10:00:00Z", nil))
|
|
||||||
var page logsPage
|
|
||||||
_ = json.Unmarshal(rec.Body.Bytes(), &page)
|
|
||||||
c, _ := decodeCursor(page.NextCursor)
|
|
||||||
if rec.Code != http.StatusOK || !page.Waiting || len(page.Rows) != 0 || !c.T.Equal(time.Date(2026, 11, 1, 10, 0, 0, 0, time.UTC)) || len(vl.queries) != 0 {
|
|
||||||
t.Fatalf("status %d, waiting %v, %d rows, cursor %v, %d queries", rec.Code, page.Waiting, len(page.Rows), c.T, len(vl.queries))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -142,10 +142,6 @@ func main() {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
log.Fatalf("tags: %v", err)
|
log.Fatalf("tags: %v", err)
|
||||||
}
|
}
|
||||||
tokens, err := LoadTokenStore(filepath.Join(cfg.dataDir, "api_tokens.json"))
|
|
||||||
if err != nil {
|
|
||||||
log.Fatalf("api tokens: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var spool *Spool
|
var spool *Spool
|
||||||
if cfg.spoolMax > 0 {
|
if cfg.spoolMax > 0 {
|
||||||
@@ -204,7 +200,7 @@ func main() {
|
|||||||
}
|
}
|
||||||
mux := http.NewServeMux()
|
mux := http.NewServeMux()
|
||||||
presets := getenv("PRESETS_FILE", filepath.Join(cfg.dataDir, "presets.json"))
|
presets := getenv("PRESETS_FILE", filepath.Join(cfg.dataDir, "presets.json"))
|
||||||
api := &API{store: store, hub: hub, tags: tags, presets: presets, rdns: rdns, allowPurge: cfg.allowPurge, exportMax: cfg.exportMax, syslog: syslogSrv, tokens: tokens}
|
api := &API{store: store, hub: hub, tags: tags, presets: presets, rdns: rdns, allowPurge: cfg.allowPurge, exportMax: cfg.exportMax, syslog: syslogSrv}
|
||||||
if cfg.dockerLogs {
|
if cfg.dockerLogs {
|
||||||
dm, err := NewDockerManager(cfg.dockerHost, cfg.dataDir, cfg.backfill, sink)
|
dm, err := NewDockerManager(cfg.dockerHost, cfg.dataDir, cfg.backfill, sink)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -230,10 +226,9 @@ func main() {
|
|||||||
go api.files.Run(ctx)
|
go api.files.Run(ctx)
|
||||||
api.selfLog = selfLog
|
api.selfLog = selfLog
|
||||||
api.Routes(mux)
|
api.Routes(mux)
|
||||||
mux.Handle("GET /", staticFiles(static))
|
mux.Handle("GET /", http.FileServer(http.FS(static)))
|
||||||
|
|
||||||
app := readOnly(mux)
|
handler, err := newAuth(cfg.auth, readOnly(mux))
|
||||||
handler, err := newAuth(cfg.auth, app)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Fatalf("auth: %v", err)
|
log.Fatalf("auth: %v", err)
|
||||||
}
|
}
|
||||||
@@ -246,8 +241,7 @@ func main() {
|
|||||||
if !authOn {
|
if !authOn {
|
||||||
log.Printf("warning: no authentication (AUTH_USER is empty): anyone who can reach %s can read the logs and change the settings", cfg.httpAddr)
|
log.Printf("warning: no authentication (AUTH_USER is empty): anyone who can reach %s can read the logs and change the settings", cfg.httpAddr)
|
||||||
}
|
}
|
||||||
// API tokens (Settings > API) are checked before the login, in every auth mode.
|
handler = secure(handler, contentSecurityPolicy(static), authOn)
|
||||||
handler = secure(tokenAuth(tokens, handler, app), contentSecurityPolicy(static), authOn)
|
|
||||||
srv := &http.Server{
|
srv := &http.Server{
|
||||||
Addr: cfg.httpAddr,
|
Addr: cfg.httpAddr,
|
||||||
Handler: handler,
|
Handler: handler,
|
||||||
|
|||||||
@@ -1,44 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"crypto/sha256"
|
|
||||||
"encoding/hex"
|
|
||||||
"io/fs"
|
|
||||||
"net/http"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
// staticFiles serves the embedded UI with an ETag per file (a hash of its
|
|
||||||
// content, computed at startup). Pages, scripts and styles are revalidated on
|
|
||||||
// every load ("no-cache"): an unchanged file costs a 304 without body, and the
|
|
||||||
// files of a new version are always taken after an update. Fonts are kept for
|
|
||||||
// 30 days.
|
|
||||||
func staticFiles(static fs.FS) http.Handler {
|
|
||||||
etags := map[string]string{}
|
|
||||||
_ = fs.WalkDir(static, ".", func(path string, d fs.DirEntry, err error) error {
|
|
||||||
if err != nil || d.IsDir() {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
b, err := fs.ReadFile(static, path)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
sum := sha256.Sum256(b)
|
|
||||||
etags["/"+path] = `"` + hex.EncodeToString(sum[:8]) + `"`
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
etags["/"] = etags["/index.html"]
|
|
||||||
files := http.FileServer(http.FS(static))
|
|
||||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
if tag, ok := etags[r.URL.Path]; ok {
|
|
||||||
// http.FileServer answers If-None-Match with 304 from this header.
|
|
||||||
w.Header().Set("ETag", tag)
|
|
||||||
if strings.HasPrefix(r.URL.Path, "/fonts/") {
|
|
||||||
w.Header().Set("Cache-Control", "public, max-age=2592000")
|
|
||||||
} else {
|
|
||||||
w.Header().Set("Cache-Control", "no-cache")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
files.ServeHTTP(w, r)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@@ -1,42 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"testing"
|
|
||||||
"testing/fstest"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestStaticFilesCache(t *testing.T) {
|
|
||||||
h := staticFiles(fstest.MapFS{
|
|
||||||
"index.html": {Data: []byte("<p>v1</p>")},
|
|
||||||
"app.js": {Data: []byte("v1")},
|
|
||||||
"fonts/mono.woff2": {Data: []byte("font")},
|
|
||||||
})
|
|
||||||
get := func(path, etag string) *httptest.ResponseRecorder {
|
|
||||||
req := httptest.NewRequest("GET", path, nil)
|
|
||||||
if etag != "" {
|
|
||||||
req.Header.Set("If-None-Match", etag)
|
|
||||||
}
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
h.ServeHTTP(rec, req)
|
|
||||||
return rec
|
|
||||||
}
|
|
||||||
rec := get("/app.js", "")
|
|
||||||
tag := rec.Header().Get("ETag")
|
|
||||||
if rec.Code != http.StatusOK || tag == "" || rec.Header().Get("Cache-Control") != "no-cache" {
|
|
||||||
t.Fatalf("app.js: %d, ETag %q, Cache-Control %q", rec.Code, tag, rec.Header().Get("Cache-Control"))
|
|
||||||
}
|
|
||||||
if rec := get("/app.js", tag); rec.Code != http.StatusNotModified || rec.Body.Len() != 0 {
|
|
||||||
t.Fatalf("revalidation: %d with %d bytes, want 304 without body", rec.Code, rec.Body.Len())
|
|
||||||
}
|
|
||||||
if rec := get("/app.js", `"old"`); rec.Code != http.StatusOK || rec.Body.String() != "v1" {
|
|
||||||
t.Fatalf("stale ETag: %d %q", rec.Code, rec.Body)
|
|
||||||
}
|
|
||||||
if rec := get("/", ""); rec.Code != http.StatusOK || rec.Header().Get("ETag") == "" || rec.Header().Get("Cache-Control") != "no-cache" {
|
|
||||||
t.Fatalf("index: %d %v", rec.Code, rec.Header())
|
|
||||||
}
|
|
||||||
if rec := get("/fonts/mono.woff2", ""); rec.Header().Get("Cache-Control") != "public, max-age=2592000" {
|
|
||||||
t.Fatalf("font Cache-Control %q", rec.Header().Get("Cache-Control"))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -423,15 +423,6 @@ func queryStatus(err error) int {
|
|||||||
return http.StatusBadGateway
|
return http.StatusBadGateway
|
||||||
}
|
}
|
||||||
|
|
||||||
// PendingSpool is the number of logs waiting in the disk spool.
|
|
||||||
func (s *Store) PendingSpool() int64 {
|
|
||||||
if s.spool == nil {
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
lines, _ := s.spool.Pending()
|
|
||||||
return lines
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *Store) Stats() map[string]any {
|
func (s *Store) Stats() map[string]any {
|
||||||
st := map[string]any{
|
st := map[string]any{
|
||||||
"received": s.received.Load(),
|
"received": s.received.Load(),
|
||||||
|
|||||||
+1
-170
@@ -174,27 +174,6 @@ const I18N = {
|
|||||||
exportNote: (n) => `Up to ${n} logs matching the current filters, newest first. Dates use the time zone chosen in Settings.`,
|
exportNote: (n) => `Up to ${n} logs matching the current filters, newest first. Dates use the time zone chosen in Settings.`,
|
||||||
exporting: 'Exporting…', exportDone: (f) => `Downloaded: ${f}`, exportError: 'Export failed: ',
|
exporting: 'Exporting…', exportDone: (f) => `Downloaded: ${f}`, exportError: 'Export failed: ',
|
||||||
err_export_pipes: 'Export does not support LogsQL pipes (| …): keep only the filter part',
|
err_export_pipes: 'Export does not support LogsQL pipes (| …): keep only the filter part',
|
||||||
tabApi: 'API',
|
|
||||||
tokensTitle: 'API tokens',
|
|
||||||
tokensHelp: 'Tokens give scripts and automation tools (n8n…) read-only access to /api/v1/logs, whatever the authentication mode. Additions and revocations apply with Save. Only a hash is kept: a new token is shown once, right after Save.',
|
|
||||||
tokenCount: (n) => `${n} token${n > 1 ? 's' : ''}`,
|
|
||||||
tokenNone: 'No API token yet.',
|
|
||||||
tokenNamePh: 'Token name (n8n, backup script…)',
|
|
||||||
tokenCreate: 'Add a token',
|
|
||||||
tokenOnce: 'Copy this token now: it will not be shown again.',
|
|
||||||
tokenCopy: 'Copy',
|
|
||||||
tokenRevoke: 'Revoke',
|
|
||||||
tokenUndo: 'Undo',
|
|
||||||
tokenCreatePending: 'created on Save',
|
|
||||||
tokenRevokePending: 'revoked on Save',
|
|
||||||
tokenExampleCopy: 'Copy the command',
|
|
||||||
tokenCreated: (d) => `created ${d}`,
|
|
||||||
tokenUsed: (p) => `last used ${p.d}${p.ip ? ' from ' + p.ip : ''}`,
|
|
||||||
tokenNever: 'never used',
|
|
||||||
tokensUse: 'Usage',
|
|
||||||
tokensUseHelp: 'Send the token in an Authorization header (highlighted below). Each answer has next_cursor: pass it as since on the next call to get the following logs, without gaps or duplicates (see the README for an n8n workflow).',
|
|
||||||
err_token_name: 'Give the token a name of 1 to 80 characters',
|
|
||||||
err_token_max: 'Too many API tokens',
|
|
||||||
tabLocale: 'Localization', tabFilters: 'Patterns', tabInterface: 'Appearance', tabData: 'Database',
|
tabLocale: 'Localization', tabFilters: 'Patterns', tabInterface: 'Appearance', tabData: 'Database',
|
||||||
themeTitle: 'Theme', themeSystem: 'System', themeLight: 'Light', themeDark: 'Dark',
|
themeTitle: 'Theme', themeSystem: 'System', themeLight: 'Light', themeDark: 'Dark',
|
||||||
themeHelp: 'System follows the light/dark preference of your computer or phone.',
|
themeHelp: 'System follows the light/dark preference of your computer or phone.',
|
||||||
@@ -427,27 +406,6 @@ const I18N = {
|
|||||||
exportNote: (n) => `Jusqu'à ${n} logs correspondant aux filtres, du plus récent au plus ancien. Dates dans le fuseau choisi dans Paramètres.`,
|
exportNote: (n) => `Jusqu'à ${n} logs correspondant aux filtres, du plus récent au plus ancien. Dates dans le fuseau choisi dans Paramètres.`,
|
||||||
exporting: 'Export…', exportDone: (f) => `Téléchargé : ${f}`, exportError: 'Échec de l\'export : ',
|
exporting: 'Export…', exportDone: (f) => `Téléchargé : ${f}`, exportError: 'Échec de l\'export : ',
|
||||||
err_export_pipes: 'L\'export ne gère pas les pipes LogsQL (| …) : gardez seulement la partie filtre',
|
err_export_pipes: 'L\'export ne gère pas les pipes LogsQL (| …) : gardez seulement la partie filtre',
|
||||||
tabApi: 'API',
|
|
||||||
tokensTitle: "Jetons d'API",
|
|
||||||
tokensHelp: "Les jetons donnent aux scripts et outils d'automatisation (n8n…) un accès en lecture seule à /api/v1/logs, quel que soit le mode d'authentification. Ajouts et révocations s'appliquent avec Valider. Seule une empreinte est conservée : un nouveau jeton n'est affiché qu'une fois, juste après Valider.",
|
|
||||||
tokenCount: (n) => `${n} jeton${n > 1 ? 's' : ''}`,
|
|
||||||
tokenNone: "Aucun jeton d'API pour l'instant.",
|
|
||||||
tokenNamePh: 'Nom du jeton (n8n, script de sauvegarde…)',
|
|
||||||
tokenCreate: 'Ajouter un jeton',
|
|
||||||
tokenOnce: "Copiez ce jeton maintenant : il ne sera plus affiché.",
|
|
||||||
tokenCopy: 'Copier',
|
|
||||||
tokenRevoke: 'Révoquer',
|
|
||||||
tokenUndo: 'Annuler',
|
|
||||||
tokenCreatePending: 'créé en validant',
|
|
||||||
tokenRevokePending: 'révoqué en validant',
|
|
||||||
tokenExampleCopy: 'Copier la commande',
|
|
||||||
tokenCreated: (d) => `créé le ${d}`,
|
|
||||||
tokenUsed: (p) => `dernière utilisation ${p.d}${p.ip ? ' depuis ' + p.ip : ''}`,
|
|
||||||
tokenNever: 'jamais utilisé',
|
|
||||||
tokensUse: 'Utilisation',
|
|
||||||
tokensUseHelp: "Envoyez le jeton dans un en-tête Authorization (surligné ci-dessous). Chaque réponse contient next_cursor : passez-le dans since à l'appel suivant pour obtenir les logs qui suivent, sans trou ni doublon (voir le README pour un workflow n8n).",
|
|
||||||
err_token_name: 'Donnez au jeton un nom de 1 à 80 caractères',
|
|
||||||
err_token_max: "Trop de jetons d'API",
|
|
||||||
tabLocale: 'Localisation et langue', tabFilters: 'Motifs de recherche', tabInterface: 'Affichage', tabData: 'Base de données',
|
tabLocale: 'Localisation et langue', tabFilters: 'Motifs de recherche', tabInterface: 'Affichage', tabData: 'Base de données',
|
||||||
themeTitle: 'Thème', themeSystem: 'Système', themeLight: 'Clair', themeDark: 'Sombre',
|
themeTitle: 'Thème', themeSystem: 'Système', themeLight: 'Clair', themeDark: 'Sombre',
|
||||||
themeHelp: 'Système suit la préférence clair/sombre de votre ordinateur ou de votre téléphone.',
|
themeHelp: 'Système suit la préférence clair/sombre de votre ordinateur ou de votre téléphone.',
|
||||||
@@ -2706,129 +2664,6 @@ $('#purgeBtn').addEventListener('click', async () => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
/* ================= API tokens ================= */
|
|
||||||
|
|
||||||
// Creations and revocations wait for Save like the other settings
|
|
||||||
// (settings.pending.tokens = { create: [names], revoke: [ids] }); a new token
|
|
||||||
// is shown once, after Save, with a curl example using it.
|
|
||||||
const tokens = { list: null, error: null, fresh: [] };
|
|
||||||
|
|
||||||
const pendingTokens = () => settings.pending.tokens || { create: [], revoke: [] };
|
|
||||||
function pendTokens(p) {
|
|
||||||
if (p.create.length || p.revoke.length) settings.pending.tokens = p;
|
|
||||||
else delete settings.pending.tokens;
|
|
||||||
renderTokens();
|
|
||||||
renderSaveState();
|
|
||||||
}
|
|
||||||
|
|
||||||
function curlExample(token) {
|
|
||||||
const url = `${location.origin}/api/v1/logs?query=severity:err&since=15m&limit=1000`;
|
|
||||||
return {
|
|
||||||
text: `curl -H "Authorization: Bearer ${token}" \\\n "${url}"`,
|
|
||||||
html: `curl -H "Authorization: Bearer <mark>${esc(token)}</mark>" \\\n "${esc(url)}"`,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
function renderTokens() {
|
|
||||||
const el = $('#tokenList');
|
|
||||||
const day = (x) => fmtFull(x).replace(/[.,]\d{3}(?=\D*$)/, '');
|
|
||||||
const p = pendingTokens();
|
|
||||||
$('#tokenCount').textContent = tokens.list ? t('tokenCount', tokens.list.length) : '';
|
|
||||||
|
|
||||||
// Tokens shown once after Save, and the curl example with the last one.
|
|
||||||
const fresh = tokens.fresh;
|
|
||||||
$('#tokenNew').hidden = !fresh.length;
|
|
||||||
$('#tokenNew').innerHTML = fresh.length ? `<p class="small">${esc(t('tokenOnce'))}</p>` + fresh.map((f, i) =>
|
|
||||||
`<div class="token-secret"><span class="token-label">${esc(f.name)}</span><code>${esc(f.token)}</code>`
|
|
||||||
+ `<button class="btn" type="button" data-copy-token="${i}">${esc(t('tokenCopy'))}</button></div>`).join('') : '';
|
|
||||||
$('#tokenExample').innerHTML = curlExample(fresh.length ? fresh[fresh.length - 1].token : 'lst_…').html;
|
|
||||||
|
|
||||||
if (!tokens.list) {
|
|
||||||
el.innerHTML = tokens.error ? `<div class="token-empty bad">${esc(tokens.error)}</div>` : '';
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
let html = tokens.list.map((k) => {
|
|
||||||
const gone = p.revoke.includes(k.id);
|
|
||||||
const used = k.last_used ? t('tokenUsed', { d: day(k.last_used), ip: k.last_ip }) : t('tokenNever');
|
|
||||||
const info = gone ? t('tokenRevokePending') : `${t('tokenCreated', day(k.created))} · ${used}`;
|
|
||||||
return `<div class="token-row${gone ? ' revoked' : ''}"><div><b>${esc(k.name)}</b> <code>${esc(k.hint)}…</code></div>`
|
|
||||||
+ (gone
|
|
||||||
? `<button class="btn" type="button" data-unrevoke="${esc(k.id)}">${esc(t('tokenUndo'))}</button>`
|
|
||||||
: `<button class="btn danger" type="button" data-revoke="${esc(k.id)}">${esc(t('tokenRevoke'))}</button>`)
|
|
||||||
+ `<div class="muted">${esc(info)}</div></div>`;
|
|
||||||
}).join('');
|
|
||||||
html += p.create.map((name, i) => `<div class="token-row pending"><div><b>${esc(name)}</b></div>`
|
|
||||||
+ `<button class="btn" type="button" data-uncreate="${i}">${esc(t('tokenUndo'))}</button>`
|
|
||||||
+ `<div class="muted">${esc(t('tokenCreatePending'))}</div></div>`).join('');
|
|
||||||
el.innerHTML = html || `<div class="token-empty">${esc(t('tokenNone'))}</div>`;
|
|
||||||
}
|
|
||||||
|
|
||||||
async function loadTokens() {
|
|
||||||
try {
|
|
||||||
tokens.list = await api('/api/tokens');
|
|
||||||
tokens.error = null;
|
|
||||||
} catch (e) {
|
|
||||||
tokens.list = null;
|
|
||||||
tokens.error = e.data?.code === 'read_only' ? '' : e.message;
|
|
||||||
}
|
|
||||||
renderTokens();
|
|
||||||
}
|
|
||||||
|
|
||||||
// Save: revocations first, then creations; the new tokens are shown once.
|
|
||||||
// What failed stays pending, so Save can try it again.
|
|
||||||
async function saveTokens() {
|
|
||||||
const p = pendingTokens();
|
|
||||||
const errors = [];
|
|
||||||
const left = { create: [], revoke: [] };
|
|
||||||
for (const id of p.revoke) {
|
|
||||||
try {
|
|
||||||
await api('/api/tokens/' + encodeURIComponent(id), { method: 'DELETE' });
|
|
||||||
} catch (e) {
|
|
||||||
if (!/not found/.test(e.message)) { errors.push(e.message); left.revoke.push(id); }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const fresh = [];
|
|
||||||
for (const name of p.create) {
|
|
||||||
try {
|
|
||||||
const res = await api('/api/tokens', { method: 'POST', body: { name } });
|
|
||||||
fresh.push({ name: res.name, token: res.token });
|
|
||||||
} catch (e) { errors.push(e.message); left.create.push(name); }
|
|
||||||
}
|
|
||||||
tokens.fresh = fresh;
|
|
||||||
if (errors.length) settings.pending.tokens = left;
|
|
||||||
else delete settings.pending.tokens;
|
|
||||||
await loadTokens();
|
|
||||||
if (errors.length) throw new Error(errors.join(' · '));
|
|
||||||
}
|
|
||||||
|
|
||||||
$('#tokenForm').addEventListener('submit', (ev) => {
|
|
||||||
ev.preventDefault();
|
|
||||||
const name = $('#tokenName').value.trim();
|
|
||||||
if (!name) { $('#tokenName').focus(); return; }
|
|
||||||
const p = pendingTokens();
|
|
||||||
pendTokens({ create: [...p.create, name], revoke: p.revoke });
|
|
||||||
$('#tokenName').value = '';
|
|
||||||
});
|
|
||||||
|
|
||||||
$('#tokenNew').addEventListener('click', (ev) => {
|
|
||||||
const b = ev.target.closest('[data-copy-token]');
|
|
||||||
if (b) copy(tokens.fresh[b.dataset.copyToken].token);
|
|
||||||
});
|
|
||||||
|
|
||||||
$('#tokenExampleCopy').addEventListener('click', () => {
|
|
||||||
const fresh = tokens.fresh;
|
|
||||||
copy(curlExample(fresh.length ? fresh[fresh.length - 1].token : 'lst_…').text);
|
|
||||||
});
|
|
||||||
|
|
||||||
$('#tokenList').addEventListener('click', (ev) => {
|
|
||||||
const p = pendingTokens();
|
|
||||||
const b = ev.target.closest('[data-revoke], [data-unrevoke], [data-uncreate]');
|
|
||||||
if (!b) return;
|
|
||||||
if (b.dataset.revoke) pendTokens({ create: p.create, revoke: [...p.revoke, b.dataset.revoke] });
|
|
||||||
else if (b.dataset.unrevoke) pendTokens({ create: p.create, revoke: p.revoke.filter((id) => id !== b.dataset.unrevoke) });
|
|
||||||
else pendTokens({ create: p.create.filter((_, i) => i !== Number(b.dataset.uncreate)), revoke: p.revoke });
|
|
||||||
});
|
|
||||||
|
|
||||||
/* ================= Tag management ================= */
|
/* ================= Tag management ================= */
|
||||||
|
|
||||||
const PALETTE = ['#6366f1', '#0ea5e9', '#14b8a6', '#a855f7', '#ec4899', '#eab308', '#64748b', '#f97316'];
|
const PALETTE = ['#6366f1', '#0ea5e9', '#14b8a6', '#a855f7', '#ec4899', '#eab308', '#64748b', '#f97316'];
|
||||||
@@ -2993,8 +2828,6 @@ $('#settingsBtn').addEventListener('click', () => {
|
|||||||
loadDocker();
|
loadDocker();
|
||||||
renderDbStats();
|
renderDbStats();
|
||||||
loadDbStats();
|
loadDbStats();
|
||||||
tokens.fresh = [];
|
|
||||||
loadTokens();
|
|
||||||
showSettingsTab(store.get('settingsTab', 'locale'));
|
showSettingsTab(store.get('settingsTab', 'locale'));
|
||||||
$('#settingsDlg').showModal();
|
$('#settingsDlg').showModal();
|
||||||
});
|
});
|
||||||
@@ -3044,7 +2877,6 @@ const SOURCE_SAVE = {
|
|||||||
selflog: async () => { selfState = await api('/api/selflog', { method: 'PUT', body: { enabled: selfState.enabled, min_level: selfState.min_level || 'warning' } }); },
|
selflog: async () => { selfState = await api('/api/selflog', { method: 'PUT', body: { enabled: selfState.enabled, min_level: selfState.min_level || 'warning' } }); },
|
||||||
docker: async () => { dockerState = await api('/api/docker', { method: 'PUT', body: settings.pending.docker }); },
|
docker: async () => { dockerState = await api('/api/docker', { method: 'PUT', body: settings.pending.docker }); },
|
||||||
files: saveFiles,
|
files: saveFiles,
|
||||||
tokens: saveTokens,
|
|
||||||
};
|
};
|
||||||
|
|
||||||
async function saveSettings() {
|
async function saveSettings() {
|
||||||
@@ -3123,7 +2955,6 @@ function cancelSettings() {
|
|||||||
settings.orig = {};
|
settings.orig = {};
|
||||||
fileDrafts = [];
|
fileDrafts = [];
|
||||||
reloadSources();
|
reloadSources();
|
||||||
renderTokens();
|
|
||||||
renderSaveState();
|
renderSaveState();
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -3198,7 +3029,7 @@ fetch('/auth/me').then((res) => (res.ok ? res.json() : null)).then((me) => {
|
|||||||
}
|
}
|
||||||
if (me.role === 'viewer') {
|
if (me.role === 'viewer') {
|
||||||
document.body.classList.add('read-only');
|
document.body.classList.add('read-only');
|
||||||
for (const p of document.querySelectorAll('[data-panel="filters"], [data-panel="sources"], [data-panel="data"], [data-panel="api"]')) {
|
for (const p of document.querySelectorAll('[data-panel="filters"], [data-panel="sources"], [data-panel="data"]')) {
|
||||||
for (const s of p.querySelectorAll('.set-section')) s.inert = true;
|
for (const s of p.querySelectorAll('.set-section')) s.inert = true;
|
||||||
const note = document.createElement('p');
|
const note = document.createElement('p');
|
||||||
note.className = 'ro-note';
|
note.className = 'ro-note';
|
||||||
|
|||||||
@@ -197,10 +197,6 @@
|
|||||||
<svg viewBox="0 0 24 24"><ellipse cx="12" cy="5" rx="8" ry="3"/><path d="M4 5v14c0 1.7 3.6 3 8 3s8-1.3 8-3V5M4 12c0 1.7 3.6 3 8 3s8-1.3 8-3"/></svg>
|
<svg viewBox="0 0 24 24"><ellipse cx="12" cy="5" rx="8" ry="3"/><path d="M4 5v14c0 1.7 3.6 3 8 3s8-1.3 8-3V5M4 12c0 1.7 3.6 3 8 3s8-1.3 8-3"/></svg>
|
||||||
<span data-i18n="tabData">Database</span>
|
<span data-i18n="tabData">Database</span>
|
||||||
</button>
|
</button>
|
||||||
<button type="button" role="tab" data-tab="api">
|
|
||||||
<svg viewBox="0 0 24 24"><circle cx="7.5" cy="15.5" r="4.5"/><path d="m10.7 12.3 9.3-9.3M16 7l3 3M18 5l2 2"/></svg>
|
|
||||||
<span data-i18n="tabApi">API</span>
|
|
||||||
</button>
|
|
||||||
</nav>
|
</nav>
|
||||||
<div id="setActions" class="set-actions">
|
<div id="setActions" class="set-actions">
|
||||||
<button id="setSave" class="btn save" type="button" disabled>
|
<button id="setSave" class="btn save" type="button" disabled>
|
||||||
@@ -450,31 +446,6 @@
|
|||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- API tokens -->
|
|
||||||
<div class="set-panel wide" role="tabpanel" data-panel="api" hidden>
|
|
||||||
<section class="set-section">
|
|
||||||
<div class="sec-head">
|
|
||||||
<h3 data-i18n="tokensTitle">API tokens</h3>
|
|
||||||
<span id="tokenCount" class="muted small"></span>
|
|
||||||
</div>
|
|
||||||
<p class="muted small" data-i18n="tokensHelp"></p>
|
|
||||||
<div id="tokenList" class="token-list"></div>
|
|
||||||
<form id="tokenForm" class="token-form">
|
|
||||||
<input id="tokenName" type="text" class="dk-filter" maxlength="80" autocomplete="off" spellcheck="false" data-i18n-ph="tokenNamePh" data-i18n-aria="tokenNamePh">
|
|
||||||
<button id="tokenCreate" class="btn" type="submit" data-i18n="tokenCreate">Add a token</button>
|
|
||||||
</form>
|
|
||||||
<div id="tokenNew" class="token-new" hidden></div>
|
|
||||||
</section>
|
|
||||||
<section class="set-section">
|
|
||||||
<h3 data-i18n="tokensUse">Usage</h3>
|
|
||||||
<p class="muted small" data-i18n="tokensUseHelp"></p>
|
|
||||||
<div class="token-example-box">
|
|
||||||
<pre id="tokenExample" class="token-example"></pre>
|
|
||||||
<button id="tokenExampleCopy" class="btn" type="button" data-i18n="tokenExampleCopy">Copy the command</button>
|
|
||||||
</div>
|
|
||||||
</section>
|
|
||||||
</div>
|
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
+11
-38
@@ -392,17 +392,18 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
|
|||||||
display: grid;
|
display: grid;
|
||||||
grid-template-columns: max-content max-content max-content 4.6rem minmax(5rem, 9rem) minmax(4rem, 8rem) var(--codes-w) 1fr;
|
grid-template-columns: max-content max-content max-content 4.6rem minmax(5rem, 9rem) minmax(4rem, 8rem) var(--codes-w) 1fr;
|
||||||
gap: 0 12px; align-items: baseline;
|
gap: 0 12px; align-items: baseline;
|
||||||
padding: 3px 14px 3px 11px;
|
padding: 5px 14px 5px 11px;
|
||||||
border-left: 3px solid transparent;
|
border-left: 3px solid transparent;
|
||||||
border-bottom: 1px solid var(--border);
|
border-bottom: 1px solid var(--border);
|
||||||
font-family: var(--log-font, var(--mono)); font-size: var(--log-size); line-height: 1.35;
|
font-family: var(--log-font, var(--mono)); font-size: var(--log-size);
|
||||||
font-variant-ligatures: none; /* keep -> != etc. as typed */
|
font-variant-ligatures: none; /* keep -> != etc. as typed */
|
||||||
cursor: pointer;
|
cursor: pointer;
|
||||||
}
|
}
|
||||||
.row:last-child { border-bottom: 0; }
|
.row:last-child { border-bottom: 0; }
|
||||||
/* Settings > Interface > Density: compact fits about 50% more lines on screen */
|
/* Settings > Interface > Density: compact fits about 50% more lines on screen */
|
||||||
:root[data-density="compact"] .row { padding-top: 1px; padding-bottom: 1px; line-height: 1.2; }
|
:root[data-density="compact"] .row { padding-top: 1px; padding-bottom: 1px; line-height: 1.25; }
|
||||||
:root[data-density="compact"] .row :is(.src-badge, .sev, .codes b) { line-height: 1.15; }
|
:root[data-density="compact"] .row .sev { padding-top: 0; padding-bottom: 0; line-height: 1.3; }
|
||||||
|
:root[data-density="compact"] .row .codes b { padding-top: 0; padding-bottom: 0; line-height: 1.2; }
|
||||||
.row:hover { background: var(--row-hover); }
|
.row:hover { background: var(--row-hover); }
|
||||||
.row:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; }
|
.row:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; }
|
||||||
.row.new { animation: flash 1.2s ease-out; }
|
.row.new { animation: flash 1.2s ease-out; }
|
||||||
@@ -420,7 +421,7 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
|
|||||||
.sev {
|
.sev {
|
||||||
justify-self: start; max-width: 100%; overflow: hidden; text-overflow: ellipsis;
|
justify-self: start; max-width: 100%; overflow: hidden; text-overflow: ellipsis;
|
||||||
font-size: 10.5px; font-weight: 700; text-transform: uppercase; letter-spacing: .03em;
|
font-size: 10.5px; font-weight: 700; text-transform: uppercase; letter-spacing: .03em;
|
||||||
padding: 0 6px; border-radius: 5px; line-height: 1.3;
|
padding: 1px 6px; border-radius: 5px;
|
||||||
color: var(--sev); background: color-mix(in srgb, var(--sev) 14%, transparent);
|
color: var(--sev); background: color-mix(in srgb, var(--sev) 14%, transparent);
|
||||||
}
|
}
|
||||||
.sev-crit { --sev: var(--tag-error); --sev-text: var(--tag-error-text); }
|
.sev-crit { --sev: var(--tag-error); --sev-text: var(--tag-error-text); }
|
||||||
@@ -447,11 +448,12 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
|
|||||||
.row.src-audit, .row.src-logstream, .row.src-file { border-left-color: var(--src); background: color-mix(in srgb, var(--src) var(--src-tint), transparent); }
|
.row.src-audit, .row.src-logstream, .row.src-file { border-left-color: var(--src); background: color-mix(in srgb, var(--src) var(--src-tint), transparent); }
|
||||||
.row.src-audit:hover, .row.src-logstream:hover, .row.src-file:hover { background: color-mix(in srgb, var(--src) 13%, var(--row-hover)); }
|
.row.src-audit:hover, .row.src-logstream:hover, .row.src-file:hover { background: color-mix(in srgb, var(--src) 13%, var(--row-hover)); }
|
||||||
.src-badge {
|
.src-badge {
|
||||||
justify-self: start; box-sizing: content-box; width: 6ch; padding: 0 4px; border-radius: 4px; text-align: center;
|
justify-self: start; box-sizing: content-box; width: 6ch; padding: 1px 4px; border-radius: 4px; text-align: center;
|
||||||
font-family: var(--log-font, var(--mono)); font-size: 10px; font-weight: 700; letter-spacing: .02em; line-height: 1.3;
|
font-family: var(--log-font, var(--mono)); font-size: 10px; font-weight: 700; letter-spacing: .02em; line-height: 1.35;
|
||||||
color: var(--src); background: color-mix(in srgb, var(--src) 16%, transparent);
|
color: var(--src); background: color-mix(in srgb, var(--src) 16%, transparent);
|
||||||
box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--src) 40%, transparent);
|
box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--src) 40%, transparent);
|
||||||
}
|
}
|
||||||
|
:root[data-density="compact"] .row .src-badge { padding-top: 0; padding-bottom: 0; line-height: 1.3; }
|
||||||
.row .raw {
|
.row .raw {
|
||||||
margin-top: 2px; font-size: .85em; line-height: 1.35; color: var(--muted);
|
margin-top: 2px; font-size: .85em; line-height: 1.35; color: var(--muted);
|
||||||
white-space: pre-wrap; word-break: break-all;
|
white-space: pre-wrap; word-break: break-all;
|
||||||
@@ -463,10 +465,10 @@ body.col-resizing, body.col-resizing * { cursor: col-resize !important; user-sel
|
|||||||
.row .codes { white-space: nowrap; overflow: hidden; }
|
.row .codes { white-space: nowrap; overflow: hidden; }
|
||||||
.row .codes > span { display: inline-flex; gap: 3px; vertical-align: 1px; }
|
.row .codes > span { display: inline-flex; gap: 3px; vertical-align: 1px; }
|
||||||
.row .codes b, .tag-code {
|
.row .codes b, .tag-code {
|
||||||
min-width: 2.2ch; padding: 0 4px; border-radius: 5px; text-align: center;
|
min-width: 2.2ch; padding: 1px 4px; border-radius: 5px; text-align: center;
|
||||||
/* same font as the severity badges (the log font) */
|
/* same font as the severity badges (the log font) */
|
||||||
font-family: var(--log-font, var(--mono)); font-size: 10.5px; font-weight: 700; letter-spacing: .03em;
|
font-family: var(--log-font, var(--mono)); font-size: 10.5px; font-weight: 700; letter-spacing: .03em;
|
||||||
line-height: 1.3; font-variant-numeric: tabular-nums;
|
line-height: 1.35; font-variant-numeric: tabular-nums;
|
||||||
background: var(--code-bg); color: #0b0f17;
|
background: var(--code-bg); color: #0b0f17;
|
||||||
}
|
}
|
||||||
.row .codes b.more { background: none; box-shadow: inset 0 0 0 1px var(--code-bg); color: var(--muted); }
|
.row .codes b.more { background: none; box-shadow: inset 0 0 0 1px var(--code-bg); color: var(--muted); }
|
||||||
@@ -912,32 +914,3 @@ body.login-page {
|
|||||||
.login-card input:focus { outline: 0; border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); }
|
.login-card input:focus { outline: 0; border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); }
|
||||||
.login-card .btn.primary { height: 38px; justify-content: center; margin-top: 6px; font-size: 14px; }
|
.login-card .btn.primary { height: 38px; justify-content: center; margin-top: 6px; font-size: 14px; }
|
||||||
.login-err { margin: 0; color: var(--sev-err); font-size: 13px; }
|
.login-err { margin: 0; color: var(--sev-err); font-size: 13px; }
|
||||||
|
|
||||||
/* API tokens (Settings > API) */
|
|
||||||
.token-list { display: flex; flex-direction: column; gap: 4px; margin: 12px 0 10px; }
|
|
||||||
.token-row {
|
|
||||||
display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 2px 12px; align-items: center;
|
|
||||||
padding: 8px 10px; border-radius: 8px; background: var(--panel-2); font-size: 12.5px;
|
|
||||||
}
|
|
||||||
.token-row b { font-weight: 600; overflow-wrap: anywhere; }
|
|
||||||
.token-row code { font-family: var(--mono); font-size: 11.5px; color: var(--muted); }
|
|
||||||
.token-row .muted { grid-column: 1; font-size: 11.5px; }
|
|
||||||
.token-row .btn { grid-row: 1 / span 2; grid-column: 2; height: 28px; font-size: 12px; }
|
|
||||||
.token-empty { color: var(--muted); font-size: 12.5px; }
|
|
||||||
.token-form { display: flex; gap: 10px; align-items: center; }
|
|
||||||
.token-form .btn { height: 28px; font-size: 12px; }
|
|
||||||
.token-new { margin-top: 12px; padding: 10px 12px; border-radius: 8px; border: 1px solid var(--accent); background: var(--accent-soft); }
|
|
||||||
.token-secret { display: flex; gap: 10px; align-items: center; margin-top: 8px; }
|
|
||||||
.token-label { font-size: 12.5px; font-weight: 600; white-space: nowrap; }
|
|
||||||
.token-row.revoked b, .token-row.revoked code { text-decoration: line-through; opacity: .6; }
|
|
||||||
.token-row.pending { border: 1px dashed var(--accent); }
|
|
||||||
.token-example-box { display: flex; gap: 10px; align-items: flex-start; margin-top: 8px; }
|
|
||||||
.token-example-box .token-example { flex: 1; min-width: 0; margin: 0; }
|
|
||||||
.token-example-box .btn { height: 28px; font-size: 12px; }
|
|
||||||
.token-example mark { background: var(--accent-soft); color: var(--accent); border-radius: 3px; padding: 0 2px; font-weight: 600; }
|
|
||||||
.token-secret code { flex: 1; min-width: 0; font: 12px var(--mono); overflow-wrap: anywhere; user-select: all; }
|
|
||||||
.token-secret .btn { height: 28px; font-size: 12px; }
|
|
||||||
.token-example {
|
|
||||||
margin: 8px 0 0; padding: 8px 12px; border-radius: 8px; background: var(--panel-2);
|
|
||||||
font: 12px var(--mono); white-space: pre-wrap; overflow-wrap: anywhere;
|
|
||||||
}
|
|
||||||
Reference in new issue
Block a user